The newly established clearinghouse to approve Family First Act services has looked at three kinship navigators, one-stop shops set up to help kin who take in a grandchild, nephew or family friend.
It has rejected all three, with one more currently under consideration. And frustration is mounting with the way the clearinghouse is going about the process.
“We see a significant need for technical assistance to kinship navigator programs across the country who are working diligently to design and complete evaluations that meet the criteria,” said Jaia Lent, deputy executive director of Generations United, in an email to The Imprint. “Many have contacted the clearinghouse with clarifying questions, but the clearinghouse has simply referred them back to the handbook which does not address many of their questions.”
The clearinghouse announced last week that Ohio’s Kinship Supports Intervention model “does not meet the criteria” to be rated as “Promising,” the lowest threshold of evidence that a program must receive to be fundable under the new law. The stakes here are significant for three key reasons:
- Kinship navigators (there are about six dozen around the country) are really the front line in this country to helping relatives asked by child welfare agencies to care for children. Some of these relatives are licensed foster parents, but most of them are not, and do not receive much in the way of direct support from child welfare agencies.
- A kinship model approved by the clearinghouse could be paid for in a 50-50 match with the federal government. Congress has been approving $20 million nationwide for creation or support of kinship navigators in recent years – much more could flow to existing navigators under an even split, and states would be more enticed to invest additional funds as well.
- Were a version of navigator to be approved for matched federal funds, it would immediately become the template for expansion of these programs to more places in the country. And a lot of existing navigator programs would likely consider adopting the approved model to be the core of its operation.
The rejection of two versions reviewed last year, from New Jersey and Florida, were less of a surprise – they lacked the kind of robust studies necessary to gain a Promising rating. But the Ohio model came with a recently completed study that researchers felt would earn it approval. It already has obtained Promising status from a similar body, the California Evidence-Based Clearinghouse on Child Welfare.
The evaluation, which studies outcomes for more than 8,000 children, compared those placed with relatives receiving Ohio Kinship Supports Intervention to children in two groups – those placed with relatives who did not get the services, and nonrelative foster parents.
The evaluation found that compared to both groups, the children in homes supported by the kinship navigator experienced shorter and more stable stays outside of their home.
So what was the problem? The Family First clearinghouse reviewed the study, and rated it to be of “Low” quality. That stops the process in its tracks – under the clearinghouse’s rules, a study cannot be considered in the case for approval unless it has at least a “Medium” rating.
The clearinghouse includes a line in its review of the program explaining, in research-ese, why it deemed the study’s structure to be insufficient:
This study received a low rating because baseline equivalence of the intervention and comparison groups was necessary and not demonstrated.
Translation after speaking to some researchers on background: For a quality study, one must be able to demonstrate that the groups of people being compared are basically the same. If one comparison group was mostly middle class and another include mostly poor families, for example, it would make it hard to know the true effect of the program.
But authors in the Ohio navigator study used a research technique called propensity score matching to neutralize any differences in the three groups. That is not as good as a randomized control trial (RCT) approach, the gold standard for such ventures, but RCTs are more expensive and not always feasible.
“You can’t get much closer to Promising than what this study tried to do using a quasi-experimental design,” said Mark Testa, a veteran child welfare researcher who co-edited the journal that published the Ohio evaluation. “It probably would have sealed the deal if they had done an RCT that showed improvements.”
Cailin Wheeler, a co-author on the evaluation, said she disagreed with the clearinghouse’s assessment. The study was done before the clearinghouse existed, she said, and she has since rerun data using standards for baseline equivalence suggested by the clearinghouse.
“As expected, the findings remain the same,” Wheeler said, in an email to The Chronicle. “We are hopeful that the clearinghouse will be open to providing us with additional feedback beyond what was posted on the website, and to reviewing our updated analyses for potential reconsideration as a promising practice.”
But advocates for relative caregivers have said the clearinghouse has not been great about communicating around research issues with the navigator programs.
“Many kinship navigator programs need answers to questions and technical assistance to help their programs meet the clearinghouse requirements,” said Lent, of Generations United.
Lent said she understood the need to avoid conflicts of interest by the clearinghouse – which is managed on contract by a firm called Abt Associates – but said those kinds of questions should be routed to federal agencies that could provide insight.