Credit: The New York Times
Youth Services Insider thought it was worth directing readers to this New York Times op-ed piece from last week by child welfare policy expert Ron Haskins. The Times doomed him to traffic mediocrity by posting it on New Year’s Eve.
Haskins, a veteran Republican staffer on Capitol Hill who now serves as a senior fellow at the Brookings Institution, penned an argument for the movement toward requiring a base of concrete evidence before funding could flow to social services programs. He credits the administration of George W. Bush for beginning that movement in Washington, and credits President Obama for pursuing that standard.
Something you don’t typically hear from a conservative thinker: “It’s imperative that the new Congress reject efforts by some Republicans to cut the Obama administration’s evidence-based programs.”
It is a well-written and short piece that we’d recommend to readers. This paragraph near the end of the column really hits home why this is a big issue:
“The evidence-based movement separates the wheat from the chaff. If this movement spreads to more federal programs, especially the big education, employment and health programs supported by formula-based grants, we can expect consternation and flailing as many program operators discover that their programs are part of the chaff.”
If anyone in youth services still thinks that federal programs will continue to be funded without any evidence that they work, they are out of their minds. If there is a Republican president and Congress, youth-serving programs with big federal funding lines will be called to account soon.
Having said that, here are two thoughts on why the evidence-based movement is so good in theory and so tough in the real world.
Lack of agreement on the word “works”
It should be easy for everyone to come around to the idea that the federal government shouldn’t fund programs that do not work. A harder conversation is the one about what it means to “work.”
Haskins suggests that Head Start, a multi-billion dollar federal expenditure championed by many, as one that might not survive a full evidence-based movement. Head Starts critics point to a national, randomized control study to show that the academic gains made by Head Start participants fade by third grade. Head Start’s defenders point to other studies showing long-term socio-emotional assets connected to Head Start. (W. Steven Barnett, director of the National Institute for Early Education Research at Rutgers University, covers this well here.)
Another example: Recidivism, a word that constantly pops up when it comes time to discuss the effectiveness of juvenile justice interventions. On a basic level recidivism measures whether someone did something wrong after an intervention aimed at stopping them from doing it.
But there are a lot of ways to mark that. Do you count juvenile rearrests, or only instances when they are adjudicated/convicted again? Do you track recidivism over one year, three years? Do you only count offenses that rise above a misdemeanor?
And that is all stuff to figure out before you get to the question of what recidivism rate suggests that a certain program or intervention is “not working.” How many juvenile offenders out of ten need to stay recidivism-free to say that a program they participated in is “working”?
It doesn’t even end there. What about a youth who was rearrested after a stint with a community-based program, but also finished high school and got a job at a grocery store right after graduation? That juvenile recidivated, and did not go to college. But he completed high school and is employed with possibility to move up.
Is he a win or a loss for that program in the game of “Does It Work?”
It is worth noting that the Office of Juvenile Justice and Delinquency Prevention once set out to establish a standard process for measuring recidivism. As far as YSI knows, that project died on the vine.
The Politics of Chaff Identification
YSI agrees with Haskins that an evidence-based movement does drive some to defend programs they’ve supported or worked on for years, regardless of what the evidence shows.
But the political reality of effectiveness-gauging creates another defensive group with a more practical belief: That some money going to questionably effective programs is better than no money going to no programs. There is a real fear that when it comes time to identify the programs without evidence, the money won’t be reinvested in programs with evidence; it will just be captured as Beltway belt-tightening.
Haskins’ own metaphor reinforces that fear somewhat: wheat is reaped for its value, and chaff is cast aside.
That flies in the face of two truths about things that “don’t work” in our personal lives. The first is that some of them just need to be fixed; they aren’t permanently broken. And when things are permanently broken, we tend to replace them.
Youth Services Insider is mostly written by Chronicle Editor John Kelly.
