Child welfare agencies across the country are turning to predictive analytics, but few families know they are subject to them
Our country’s latest reckoning with structural racism has involved critical reflection on the role of the criminal justice system, education policy and housing practices in perpetuating racial inequity. The child welfare system — or, as some are now more appropriately calling it, the family regulation system — needs to be added to this list, along with the algorithms, risk assessments and predictive analytics working behind the scenes.
That’s why the American Civil Liberties Union has conducted a nationwide survey to learn more about these tools. And what we found makes us concerned that instruments that can lead to the separation of families are being built and deployed without the input of the communities they will impact.
Many child welfare agencies have begun turning to risk assessment tools for reasons ranging from wanting the ability to predict which children are at higher risk for maltreatment to improving agency operations. Allegheny County, Pennsylvania, has been using the Allegheny Family Screening Tool since 2016. This tool generates a risk score for complaints received through the county’s child maltreatment hotline by looking at whether certain characteristics of the agency’s past cases are also present in the complaint allegations.
Key among these characteristics are family member demographics and prior involvement with the county’s child welfare, jail, juvenile probation and behavioral health systems. Intake staff then use this risk score as an aid in deciding whether or not to follow up on a complaint with a home study or a formal investigation, or to dismiss it outright.
Like their criminal justice analogues, however, child welfare risk assessment tools do not predict the future. For instance, a recidivism risk assessment tool measures the odds that a person will be arrested in the future, not the odds that they will actually commit a crime. Just as being under arrest doesn’t necessarily mean you did something illegal, a child’s removal from the home, often the target of a prediction model, doesn’t necessarily mean a child was in fact maltreated.
We examined how many jurisdictions across the 50 states, D.C. and U.S. territories are using one category of predictive analytics tools: models that systematically use data collected by jurisdictions’ public agencies to attempt to predict the likelihood that a child in a given situation or location will be maltreated. Here’s what we found:
- Local or state child welfare agencies in at least 26 states plus D.C. have considered using such predictive tools. Of these, jurisdictions in at least 11 states are currently using them.
- Large jurisdictions like New York City, Oregon and Allegheny County have been using predictive analytics for several years now.
- Some tools currently in use, such as Allegheny County’s, are used when deciding whether to refer a complaint for further agency action, while others are used to flag open cases for closer review because the tool deems them to be higher-risk scenarios.
Despite the growing popularity of these tools, few families or advocates have heard about them, much less provided meaningful input into their development and use. Yet countless policy choices and value judgments are made in the course of creating and using the tool, any or all of which can impact whether the tool promotes “fairness” or reduces racial disproportionality in agency action.
Moreover, like the tools we have seen in the criminal legal system, any tool built from a jurisdiction’s historical data runs the risk of continuing and increasing existing bias. Historically over-regulated and over-separated communities may get caught in a feedback loop that quickly magnifies the biases in these systems. Who decides what “high risk” means? And when a caseworker sees a “high” risk score for a Black person, do they respond in the same way as they would for a white person?
Ultimately, we must ask whether these tools are the best way to spend hundreds of thousands, if not millions of dollars, when such funds are urgently needed to help families avoid the crises that lead to abuse and neglect allegations.
It’s critical that we interrogate these tools before they become entrenched, as they have in the criminal justice system. Information about the data used to create a predictive algorithm, the policy choices embedded in the tool, and the tool’s impact both system-wide and in individual cases are some of the things that should be disclosed to the public before a tool is adopted and throughout its use. In addition to such transparency, jurisdictions need to make available opportunities to question and contest a tool’s implementation or application in a specific instance if our policymakers and elected officials are to be held accountable for the rules and penalties enforced through such tools.
In this vein, the ACLU has requested data from Allegheny County and other jurisdictions to independently evaluate the design and impact of their predictive analytics tools and any measures they may take to address fairness, due process and civil liberty concerns.
It’s time that all of us ask our local policymakers to end the unnecessary and harmful policing of families through the family regulation system.