The 2016 death of 11-year-old Yonatan Aguilar reignited a years-long debate about the strengths and weaknesses of a tool used by social workers across Los Angeles County to determine the risk a child will be abused.
On Tuesday, the Los Angeles County Board of Supervisors voted to keep the risk-assessment protocol, called Structured Decision Making (SDM), in use through 2020, with a new focus on training aimed at addressing concerns with its use.
“This will help to better protect the county’s most vulnerable children preventatively and diagnostically,” said Supervisor Mark Ridley-Thomas in an email.
Following Yonatan’s death last year, Ridley-Thomas and former Supervisor Michael Antonovich issued a motion calling on the county’s Office of Child Protection (OCP) to evaluate the use of SDM by the county’s Department of Children and Family Services (DCFS) and compare it with the nascent application of predictive analytics in child maltreatment detection.
Developed by the nonprofit National Council on Crime and Delinquency (NCCD), SDM is similar to the “actuarial” questionnaires that insurers use to determine rates. It is used in jurisdictions across the country, including all 58 counties in California. L.A.’s system consists of six questionnaires that social workers fill out to determine both the risk of immediate harm and the likelihood of future abuse. Predictive analytics tools, like one being used in Allegheny County, Pennsylvania, are entirely automated, aggregating spools of public records to gauge the risk that a child will be abused.
This became the center of some controversy. While L.A. County questioned the merits of SDM, NCCD defended its technology and laid the blame for Yonatan’s death on DCFS’ implementation of the tool. NCCD has enjoyed an exclusive contract to provide mechanized risk analysis support to DCFS, and the department’s former director, Philip Browning, suggested the desire to preserve that contract might be part of the reason for the pushback from the nonprofit.
“I think they [NCCD] know we’re interested in a predictive analytics tool, and I think they see that as competition, and they are concerned about that,” Browning said in an interview last year. “And I think they have a proprietary interest in continuing what they’re doing.”
In May, the OCP issued a report recommending that DCFS continue using SDM, but with some changes. Of the report’s five recommendations, three were focused on improving the county’s use of SDM, predominantly through increased oversight and training.
The board’s decision to approve the motion came after DCFS worked with the OCP to ensure that any new contract with NCCD would allow for the increased oversight described in the May report.
“We’re working with NCCD to reinvigorate our training to have more accountability and quality checks,” said Diane Iglesias, senior deputy director at DCFS. The department is discussing with NCCD how best to roll out the training, and Iglesias expects they will have a contract before the board early next year. Some 4,000 social workers will receive this enhanced training, and the department is looking into what other staff might benefit from a better understanding of the tool.
The contract approved in Tuesday’s motion set aside a maximum budget of $178,655 per year for the three-year term, just slightly more than the contract ending this month. The department allocated 10 percent of its 2017 SDM contract budget to begin planning for the enhanced implementation.
OCP Executive Director Michael Nash said that a key aim of the training is “so social workers can truly understand the value of SDM … with the understanding that SDM doesn’t replace social workers’ knowledge and common sense.”
When used properly, SDM has been shown to increase the accuracy of risk assessments, according to a study promoted on NCCD’s website. However, DCFS social workers are not obligated to act based on the SDM recommendations; in fact, Yonatan’s tragic death highlighted a problematic discrepancy between DCFS’s policy and the protocol set forth in the SDM manual.
In four separate referrals between 2009 and 2012, the system categorized Yonatan’s risk of future abuse as “high.” However, the safety assessments all four times indicated that “no safety threats” were identified, so the investigations were closed.
While NCCD recommends opening a case in essentially all circumstances that are categorized by SDM as “high” or “very high” risk, DCFS policy mandates that “[u]nder no circumstances are users to select the client disposition of ‘Open New Case’ for a child without a substantiated allegation [of abuse or neglect].”
For an allegation to be “substantiated,” the investigating social worker must determine that the child or children are in imminent risk of harm, in which case the worker will either detain them on the spot or set up a “safety plan” to mitigate safety threats until they can get a warrant to remove the child or children. If no immediate danger is identified, DCFS does not have the legal right to take a child from their home.
A key finding of the OCP report, though, was that unsubstantiated reports of abuse resulted in near-fatal incidence at almost precisely the same rate as allegations that were substantiated. Moreover, within the sample of 1,225 DCFS referrals analyzed, claims that were not substantiated were followed by a fatality nearly twice as often as those that were.
Nash, who recommended in his report that DCFS reevaluate this policy, said he’s confident that it will be changed, adding that OCP will work alongside DCFS and NCCD as these changes are implemented.
“We’ll be at the table with them, and we’ll make sure this is part of the conversation,” he said.
This renewed investment in SDM does not indicate a move away from considering predictive analytics as a future mechanism for identifying kids at risk of abuse. California is looking into testing a model to be used at the referral hotline level to help determine whether an investigation is appropriate.
“Predictive analytics is on a parallel track,” Nash said. “I don’t know if it will eventually replace SDM or be used alongside it. But we’re just not there yet.”