In response to the death of an 11-year-old boy, the Los Angeles County Office of Child Protection will evaluate how the county’s child protection system measures risk and report to the Board of Supervisors in 30 days.
During a board meeting yesterday, the five county supervisors discussed the need to evaluate how social workers in the Department of Children and Family Services (DCFS) assess risk and to compare the tools they use with potential alternatives.
The motion, introduced by Supervisors Michael Antonovich and Mark Ridley-Thomas, called to give the Office of Child Protection 60 days to report back, but the supervisors amended the timeline for the report to 30 days before they voted to approve the motion.
The supervisors spoke about Yonatan Aguilar, who was found dead in his Echo Park home last month, and how DCFS had conducted a handful of investigations of suspected abuse at his home. During these visits, a risk and safety assessment tool used by social workers determined that Aguilar fit into a “high risk” category for child maltreatment, but that he was not in danger of actual harm.
The tool used, called Structured Decision Making (SDM), is an assessment tool that all DCFS social workers use during investigations into child abuse and neglect. It prompts social workers to answer multiple choice questions that collectively categorize a child’s risk level as high, moderate or low and gauge whether a child is in danger. Because social workers using SDM did not determine that Aguilar was in danger, the supervisors want to look at SDM’s effectiveness and whether social workers would benefit from another tool, such as a predictive analytics tool wherein algorithms crunch big data to determine the likelihood that an outcome will occur.
Michael Nash, director of the county’s Office of Child Protection, said he is not sure if the Aguilar case calls into question the effectiveness of SDM. However, he wants to investigate SDM and its alternatives, he said.
“I do have some questions about the tool,” Nash said. “I have questions about the strengths and weaknesses of that tool. I have questions about how we train [social workers] on that tool here in Los Angeles County.”
“SDM was utilized in this case, and there’s no suggestion I’ve seen thus far that it was utilized incorrectly,” Nash added in a phone interview after the meeting. “There may be questions about the response to the assessment, but there’s no suggestion at this point that the assessment was inaccurate.”
During the board meeting, Ridley-Thomas said that the county needs to look further into technological advancements, such as tools that use predictive analytics.
In 2013, in an effort to help investigators make better decisions, DCFS contracted with SAS, the world’s largest private software firm, to test predictive analytics.
The experiment, dubbed AURA, or Approach to Understanding Risk Assessment, tracked child deaths, near fatalities and “critical incidents” in 2011 and 2012. Using a mix of data including, but not limited to, prior child abuse referrals, involvement with law enforcement, as well as mental health records and alcohol and substance abuse history, SAS statisticians created a risk score from one to 1,000, wherein high numbers demark high risk.
Ridley-Thomas said that the county has paid too little attention to the idea of using predictive analytics in child protection.
“We haven’t put enough energy into it, intellectually or otherwise,” Ridley-Thomas said.
The supervisors also spoke about the need for sufficient training and the importance of a social worker’s judgement in the risk assessment process.
“If the user who inputs the data isn’t adequately trained, what good is the system?” asked Supervisor Hilda Solis.
Nash said he wants to reach out to a wide variety of people to discuss SDM, including county employees, academics and nonprofit leaders.
“I think it’s an important issue and it’s one that I’ve already been looking at,” Nash said. “It’s just too bad that it comes within the context of a dead child.”
Note: An earlier version of this article stated that SDM did not determine Aguilar was in danger. Decisions not to take further action in the Aguilar case were based on decisions made by social workers at DCFS. It is not clear what results SDM produced regarding the boy’s safety and how those results were used.