By Darian Woods
Imagine that in a Child, Youth and Family (CYF) call center in Auckland, New Zealand, operators are interpreting rows of numbers on a computer monitor. Among the data points: The age of mothers on a benefit, the date of their first benefit payment, family type and 129 other details.
These numbers would be fed through a data tool that predicts the risk that a child will be maltreated. The tool may soon mean that, when a phone call comes in reporting suspected child abuse, a social worker may decide to spend more time working with a home flagged as high-risk.
“We now have a golden opportunity in the social sector to use data analytics to transform the lives of vulnerable children,” former Minister of Social Development Paula Bennett told statisticians in a speech at a Statistical Analysis System (SAS) users conference in Wellington last year.
These child abuse data analytics will be a world first. Using the tool in New Zealand and globally requires confronting hard ethical issues, including the possibility that minorities such as Māori may be over-represented in prioritized cases.
Some on the front line are urging for more transparency. This includes Dr. Ed Mitchell and Dr. Patrick Kelly. Mitchell is a University of Auckland child health researcher whose work on sudden infant death inspired New Zealand’s “back to sleep” program to prevent crib death.
Kelly is the director of Auckland District Health Board’s child abuse unit Te Puaruruhau. Te Puaruruhau is an office across the road from the Starship Children’s Hospital bringing together law enforcement, CYF workers and pediatricians to care for abused children.
These child health experts wish to understand the tool better by opening it up to closer peer review.
Using data to predict abuse
Auckland University of Technology professor Dr. Rhema Vaithianathan was the lead researcher developing the Predictive Risk Modeling (PRM) tool that may soon support decisions at child abuse call centers. She described how a medical professional can walk through a maternity ward and be able to identify, out of five children whose families are on a benefit, one will have a 40 percent or higher chance of being abused.
“We see this train wreck in slow motion and no one’s doing anything about it until that first call comes in,” Vaithianathan said. “The question is, do we have an obligation to do something?”
In 2012 Vaithianathan led a team of researchers from University of Auckland and partnered with the Ministry of Social Development (MSD) to analyze government data in the hope of identifying children at risk of abuse. The formula found that, with existing information about families receiving government assistance, analysts could predict child maltreatment with about the same accuracy as a mammogram detects breast cancer.
The paper discussed how this may mean that vulnerable families could be offered programs such as parenting classes ahead of time, preventing the likelihood of these children being physically, emotionally or sexually abused.
A new paper by MSD officials, building on this model, has been published in the latest American Journal of Preventive Medicine, concluding that trials “are needed to establish whether risks can be mitigated.”
146,657 Calls
New Zealand government appears likely to move forward in the near future with using data to screen the vast numbers of calls to CYF, most of which are assessed as not requiring further action. In the fiscal year ending 2014 there were nearly 147,000 allegations filed with the department; 54,000 of these were followed up with further action.
A single mother in Gisborne, who spoke under the condition of anonymity, was investigated by CYF last year when her daughter’s father called the police claiming abuse was occurring in her house. While it was quickly apparent that this was a false claim by the father due to a relationship breakdown, social workers were obliged by internal policies to spend hours talking with the woman and her daughter in her house.
“They said they had to give this case high priority because it came through the police,” she said. “I couldn’t help but wonder who else was missing out because their calls went through CYF.”
The hope by analytics supporters is that a data-driven approach will sort the thousands of calls into various levels of urgency based on what is known about families with certain characteristics.
Rushing In?
Dr. Kelly was careful to point out that he can understand why the Ministry of Social Development is considering using this data.
“You could argue that the current way is arbitrary and unreliable, and I can’t disagree with that,” Kelly said.
Yet he holds a real unease that New Zealand might be rushing into this. “The lack of published detail means that this model hasn’t been opened up to critical review,” he said. “I’ve seen a whole lot of interventions. The universal failing is the failure to evaluate. MSD still doesn’t know, of all the programs that were implemented, which ones worked.”
Further, he is concerned about the way this might actually work on the ground. If this tool is used to prioritize reported abuse differently, there will be situations in which a social worker’s experience contradicts the findings of the tool. In which situations will the tool be superior to social worker judgment?
Kelly offered a solution, echoed by officials in the Ministry of Social Development: run this as a randomly controlled trial. Half of calls to CYF could be randomly assigned to social workers using current practices, while the other half are assigned to social workers using the decision support tool. MSD can then investigate whether this actually had an effect in reducing assaults and neglect on children.
Kelly cited one program underpinned by strong international evidence, such as the Nurse-Family Partnership program, which is already given to targeted families.
“We already have entry criteria for targeting programs. If you’re going to use alternative entry criteria, why not test it?”
Disproportionate Demographics
Stigmatization of minorities and families receiving government assistance is one issue. This tool, according to child health researcher Mitchell, may perpetuate stereotypes.
“There is enough prejudice as it is,” Mitchell said. While admitting that receiving public assistance is strongly correlated with child maltreatment, he worries about the tool reinforcing biases against such families.
“I hate the idea of targeting this group,” he said.
Māori children make up only 22 percent of New Zealand’s child population, but represent close to half of children in out-of-home care. This tool does not include race as a predictor, but other characteristics where Māori are over-represented may mean that Māori are more likely to appear in the high-risk category.
MSD’s latest paper on predictive modeling says, “given their over-representation, Māori would experience more keenly both benefits and harms.” A Cabinet paper from 2014 says that the tool over-identified Māori children and under-identified Pākehā (non-Māori) children. The paper states that MSD is “working through a number of options for addressing this.”
Stigma may be an issue, though Vaithianathan and Kelly said that this tool is unlikely to make matters worse.
“Stigmatization already exists,” said Starship Children’s Hospital pediatrician Dr. Patrick Kelly.
Vaithianathan is mindful of the ethical issues surrounding data and child welfare, saying that these conversations require statisticians, social work practitioners, ethics specialists and experts on disparities to come together. She has spoken personally with Kelly about his concerns.
“We’re getting these people around the table to negotiate,” she said. “The social workers are critical to speak to the ethics point of view. They know what’s happening on the front line.”
Ahead of the Curve
“New Zealand is definitely on the cutting edge,” said Assistant Professor at the School of Social work at University of Southern California, Emily Putnam-Hornstein, who worked on the 2012 PRM research and has also studied predictors of child abuse in California.
New Zealand’s advantage is that rich data on families are integrated: Both child protection services and welfare payments come from the same department. In the United States, for example, child protection services data are collected at the state level, but held in many different formats that are not easily compatible with each other.
In the United States, Pennsylvania’s Allegheny County (Pittsburgh) is attempting a similar predictive screening process.
New Zealand and Allegheny County’s experiences could provide lessons for reducing child abuse across the world. Researchers working on the two systems are already collaborating.
Seductively attractive data
We are entering new territory with using data to triage abuse calls. The stakes are high. Children’s lives are literally at stake. There is palpable tension between getting this done as soon as possible and getting this done with full evaluation and ethical considerations.
“It’s important for us not to be overly sensitive to this issue,” Vaithianathan said. “We have evidence that smoking during pregnancy is risky, for example. We have to respect women’s decision-making and put that evidence in front of them.”
On the other hand, Kelly warned that moving from theory to practice has failed in the past.
“I’ve seen a whole lot of interventions,” Kelly said. “The universal failing is the failure to evaluate. MSD still doesn’t know, of all the programs that were implemented, which ones worked.”
Both parties agreed that the government holds vast data that, if harnessed correctly, could reduce children being assaulted and neglected.
“The data is seductively attractive,” Kelly said.
Darian Woods is a Master of Public Policy candidate at the Goldman School of Public Policy at University of California, Berkeley. He is an editor at the School’s PolicyMatters Journal. This piece was written as part of the graduate course, Journalism for Social Change.