Davene Roseborough still can\u2019t believe the number of foster homes she lived in that had locks on the refrigerators and cabinets. At one spot, Roseborough was barred from eating dinner if she wasn\u2019t home by 6 p.m. That place didn\u2019t last long, though. She was bounced from home to home from the time she entered foster care at the age of eight to the time she aged out at 18. Those years were characterized by rampant physical and sexual abuse, for which her foster parents received a $1,200 monthly stipend. By the time she had her own child, she exemplified the kind of mother scrutinized by the very family policing system that raised her\u2014Black, former system involvement, survivor of abuse, mental health struggles. Sure enough, her son was eventually taken from her and placed in the foster system for five years, where he also experienced abuse.<\/p>\n
The Administration for Children\u2019s Services, the New York City agency tasked with investigating allegations of child abuse and neglect, has been algorithmically encoding this kind of profiling since 2018. Before DOGE lived and died, before ICE contracted Palantir to supercharge mass deportation, before ChatGPT even launched, ACS was already using AI. What started with a single predictive analytics score meant to identify children at a high risk of severe harm has since grown into a six-algorithm suite<\/a> of surveillance, which operates with very little public oversight.<\/p>\n Yearly AI disclosures<\/a> required by city law don\u2019t include detailed information about what data the algorithms are trained on, or how exactly they are used by caseworkers. Despite the lack of transparency, what is evident is that the algorithms function as an expensive way to perpetuate the status quo\u2014encoding extant and extreme human biases in the family policing system into algorithmic ones, and enabling a policing agency to claim it has the technical capacity to oversee the welfare of New York\u2019s nearly 2 million children.<\/p>\n ACS says<\/a> there\u2019s no need for public oversight of the algorithms because they simply prioritize cases for ACS employees to review, rather than make final decisions. It was only last summer that an investigation<\/a> by The Markup revealed that the originating predictive algorithm uses 279 variables to assign families a risk score, based on cases from 2013 and 2014 that ended in a child being severely harmed. Though there are some ostensibly reasonable factors, like if a child tested positive for drugs or alcohol, many are completely outside a parent\u2019s control, nakedly use proxies for race, or otherwise indicate poverty rather than abuse. The mother\u2019s age, the neighborhood the family lives in, and whether the parent was abused as a child are all factored in. So is the state of the family\u2019s home, the family\u2019s \u201cfinancial resource management,\u201d and whether a parent is \u201cunable\/unwilling to meet child\u2019s basic needs for food, clothing, shelter and\/or medical care,\u201d which quite literally equates conditions of poverty with neglectful or abusive parenting choices. Past involvement with ACS, including involvement that did not result in any sort of finding of abuse or neglect, is also a factor, meaning that ACS subjecting a parent to surveillance serves as its own justification.<\/p>\n When it comes to the family policing system, the problem is obvious: Parents are punished for being poor, Black, or both\u2014now with the help of algorithms. Stevie Glaberson, a former family defense attorney at Brooklyn Defender Services and the current Director of Research & Advocacy at the Center on Privacy & Technology at Georgetown Law, said inputs that penalize poverty and race emphasize the way AI takes in human prejudice and spits it right back out. \u201cData doesn’t just exist in the world. Choices are made at every point. Who are we watching? What are we collecting? What are we noting? And what societal conditions does it reflect?\u201d Glaberson said. \u201cWith child welfare, it always, always will reflect racial, poverty, and disability-based biases.\u201d<\/p>\n ACS\u2019s own internal technical audit<\/a> of the original algorithm \u201cfound substantial variation across race\/ethnic groups,\u201d which the agency says reflects genuine differences among racial groups and also \u201csome implicit and systemic biases.\u201d The \u201csome\u201d in question: A 2020 racial equity audit<\/a> commissioned by ACS described \u201ca predatory system that specifically targets Black and brown parents\u201d where \u201csymptoms of poverty are frequently punished as signs of neglect.\u201d Neglect, as defined by ACS, includes things like a lack of adequate food, clothing, shelter, and medical care. The audit surveyed dozens of ACS\u2019s own workers, who consistently reported that allegations of neglect permanently entangle poor families with the family policing system.<\/p>\n ACS is a relatively recent development in New York City\u2019s history. Then-Mayor Rudy Giuliani created the agency in 1996, following public outcry<\/a> after a 6-year-old girl was beaten to death by her mother despite attempts by her relatives, neighbors, and teachers to warn officials that her life was in danger. Fear of child abuse and death, the beating heart of the agency\u2019s origins, still motivates much of the justification for the status quo. The New York Post meets its annual quota for filth by hammering ACS for failing to prevent one or two shocking cases of abuse every year. But 30 years after the agency was created, abuse cases are not the focus of most of its investigations. In 2023, 75 percent of ACS cases alleged neglect alone<\/a>. ACS found almost 80 percent of cases to be unsubstantiated that year. Year after year<\/a>, neglect cases and unsubstantiated cases make up the vast majority of the family policing system.<\/p>\n As mayor, Giuliani cut more than $1.6 billion from public schools in his first three budgets. He cut subsidies to city hospitals, which have a mandate to serve poor New Yorkers, by nearly 85 percent. At the same time, Giuliani oversaw a more than $1 billion increase<\/a> in the NYPD\u2019s budget to help oversee and enact<\/a> broken windows policing. By creating ACS while he decimated the city\u2019s social safety net, Giuliani contorted children from a collective responsibility to an individual one. The negative consequences of social abandonment were transmuted into child neglect, a crime for which individual parents could be blamed.<\/p>\n But the question remains: When a parent does not have the resources to care for a child, what should happen to that child? For Black and Latino children, the city\u2019s answer is usually intervention by ACS. While only 48 percent of NYC\u2019s population is Black or Latino, 82 percent of ACS investigations<\/a> were of Black or Latino families in 2024. The same year, more than 86 percent<\/a> of the children in New York\u2019s foster care system were Black or Latino. Poverty rates also correlate heavily<\/a> with ACS investigation rates; the poorer the district, the higher the ACS investigation rate. These disparities existed before ACS began using AI, and they still exist, years later.\u00a0<\/p>\n Richard Wexler, the executive director of the National Coalition for Child Protection Reform, once described risk algorithms as self-fulfilling prophecies rather than predictions. \u201cProponents say the model is successful if it flags cases that are more likely to result in substantiation or more likely to result in placement in foster care. But if a family is poor, and the poverty is confused with neglect, and a year later the family is still poor, of course there\u2019s a good chance the poverty will be confused with neglect again,\u201d he said<\/a>.<\/p>\n Still, ACS argues<\/a>, \u201cpredictive risk models have the potential to improve child welfare outcomes by improving resource allocation.\u201d Every year, ACS receives<\/a> more than 65,000 calls to the city\u2019s child protection hotline. At any given time, the agency has roughly 50,000 active cases open. This enormous mandate is why even the most clear-eyed reckonings with the biases of the family policing system are often presented alongside the plight of the ACS worker.<\/p>\n In a New Yorker article<\/a> published in 2017, Larissa MacFarquhar initially writes from the perspective of a parent forced to open the door for an ACS caseworker, who proceeds to strip your kids to search them for bruises, and who may take them away immediately and tell them false things about you, forever traumatizing them and subjecting you to terrifying and tedious humiliations to get them back. MacFarquhar follows this scene by writing from the point of view of an ACS caseworker, faced with the impossible challenge of making a home visit and quickly deciding if a parent is neglectful or simply poor. If you make a mistake, you might risk a child\u2019s life, and if you make the difficult choice to take a child away, you might witness unimaginable anguish.<\/p>\n To be fair, it is impossible for a worker to do this job. ACS, as a policing agency, is not especially capable of fostering social health, well-being, or even safety. That\u2019s not what it was made to do. For Joyce McMillan, the executive director of Just Making A Change for Families and a parent who has faced ACS investigations, the outcomes speak for themselves.<\/p>\n \u201cChildren age out into homelessness. There are a gazillion lawsuits for children being beaten and battered and sexually abused, and all of these other things while in the foster system,\u201d McMillan told me over Zoom. \u201cThen we have children who don’t graduate high school because they change homes regularly, and every time they change homes, they change schools, caregivers, their entire surroundings, they\u2019re completely disoriented. Many of these children, over half of them<\/a>, end up involved in either the juvenile justice system or the adult prison industrial complex.\u201d<\/p>\n Those inclined to accept these outcomes as necessary evils to prevent the death of even one child don\u2019t have much evidence to justify the trade. Last year, a major study<\/a> of 3.4 million records of children in foster care found no relationship between removing kids from allegedly abusive or neglectful homes and child mortality. It didn\u2019t go down when more kids were put in foster homes, and it didn\u2019t go up when more families were kept together.\u00a0<\/p>\n The city doesn\u2019t have the money to support families. That\u2019s why it has to spend billions<\/a> of dollars on policing them. But the city also doesn\u2019t have the money to police all the families it needs to, so it has to spend even more money on AI to decide which families need policing the most. Austerity, disaster, then AI acting as austerity by another name, on and on in a perfect loop.\u00a0<\/p>\n \u201cThe allure of using AI and algorithmic decision making in government services is very much a product of the assault on government over the last couple of political generations and accompanying hollowing out of government’s ability to serve people,\u201d said Kevin De Liban, a former Legal Aid attorney and the founder of TechTonic Justice, a nonprofit focused on how AI shapes the lives of low-income Americans. A cobwebbed social safety net results in policing agencies like ACS improperly and impossibly tasked with social welfare. It\u2019s only too easy for enterprising AI adventists to promise to use algorithms to ease the burden.\u00a0<\/p>\n Erin Dalton, New York\u2019s current head of Social Services, pioneered predictive algorithms in family policing while serving as the director of the Allegheny County Department of Human Services in Pennsylvania. Allegheny\u2019s algorithm is infamous<\/a> for targeting low-income and Black families. In its first iteration, the Allegheny family screening tool discriminated against people for using the kind of public benefits that Dalton is now in charge of administering to New Yorkers. When defending the algorithm to the Associated Press, Dalton said<\/a>, \u201cWorkers, whoever they are, shouldn\u2019t be asked to make, in a given year, 14, 15, 16,000 of these kinds of decisions with incredibly imperfect information.\u201d<\/p>\n Concern about workload is so great that one of the six predictive algorithms used<\/a> by ACS \u201cestimates weekly caseloads by area for the next 12 months\u201d to “identify how busy certain neighborhoods are likely to be\u201d and \u201coptimize workforce distribution across different boroughs.\u201d There will never be an algorithm that can reverse the societal-level impact of state abandonment. But we will do anything\u2014including profile entire neighborhoods into literal heat maps of alleged future child abuse\u2014other than fund universal social goods.\u00a0<\/p>\n Despite growing recognition of the system\u2019s failures, there is still far greater political willpower to pursue punishment for families than soaring anti-poverty measures. It\u2019s somehow easier to look at the mess of family policing as it exists and promise to use AI to make it slightly better, for the workers who run it. In May, the Administration for Children and Families at the U.S. Department of Health and Human Services announced<\/a> a $6 million investment \u201cto expand states\u2019 use of technology solutions, including predictive analytics.\u201d<\/p>\n This may seem too grim a diagnosis, given the significant reduction<\/a> in child removals<\/a> and increased emphasis on ostensibly voluntary preventive services. The same year ACS began using AI, Congress passed the Family First Prevention Services Act, which prioritizes keeping families together and restricting foster placements. In recent years, ACS has spent as much on preventive services<\/a> as it has on investigations, investing<\/a> more than $300 million a year on things like parenting classes and therapeutic mental healthcare. Mayor Zohran Mamdani appointed a new ACS commissioner who lived in foster care herself and champions<\/a> preventive services that avoid hastily accusing families of child abuse or neglect and separating them.\u00a0<\/p>\n But families and legal advocates say that just because there are fewer child removals doesn\u2019t mean there\u2019s less family surveillance, or that \u201csupport\u201d means material support. \u201cWhat often happens is when a family comes to the attention of [ACS], a service plan gets generated, not often with their input, and they\u2019re often these cookie cutter service plans,\u201d Glaberson said. \u201cThings like take a parenting class, do a mental health evaluation, go to therapy, and comply. Compliance is a huge thing, and drug testing, even if there\u2019s no allegations of drug use. And so families are sent to run around town to comply with all these services, without regard to whether they have jobs and whether they have the time for stuff that is not relevant at all to their needs or experience.\u201d<\/p>\n The hundreds of millions that ACS spends on preventive\u00a0services focus heavily on mental health rather than material needs.\u00a0Ashleigh Washington, a social worker with a doctorate in social welfare from the City University of New York, points to a preventive service that addresses domestic violence as illustrative of the difference. “You’re working with the harming party, the non-harming party, and the child. There’s caseworkers assigned to all aspects of that. You have a clinical supervisor, you have a deputy director, an assisted deputy director, a team of case planners, a clinical supervisor, social workers,\u201d\u00a0Washington said. \u201cThere is probably very little in those budget lines for if the person needs to move out of the state to get away from an abusive partner.\u201d<\/p>\n Kiara Harris, a mother of a daughter with autism, came to the attention of ACS because of domestic violence, and she agreed to preventive services for fear that saying no might lead ACS to take her child away. She also hoped the services might help her secure diapers and other basic necessities. Instead, they took the form of frequent unannounced visits to her home and her daughter\u2019s school and doctor\u2019s office. This is the kind of support ACS is using algorithms to flag families for. Harris compared the preventive services to living with her abuser. \u201cThat’s like second nature, learning how having to mask how you feel or act like something isn’t bothering you. Being in that relationship, that became survival,\u201d Harris said. \u201cI remember being really, really depressed and being extra mindful to not show any type of weaknesses in front of them when they would come and do the visits.\u201d<\/p>\n ACS claims to use AI to meet some material needs. Shruti Pandey, the Director of Predictive Analytics at ACS, describes<\/a> the agency\u2019s algorithms as tools to \u201cflag high-risk cases for support,\u201d emphasizing a certain service sensibility. One of the algorithms, Pandey says, predicts which families are most likely to become unhoused and allows ACS to prioritize them for housing vouchers. But ACS only has<\/a> 100 of these to distribute. The algorithm serves as a means-testing triage system for suffering. Which families deserve to win the lottery and secure one of these 100 vouchers? Is it too grotesque for a human to decide? Would it be easier for everyone if an algorithm could tell us who is more or less worthy of being left behind? Algorithmic prediction helps obscure the political choice at play, painting over indefensible abandonment with the figment of mathematical rigor.<\/p>\n Though AI marks the latest update in this trend, we\u2019ve long ignored the fundamental contradiction between policing and support. We\u2019ve banged our heads against the wall of this lesson for decades, with succeeding waves of policy choices that do anything but address the root cause of the vast majority of suffering among children: poverty. Davene Roseborough has waited almost her whole life for something to change. \u201cI’m almost 50 years old. This happened to me when I was eight, so that’s about 40 years ago,\u201d she said. \u201cAnd children are still not getting what they need. They’re still not getting the support that they need.\u201d<\/p>\n Recommended<\/p>\n","protected":false},"excerpt":{"rendered":"Davene Roseborough still can\u2019t believe the number of foster homes she lived in that had locks on the…\n","protected":false},"author":2,"featured_media":314876,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[9,24,55,54,56],"class_list":["post-314875","post","type-post","status-publish","format-standard","has-post-thumbnail","category-new-york-city","tag-new-york","tag-new-york-city","tag-new-york-city-headlines","tag-new-york-city-news","tag-ny"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/posts\/314875","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/comments?post=314875"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/posts\/314875\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/media\/314876"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/media?parent=314875"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/categories?post=314875"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ny\/wp-json\/wp\/v2\/tags?post=314875"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}