A year ago, California officials had to report under a new state law how they used automated systems to make important decisions about people\u2019s lives.\u00a0<\/p>\n
They said they never did \u2014 a startling answer for a number of reasons, sources told CalMatters at the time<\/a>, including that there were several prominent examples to the contrary.<\/p>\n Now, the state has issued a more expansive answer: It is currently using six automated systems to make consequential decisions about the lives of Californians.<\/p>\n The systems are used to do things like:<\/p>\n Predict whether incarcerated people will re-offend\u00a0Evaluate whether unemployment claims are fraudulentRemotely administer exams for California State University students\u00a0Detect when college students use generative AI to write assignments.<\/p>\n That\u2019s according to a report released Friday<\/a> by the state\u2019s technology department. The report is required under a 2023 law mandating that<\/a> that state agencies annually disclose their use of \u201chigh-risk automated decision systems,\u201d which the law defines as systems \u201cused to assist or replace human discretionary decisions that have a legal or similarly significant effect, including decisions that materially impact access to, or approval for, housing or accommodations, education, employment, credit, health care, and criminal justice.\u201d<\/p>\n Though the report names six high-risk systems in use today, state agencies have used some for several years now. Those include COMPAS, which has been used by the California Department of Corrections and Rehabilitation to assign recidivism scores to inmates for at least a decade<\/a>.<\/p>\n The technology department said in the report that it found more systems for its report this year because it evaluated responses from state agencies more thoroughly, including by meeting with agencies and questioning them about their systems.<\/p>\n In addition to the six high-risk systems, the department\u2019s report disclosed an additional six systems initially flagged as high risk but later determined not to be. One was AI used for legislative bill analysis by the California Department of Finance.\u00a0<\/p>\n The report also notes two high-risk systems that are not currently in use: the Department of Cannabis Control is developing artificial intelligence to analyze whether marijuana packaging violates a law against appealing to children and California State University discontinued use of a language model for reviewing job applications.<\/p>\n Results of the second annual survey come after cities like San Jose and San Francisco released their first AI inventories in recent months. They also come at a time when California-based AI companies like Anthropic and OpenAI are going public and seeking government contracts. Americans are split on whether they trust AI<\/a> and surveys last year by TechEquity<\/a> and Carnegie California found<\/a> that the majority of Californians want safety over innovation. A Gallup poll to evaluate the opinions of Americans found similar results.<\/p>\n What\u2019s missing<\/p>\n While the newly-released report shares more information than last year\u2019s, several questions remain about the state\u2019s use of artificial intelligence and other automated systems.<\/p>\n The report does not include generative AI pilot projects underway with support from the governor\u2019s office to do things like help businesses file taxes, support state employees who work on homelessness, and an AI assistant named Poppy that uses language models like Anthropic\u2019s Claude to do things like draft documents, research policy, or build custom AI tools, according to a state website<\/a>. The website says that 67 state departments provided input during the pilot phase and statewide rollout of Poppy begins next month<\/a>.<\/p>\n The 2023 law mandating the annual high-risk systems report excludes reporting by a number of state agencies, including the judicial branch and the University of California college system. Reporting by CalMatters last month<\/a> found that a majority of the roughly 60 courts that operate statewide have adopted generative AI use policies. Courts in Los Angeles and Riverside counties have begun testing<\/a> an AI tool to act as a clerk, drafting orders and producing research memos.<\/p>\n CalMatters is compiling an inventory of automated decisionmaking systems in use by state and local agencies throughout California in order to provide transparency into how governments are using decisionmaking systems and AI. Know about an AI system in use by a state or local agency? Email khari@calmatters.org<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"A year ago, California officials had to report under a new state law how they used automated systems…\n","protected":false},"author":2,"featured_media":363889,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[2007,2080,78234,7,9,8,6752,60829,721,1454,10020,47326,38669],"class_list":["post-363888","post","type-post","status-publish","format-standard","has-post-thumbnail","category-california","tag-ai","tag-artificial-intelligence","tag-automation","tag-california","tag-california-headlines","tag-california-news","tag-calmatters","tag-generative-artificial-intelligence","tag-law","tag-poppy","tag-state-government","tag-state-news","tag-state-and-regional"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/363888","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/comments?post=363888"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/363888\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media\/363889"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media?parent=363888"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/categories?post=363888"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/tags?post=363888"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}