A California State Auditor report released Tuesday found that poor planning, inadequate testing and weak oversight contributed to widespread technical failures during the February 2025 California Bar Exam, disrupting one of the state’s most important professional licensing exams.The report concluded that many of the problems experienced by thousands of aspiring attorneys were preventable and stemmed from the State Bar of California’s rollout of a completely new exam format rather than a lack of preparation by examinees.For years, California relied on the National Conference of Bar Examiners’ multiple-choice questions and primarily administered the exam in person. Seeking greater flexibility and lower costs, the State Bar transitioned to a California-specific exam that introduced newly written multiple-choice questions, a remote testing option and new vendors to develop, administer and validate the exam.According to the audit, that transition occurred on a compressed timeline. The California Supreme Court approved the new exam format in late October 2024, and the State Bar finalized its agreement with testing vendor Meazure Learning about a month later. Less than three months after that, thousands of applicants sat for the new exam.The auditor concluded the State Bar failed to adequately test the new system before launch and did not provide sufficient oversight of its vendors. It also found the agency rushed parts of its procurement process while implementing the new exam.Technical failures and flawed questionsThe audit found many examinees experienced computer freezes, crashes, error messages and other technical interruptions during the exam. Nearly half of the approximately 2,700 examinees who responded to a State Bar survey reported their computers froze or crashed during testing, while roughly 1,200 reported receiving error messages during the multiple-choice portion of the exam.The report also identified problems with the exam itself.Instead of relying on nationally developed multiple-choice questions, the State Bar contracted with Kaplan to create new California-specific questions and used ACS Ventures to provide psychometric analysis and assist with question development. The audit found ACS Ventures used artificial intelligence to help develop some questions and concluded the State Bar’s review process did not adequately validate the content before the exam was administered.Following the exam, ACS Ventures identified performance issues with 40 of the 200 multiple-choice questions. The State Bar ultimately removed 29 questions from scoring after determining they should not count toward applicants’ results.Because of the widespread problems, the State Bar also adjusted how the February exam was scored. The auditor estimated the pass rate increased from about 36% to approximately 65% after those changes.Examinees describe chaotic experienceStephen Zendejas, who took the February exam, described the experience as “traumatizing.””The laptops then crashed on you,” Zendejas said. “When you get these variables that haven’t been given to other test takers before, that’s really, really difficult.”Chris Micheli, a legislative advocate and adjunct professor at the University of the Pacific’s McGeorge School of Law, said the report underscores the importance of proper oversight when administering a high-stakes licensing exam.”What stood out to me most is the fact that there was probably inadequate oversight over the people contracted to develop the bar questions,” Micheli said. “Perhaps you start at a much earlier time period.”State Bar accepts recommendationsThe audit also found the State Bar failed to achieve one of the primary goals behind the new exam format: reducing costs.According to the report, the February 2025 exam cost at least $5.1 million to administer, and remediation efforts, including refunds and fee waivers offered to affected examinees, resulted in at least $4 million in lost revenue.In a statement, the State Bar said it agrees with all six of the Auditor’s recommendations and acknowledged what it called the “unacceptable experiences” examinees faced.The agency said it has already begun strengthening its contracting, vendor oversight and exam development processes. It has also restructured its Office of Admissions, created new leadership positions focused on exam development and administration, and is developing a comprehensive framework for future bar exams.”I just hope that nobody that has to take the exam ever has to experience what the test takers of February 2025 had to experience,” Zendejas said.See more coverage of top California stories here | Download our app | Subscribe to our morning newsletter | Find us on YouTube here and subscribe to our channel

, Calif. —

A California State Auditor report released Tuesday found that poor planning, inadequate testing and weak oversight contributed to widespread technical failures during the February 2025 California Bar Exam, disrupting one of the state’s most important professional licensing exams.

The report concluded that many of the problems experienced by thousands of aspiring attorneys were preventable and stemmed from the State Bar of California’s rollout of a completely new exam format rather than a lack of preparation by examinees.

For years, California relied on the National Conference of Bar Examiners’ multiple-choice questions and primarily administered the exam in person. Seeking greater flexibility and lower costs, the State Bar transitioned to a California-specific exam that introduced newly written multiple-choice questions, a remote testing option and new vendors to develop, administer and validate the exam.

According to the audit, that transition occurred on a compressed timeline. The California Supreme Court approved the new exam format in late October 2024, and the State Bar finalized its agreement with testing vendor Meazure Learning about a month later. Less than three months after that, thousands of applicants sat for the new exam.

The auditor concluded the State Bar failed to adequately test the new system before launch and did not provide sufficient oversight of its vendors. It also found the agency rushed parts of its procurement process while implementing the new exam.

Technical failures and flawed questions

The audit found many examinees experienced computer freezes, crashes, error messages and other technical interruptions during the exam. Nearly half of the approximately 2,700 examinees who responded to a State Bar survey reported their computers froze or crashed during testing, while roughly 1,200 reported receiving error messages during the multiple-choice portion of the exam.

The report also identified problems with the exam itself.

Instead of relying on nationally developed multiple-choice questions, the State Bar contracted with Kaplan to create new California-specific questions and used ACS Ventures to provide psychometric analysis and assist with question development. The audit found ACS Ventures used artificial intelligence to help develop some questions and concluded the State Bar’s review process did not adequately validate the content before the exam was administered.

Following the exam, ACS Ventures identified performance issues with 40 of the 200 multiple-choice questions. The State Bar ultimately removed 29 questions from scoring after determining they should not count toward applicants’ results.

Because of the widespread problems, the State Bar also adjusted how the February exam was scored. The auditor estimated the pass rate increased from about 36% to approximately 65% after those changes.

Examinees describe chaotic experience

Stephen Zendejas, who took the February exam, described the experience as “traumatizing.”

“The laptops then crashed on you,” Zendejas said. “When you get these variables that haven’t been given to other test takers before, that’s really, really difficult.”

Chris Micheli, a legislative advocate and adjunct professor at the University of the Pacific’s McGeorge School of Law, said the report underscores the importance of proper oversight when administering a high-stakes licensing exam.

“What stood out to me most is the fact that there was probably inadequate oversight over the people contracted to develop the bar questions,” Micheli said. “Perhaps you start at a much earlier time period.”

State Bar accepts recommendations

The audit also found the State Bar failed to achieve one of the primary goals behind the new exam format: reducing costs.

According to the report, the February 2025 exam cost at least $5.1 million to administer, and remediation efforts, including refunds and fee waivers offered to affected examinees, resulted in at least $4 million in lost revenue.

In a statement, the State Bar said it agrees with all six of the Auditor’s recommendations and acknowledged what it called the “unacceptable experiences” examinees faced.

The agency said it has already begun strengthening its contracting, vendor oversight and exam development processes. It has also restructured its Office of Admissions, created new leadership positions focused on exam development and administration, and is developing a comprehensive framework for future bar exams.

“I just hope that nobody that has to take the exam ever has to experience what the test takers of February 2025 had to experience,” Zendejas said.

See more coverage of top California stories here | Download our app | Subscribe to our morning newsletter | Find us on YouTube here and subscribe to our channel