Five years ago, there were high hopes for a new San Diego County office tasked with analyzing how well county programs are working.
But despite a nearly 20-person staff and $17 million spent in the years since, the office has released only one report.
Perceived shortcomings at the county’s Office of Evaluation, Performance and Analytics, or OEPA, have drawn bipartisan criticisms from the Board of Supervisors, with calls ranging from overhauling the office to eliminating it outright.
When it was created in 2021, the office was billed as a groundbreaking change. For the first time, county leaders would have real-time data on the outcomes of key programs from a team of dedicated researchers.
But it wasn’t until April that supervisors got their first report from OEPA.
The report looked at a now long discontinued program that gave a one-time payment of $4,000 to about 2,200 households. OEPA researchers ultimately found that for those who received the no-strings-attached subsidy, it did not change their reliance on homelessness services or other social services programs such as CalFresh.
Findings like that are valuable to policymakers, said Supervisor Terra Lawson-Remer, a Democrat who first proposed creating the office in 2021.
But while Lawson-Remer is still a believer in OEPA’s potential, she conceded that its current form isn’t working.
“It’s moving way too slowly,” Lawson-Remer said. “It’s doing way too few studies. It’s just stuck in the mud.”
Republican Supervisor Joel Anderson, who originally supported creating the office, now calls it “a huge failure” with leadership that is “grossly incompetent.”
“How do you define this as a successful program in any way, shape or form?” Anderson said. “My colleagues are willing to allow money to be squandered and wasted rather than address the fact that this program doesn’t work.”
County Chief Evaluation Officer Ricardo Basurto-Dávila, who runs the office and is paid $242,000 a year, declined requests for comment.
County administration stands by OEPA’s work.
County spokesperson Sarah Sweeney held up the office’s other efforts, like a data dashboard that aims to measure the county’s performance across categories like community, empower, equity, justice and sustainability.
OEPA staff also provide support to other county data platforms and train county staff on how to evaluate programs, Sweeney said.
“Taken together, OEPA’s contributions position the county as a national leader in evidence‑based government,” she added. “Few jurisdictions have built comparable structures for evaluation, enterprise data management and analytic capacity.”
While supervisors and county staff disagree over the office’s success, voters will very soon get a say in its future.
Changes to who controls the office is on the ballot as part of a sweeping measure to reshape the structure and distribution of power in county government.
The ballot measure to rewrite the county’s charter, known as Measure A, includes seismic changes like extending term limits for supervisors from two terms to three, letting sitting supervisors run for an extra term.
The measure, if passed, would also create an ethics commission and a new budget analyst office and would give supervisors the power to confirm and remove top county officials.
But a lower-profile part of the package would establish an “independent program auditor.”
In reality, that section of the charter rewrite would give supervisors power over OEPA and take it away from county administration, according to Lawson-Remer, the lead architect of the charter rewrite.
Exact details for how OEPA will be transformed into a program auditor’s office will be up to a future task force charged with implementing changes to the county’s charter.
San Diego County Supervisor Terra Lawson-Remer at a meeting in April. (Nelvin C. Cepeda / The San Diego Union-Tribune)
But Lawson-Remer thinks OEPA’s placement in the county’s vast bureaucracy is what’s holding it back.
“It doesn’t operate with authority that comes from either the board or an independent elected,” Lawson-Remer said. “It’s fundamentally related to the whole premise of the charter, which is that right now, all the staff kind of operate without a lot of accountability to voters.”
A recent San Diego Union-Tribune poll found 71% of registered voters were supportive of an independent auditor’s office. But a smaller share of respondents were enthusiastic about the charter package as a whole, with 40% saying they supported it and another 25% saying they were undecided.
In June, supervisors voted to place the charter rewrite on the ballot. But in the run-up to the vote, Anderson introduced a rival proposed ballot measure, though supervisors ultimately backed Lawson-Remer’s plan over his.
Under Anderson’s plan, the program auditor would have been an elected official free of any influence from the Board of Supervisors.
Anderson thinks OEPA should be defunded. Should the charter rewrite pass, he worries a program auditor under the control of supervisors would be subservient to the county’s Democratic majority.
“Do you think they’re going to ever audit anything that’s going to put their leadership in a bad light?” Anderson said.
Supervisor Monica Montgomery Steppe, who backed putting Lawson-Remer’s plan on the ballot, said the charter rewrite is an opportunity to fit OEPA into a new county government structure.
“I supported putting these reforms before voters because strong, independent oversight is an important part of accountable government,” Montgomery Steppe said.
The Board of Supervisors’ two other members — Paloma Aguirre and Jim Desmond — did not respond to requests for comment.
Evaluating the evaluators
Evaluating the impact of government programs is a relatively new endeavor in the field of economics, born out of monitoring the success of public health and anti-poverty programs in developing nations.
But the migration to local governments has been slower.
With a budget of $9.2 billion, San Diego County’s foray into program evaluation makes it “pretty unique” among local governments, said UC San Diego professor Gordon McCord, who has worked with OEPA.
“It’s on the order of the national budgets of some of the poorest countries in the world,” McCord said of the county. “The fact that the county wants to put a few of those resources into measuring whether they are spending their money well seems quite reasonable.”
At OEPA, its annual budget has averaged about $4.6 million since being created in 2021.
County Supervisor Joel Anderson addressing constituents during a town hall meeting in Rancho Bernardo. (Amy Stark Shireman)
After the county hired Basurto-Dávila in 2022, a 20-person staff was built out over the following two years.
Positions in the office include three department heads, seven researchers and other administrative staff, though total staff has been reduced to 16 people due to consolidation with the county’s IT office and the elimination of two vacant jobs.
All told, the county has spent $17.4 million on OEPA since its inception.
According to a strategic plan from the office and its website, its staff is currently evaluating six county programs. Similar to the county’s one-time $4,000 payments, some of the programs it’s examining guarantee direct subsidies to residents.
One gives $500 a month to about 450 families who are at risk of having their children taken into the foster care system. The study intends to determine if the program is cost-effective and if it ultimately reduces incidents of child abuse or neglect and improves families’ quality of life.
Another report will look at the success of a program that gives $500 a month to hundreds of low-income older adults in an effort to prevent them from becoming homeless, an increasingly common anti-homelessness strategy known as shallow rent subsidies.
But there’s a significant turnaround time for these ongoing evaluations. Reports for the two programs were scheduled to take four and a half years to produce, and OEPA plans to release them in 2028 at the latest, according to the office’s strategic plan.
The county says these long timelines are necessary to guarantee a thorough deep dive into the data and subsequent follow-up to see if county dollars made a true difference in peoples’ lives.
“A shorter turnaround for impact evaluations would require using incomplete data and eliminating the post‑program follow‑up work,” Sweeney said. “This would limit reliability and could result in misleading conclusions.”
McCord agrees, adding that how many reports the office has produced is not the best metric on which to evaluate its success after only five years.
“The idea of getting real-time information from a program is kind of a misnomer,” he said. “In terms of an impact evaluation perspective, you have to hold the line and say ‘Sorry. We would all like to know in real time whether programs work, but that’s just not the way the world works.’”
Supervisors don’t think that’s good enough, especially when they’re making decisions now about whether to continue funding these programs.
Long turnaround times are also at odds with the legislation that originally created the office, which called for the collection of “timely information” so that officials could “make mid-course corrections and policy adjustments.”
“Their reports are of no use unless we get them in a timely fashion so that we can use them in making future decisions,” Anderson said.
For example, supervisors voted in March to spend $2.8 million of county reserves to prolong its shallow rent program.
By then, OEPA’s report on the program had already been underway for nearly three years.