When Oded Oren was working as a public defender in the Bronx, he began to notice patterns in the behavior of judges he appeared before — such as which ones carefully engage with legal arguments or impose harsher sentences.
That information, he told amNewYork Law, gradually accumulates for practitioners, expanding among circles of colleagues and lawyer friends. But he kept wondering why it should remain siloed among lawyers when it is the public in New York state that is tasked with holding judges accountable through our system of elections and appointments.
That thought eventually gave rise to Scrutinize, the platform he founded that analyzes court data to provide a window into the overall decisions of more than 1,200 state judges. Since launching its online judicial database last year, the platform has attracted over 150,000 users to scour the records of the bench, looking for insight into what kinds of decisions judges are making.
“There are often discussions of the role of prosecutors, the role of police, and the role of specific legislation statutes … But what is lacking in that conversation is a discussion of the role of judges and their power,” Oren said. “I think that maps out to other areas of the law, not just criminal law, but civil, family, housing, and so on.”
Oren announced Tuesday that Scrutinize is growing. After receiving a $200,000 award from The David Prize, Oren is planning to expand Scrutinize in phases beyond criminal court judges into the civil and appellate judiciary.
He will also launch another tool called The Court of Appeals Decoded — an AI-powered database that tracks the positions and voting patterns of the judges on New York’s highest court to guide future nomination processes. This tool will be the first phase of Scrutinize’s new programming.
Scrutinize aims to bring transparency to judicial process, and voters
The overall goal of Scrutinize, as Oren describes it, hinges on the availability of public information on New York’s judiciary. In increasing transparency, Oren has also hoped to increase the possibilities to hold judges accountable for their conduct. To accomplish this, he’s had to think about appealing to a broad audience.
In New York City and state, judges find their way to the bench through a mix of public elections and gubernatorial or mayoral appointments, depending on the court level.
In Civil and Supreme Courts across the state, and in city and family courts outside New York City, the bench is an elected office. In the appellate division and the Court of Appeals, judges are appointed by the governor — in the case of the high court — through a nomination process overseen by the legislature. In New York City, on the other hand, the mayor appoints judges to the city’s family and criminal courts, as well as interim judges to its civil courts.
For the purpose of greater civic understanding generally, and specifically around elections, Oren knew he had to make Scrutinize accessible to the general public, but in the appointed positions, Oren said that his audience is also the decision-makers involved in judicial selection, including committee members, legislators, executive officials, and judicial delegates.
One level removed from the actual decision-makers are the grassroots organizers who can use court data to build accountability campaigns. He touted the fact that the data has been used in a number of different judicial accountability campaigns in the relatively short period of time the website has been live.
“The more that the public is aware of what is happening, and the more that public servants know that they’re being watched and monitored by the public, the better our democracy is going to work and the more trust that people will have in their elected officials, and their public servants, and specifically here in their courts,” Oren said.
Accountability in the form of judicial elections comes with tension. Judicial elections inherently create strain between political pressure that voters and political groups put on judges and the idea that judges should operate with complete independence in how they interpret the law.
But Oren said that ultimately he believes the type of factual information he’s sharing is not at odds with judicial independence.
“Where exactly do you put the line between accountability and judicial independence is an issue open for debate,” he said. “What I am sure of is that putting more information out there for people to be able to understand what judges are doing, how they’re deciding their cases, how they’re exercising their discretion when they have it, and how they’re choosing to interpret the law in one way versus another — information like that is always useful and always important to have in a democratic system.”
There are plenty of other obstacles to judicial accountability outside of the world of data, Oren said. He recognized, for instance, that transparency alone cannot solve structural problems in judicial selection, including uncompetitive elections, party-controlled nomination systems, and limited opportunities for voters to influence appointments. But he said he sees public information as the necessary first step.
Oded OrenPhoto via Scrutinize
COVID origins of Scrutinize
Oren initially began to work on Scrutinize during the COVID-19 pandemic, when a slower court schedule gave him time to teach himself coding and data analysis. Oren initially explored publicly available information, then began to file public information requests to be able to look at court data at the individual-judge level and identify patterns over time.
Some of the gaps in public information were surprisingly basic. One early freedom of information request Oren had to make was to produce a statewide list of judges, including their counties and term start and end dates—information he said had not previously been available in a centralized, current public database.
Years down the road, Oren continues to operate the site as the sole full-time employee, leveraging contractors and partnerships like the NYCLU, Harvard Law, NYU Law, and Reinvent Albany to scale its operations.
The other way that Oren is able to surpass his status as a one-man team is through a reliance AI, with what he called a rigorous system of legal testing.
For the Court of Appeals Decoded initiative, the new technology will extract information from about 15 years of New York Court of Appeals decisions—covering the period since the court’s longest-serving current judge, Justice Rivera, joined the bench.
In addition to individual judges’ records and voting patterns, Oren said that the platform will generate the Court of Appeals’ overall record on particular legal subjects, like consumer protection, free speech or criminal justice. The goal is to create a summary of how judges’ positions develop across multiple cases, rather than relying on a single high-profile decision.
Appellate cases analysis differs from trial-court analysis because appellate judges usually decide cases in panels of multiple judges. Their records are therefore reflected in collective decisions, so Scrutinize has a challenge to identify each judge’s legal positions as accurately as possible within those constraints.
After the platform first rolls out the Court of Appeals data, Oren plans to unveil data on New York’s intermediate appellate courts.
Underlying these expansions, Oren said, is the central argument is that people cannot effectively reform or participate in judicial-selection systems unless they understand why judges matter in their daily lives. By translating dense legal records into accessible bites, Scrutinize hopes to give New Yorkers a stronger metric for deciding which judges should remain in power.
“Before you get into citizen involvement in reshaping the process, that is … completely defective — before you get to that, you have to have a citizenry that cares and is involved and understands what the stakes are. And the precondition for that is information, is data, right?” Oren said. “You will find it much harder to get involved in the Court of Appeals if you don’t really understand why their decisions are shaping your today and your tomorrow, and that’s what we try to bridge initially.”
Oded Oren