A thesis project from a computer science Ph.D. student at UC San Diego in La Jolla offers a new framework using a team of specialized artificial intelligence “agents” to read research papers through a pdf or website link and automatically generate code that can be shared from institution to institution with fewer barriers.
The concept, called NERFIFY, is the brainchild of first author Seemandhar Jain, with support from co-authors Keshav and Kunal Gupta and oversight from UCSD professor Manmohan Chandraker.
It is intended to provide the ability to more efficiently read neural radiance field, or NeRF, papers and translate them into code.
It will be presented at the Institute of Electrical and Electronics Engineers and Computer Vision Foundation’s Conference on Computer Vision and Pattern Recognition June 3-7 in Denver.
NERFIFY cuts out the extensive process of reverse-engineering individual papers and reduces a “bottleneck” of innovation across different fields, Jain said.
“The main benefit of this method is you can cut out the cost of reproducibility of a paper from weeks to a couple of minutes,” he said. “So you can save a lot of time in order to replicate the paper.”
NeRF is a relatively new neural network allowing for the construction of three-dimensional images from two-dimensional ones. With a lack of available code, work in this field takes extensive effort, Jain said.
The new platform differs from frontier models, or large-scale AI models defined by costly training and wide reach, according to Jain and Chandraker.
“This particular project … came about as us trying to figure out how we can take codified knowledge — essentially knowledge which has been written down by humans — and translate that into something which is a working piece of code,” Chandraker said.
“Frontier models like ChatGPT and so on are really good at giving us answers to questions that you’re asking — they’re very good at analysis,” he said. “But we can think of NERFIFY as basically a team of [AI] researchers that are working together.
Manmohan Chandraker, a professor in UC San Diego’s Computer Science & Engineering Department, is the adviser for student Seemandhar Jain’s Ph.D. thesis project. (Provided by Manmohan Chandraker)
“What NERFIFY does with this multi-agent framework is think of each of these steps in the research process as being one agent. And each of these agents can then go and deploy specific tools.”
Jain said accuracy is just as important to NERFIFY.
“Frontier models … fail often,” he said. “We wanted to propose a solution which can work most of the time, and we wanted to convert [papers] into code.”
The process of developing NERFIFY began with a couple of months of research, followed by two to three months of experimentation, Jain said.
If AI can reliably convert published science into runnable code, it can fundamentally change how collaboration can look across institutions from Nairobi, Kenya, and São Paulo, Brazil, to U.S. research giants such as Stanford University and the Massachusetts Institute of Technology, regardless of the amount of resources they have at their disposal, Jain said.
Chandraker emphasized the importance of establishing safety measures to prevent a proliferation of plagiarism or excess automation.
“We need to have guardrails where we are constraining them to produce a sort of output that we think are verifiable, that we think are solving a particular, responsible goal,” Chandraker said.
“I hope human subjectivity and knowledge that the scientific community brings is something that does provide these guardrails, where we are evaluating ideas on their scientific merit.” ♦