For most of human history, mapping the universe meant staring up at the night sky. Today, it means crunching trillions of data points drawn from the faint light of billions of galaxies.
However, as cosmic surveys like the Dark Energy Spectroscopic Instrument (DESI) and Europe’s Euclid mission gather vast amounts of data, astronomers are hitting a computational wall.
Now, building on these challenges, an international team led by physicist Dr. Marco Bonici at the University of Waterloo has unveiled a breakthrough tool that could redefine how cosmologists turn that data into understanding.
Called “Effort.jl,” it is a high-speed, differentiable “emulator”—a kind of machine-learning model that mimics the behavior of extremely complex cosmological simulations. According to the developers, Effort.jl is capable of mapping the large-scale structure of the universe in a fraction of the time traditional methods require.
Their results, published in the Journal of Cosmology and Astroparticle Physics (JCAP) in September 2025, show that Effort.jl can perform calculations up to three orders of magnitude faster than conventional pipelines while maintaining near-perfect accuracy.
“Using Effort.jl, we can run through complex data sets on models like EFTofLSS, which have previously needed a lot of time and computer power,” Dr. Bonici said in a statement. “With projects like DESI and Euclid expanding our knowledge of the universe and creating even larger astronomical datasets to explore, Effort.jl allows researchers to analyze data faster, inexpensively, and multiple times while making small changes based on nuances in the data.”
Modern cosmology is driven by vast datasets—from galaxy clustering surveys to measurements of cosmic microwave background ripples. To interpret this data, researchers rely on the Effective Field Theory of Large-Scale Structure (EFTofLSS), a mathematical framework that links the visible distribution of galaxies to the invisible scaffolding of dark matter.
One problem is that EFTofLSS computations are incredibly expensive. Traditional tools, such as CLASS, CAMB, and pyBird, can take seconds to minutes for each likelihood evaluation, and full Bayesian analyses require millions of such calculations. That means even with supercomputers, generating a high-resolution “map” of cosmic structure can take days or weeks.
Dr. Bonici’s team built Effort.jl to address this issue. Written in the Julia programming language, often prized in the scientific community for combining Python-like ease with C-level speed, the new emulator replaces the slowest steps of these analyses with a neural network that predicts their outputs almost instantly.
At its core, Effort.jl acts as a surrogate model for the EFTofLSS, using a carefully trained neural network to reproduce the galaxy power spectrum. This is the statistical fingerprint of how galaxies cluster across cosmic scales. By combining physics-based preprocessing with machine learning, it achieves both speed and precision.
One of its key design features is that it remains fully differentiable, allowing scientists to utilize powerful gradient-based inference techniques, such as Hamiltonian Monte Carlo (HMC). These methods can navigate complex, high-dimensional parameter spaces far more efficiently than the random-walk algorithms most cosmologists still use.
In the team’s tests, Effort.jl could compute the galaxy power spectrum in about 15 microseconds on a single CPU core. When paired with modern probabilistic programming frameworks such as Turing.jl, it achieved Bayesian convergence in roughly ten minutes on a laptop. This is compared to many hours on computing clusters using older software.
To validate its accuracy, the researchers applied Effort.jl to the PT-challenge simulations, a massive suite of high-precision cosmological mock universes, as well as the real-world Baryon Oscillation Spectroscopic Survey (BOSS) data.
In both cases, Effort.jl’s results agreed “within Monte Carlo noise” with those produced by pybird, the current workhorse of EFTofLSS analysis. Yet it finished in a fraction of the time.
“The entire analysis with Effort.jl required approximately ten minutes on a laptop and converged to the same posterior distribution as pybird,” the researchers write. “We stress that, given the large volume of the PT-challenge, this is a very stringent test of the accuracy [of] our emulator.”
Put simply, Effort.jl allows scientists to perform simulations that once required a supercomputer on a personal laptop.
While Effort.jl relies on neural networks, it isn’t what scientists call a “black-box AI.” In typical black-box systems, an algorithm may learn to make accurate predictions, but it does so without offering any insight into how those answers arise. This is a significant drawback in a field like cosmology, where physical understanding is as important as precision.
In contrast, Effort.jl employs what the authors describe as “physics-based preprocessing”—scaling and normalizing its inputs and outputs according to established physical laws, such as the evolution of the cosmic growth factor D(z) with redshift.
This approach anchors the neural network in established theory, enabling it to focus solely on the complex, nonlinear features that are most challenging to model. The result is an emulator that combines the interpretability of physics with the efficiency of AI—delivering speed without sacrificing scientific transparency.
Researchers even experimented with symbolic regression, a method that searches for human-readable equations instead of opaque neural weights. By finding an analytical approximation for the cosmic growth factor, they cut one of the emulator’s remaining bottlenecks, reducing a 150-microsecond calculation to just 200 nanoseconds.
“We were able to validate the predictions coming out of Effort.jl by aligning them with those coming out of EFTofLSS,” Dr. Bonici said. “The margin of error was small and showed us that the calculations coming out of Effort.jl are strong. Effort.jl can also handle observational quirks like distortions in data and can be customized very easily to the needs of the researcher.”
With instruments like DESI now cataloging tens of millions of galaxies and Euclid returning its first sky maps from orbit, cosmologists are entering what many call the “petabyte era” of data. The challenge isn’t collecting information anymore—it’s keeping up with it.
This is where Effort.jl could become indispensable. By making advanced modeling orders of magnitude faster, it enables researchers to run more exhaustive parameter scans, test exotic theories of dark energy, and integrate multiple datasets. This could include galaxy clustering or the cosmic microwave background. All in a single coherent analysis.
Because Effort.jl was built on top of AbstractCosmologicalEmulators.jl, its architecture is modular. This means the same framework can be retrained for other cosmological codes or even for entirely different physical domains, such as plasma physics and quantum materials. The team is already developing a Python-compatible JAX version to broaden adoption.
In practical terms, tools like Effort.jl could significantly reduce the time between data collection and discovery, enabling scientists to test new cosmological models nearly as quickly as telescopes can gather information.
That means faster updates to our estimates of the universe’s expansion rate, more precise measurements of dark matter’s distribution, and tighter constraints on the elusive properties of dark energy—the mysterious force driving the cosmos apart.
By streamlining this process, Effort.jl doesn’t just help researchers crunch numbers; it helps refine the very story of how the universe evolved and where it might be headed.
Equally as important is that the same computational advances that make Effort.jl so powerful—fast, interpretable machine learning combined with physics-based modeling—are already finding uses beyond cosmology.
Similar techniques could enhance weather and climate models, refine medical imaging, or even expedite the development of next-generation materials and energy systems.
By teaching computers to understand the universe in a manner similar to scientists, Effort.jl could help humanity make sense of everything, from the largest cosmic structures to the smallest technologies that shape daily life.
“The modular structure of Effort.jl is sufficiently general to support compatibility with other EFT-based codes,” the researchers conclude. “This flexibility opens the door for training Effort.jl to emulate these codes as well, broadening its application and usability.”
Ultimately, the breakthrough isn’t just about making cosmology faster—it’s about changing how scientists interact with data itself. By merging physics-informed machine learning with next-generation probabilistic programming, Effort.jl transforms the process of cosmic inference from a computationally intensive task into something nearly interactive.
For the first time, researchers could explore the universe’s deepest questions, from the nature of dark energy to the sum of neutrino masses, in real time.
The team has made Effort.jl freely available to the public, continuing a growing trend toward open, collaborative science in cosmology. The software is available on GitHub under an open-source MIT license, complete with documentation and example data for researchers to build upon.
By sharing the code behind their breakthrough, the authors aim to accelerate discovery not only within their own field, but across any discipline where understanding complex systems—from the cosmos to quantum materials—requires both speed and transparency.
“While previous codes have laid the groundwork,” researchers conclude, “Effort.jl offers a distinct advantage for analyses centered on gradient-based techniques, providing a robust and flexible toolkit tailored to the evolving needs of modern cosmological research.”
Tim McMillan is a retired law enforcement executive, investigative reporter and co-founder of The Debrief. His writing typically focuses on defense, national security, the Intelligence Community and topics related to psychology. You can follow Tim on Twitter: @LtTimMcMillan. Tim can be reached by email: tim@thedebrief.org or through encrypted email: LtTimMcMillan@protonmail.com