Newswise \u2014 X-ray tomography is a powerful tool that enables scientists and engineers to peer inside of objects in 3D, including computer chips and advanced battery materials, without performing anything invasive. It\u2019s the same basic method behind medical CT scans. Scientists or technicians capture X-ray images as an object is rotated, and then advanced software mathematically reconstructs the object\u2019s 3D internal structure. But imaging fine details on the nanoscale, like features on a microchip, requires a much higher spatial resolution than a typical medical CT scan \u2014 about 10,000 times higher.<\/p>\n
The Hard X-ray Nanoprobe<\/a> (HXN) beamline at the National Synchrotron Light Source II<\/a> (NSLS-II), a U.S. Department of Energy (DOE) Office of Science user facility at DOE\u2019s Brookhaven National Laboratory, is able to achieve that kind of resolution with X-rays that are more than a billion times brighter than traditional CT scans.<\/p>\n Tomography only works well when these projection images can be taken from all angles. In many real-world cases, however, that\u2019s impossible. For example, scientists can\u2019t spin a flat computer chip around 180 degrees without blocking some of the X-rays. When parallel to the surface at high angles, fewer X-rays can penetrate the chip, limiting the viewing angles of the measurement. The missing data from this angular range produces a \u201cblind spot,\u201d leading the reconstruction software to produce blurry, distorted images.<\/p>\n \u201cWe call this the \u2018missing wedge\u2019 problem,\u201d said Hanfei Yan, lead beamline scientist at the HXN beamline and corresponding author of this work. \u201cFor decades, this problem has limited the applications of X-ray and electron tomography in many areas of science and technology.\u201d<\/p>\n To solve the problem, researchers at NSLS-II have developed a new method called the perception fused iterative tomography reconstruction engine (PFITRE). This novel approach combines the physics of X-rays with the power of artificial intelligence (AI). The team trained a convolutional neural network, a type of AI model that automatically learns data patterns, with simulated data. Convolutional neural networks use convolutional layers to detect important features, such as edges, textures, or shapes, and combine these features to make predictions, like identifying what\u2019s in an image. The AI component captures perceptual knowledge about the sample, what the team expects the solution should look like, and uses it to improve the reconstructed image based on that knowledge. Meanwhile, the physics-based model checks that the results still make sense scientifically. This process repeats several times until results from the AI and physics components converge, producing a reconstruction that is both accurate and visually clear. Their results were recently published in npj Computational Materials<\/a>.<\/p>\n Better vision requires extensive training<\/p>\n Unlike image correction in consumer tech, like cell phone cameras, scientific imaging must preserve accuracy, not just appearance. Scientists needed to devise a method to ensure that the corrected image is still consistent with the physical model and data. To do this, NSLS-II scientists embedded the AI into an iterative solving engine, a mathematical tool that tackles complex problems by repeatedly trying improved solutions, step by step, until it gets close enough to the right answer. The embedded AI acts as a \u201csmart\u201d regularizer, a function that limits overcorrection, leveraging physics-based modeling to ensure that the reconstructions stay faithful to the actual X-ray measurements.<\/p>\n \u201cWe didn\u2019t want an AI that just makes better images. We wanted an AI that works hand-in-hand with physics, so that the results are both visually clear and scientifically trustworthy,\u201d said Chonghang Zhao, a postdoc at HXN and lead author of this work. \u201cThat\u2019s the power of our method \u2014 combining the sophistication of AI with the physical model to ensure fidelity.\u201d<\/p>\n The AI in PFITRE is built on a type of neural network called a U-net architecture, an encoder-decoder design that is popular for general image processing. The encoder stage learns and detects essential features, such as the edges, textures, and shapes of an input image, and the decoder stage rebuilds the image using those features to restore details and correct distortions. The researchers enhanced the ability of U-net with structural modifications called residual dense blocks and dilated convolutions. These help the network capture information across multiple scales from fine textures to larger structures, making the network more suitable for handling the missing wedge problem in tomography. A model like this can\u2019t learn on its own, though. It needs significant amounts of data to train on.<\/p>\n Real scientific microscopy datasets are too limited for effective training in a specific AI model like PFITRE, so the team relied on synthetic data. They generated training datasets by using natural images, simulated patterns, and scanning electron microscope images of circuits as samples being imaged. To make the training as realistic as possible, they introduced a \u201cdigital twin\u201d of the experiment and created virtual data that mimics real-world conditions. They intentionally included noise, misalignment, and other imperfections so the AI could handle physical data.<\/p>\n Impacting the future of imaging<\/p>\n While there is still work to be done to perfect this method, the benefits are clear. Samples that were once inaccessible due to their size or geometry can now produce informative data. A larger field of view allows more of a sample to be analyzed without falling victim to the missing wedge. This method could also prove beneficial in experiments where fewer measurements are required, enabling faster in situ studies and reducing radiation dose in sensitive samples.<\/p>\n \u201cThis opens the door to detailed imaging of samples that couldn\u2019t be studied before. That\u2019s a very big step forward,\u201d Yan said. \u201cWhether it\u2019s diagnosing defects in microchips or understanding why a battery degrades, PFITRE allows us to see details under conditions that were previously considered infeasible.\u201d<\/p>\n While PFITRE is a major advance, the team acknowledges room for improvement. Currently, the method processes 3D objects slice by slice. Expanding it into a full 3D approach would further enhance consistency but require more computation. Another challenge is including more artifacts, like ones from faulty pixels or sample movement, to broaden its application range. Like other AI models, it can\u2019t fix issues that have not been \u201cseen\u201d before. To address this, future work will incorporate building a richer training dataset that includes many types of artifacts and developing ways for the model to learn more effectively with less training.<\/p>\n This new, powerful 3D image analysis method has the potential to accelerate discoveries across many fields, from developing faster, more efficient microchips to synthesis of new materials and even biomedical applications. As machine learning and synchrotron science continue to evolve together, tools like this will sharpen scientists\u2019 view of the microscopic world to address some of society\u2019s greatest scientific challenges.<\/p>\n Brookhaven National Laboratory is supported by the Office of Science of the U.S. Department of Energy. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit science.energy.gov<\/a>.\u00a0<\/p>\n Follow @BrookhavenLab on social media. Find us on Instagram<\/a>, LinkedIn<\/a>, X<\/a>, and Facebook<\/a>.\u00a0<\/p>\n \u00a0<\/p>\n \t\t\t\t\t\t\t\t\t\t