Recent academic research involves the daily analysis and generation of vast amounts of data using supercomputers. However, the research process and results are often left to manual recording and management by researchers, raising concerns regarding reproducibility, fairness, and efficiency. Furthermore, from the perspective of promoting open science, there is a demand for greater transparency in the research process and the implementation of an audit trail management system.
In response, a research group led by Susumu Date, Director at The Joint Research Laboratory for Integrated Infrastructure of High Performance Computing and Data Analysis —established in 2021 by The University of Osaka D3 Center and NEC—developed SCUP-HPC (*3), a new technology that records and manages the computational provenance executed on supercomputers.
SCUP-HPC tracks, records, manages, and visualizes computational provenance—tracking what data is accessed by which programs and what data is generated—in cluster-type supercomputers where multiple high-performance computers are connected via high-speed networks, while minimizing the impact on the supercomputer’s performance.
This enables “Scientific Computing Unifying Provenance – High Performance Computing,” which integrates computational provenance on supercomputers, and is expected to dramatically improve the productivity of researchers using supercomputers for scientific computing tasks such as simulations and AI training.
Furthermore, by utilizing the provenance management and search service provided by SCUP-HPC, authorized users will be able to perform searches using history IDs and view visualized computational provenance. This will enable researchers to include the computation history ID in their paper acknowledgments (*4), confirming that the research results were computed using OCTOPUS, thereby helping to ensure the integrity of academic research.
NEC plans to commercialize a provenance management system for supercomputers utilizing SCUP-HPC in the future.