Medical Physics Simulation provides both classical physics-based solvers and generative world-model-based simulators for real-time simulation of device\u2013anatomy interactions\u2014enabling RL-ready policy training across healthcare robotics segments. Both modules provide a modular research and engineering platform for interactive surgical and interventional simulation, robot learning, synthetic data generation, and procedure development. They are well suited for machine learning workflows because both simulation and learning run on the same GPU, avoiding the overhead of repeated CPU\u2013GPU memory transfers.<\/p>\n
Classical solvers\u00a0\u00a0<\/a><\/p>\n The Endoluminal Simulation Module<\/a> is released for general availability in Isaac for Healthcare. It enables real-time simulation of diagnostic and interventional procedures involving long, flexible surgical instruments navigating through endoluminal cavities. It is implemented as a standalone package, so it can be integrated independently into different workflows and environments. This initial release shows catheter navigation through the vascular system under fluoroscopic guidance.<\/p>\n The module is implemented in Python using NVIDIA Warp<\/a> and Newton Physics<\/a>. The flexible instruments are modelled as Cosserat rods, providing a solid theoretical foundation for simulating materials\u2019 bending, twisting, and stretching deformation. To capture the complex nonlinear dynamics of these one-dimensional rods efficiently on the GPU, extended position-based dynamics (XPBD) was selected as the primary simulation method.\u00a0<\/p>\n XPBD typically relies on local, iterative constraint projections. For long instruments, however, local projections may require many iterations to propagate motion from the controlled proximal end to the distal tip. This module instead assembles the coupled rod constraints into a matrix system.\u00a0<\/p>\n Each rod segment contributes six constraint equations\u2014three for stretch and shear and three for bending and twisting\u2014producing a block-tridiagonal XPBD system with 6 \u00d7 6 blocks. The Thomas algorithm solves this system in linear time with respect to instrument length, while independent instruments are processed concurrently across vectorized environments on the GPU.\u00a0<\/p>\n This globally coupled solver propagates inputs such as proximal insertion and rotation along the entire instrument within each simulation step, even for very long instruments, enabling immediate response to user manipulations at the distal end.<\/p>\n \t\tFigure 1. Catheter insertion and steering through a vascular model<\/p>\n Simulating catheter-vessel interactions<\/p>\n Catheter\u2013vessel interaction is also evaluated in parallel on the GPU. Signed closest-point queries are evaluated directly against the patient-specific triangle mesh, avoiding a separately precomputed distance field. Instrument samples are projected back into the lumen when they penetrate the wall while retaining tangential sliding motion. The current release supports rigid, one-way containment; two-way coupling to deformable vessel walls is planned for a future release.<\/p>\n Control inputs are expressed directly through the rod\u2019s boundary conditions and rest state. Insertion advances the constrained proximal end along the introducer axis, hub rotation sets its orientation, and distal steering modifies the rest curvature of the final rod segments rather than applying an artificial external force. The bend-and-twist constraints then propagate these inputs through the instrument.<\/p>\n Connecting physics, imaging, and robot learning<\/p>\n The Endoluminal Simulation Module maintains compatibility with other Newton solvers through a common API. Torch\u2013Warp interoperability, provided through wp.from_torch and wp.to_torch keeps the control and reinforcement-learning interface zero-copy and fully GPU-resident. This enables straightforward integration with NVIDIA Isaac Lab, a framework designed for large-scale robot learning. Isaac Lab manages environment orchestration and rollout control, while the solver executes the GPU-based dynamics and returns device-resident tensors for observation, reward, and control loops.<\/p>\n This co-design of physics and imaging supports high-throughput policy training at scale, including experiments with 512 parallel environments and 1,500 training iterations. Reported performance is approximately 1,300 Hz for single-environment physics, 60 Hz for physics simulation across 512 environments, and 63 frames per second for the complete simulation and rendering loop at a resolution of 256 \u00d7 256 pixels. The platform also provides a foundation for future policy training directly from fluoroscopic images and other pixel-based observations.<\/p>\n \t\tFigure 2. Simulated catheter navigation in patient-specific vasculature<\/p>\n The catheter navigation workflow runs as a unified simulation loop that combines two coupled patient-specific components. The vasculature-digital-twin<\/a> package generates the anatomical attenuation\/geometry artifacts (mu_volume, vessel mask\/mesh, and centerline), and the endoluminal simulation<\/a> package advances catheter mechanics under XPBD constraints inside that anatomy. For each frame, the solver state is transformed into renderable catheter segments and composited by fluorosim<\/a> into the same CT-derived imaging domain. This keeps the physics and fluoroscopy synchronized in one end-to-end loop while additional detector-response effects improve the realism of the resulting fluoroscopic images.<\/p>\n \t\tFigure 3. Synchronized catheter physics and fluoroscopic imaging<\/p>\n Surgical Simulation Module\u00a0<\/a><\/p>\n The Surgical Simulation Module available through early access is a real-time GPU-accelerated surgical simulation framework, designed to model soft-tissue deformation and surgical interactions with haptic feedback and rendering within a unified simulation pipeline.<\/p>\n Similar to the Endoluminal Simulation Module, it is also implemented in Python using NVIDIA Warp<\/a> and Newton Physics<\/a>. Unlike monolithic physics engines, it organizes its functionality as explicitly ordered simulation systems. Each procedure is configured through the sequence of deformation, collision, grasping, cutting, clipping, diathermy, and rendering operations. This makes the physical GPU pipeline inspectable, configurable, and easier to extend.<\/p>\n \t\tFigure 4. Robotic interaction with deformable soft tissue<\/p>\n Soft tissue is represented using tetrahedral meshes. Deformation is solved with position-based dynamics using distance- and volume-preserving constraints. Corrections are accumulated and averaged on the GPU, reducing constraint-ordering artifacts and supporting efficient parallel execution (Jacobi solver).\u00a0<\/p>\n Additionally, both simulation and visualization loops remain on the GPU, including deformation, collision queries, grasping, thermal diffusion, topology updates, and surface reconstruction. NVIDIA CUDA graph capture further reduces kernel-launch overhead, while direct GPU-to-renderer data transfer (CUDA interop) avoids unnecessary CPU\/GPU copies. This enables real-time simulation at more than 30 frames per second with eight physics substeps and eight internal deformation constraint iterations on one consumer-grade GPU.\u00a0<\/p>\n The initial release of this module supports an end-to-end cholecystectomy procedure (gallbladder removal). The simulation begins with the robotic instruments inserted into the inflated abdominal cavity, visualizing key structures like the liver, gallbladder, and connective fatty tissue. It involves dissection of the hepatocystic triangle, clipping and cutting the cystic duct and artery, and separating the gallbladder from the liver.<\/p>\n \t\tFigure 5. Simulated robotic cholecystectomy<\/p>\n Learned world dynamics and generative physics for surgical simulator\u00a0<\/a><\/p>\n The generative approach to medical physics simulation is a complementary technology to physics solvers. Rather than explicitly programming every interaction, a generative world model learns from video and other data how real scenes look and how they change over time. During training, it learns relationships between visual observations, motion, and conditioning signals such as robot actions. At inference time, it uses this learned world knowledge to predict the next video frames.\u00a0<\/p>\n Because the result is produced in the visual space seen by cameras and robotic policies, the model can reproduce scene detail and appearance that would be difficult to build by hand. It also avoids much of the manual work needed to create every asset, material, and lighting setup for a conventional simulator. The tradeoff is that these predictions are learned from data: they can be visually realistic without providing the explicit physical guarantees of a numerical solver.<\/p>\n NVIDIA Cosmos provides the world foundation models for this approach. Cosmos video models use a flow-based diffusion-transformer architecture with a rectified-flow formulation. During generation, the model iteratively transforms noise in a compressed video representation into a coherent sample from the learned data distribution. A video tokenizer compresses pixel-space frames into this latent representation for efficient modeling and decodes the generated latents back into video. Multimodal conditioning makes the generated future controllable. For medical applications, post-training on domain data adapts the broad visual model before specialized anatomy, devices, procedures, and dynamics.<\/p>\n Generative simulation with Cosmos-H<\/a><\/p>\n The Cosmos-H family shows several forms of generative medical physics simulation. Cosmos-H-Surgical-Predict forecasts plausible future surgical video from an initial image and text. Cosmos-H-Surgical-Transfer generates surgical video guided by controls such as depth, edges, segmentation, or blur. Cosmos-H-Surgical-Simulator predicts future frames conditioned directly on surgical-robot kinematics.\u00a0<\/p>\n Together, these models support use cases ranging from synthetic-data generation and controlled domain transfer to action-conditioned robot training and policy evaluation. They do not replace explicit physics in every setting, but reduce scene-authoring burden and provide realistic, observation-level simulations where complete first-principles modeling is impractical.<\/p>\n Cosmos-H-Dreams extends this approach to interactive simulation. It distills a teacher fine-tuned from Cosmos-H-Surgical-Simulator into a causal, few-step student that autoregressively streams future surgical video in response to robot actions. Served through the FlashDreams inference engine, it turns a high-quality offline world model into a real-time environment with frame generation at more than 30 frames per second. The environment can be controlled by a human operator or queried in closed loop by a learned robotics policy on a single workstation GPU.<\/p>\n Video 1. Interactive simulation of tissue suturing with Cosmos-H-Dreams<\/p>\n Why healthcare robotics needs both approaches<\/a><\/p>\n Isaac for Healthcare supports both classical and generative simulation to address the complementary requirements of healthcare robotics. To provide user trust and quantitative quality assessment, which are the foundations of any healthcare solution, the physically consistent and deterministically modeled environments provided by classical simulation are essential. Generative simulation, in turn, produces diverse and visually realistic environments at scale that classical simulation cannot, enabling clinical applications in both data synthesis and real-time clinical workflows.<\/p>\n Get started with medical physics simulation<\/a><\/p>\n Medical Physics Simulation in Isaac for Healthcare provides developers with the building blocks to create patient-specific digital twins, simulate device\u2013anatomy interactions, generate synthetic experiences, and train intelligent healthcare robotics systems. The following resources help you get started with each simulation capability.<\/p>\n Build an Endoluminal Robotics Workflow<\/a><\/p>\n Build patient-specific catheter navigation simulations using GPU-accelerated physics, anatomical digital twins, and simulated medical imaging.<\/p>\n Generate and create a vasculature digital twin<\/a>.<\/p>\n Incorporate a catheter-vasculature interaction solver<\/a>.<\/p>\n Simulate X-ray perception using X-ray sensor simulation<\/a>.<\/p>\n Build the end-to-end catheter navigation workflow<\/a>.\u00a0<\/p>\n Cosmos-H Dreams generative surgical simulation<\/a><\/p>\n Cosmos-H Dreams enables scalable generation of realistic surgical experiences using world foundation models. Developers can use pretrained models, run interactive simulation, or adapt the framework to new procedures and robotic embodiments.<\/p>\n Run Cosmos-H Dreams. Download the Cosmos-H Dreams model. Create a custom Cosmos-H Dreams model. Extend Cosmos-H:<\/p>\n Fine-tune Cosmos-H-Surgical-Simulator on your data by following its training guide.<\/p>\n Distill the fine-tuned checkpoint to a few-step diffusion schedule for real-time Deploy the distilled checkpoint with![\"Two](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2026\/07\/Catheter-insertion.gif\")
<\/p>\n![\"Catheter](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2026\/07\/Catheter-navigation.gif\")
<\/p>\n![\"Simulated](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2026\/07\/Catheter-path-.gif\")
<\/p>\n![\"Robotic](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2026\/07\/Tissue.gif\")
<\/p>\n![\"A](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2026\/07\/Gall-bladder.gif\")
<\/p>\n
\n Get started with pretrained checkpoints, examples, and interactive simulation workflows in the
\n GitHub repository<\/a>.<\/p>\n
\n Access pretrained model checkpoints and configurations from the
\n Hugging Face checkpoint<\/a>.<\/p>\n
\n Cosmos-H Dreams is not limited to the procedures and robots included with the framework.
\n Choose one of two paths to build a real-time simulator for your own embodiment:<\/p>\n
\n throughput using the
\n distillation guide<\/a>.<\/p>\n
\n –pipeline.diffusion-model.transformer.checkpoint-path.<\/p>\n