A new AI system is transforming surgical imaging by instantly matching X-rays with preoperative 3D scans, enhancing precision and safety in minimally invasive procedures.
To improve surgical device localisation, clinicians often manually align X-rays with preoperative 3D images such as CT scans or MRIs.
Until now, AI tools designed to facilitate this alignment often struggle to perform reliably across diverse patients, limiting their clinical utility.
This new system, developed by scientists and clinicians at the Massachusetts Institute of Technology and collaborators, uses an AI model that adapts to each patient in about five minutes. This model automatically registers a patient’s X-rays with 3D scans in seconds, achieving sub-millimetre accuracy.
Named xvr (X-ray volume registration), it outperformed existing AI methods by an order of magnitude across a wide range of patients, body parts, and medical procedures.
In many minimally invasive surgical procedures, such as angioplasty to open blocked arteries, clinicians insert instruments through a tiny incision and use a high-speed mobile X-ray scanner to generate images that let them visualise the procedure from any angle.
But to guide surgical tools without accidentally damaging other tissue, clinicians must align real-time X-rays with the patient’s preoperative MRI or CT scan. This process, called registration, helps them determine where the tool is in relation to anatomical structures.
Vivek Gopalakrishnan, a postdoc in the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL), a recent graduate of the Harvard-MIT Program in Health Sciences and Technology and lead author of a paper on xvr, said. ‘It takes decades of training for a clinician to become skilled enough to see grainy, 2D images and understand how everything is oriented. We want to make these 2D X-rays more informative, so it becomes safer and easier to do these life-saving procedures.’
Rather than building a machine-learning model that works for all patients, the researchers built a model designed to adapt extremely well to a specific patient.
xvr takes one patient’s preoperative 3D scan, like an MRI or CT, and uses it to generate thousands of synthetic X-rays from many angles, producing about 1,000 images each second. It uses a physics-based simulation of the X-ray process to ensure these synthetic images are realistic.
The xvr framework uses these simulated data to train an AI model that can align a patient’s 2D X-rays with their 3D scan in seconds.
While such a registration model is highly accurate, training it from scratch for each patient would take about 12 hours, making it impossible to deploy in an emergency. To speed up the process, the researchers used xvr to pretrain a more versatile AI system, called a foundation model, that can quickly adapt to each new patient.
They collected whole-body 3D medical scans from more than 2,000 patients, covering a wide range of ages, imaging modalities, and regions. xvr used these diverse data to generate synthetic X-rays and train a foundation model to perform 2D/3D registration.
This pretrained model can adapt to a new patient in about five minutes and perform registration with the same accuracy as if it were trained from scratch.
In the future, the researchers hope to focus on making xvr faster for real-time deployment, conducting further studies to verify its reliability in additional situations, and extending the system to handle more complex scenarios, such as moving body parts.
Published: 19.10.2026
