An automated ‘third hand’ for surgeons has been hailed as helping in routine tasks like retracting tissues, picking up gauze and clipping blood vessels.
The technology has the potential to enable the automated robotic arm to assist with complex procedures.
A multidisciplinary research team from The Chinese University of Hong Kong (CUHK)’s Faculty of Engineering and Faculty of Medicine (CU Medicine) has developed new artificial intelligence (AI)-powered surgical robot automation techniques, completing the world’s first multi-task surgical automation tests on a live animal.
The research has been published in the prestigious multidisciplinary research journal Science Robotics.
Professor Dou Qi, Assistant Professor from CUHK’s Department of Computer Science and Engineering, who led the study, said: ‘Traditional surgical automation approaches often relied on additional sensors or predefined models, which limited their clinical applicability. We used innovative AI techniques to create a brand-new embodied intelligence framework for surgical robot automation, contributing a data-driven and purely vision-based solution that is the first of its kind globally.’
This surgical embodied intelligence framework can analyse endoscopic images in real-time, without the need for additional sensors.
The framework integrates advanced visual foundation models, reinforcement learning and visual serving techniques to achieve accurate, efficient and safe automation of various surgical tasks.
Its foundation-model-based visual perception enables it to perform surgical scene understanding and depth estimation robustly in practice.
The reinforcement learning-based control policy was trained using SurRoL, an embodied AI simulator developed by the team. The simulation-trained policy can be directly deployed in real-world robots via zero-shot sim-to-real transfer. In this research, the developed AI system has been seamlessly integrated into the Sentire Surgical System, which has distinctive AI-readiness and AI-friendly characteristics.
This data-driven paradigm eliminates task-specific engineering, providing a general-purpose solution for versatile surgical autonomy through embodied AI, accelerating the translation from concept to pre-clinical testing.
The research team conducted in vivo testing of the AI system using a live animal model that replicated clinical surgical conditions.
The system successfully performed multiple autonomous surgical tasks, including tissue retraction, gauze picking and blood vessel clipping – actions that surgeons regularly perform during operations.
Dr Yip Hon-chi, assistant professor from the Department of Surgery at CU Medicine, who led the animal testing, said: ‘This represents a breakthrough in AI-powered surgical robot automation, validated across diverse tasks and environmental conditions. Our system demonstrates remarkable generalisability, maintaining stable performance despite environmental changes such as different tissue appearances and varying lighting conditions.’
By automating routine tasks with an AI assistant, the system can potentially greatly reduce surgeon workload, enhance overall surgical efficiency, and shorten procedure times for patients.
The InnoHK Multi-Scale Medical Robotics Centre (MRC) played a pivotal role in this ground-breaking research.
SurRoL was developed through a strategic collaboration between CUHK and Johns Hopkins University (JHU) in the US, fostered by the MRC’s international network.
The research team open-sourced the surgical embodied AI software infrastructure to the global surgical robotics research community in 2021, and it has since been adopted by numerous prestigious research institutions worldwide.
Professor Samuel Au Kwok-wai, Co-director of MRC and Professor from CUHK’s Department of Mechanical and Automation Engineering, said: ‘This work exemplifies the exceptional innovations that can emerge from international collaborations cultivated by the MRC. The research has achieved pioneering advancements in AI-powered surgical robot automation.’
The live animal experiments were conducted in the MRC’s hybrid operating room, which provided professional support for pre-clinical evaluation. This environment allowed the surgeon to rigorously test the newly developed AI algorithms under conditions that closely resemble actual surgical settings.
Professor Philip Chiu Wai-yan, Co-director of MRC and Dean of CU Medicine, said: ‘The MRC creates a unique synergy of engineering innovation and surgical expertise, significantly accelerating the journey from laboratory concepts to pre-clinical studies. This engineer-clinician collaborative research showcases the transformative potential of AI co-pilots in robotic surgery, positioning CUHK at the forefront of the global advancement of surgeon-AI-robot partnerships.’
