A human-AI collaboration that categorises surgical feedback in live operating room (OR) settings could revolutionise surgical training.
Researchers claim the innovative approach offers key insights into how feedback could better shape trainee behaviour and responses.
Combining unsupervised machine learning techniques with expert human interpretation offers a scalable and precise framework for analysing real-world surgical feedback, they said.
As a result, it addresses some long-standing challenges in surgical education.
The findings underscore the critical role of targeted feedback in promoting behavioural adjustments among trainees.
Key findings include:
• Actionable feedback drives behaviour: Topics like ‘handling bleeding’ had the most substantial impact, prompting immediate trainee adjustments due to the urgency of intraoperative bleeding management. Other high-impact areas included ‘sweeping techniques’ and ‘positioning and height guidance’, highlighting their critical importance of immediacy during procedures.
• Precision matters: Feedback on technical execution, such as instrument positioning and retraction, proved especially effective in driving trainee improvement.
• Not all feedback leads to action: Comments focused on discussion, such as ‘trainer’s queries’ or general affirmations, were less likely to result in immediate changes, underscoring the need for actionable, task-specific feedback.
To address the complexities of live surgical feedback, the researchers developed a human-AI collaborative refinement process that leverages the unsupervised learning capabilities of BERTopic, an advanced topic modelling framework.
BERTopic uses pre-trained text embeddings to capture semantic meaning. This makes it particularly effective for analysing short, context-rich feedback typical of surgical interactions.
The AI component automatically grouped feedback instances into clinically relevant topics, reducing reliance on labour-intensive manual annotation.
Experts then refined these AI-generated topics to ensure clinical clarity and relevance. This hybrid approach combined AI’s scalability with human interpretation, resulting in actionable insights.
The study’s findings have far-reaching implications for surgical education. By identifying feedback topics strongly associated with trainee behaviour, the research provides a data-driven foundation for enhancing feedback delivery in the OR to maximise trainee growth.
Future systems could provide alerts for ineffective feedback and/or suggest context-specific improvements to boost communication dynamics.
While the study represents significant progress, researchers now call for further exploration, including integrating video and audio data to deepen insights.
