Projects

Lifelong Learning for Therapeutic Endoscopy

PI: Daniel Hashimoto, Galen Leung (GI), Gregory Ginsberg (GI), Eric Eaton (SEAS)

Funding: American Society for Gastrointestinal Endoscopy

While different deep learning approaches have been successful in identifying phases of surgery or instruments, existing approaches are limited to individual operations. However, many procedures share steps or are performed on the same organ(s). Such interconnections offer the potential for improved clinical benefit by incorporating recent advances in lifelong deep learning, which enables transfer of knowledge across procedures and continual improvement over time. This project investigates knowledge transfer across “third space” therapeutic endoscopic procedures.

Transfer Learning for Automated Surgical Feedback

PI: Daniel Hashimoto

Funding: American Surgical Association Foundation

Video-based coaching and assessment offer the opportunity to advance surgical performance. Yet, its potential is limited by difficulties in applying it to a broad audience due to the need for many human coaches. Artificial intelligence (AI) could provide automated surgical video analysis to provide coaching and assessment with reduced human input. However, current surgical AI methods focus on one task or one type of procedure at a time. This approach does not account for the interrelated nature of complex phenomena in an operation, preventing AI from providing actionable automated feedback on performance. In contrast, surgeons learn surgical principles and techniques that translate to different operations. This project develops a multitask AI model that can analyze and interpret a surgeon's technique in a way that is analogous to surgeons learning surgical principles, enabling algorithms to transfer knowledge learned in one operation to another. 

Metrics Revolutions: Clinically Relevant Outcome Measures for Computer Vision in Surgery

PI: Daniel Hashimoto, Lena Maier-Hein (Univ of Heidelberg), Amin Madani (UHN), Stefanie Speidel (Univ of Dresden), Dan Stoyanov (UCL)

Building off the Metrics Reloaded project to provide guidance to researchers on appropriate selection of metrics to evaluate the performance of visual models in biomedical imaging tasks, the Metrics Revolutions project seeks to outline more specific, clinically oriented metrics to assess the performance of models that work with visual surgical data.

Patient Perceptions of Therapeutic AI

PI: Daniel Hashimoto

         While much of the effort in applying AI to surgery has focused on developing methods for annotating ground truth in surgery and building visual models for the analysis of surgical video, few efforts have sought to determine how patients feel about the possibility of AI influencing the decisions that their physicians may make during a therapeutic procedure such as surgery. This work surveys the landscape of existing literature and seeks the input of a wide range of patients to understand better how patients understand and perceive the role of AI in their care.