Toward Robust AR Surgical Navigation: Eliminating External Trackers and Ensuring OST-HMD Alignment Under Various Operating Room Lighting Conditions

Abstract

Existing integrations of augmented reality (AR) into surgical navigation systems heavily depend on external tracking systems prone to occlusion and increased complexity of setting up the devices. These issues can compromise the accuracy and usability of AR navigation systems in a clinical operating room. Additionally, the impact of illumination exposure on AR system performance in a surgical room remains underexplored. Current methods for addressing the misalignment issue of AR contents in optical-see-through head-mounted displays (OST-HMDs) also lack sufficient precision, particularly for depth orientation. This study evaluated two inside-out tracking systems, the monocular tracker and stereo trackers, using the RGB and grayscale cameras of Microsoft HoloLens 2. Experimental assessments showed that the proposed stereo tracker achieved superior accuracy, with translational and rotational error of 0.73 mm and 0.15 degrees, respectively, outperforming the proposed monocular tracker and existing reported literature. The proposed automated method to mitigate the impact of excessive light exposure can maintain a 98% marker detection rate across various lighting conditions and marker configurations. These results showed that the proposed techniques can eliminate the dependence on external trackers by providing robust performance in any surgical setting. A phantom study on AR-guided needle insertion further validated the efficacy of the proposed system, achieving an accuracy of 1.65 +/- 0.16 mm and 2.69 +/- 0.19 degrees. These results meet the precision requirements for clinical surgeries, highlighting the potential of the proposed AR system to improve surgical outcomes.