Automatic exposure control for HoloLens 2 camera: detection of reflective markers for robust tool tracking under surgical light

Abstract

Introduction of surgical navigation into the operating room presents challenges related to lighting. While augmented reality (AR) has the potential to aid in surgery, most research on this subject have been conducted in preclinical settings with controlled lighting. HL2 (HoloLens 2), an OST-HMD (optical see-through head-mounted device) for AR commonly used in surgery, experiences performance limitations when exposed to direct and intense lighting. To tackle this problem, our research presents a real-time method to modify the exposure settings of the HL2 camera and cater to alterations in lighting conditions. The proposed algorithm identifies the brightest spot in a series of frames and then calculates its luminance level. Using this information, the process automatically determines the exposure time at a predefined ISO value while minimizing image artifacts. To assess the efficacy of this approach, we conducted experiments utilizing two different surgical lights and two distinct marker designs, both under static and dynamic conditions. The experimental findings suggest that both luminaires can achieve a detection rate of 93.59 +/- 7.58% (p = 0.001*) for smaller marker geometries. Conversely, for larger geometries, the detection rate was 91.90 +/- 9.02%, with p-values of 0.0137* and 0.0189*, respectively, under each luminaire source, respectively, can be achieved.