Preliminary study of AR navigation-guided core-needle breast biopsy

Abstract

Accurate tumor localization has remained to be one of the biggest challenges in core-needle biopsy due to the malleability and texture-heterogeneity of tissues surrounding a tumor. To overcome these limitations, we explored the effectiveness of augmented reality (AR)-guidance to navigate the core-needle insertion in breast biopsy thanks to the enhanced technology of AR. Our proposed framework consists of two independent functions: tumor localization and needle pose tracking in real-time. First, tumor localization was achieved by employing the two-front environmental sensors of HoloLens 2 (HL2) for tracking the passive-reflective markers. Given the geometrical information of the reference markers, the stereo-correspondences and the marker pose can be calculated efficiently for overlaying the corresponding virtual models in HL2. Second, an external RGB camera was used to track the pose of a needle. A You-Only-Look-Once (YOLO) model with a customized training set of 200 images was employed to detect the collinear pinktape points attached to the needle. The interpoint-distance between these points will be utilized to estimate the pose of a needle that will be sent to HL2 using TCP/IP communication. Finally, real-time navigation of needle insertion can be rendered in HL2 using a more accurate transformation to assist the breast biopsy. Overall, the preliminary results demonstrate that the whole process can be executed in a reasonable running time of less than two seconds.