Development of surgical assistant navigation system using Augmented Reality glasses

Development of surgical assistant navigation system using Augmented Reality glasses
2D/3D registration; Augmented Reality; AR glasses; Face landmark; Surgical assistant system
Issue Date
ACDDE 2018 (Asian Conference on Design and Digital Engineering 2018)
Recently, interest in Augmented Reality (AR) technology has increased, followed by the development of related devices and diverse rising industry utilizing it. The medical field is not the exception and various researches to integrate AR technology into medical application have been conducted. In the conventional surgical environment, the surgical site of the patient locate in the operating table. However, medical images and information to help the surgery are displayed on physically separated screens. Such a separation can disturb the concentration of surgeon and decrease the safety of patient during surgery. In this study, we proposed an surgical guidance system that simultaneously displays various information of patient and surgery on the patient's face using AR glasses to overcome the suggested limitation. By registrating the 3D scanned face mesh and the 2D face image of the patient obtained from the AR glasses, surgical data and additional information can be overlaid on the face. The aligned 3D and 2D face is obtained by matching landmarks extracted from 3D mesh and 2D image respectively. Since landmarks are obtained from 2D image through AR glasses, there is no need to attach markers on the patient’s face. In addition, the visualization through AR glasses is invariant to the surgeon's pose and the motion of the patient during the surgery. The proposed system is expected to be utilized in the operation room to assist the surgery and guarantee the safety of patient. It can be more improved by integrating visualization of additional medical image or tracking of surgeon’s hand and surgical instrument.
Appears in Collections:
KIST Publication > Conference Paper
Files in This Item:
There are no files associated with this item.
RIS (EndNote)
XLS (Excel)


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.