Lee, Cheon Kim, Sung-Yeol Choi, Byeongho Kwon, Yong-Moo Ho, Yo-Sung 2024-01-20T12:02:48Z 2024-01-20T12:02:48Z 2021-09-05 2013-07 0091-3286 https://pubs.kist.re.kr/handle/201004/127913 When the three-dimensional (3-D) video system includes a multiview video generation technique using depth data to provide more realistic 3-D viewing experiences, accurate depth map acquisition is an important task. In order to generate the precise depth map in real time, we can build a camera fusion system with multiple color cameras and one time-of-flight (TOF) camera; however, this method is associated with depth errors, such as depth flickering, empty holes in the warped depth map, and mixed pixels around object boundaries. In this paper, we propose three different methods for depth error reduction to minimize such depth errors. In order to reduce depth flickering in the temporal domain, we propose a temporal enhancement method using a modified joint bilateral filtering at the TOF camera side. Then, we fill the empty holes in the warped depth map by selecting a virtual depth and applying a weighted depth filtering method. After hole filling, we remove mixed pixels and replace them with new depth values using an adaptive joint multilateral filter. Experimental results show that the proposed method reduces depth errors significantly in near real time. (c) The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI. English SPIE-SOC PHOTO-OPTICAL INSTRUMENTATION ENGINEERS SPECIAL-ISSUE Depth error compensation for camera fusion system Article 10.1117/1.OE.52.7.073103 1 OPTICAL ENGINEERING, v.52, no.7 OPTICAL ENGINEERING 52 7 scie scopus 000322478600006 2-s2.0-84880478274 Optics Optics Article SPECIAL-ISSUE camera fusion system time-of-flight camera depth camera depth map generation depth error reduction