CNN-Based Denoising, Completion, and Prediction of Whole-Body Human-Depth Images

Abstract

Three-dimensional human shape reconstruction is important in many applications, such as virtual or augmented reality (VR/AR), virtual clothing fitting, and healthcare. In this paper, we propose a learning-based method for reconstructing a whole-body point cloud from a single front-view human-depth image. Because actual depth images typically suffer from noise and missing data, an accurate point cloud cannot be reasonably obtained by simply predicting a back-view depth image. To solve this problem, we propose to use convolutional neural networks that not only predict a back-view depth image but also refine the input front-view depth image. To train the networks, we propose a carefully designed method for generating synthetic but realistic human-depth images with noise and missing data. Experiments show that the proposed method is effective for obtaining seamless whole-body point clouds. In addition, the experiments show that the networks trained on the synthetic depth images are ready for application to actual depth images.