Adaptive Precision-Enhancing Hand Rendering for Wearable Fingertip Tracking Devices

Abstract

We introduce a 3D hand rendering framework to reconstruct a visually realistic hand from a set of fingertip positions. One of the key limitations of wearable fingertip tracking devices used in VR/AR applications is the lack of detailed measurements and tracking of the hand, making the hand rendering difficult. The motivation for this paper is to develop a general framework to render a visually plausible hand given only the fingertip positions. In addition, our framework adjusts the size of a virtual hand based on the fingertip positions and device’s structure, and reduces a mismatch between the pose of the rendered and user’s hand by retargeting virtual finger motions. Moreover, we impose a new hinge constraint on the finger model to employ a real-time inverse kinematic solver. We show our framework is helpful for performing virtual grasping tasks more efficiently when only the measurements of fingertip positions are available.