A Generative Neural Network for Learning Coordinated Reach-Grasp Motions

Abstract

A neural network for generating coordinated reach-grasp motions is proposed, based on a type of generative neural network called the conditional restricted Boltzmann machine (CRBM). Given demonstrations of humans reaching and grasping various target objects of different shapes and poses, a mixture-type CRBM model is first used to learn and cluster the reach-grasp motions into different movement types. A novel variant of CRBM, called CRBM-l, is then proposed, in which the CRBM network is augmented with a control variable that is adjustable for different target objects. A CRBM-l trained with the previously obtained movement-specific data is then used to generate real-time reach-grasp motions for new target objects, by appropriately adjusting the control variable. The generated reach-grasp motions are then fine-tuned taking into account the contact states between the object and the hand/fingers. The versatility and efficiency of our reachgrasp motion generation method is validated through systematic experiments involving a diverse set of target objects.