Real-Time Mapping of Sensed Textures into Vibrotactile Signals for Sensory Substitution

Abstract

This paper presents a method to transform a surface texture sample sensed with a force-torque sensor into a vibrotactile stimulus in real time, as a technique to let a hand amputee feel the surface of objects. We built a convolution neural network with the contact force for real-time texture classification and haptic rendering. The neural network was constructed from the contact force between the force-torque sensor and sliding physical texture samples with three wavelengths. Once the classifier is constructed and if the force-torque sensor moves over a texture, the classified texture is mapped to a sinusoidal source signal generated with a DAQ board. We mapped the textures with the wavelengths of 3.14, 6.28, and 9.42 mm into sinusoids with the frequency of 150, 100 and 50 Hz. Then, the source signal is amplified and drives a piezoelectric actuator installed on a user's forearm, to provide a vibrotactile stimulus corresponding to the sensed texture.