Lower limb joint angle estimation using a single RGB camera

Abstract

Accurate measurement of lower-limb joint angles is essential for gait analysis, rehabilitation, and early detection of degenerative disease. Current marker-based approaches using wearable sensors not only require complex setups but also intricate calibrations, hindering widespread application of them in daily life. This study aimed to propose a vision-based estimation of hip, knee, and ankle angles directly from single-view RGB frames. An end-to-end model that integrated a convolutional neural network for spatial feature extraction with a transformer module to capture temporal gait dynamics was built, and a novel loss function that jointly penalizes angle and angular-velocity errors improved overall consistency and accuracy of the estimation. The performance of the developed model was compared with that of the 3D pose method, and its robustness was examined by varying camera perspectives, walking speeds, and abnormality in gait patterns. The proposed model achieved the intraclass correlation coefficients above 0.92 across varying viewing angles, walking speeds, and gait patterns. The model outperformed the two-stage baseline of the 3D pose method. The proposed model not only eliminates the need for complex setups and sensors but is also less prone to inaccuracies compared to the two-stage pipeline method. When applied to clinical and home-based environments, it can offer a cost-effective and reliable solution for gait assessment.