A vision-based system analyzing sit-to-stand and its clinical utility in classifying Parkinson’s disease

Abstract

Vision-based motion analysis, assessing Sit-to-Stand (STS) tasks, has emerged as a promising alternative to traditional sensor-based methods, which often require wearable devices and controlled environments. Yet, challenges regarding heavy dependency on views and human pose estimation persisted. This study aimed to develop and validate a view-independent vision-based system that can detect STS events and estimate vertical pelvic displacement using the videos of the STS task. Employing the features extracted from the system, multiclass classification for the severity of Parkinson’s Disease (PD) was conducted to validate the clinical utility of the developed system. Using a dataset comprising individuals with diverse ages and motor abilities, a Motion Guided Masked Autoencoder (MGMAE) was integrated into a multi-task learning pipeline to detect STS events and estimate vertical pelvic displacement directly from RGB videos. The MGMAE model achieved a mean Average Precision (mAP) of 0.952 for detecting the STS events and an Intraclass Correlation Coefficient (ICC) of 0.986 for the estimation of pelvic displacement. The BiGRU model achieved the highest accuracy in subject-wise classification of PD severity with an F1 score of 0.853, with an accuracy of 0.882. The proposed vision-based system can serve as a reliable and more affordable alternative to employing costly motion capture systems when assessing motor impairments. Advancing vision-based motion analysis, this study lays the groundwork for broader applications of it to other neurodegenerative and musculoskeletal conditions.