Discriminative action tubelet detector for weakly-supervised action detection

Abstract

We propose a novel framework for spatiotemporal action detection using only video -level class labels as weak supervision. Traditional fully -supervised approaches rely on labor-intensive manual annotation of bounding boxes for each frame. In contrast, collecting video -level class labels is significantly less tedious and more feasible compared to annotating frame -level sequences with bounding boxes. To address this challenge, we propose a discriminative action tubelet detector, called DAT-detector, designed to discern discriminative tubelets from action tubelet proposals (ATPs). Whereas the previous approaches have only focused on tubelet selection among the predefined object proposals, our DAT-detector prioritizes the generation of more precise action tubelets using regression and attention modules. Moreover, we introduce an ATP generation method that enhances the quality of tubelet proposals. Our approach achieves state-of-the-art performance on several benchmarks, and also demonstrates competitive performance even with fully -supervised approaches.