CabinetBot: A Context and Intention-Aware Robotic Cabinet System for Supporting Object Retrieval, Organization, and Storage

Abstract

Managing personal belongings, such as retrieving, organizing, and storing objects, is a cognitively demanding task in daily life, especially for individuals with physical or mental limitations. We present CabinetBot, a context and intention-aware robotic cabinet system that supports object management through multimodal sensing and interaction. The system integrates computer vision, hand-object interaction recognition, and large language models (LLMs) to proactively detect user behavior and respond through automated drawer actuation. In our evaluation, CabinetBot demonstrated both high object detection and action recognition accuracy (over 90%) and reliable understanding of user voice commands. This work highlights a human-centered approach to robotic assistance in everyday environments by enabling natural, adaptive support for object management-related tasks.