Multi-Level Segmentation Data Generation based on a Scene-Specific Word Tree

Abstract

We, humans, perceive the scene utilizing pre-learned language categories. Our vocabulary system inherently possesses a hierarchy, aiding humans in understanding scenes at multiple levels. For example, when a person passes by chairs and desks from a distance rather than interacting with them up close, the objects are perceived from a broader perspective and recognized as furniture at a higher category level. In this work, we propose a multi-level semantic segmentation data generation method based on a scene-specific word tree to mimic human multi-level scene recognition. Multi-level semantic segmentation data encompasses diverse levels of grouped segmented areas with different degrees of detail, from the finest level of conventional semantic segmentation to coarser levels. Our scene-specific word trees leverage linguistic hierarchies to group scene components by considering relationships between words present in the scene. Furthermore, in the proposed data generation method, each word tree is constructed within a single image, allowing us to group the objects into user-selected levels, taking into account the relative relationship between objects in that scene. We demonstrate the effectiveness of our data generation method by building a multi-level scene segmentation network and training the model with the generated dataset, which reflects the scene-specific word tree.