Lee, Hansoo Kim, Eunseo Lee, Jeongmin Joo, Hyunsoo Kwak, Sonya S. 2026-02-25T01:30:05Z 2026-02-25T01:30:05Z 2026-02-04 2025-09-30 https://pubs.kist.re.kr/handle/201004/154358 The Experience Sampling Method (ESM) allows researchers to collect ecologically valid data on users' daily experiences. However, conventional mobile ESM systems rely on visual interfaces (e.g., checkboxes, text fields), which can burden users in mobile or hands-free contexts. Voice-based alternatives address this but are often limited by rigid, rule-based interactions. We present SurveyTalk, a multimodal ESM system powered by large language models (LLMs) that enables real-time semantic mapping between speech and visual survey interfaces. The system supports voice-driven selection, clarification, navigation, and response interpretation, reducing user friction and improving data quality. Our approach enhances the accessibility, adaptability, and robustness of ESM in dynamic real-world settings. English ASSOC COMPUTING MACHINERY SurveyTalk: Voice-to-UI Mapping for In-Situ ESM via LLMs Conference 10.1145/3746058.3758377 1 38th Symposium on User Interface Software and Technology-UIST-Annual, pp.1 - 3 38th Symposium on User Interface Software and Technology-UIST-Annual 1 3 US Busan 2025-09-28 ADJUNCT PROCEEDINGS OF THE 38TH ANNUAL ACM SYMPOSIUM ON USER INTERFACE SOFTWARE AND TECHNOLOGY, UIST 2025 001612646100119 2-s2.0-105020851454