ABSTRACT VIEW
Abstract NUM 1543

ENHANCING MUSEUM ACCESSIBILITY THROUGH A SPEECH-TO-VISUAL OVERLAY APPLICATION: A PILOT STUDY OF INCLUSIVE GUIDED TOURS
M. Namatame1, S. Inagaki2
1 Kyoto Women's University (JAPAN)
2 Rikkyo University (JAPAN)
This study addresses information accessibility challenges faced by Deaf and Hard of Hearing (DHH) students in educational and public learning environments. A novel application with the SwipeTalk interface was joined, enabling real-time voice-to-text transcription with spatial annotations and freehand drawings. This tool was tested in three practical settings: two in classrooms and one in a museum. In classroom contexts, the application supported teacher-student and peer communication, especially under conditions limiting face-to-face interaction. Students reported increased engagement and ease of exchanging comments through visual annotations. In museum practice, the application was evaluated against sign language interpretation and DHH-led tours. No significant differences were found in understandability, comprehension accuracy, or satisfaction among the three methods, suggesting the application offers accessibility comparable to traditional human-mediated communication. The results highlight the potential of visual-spatial communication tools to enhance inclusive learning and promote spontaneous interaction. Further improvements are needed for stability and usability, but the approach shows promise for broader educational use.

Keywords: Information Accessibility, Visual Communication Application, Deaf and Hard of Hearing.

Event: ICERI2025
Session: Assistive Technologies
Session time: Tuesday, 11th of November from 17:15 to 18:30
Session type: ORAL