Publications

Gaze Enhanced Speech Recognition for Truly Hands-Free and Efficient Text Input During HCI

Authors:
Matheus Vieira Portela – Universidade de Brasilia, Brasilia, Brazil
David Rozado – CSIRO, QLD, Brisbane, Australia
Published at:  OzCHI ’14 Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: the Future of Design
Publisher: ACM http://dl.acm.org/citation.cfm?id=2686679
Publication date: December 2nd, 2014
Download: PDF

Abstract: The performance of current speech recognition algorithms is well below that of human speech recognition, with high number of misrecognized words in quiet environments and degrading even further in noisy ones. Therefore, hands-free interaction remains a deeply frustrating experience. In this work, we present an innovative form of correcting misrecognized words during a speech recognition task by using gaze tracking technology in a multimodal approach. We propose to employ the user’s gaze to point at misrecognized words and select appropriate alternatives. We compare the performance of this multimodal approach with traditional modalities of correcting words: usage of mouse and keyboard and usage of voice alone. The results of the user study show that whereas the proposed system is not as fast as using mouse and keyboard for correction, gaze enhanced correction significantly outperforms voice alone correction and is preferred by the users, offering a truly hands-free means of interaction.