Speech and Sketching: An Empirical Study of Multimodal Interaction

Loading...
Thumbnail Image
Date
2007
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Sketch recognition can capture the sketching component of a multimodal conversation about design, but it does not capture information conveyed in the other modalities. The informal speech that accompanies a sketch often has a considerable amount of additional information. We want to develop a digital whiteboard capable of understanding both sketching and speech, and capable of participating in a conversation similar to one that the user would have with a human design partner. We conducted a user study to help us understand what kinds of conversations users would have with a whiteboard capable of recognizing a sketch. We report results that we believe will help guide the design of an effective multimodal interface, and discuss implications for system architectures.
Description

        
@inproceedings{
:10.2312/SBM/SBM07/083-090
, booktitle = {
EUROGRAPHICS Workshop on Sketch-Based Interfaces and Modeling
}, editor = {
Michiel van de Panne and Eric Saund
}, title = {{
Speech and Sketching: An Empirical Study of Multimodal Interaction
}}, author = {
Adler, A.
and
Davis, R.
}, year = {
2007
}, publisher = {
The Eurographics Association
}, ISSN = {
1812-3503
}, ISBN = {
978-3-905674-00-2
}, DOI = {
/10.2312/SBM/SBM07/083-090
} }
Citation