Characterizing User Behavior for Speech and Sketch-based Video Retrieval Interfaces

Altıok, Ozan Can; Sezgin, Tev k Metin

Characterizing User Behavior for Speech and Sketch-based Video Retrieval Interfaces

Files

a10-altiok.pdf (3.8 MB)

Date

2017

Authors

Altıok, Ozan Can
Sezgin, Tev k Metin

Publisher

Association for Computing Machinery, Inc (ACM)

Abstract

From a user interaction perspective, speech and sketching make a good couple for describing motion. Speech allows easy speci cation of content, events and relationships, while sketching brings in spatial expressiveness. Yet, we have insu cient knowledge of how sketching and speech can be used for motion-based video retrieval, because there are no existing retrieval systems that support such interaction. In this paper, we describe a Wizard-of-Oz protocol and a set of tools that we have developed to engage users in a sketch- and speech-based video retrieval task. We report how the tools and the protocol t together using ”retrieval of soccer videos” as a use case scenario. Our so ware is highly customizable, and our protocol is easy to follow. We believe that together they will serve as a convenient and powerful duo for studying a wide range of multi-modal use cases.

        @inproceedings{10.1145:3122791.3122801
,
booktitle = {Computational Aesthetics
},
editor = {Holger Winnemoeller and Lyn Bartram
},
title = {{Characterizing User Behavior for Speech and Sketch-based Video Retrieval Interfaces
}},
author = {Altıok, Ozan Can and 
Sezgin, Tev k Metin
},
year = {2017
},
publisher = {Association for Computing Machinery, Inc (ACM)
},
ISSN = {1816-0859
},
ISBN = {978-1-4503-5080-8
},
DOI = {10.1145/3122791.3122801
}
}