Vision-Based Interaction within a Multimodal Framework
No Thumbnail Available
Date
2022
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Our contribution is to the field of video-based interaction techniques and is integrated in the home environment of the EMBASSI project. This project addresses innovative methods of man-machine interaction achieved through the development of intelligent assistance and anthropomorphic user interfaces. Within this project, multimodal techniques represent a basic requirement, especially considering those related to the integration of modalities. We are using a stereoscopic approach to allow the natural selection of d evices via pointing gestures. The pointing hand is segmented from the video images and the 3D position and orientation of the forefinger is calculated. This modality has a subsequent integration with that of speech, in the context of a multimodal interaction infrastructure. In a first phase, we use semantic fusion with amodal input, considering the modalities in a so-called late fusion state.
Description
@inproceedings{10.2312:pt.20011318,
booktitle = {10º Encontro Português de Computação Gráfica},
editor = {Joaquim Madeira and Jorge Salvador Marques and Miguel Salles Dias and Joaquim A. Jorge},
title = {{Vision-Based Interaction within a Multimodal Framework}},
author = {Sá, Vítor and Malerczyk, Cornelius and Schnaider, Michael},
year = {2022},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-193-9},
DOI = {10.2312/pt.20011318}
}