Sá, VítorMalerczyk, CorneliusSchnaider, MichaelJoaquim MadeiraJorge Salvador MarquesMiguel Salles DiasJoaquim A. Jorge2022-12-212022-12-212022978-3-03868-193-9https://doi.org/10.2312/pt.20011318https://diglib.eg.org:443/handle/10.2312/pt20011318Our contribution is to the field of video-based interaction techniques and is integrated in the home environment of the EMBASSI project. This project addresses innovative methods of man-machine interaction achieved through the development of intelligent assistance and anthropomorphic user interfaces. Within this project, multimodal techniques represent a basic requirement, especially considering those related to the integration of modalities. We are using a stereoscopic approach to allow the natural selection of d evices via pointing gestures. The pointing hand is segmented from the video images and the 3D position and orientation of the forefinger is calculated. This modality has a subsequent integration with that of speech, in the context of a multimodal interaction infrastructure. In a first phase, we use semantic fusion with amodal input, considering the modalities in a so-called late fusion state.Attribution 4.0 International LicenseEMBASSI project, 3D deictic gestures, multimodal man-machine interaction, agent-based systemsEMBASSI project3D deictic gesturesmultimodal manmachine interactionagentbased systems10.2312/pt.2001131861-677 pages