VibVid: VIBration Estimation from VIDeo by using Neural Network

Yoshida, Kentaro; Inoue, Seki; Makino, Yasutoshi; Shinoda, Hiroyuki

VibVid: VIBration Estimation from VIDeo by using Neural Network

Files

037-044.pdf (6.89 MB)

Date

2017

Authors

Yoshida, Kentaro
Inoue, Seki
Makino, Yasutoshi
Shinoda, Hiroyuki

Publisher

The Eurographics Association

Abstract

Along with advances in video technology in recent years, there is an increasing need for adding tactile sensation to it. Many researches on models for estimating appropriate tactile information from images and sounds contained in videos have been reported. In this paper, we propose a method named VibVid that uses machine learning for estimating the tactile signal from video with audio that can deal with the kind of video where video and tactile information are not so obviously related. As an example, we evaluated by estimating and imparting the vibration transmitted to the tennis racket from the first-person view video of tennis. As a result, the waveform generated by VibVid was almost in line with the actual vibration waveform. Then we conducted a subject experiment including 20 participants, and it showed good results in four evaluation criteria of harmony, fun, immersiveness, and realism etc.

        @inproceedings{10.2312:egve.20171336
,
booktitle = {ICAT-EGVE 2017 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments
},
editor = {Robert W. Lindeman and Gerd Bruder and Daisuke Iwai
},
title = {{VibVid: VIBration Estimation from VIDeo by using Neural Network
}},
author = {Yoshida, Kentaro and 
Inoue, Seki and 
Makino, Yasutoshi and 
Shinoda, Hiroyuki
},
year = {2017
},
publisher = {The Eurographics Association
},
ISSN = {1727-530X
},
ISBN = {978-3-03868-038-3
},
DOI = {10.2312/egve.20171336
}
}