FaceDrive: Facial Expression Driven Operation to Control Virtual Supernumerary Robotic Arms

Loading...
Thumbnail Image
Date
2019
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Supernumerary Robotic Limbs (SRLs) can make physical activities easier, but require cooperation with the operator. To improve cooperation between the SRLs and the operator, the SRLs can try to predict the operator's intentions. A way to predict the operator's intentions is to use his/her Facial Expressions (FEs). Here we investigate the mapping between FEs and Supernumerary Robotic Arms (SRAs) commands (e.g. grab, release). To measure FEs, we used a optical sensor-based approach (here inside a HMD). The sensors data are fed to a SVM able to predict FEs. The SRAs can then carry out commands by predicting the operator's FEs (and arguably, the operator's intention). We ran a data collection study (N=10) to know which FEs assign to which robotic arm commands in a Virtual reality Environment (VE). We researched the mapping patterns by (1) performing an object reaching - grasping - releasing task using ''any'' FEs; (2) analyzing sensors data and a self-reported FE questionnaire to find the most common FEs used for a given command; (3) classifying the FEs in FEs groups. We then ran another study (N=14) to find the most effective combination of FEs groups / SRAs commands by recording task completion time. As a result, we found that the optimum combinations are: (i) Eyes + Mouth for grabbing / releasing; and (ii) Mouth for extending / contracting the arms (i.e. a along the forward axis).
Description

        
@inproceedings{
10.2312:egve.20191275
, booktitle = {
ICAT-EGVE 2019 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments
}, editor = {
Kakehi, Yasuaki and Hiyama, Atsushi
}, title = {{
FaceDrive: Facial Expression Driven Operation to Control Virtual Supernumerary Robotic Arms
}}, author = {
Fukuoka, Masaaki
 and
verhulst, adrien
 and
Nakamura, Fumihiko
 and
Takizawa, Ryo
 and
Masai, Katsutoshi
 and
Sugimoto, Maki
}, year = {
2019
}, publisher = {
The Eurographics Association
}, ISSN = {
1727-530X
}, ISBN = {
978-3-03868-083-3
}, DOI = {
10.2312/egve.20191275
} }
Citation
Collections