FaceDrive: Facial Expression Driven Operation to Control Virtual Supernumerary Robotic Arms

Thumbnail Image
Journal Title
Journal ISSN
Volume Title
The Eurographics Association
Supernumerary Robotic Limbs (SRLs) can make physical activities easier, but require cooperation with the operator. To improve cooperation between the SRLs and the operator, the SRLs can try to predict the operator's intentions. A way to predict the operator's intentions is to use his/her Facial Expressions (FEs). Here we investigate the mapping between FEs and Supernumerary Robotic Arms (SRAs) commands (e.g. grab, release). To measure FEs, we used a optical sensor-based approach (here inside a HMD). The sensors data are fed to a SVM able to predict FEs. The SRAs can then carry out commands by predicting the operator's FEs (and arguably, the operator's intention). We ran a data collection study (N=10) to know which FEs assign to which robotic arm commands in a Virtual reality Environment (VE). We researched the mapping patterns by (1) performing an object reaching - grasping - releasing task using ''any'' FEs; (2) analyzing sensors data and a self-reported FE questionnaire to find the most common FEs used for a given command; (3) classifying the FEs in FEs groups. We then ran another study (N=14) to find the most effective combination of FEs groups / SRAs commands by recording task completion time. As a result, we found that the optimum combinations are: (i) Eyes + Mouth for grabbing / releasing; and (ii) Mouth for extending / contracting the arms (i.e. a along the forward axis).

, booktitle = {
ICAT-EGVE 2019 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments
}, editor = {
Kakehi, Yasuaki and Hiyama, Atsushi
}, title = {{
FaceDrive: Facial Expression Driven Operation to Control Virtual Supernumerary Robotic Arms
}}, author = {
Fukuoka, Masaaki
verhulst, adrien
Nakamura, Fumihiko
Takizawa, Ryo
Masai, Katsutoshi
Sugimoto, Maki
}, year = {
}, publisher = {
The Eurographics Association
}, ISSN = {
}, ISBN = {
}, DOI = {
} }