What's in a Face? Metric Learning for Face Characterization

dc.contributor.authorSendik, Omryen_US
dc.contributor.authorLischinski, Danien_US
dc.contributor.authorCohen-Or, Danielen_US
dc.contributor.editorAlliez, Pierre and Pellacini, Fabioen_US
dc.date.accessioned2019-05-05T17:41:52Z
dc.date.available2019-05-05T17:41:52Z
dc.date.issued2019
dc.description.abstractWe present a method for determining which facial parts (mouth, nose, etc.) best characterize an individual, given a set of that individual's portraits. We introduce a novel distinctiveness analysis of a set of portraits, which leverages the deep features extracted by a pre-trained face recognition CNN and a hair segmentation FCN, in the context of a weakly supervised metric learning scheme. Our analysis enables the generation of a polarized class activation map (PCAM) for an individual's portrait via a transformation that localizes and amplifies the discriminative regions of the deep feature maps extracted by the aforementioned networks. A user study that we conducted shows that there is a surprisingly good agreement between the face parts that users indicate as characteristic and the face parts automatically selected by our method. We demonstrate a few applications of our method, including determining the most and the least representative portraits among a set of portraits of an individual, and the creation of facial hybrids: portraits that combine the characteristic recognizable facial features of two individuals. Our face characterization analysis is also effective for ranking portraits in order to find an individual's look-alikes (Doppelgängers).en_US
dc.description.number2
dc.description.sectionheadersLearning Images
dc.description.seriesinformationComputer Graphics Forum
dc.description.volume38
dc.identifier.doi10.1111/cgf.13647
dc.identifier.issn1467-8659
dc.identifier.pages405-416
dc.identifier.urihttps://doi.org/10.1111/cgf.13647
dc.identifier.urihttps://diglib.eg.org:443/handle/10.1111/cgf13647
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.subjectfacial hybrids
dc.subjectface recognition
dc.subjectfeature polarization
dc.subjectneural networks CCS Concepts
dc.subjectComputing methodologies
dc.subjectNeural networks
dc.subjectImage processing
dc.titleWhat's in a Face? Metric Learning for Face Characterizationen_US
Files
Collections