CLIP-based Neural Neighbor Style Transfer for 3D Assets
| dc.contributor.author | Mishra, Shailesh | en_US |
| dc.contributor.author | Granskog, Jonathan | en_US |
| dc.contributor.editor | Babaei, Vahid | en_US |
| dc.contributor.editor | Skouras, Melina | en_US |
| dc.date.accessioned | 2023-05-03T06:02:53Z | |
| dc.date.available | 2023-05-03T06:02:53Z | |
| dc.date.issued | 2023 | |
| dc.description.abstract | We present a method for transferring the style from a set of images to the texture of a 3D object. The texture of an asset is optimized with a differentiable renderer and losses using pretrained deep neural networks. More specifically, we utilize a nearest-neighbor feature matching (NNFM) loss with CLIP-ResNet50 that we extend to support multiple style images. We improve color accuracy and artistic control with an extra loss on user-provided or automatically extracted color palettes. Finally, we show that a CLIP-based NNFM loss provides a different appearance over a VGG-based one by focusing more on textural details over geometric shapes. However, we note that user preference is still subjective. | en_US |
| dc.description.sectionheaders | Stylization and Point Clouds | |
| dc.description.seriesinformation | Eurographics 2023 - Short Papers | |
| dc.identifier.doi | 10.2312/egs.20231006 | |
| dc.identifier.isbn | 978-3-03868-209-7 | |
| dc.identifier.issn | 1017-4656 | |
| dc.identifier.pages | 25-28 | |
| dc.identifier.pages | 4 pages | |
| dc.identifier.uri | https://doi.org/10.2312/egs.20231006 | |
| dc.identifier.uri | https://diglib.eg.org:443/handle/10.2312/egs20231006 | |
| dc.publisher | The Eurographics Association | en_US |
| dc.rights | Attribution 4.0 International License | |
| dc.rights.uri | https://creativecommons.org/licenses/by/4.0/ | |
| dc.subject | CCS Concepts: Computing methodologies → Appearance and texture representations; Rasterization; Supervised learning by regression | |
| dc.subject | Computing methodologies → Appearance and texture representations | |
| dc.subject | Rasterization | |
| dc.subject | Supervised learning by regression | |
| dc.title | CLIP-based Neural Neighbor Style Transfer for 3D Assets | en_US |