SmartSketcher: Sketch-based Image Retrieval with Dynamic Semantic Reranking

dc.contributor.authorPortenier, Tizianoen_US
dc.contributor.authorHu, Qiyangen_US
dc.contributor.authorFavaro, Paoloen_US
dc.contributor.authorZwicker, Matthiasen_US
dc.contributor.editorHolger Winnemoeller and Lyn Bartramen_US
dc.date.accessioned2017-10-18T08:37:15Z
dc.date.available2017-10-18T08:37:15Z
dc.date.issued2017
dc.description.abstractWe present a sketch-based image retrieval system, designed to answer arbitrary queries that may go beyond searching for predefined object or scene categories. While sketching is fast and intuitive to formulate visual queries, pure sketch-based image retrieval often returns many outliers because it lacks a semantic understanding of the query. Our key idea is to combine sketch-based queries with interactive, semantic re-ranking of query results. We leverage progress in deep learning and use a feature representation learned for image classification for re-ranking. This allows us to cluster semantically similar images, re-rank based on the clusters, and present more meaningful query results to the user. We report on two large-scale benchmarks and demonstrate that our re-ranking approach leads to significant improvements over the state of the art. Finally, a user study designed to evaluate a practical use case confirms the benefits of our approach.en_US
dc.description.sectionheadersSketching
dc.description.seriesinformationSketch-Based Interfaces and Modeling
dc.identifier.doi10.1145/3092907.3092910
dc.identifier.isbn978-1-4503-5080-8
dc.identifier.issn1812-3503
dc.identifier.urihttps://doi.org/10.1145/3092907.3092910
dc.identifier.urihttps://diglib.eg.org:443/handle/10.2312/sbim2017a01
dc.publisherAssociation for Computing Machinery, Inc (ACM)en_US
dc.subjectInformation systems
dc.subjectImage search
dc.subjectsketch
dc.subjectbased image retrieval
dc.subjectclustering
dc.titleSmartSketcher: Sketch-based Image Retrieval with Dynamic Semantic Rerankingen_US
Files