Text2Autochrome: Text guided autochrome synthesis using generative models

Loading...
Thumbnail Image
Date
2025
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Autochrome is an early color photography technique that is highly sensitive and prone to deterioration, limiting their public display. A limited collection of digitized autochromes exists, often with defects due to their fragile nature. We applied generative AI methods, specifically Low-Rank Adaptation (LoRA), to fine-tune diffusion models, enabling efficient use of computational resources. Our curated dataset of vintage digitized autochromes showcased various styles and served as the basis for training the LoRA model, resulting in the generation of digitized autochromes that preserved the original color filter effects and characteristic granularity. By leveraging generative AI, we can utilize the multi-modal capabilities of the model, allowing each user to generate images through concept-based prompts. This approach empowers users to creatively interact with the model, producing personalized images while maintaining the historical color fidelity and structure of autochromes. This capability also enables us to generate defect-free autochromes, which can be utilized for synthetic training in autochrome restoration efforts. We evaluated our approach using the CLIPScore metric for quantitative similarity and conducted a user study for qualitative feedback on the generated images. Our results show that the fine-tuned LoRA model effectively captures the essence of autochromes, producing visually appealing images that respect the historical aesthetic. Considering the potential for misinterpretation and ethical concerns surrounding text-to-image methods using deep learning with historical photographs, we are committed to enhancing transparency by releasing our model weights and training datasets, thereby empowering the community to better understand, evaluate, and address these important issues. Further we release an interactive demo together with the fine-tuned weights available via huggingface.
Description

CCS Concepts: Computing methodologies → Neural networks

        
@inproceedings{
10.2312:dh.20253061
, booktitle = {
Digital Heritage
}, editor = {
Campana, Stefano
and
Ferdani, Daniele
and
Graf, Holger
and
Guidi, Gabriele
and
Hegarty, Zackary
and
Pescarin, Sofia
and
Remondino, Fabio
}, title = {{
Text2Autochrome: Text guided autochrome synthesis using generative models
}}, author = {
Kühn, Paul Julius
and
Sinha, Saptarshi Neil
and
Nguyen, Duc Anh
and
Horst, Robin
and
Kuijper, Arjan
and
Fellner, Dieter W.
}, year = {
2025
}, publisher = {
The Eurographics Association
}, ISBN = {
978-3-03868-277-6
}, DOI = {
10.2312/dh.20253061
} }
Citation