Kassubeck, MarcKappel, MoritzCastillo, SusanaMagnor, MarcusGuthe, MichaelGrosch, Thorsten2023-09-252023-09-252023978-3-03868-232-5https://doi.org/10.2312/vmv.20231224https://diglib.eg.org:443/handle/10.2312/vmv20231224This paper handles the highly challenging problem of reconstructing the shape of a refracting object from a single image of its resulting caustic. Due to the ubiquity of transparent refracting objects in everyday life, reconstruction of their shape entails a multitude of practical applications. While we focus our attention on inline shape reconstruction in glass fabrication processes, our methodology could be adapted to scenarios where the limiting factor is a lack of input measurements to constrain the reconstruction problem completely. The recent Shape from Caustics (SfC) method casts this problem as the inverse of a light propagation simulation for synthesis of the caustic image, that can be solved by a differentiable renderer. However, the inherent complexity of light transport through refracting surfaces currently limits the practical application due to reconstruction speed and robustness. Thus, we introduce Neural-Shape from Caustics (N-SfC), a learning-based extension incorporating two components into the reconstruction pipeline: a denoising module, which both alleviates the light transport simulation cost, and also helps finding a better minimum; and an optimization process based on learned gradient descent, which enables better convergence using fewer iterations. Extensive experiments demonstrate that we significantly outperform the current state-of-the-art in both computational speed and final surface error.Attribution 4.0 International LicenseCCS Concepts: Computing methodologies → Image-based rendering; Shape modeling; Machine learningComputing methodologies → Imagebased renderingShape modelingMachine learning10.2312/vmv.2023122433-419 pages