GCH 2023 - Eurographics Workshop on Graphics and Cultural Heritage

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/3543875

Browse

Now showing 1 - 20 of 35

High Resolution 2D-/3D-Scanning and Deep Learning Segmentation for Digitization of Fragmented Wall Paintings
(The Eurographics Association, 2023) Kroeger, Ole; Krumpek, Oliver; Koch, Paul; Pape, Martin; Schneider, Jan; Krüger, Jörg; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
The preservation and study of mural wall paintings often involve the collection of numerous fragments with unknown context. In this paper the authors present a case study involving a Roman wall painting discovered in 2013 at the European Cultural Park Bliesbruck-Reinheim. The objective of this work was to develop a semi-automated assistance system for the digitization, visualization, and digital repositioning of the Roman wall painting fragments. Therefore an easy-to-use scanner system was developed, that captures high-resolution 2D images of the front and back surfaces of the fragments, along with a height map of the backside. The contributing partners also developed a control and operating software for the scanner, as well as an automated software platform for visualization and repositioning of the digital fragments. The contributions of this paper include the introduction of a ML-based algorithm for background subtraction and segmentation of the front surface of the fragments. The technical realisation for fast and accurate image acquisition of the fragments, including sensor registration and highresolution capture, has been worked out. The system calibration process, hardware setup and data correction techniques are described in detail. Additionally, the challenges of pixel-wise image segmentation for distinguishing between background, inner contour (wall painting), and outer contour (fragment surface without painting) are discussed. Our proposed approach overcomes the limitations of training ML algorithms on high-resolution images by employing patch-wise training and leveraging small features instead of large-scale features. The digitization and segmentation process demonstrated promising results in preserving and reconstructing the roman wall painting fragments. The findings of this study contribute to the field of cultural heritage preservation and provide valuable insights as the developed equipment and methods are highly transferable to future digitization projects.
One-to-many Reconstruction of 3D Geometry of cultural Artifacts using a synthetically trained Generative Model
(The Eurographics Association, 2023) Pöllabauer, Thomas; Kühn, Julius; Li, Jiayi; Kuijper, Arjan; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Estimating the 3D shape of an object using a single image is a difficult problem. Modern approaches achieve good results for general objects, based on real photographs, but worse results on less expressive representations such as historic sketches. Our automated approach generates a variety of detailed 3D representation from a single sketch, depicting a medieval statue, and can be guided by multi-modal inputs, such as text prompts. It relies solely on synthetic data for training, making it adoptable even in cases of only small numbers of training examples. Our solution allows domain experts such as a curators to interactively reconstruct potential appearances of lost artifacts.
R-CNN based PolygonalWedge Detection Learned from Annotated 3D Renderings and Mapped Photographs of Open Data Cuneiform Tablets
(The Eurographics Association, 2023) Stötzner, Ernst; Homburg, Timo; Bullenkamp, Jan Philipp; Mara, Hubert; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Motivated by the demands of Digital Assyriology and the challenges of detecting cuneiform signs, we propose a new approach using R-CNN architecture to classify and localize wedges. We utilize the 3D models of 1977 cuneiform tablets from the Frau Professor Hilprecht Collection available as pen data. About 500 of these tablets have a transcription available in the Cuneiform Digital Library Initiative (CDLI) database. We annotated 21.000 cuneiform signs as well as 4.700 wedges resulting in the new open data Mainz Cuneiform Benchmark Dataset (MaiCuBeDa), including metadata, cropped signs, and partially wedges. The latter is also a good basis for manual paleography. Our inputs are MSII renderings computed using the GigaMesh Software Framework and photographs having the annotations automatically transferred from the renderings. Our approach consists of a pipeline with two components: a sign detector and a wedge detector. The sign detector uses a RepPoints model with a ResNet18 backbone to locate individual cuneiform characters in the tablet segment image. The signs are then cropped based on the sign locations and fed into the wedge detector. The wedge detector is based on the idea of Point RCNN approach. It uses a Feature Pyramid Network (FPN) and RoI Align to predict the positions and classes of the wedges. The method is evaluated using different hyperparameters, and post-processing techniques such as Non-Maximum Suppression (NMS) are applied for refinement. The proposed method shows promising results in cuneiform wedge detection. Our detector was evaluated using the Gottstein system and with the PaleoCodage encoding. Our results show that the sign detector performs better when trained on 3D renderings than photographs. We showed that detectors trained on photographs are usually less accurate. The accuracy on photographs improves when trained, including 3D renderings. Overall, our pipeline achieves decent results, with some limitations due to the relatively small amount of data. However, even small amounts of high-quality renderings of 3D datasets with expert annotations dramatically improved sign detection.
Hybrid XR Collaborative and Guided Experiences in Cultural Heritage: Brancacci POV Prototype
(The Eurographics Association, 2023) Pescarin, Sofia; Cerato, Ivana; D'Annibale, Enzo; Fanini, Bruno; Ferdani, Daniele; Fà, Rachele Manganelli Del; Marasco, Alessandra; Massidda, Marcello; Palombini, Augusto; Ronchi, Diego; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
In this work, we present a research prototype of a hybrid XR experience for cultural tourism, accessible by any potential device, including immersive VR headsets. It has been designed to better answer visitors' expectations and to overcome the limitations offered by traditional Virtual Tours (VT). We present the conceptual model, prototype, applied case study and users survey. The overall result has shown how the development of a collaborative and guided experience, enhanced by the use of a gamified narrative, has produced an increased perception of authenticity of the VT.
Prototyping Care: Two Case Studies
(The Eurographics Association, 2023) Clay, Arthur; Trumpy, Giorgio; Weinmann, Michael; Wetzel, Richard; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
To enable a richer presentation of cultural heritage and its needs, a shift in how artworks are exhibited is necessary. This paper explores two case studies that highlight the significant role of reproductions in showcasing restoration processes and associated technologies. This approach raises awareness about concepts of care and authenticity and their impact. It goes beyond merely displaying restored digital images that fail to capture the true state of the artworks or the artist's original intent. To achieve this, we propose employing glass layers or lenticular print technology, allowing a restored version of the artwork's original state while maintaining the ability to view the original artwork and restoration process separately.
Interaction Styles in a Multi-Modal Virtual Museum
(The Eurographics Association, 2023) Southall, Ethan; Hulusic, Vedad; Hargood, Charlie; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Virtual Reality is used for creating immersive experiences with rich interactions in many application domains, from video games, simulations, and training, to cultural heritage and educational applications. Taking advantage of this technology, the experience in traditional museums can be enhanced with digital content, the museum or their collections can be replicated for remote visitors, or entirely new virtual museums can be created. In this paper, a demo of a multi-modal virtual museum is presented and interaction discussed from the point of view of a consumer and a virtual museum creator.
Large Room Scale Augmented Reality in an Unaltered World Heritage Site
(The Eurographics Association, 2023) Olbrich, Manuel; Zapf, Andreas; Stiegemann, Christoph; Pröbe, Annika; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Augmented Reality (AR) has found use in cultural heritage exhibitions in many forms during the last decade [BBG∗22], but usually by preparing the environment with markers or focusing on smaller exhibits like single statues [KKK∗14]. In this paper we present our approach to apply the technology to a larger unaltered environment. We show how we use Augmented Reality in this cultural heritage site to superimpose what was lost in time over what is preserved today, and virtually illustrate different aspects of the creation or reconstruction. Furthermore, we explain how the application, which is purely an extension of guided tours, is set up so that the guide can control the visitor's experience and steer them to the current point of interest. In this way, the visitor moving through the room is virtually shown the originally existing rich design of the cultural heritage site and is able to access it.
Automatic Detection of Windows Reflection or Transparency Pollution in TLS Acquisitions
(The Eurographics Association, 2023) Badalyan, Edgar; Schenkel, Arnaud; Debeir, Olivier; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Three-dimensional acquisitions have been more and more used in recent years, for multiple applications, such as cultural heritage preservation. When these point clouds are generated through laser scanning, transparent and/or reflective objects such as windows can generate inexact or undesirable data. These must be cleaned up by a human, which is often time-consuming and requires experience. This work provides an insight on some methods that can be used to automate this task. It investigates the usage of Mask R-CNN with intensity images in equirectangular projections. The huge images are tiled into squares of 2048x2048 pixels for both training and prediction. The model has good performances on the test and validation sets to handle both types of problems; but also to manage the presence of a mirror in a scene.
Interactive Digital Narrative Authoring Tools and Hybrid Experiences in Cultural Heritage: An Integrated Review
(The Eurographics Association, 2023) Massidda, Marcello; Travaglini, Laura; Pescarin, Sofia; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Interactive Digital Narrative Authoring Tools (IDN-AT) are software applications that allow users, without the need for programming skills, to create, design, and publish interactive digital narrative applications. In the field of Digital Heritage and specifically in that of Virtual Museums there is an increasing need of such tools, to enable non programmers (designers, creatives and curators) to design and develop hybrid experiences in cultural contexts. This paper aims at reviewing available solutions, defining the gaps and still missing features, tools or services, with the goal of paving the way for a webXR integrated solution.
Effective Interactive Visualization of Neural Relightable Images in a Web-based Multi-layered Framework
(The Eurographics Association, 2023) Righetto, Leonardo; Bettio, Fabio; Ponchio, Federico; Giachetti, Andrea; Gobbetti, Enrico; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Relightable images created from Multi-Light Image Collections (MLICs) are one of the most commonly employed models for interactive object exploration in cultural heritage. In recent years, neural representations have been shown to produce higherquality images, at similar storage costs, with respect to the more classic analytical models such as Polynomial Texture Maps (PTM) or Hemispherical Harmonics (HSH). However, their integration in practical interactive tools has so far been limited due to the higher evaluation cost, making it difficult to employ them for interactive inspection of large images, and to the difficulty in integration cost, due to the need to incorporate deep-learning libraries in relightable renderers. In this paper, we illustrate how a state-of-the-art neural reflectance model can be directly evaluated, using common WebGL shader features, inside a multiplatform renderer. We then show how this solution can be embedded in a scalable framework capable to handle multi-layered relightable models in web settings. We finally show the performance and capabilities of the method on cultural heritage objects.
Estimating the Number of Chases used for Printing Books with Movable Metal Types
(The Eurographics Association, 2023) Kim, Dongkeun; Ahmed, Maaz; Choi, Kang-Sun; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Many books were printed with movable metal types during the Joseon Dynasty (A.D. 1392-1897), and even early books are older than the Gutenberg Bible which was the earliest book printed using movable metal type in Europe (A.D. 1455). However, there is little information about the printing technology, such as the scale of metal type at that time and the management system. In this paper, we propose an algorithm to estimate the number of chases simultaneously used to print a book with movable metal types, which can provide not only insights into the level of printing technology at that time but also important clues for estimating the number of movable metal types. In contrast to previous studies of chase count estimation based on the subjective comparison, we propose an algorithm to estimate the number of chases using the similarity of character spacing distribution in the central part of the printed sheets, assuming that the metal types for the central part were set once and reused without replacement during the production of the book. The central part of each sheet in one book is cropped. After rotational deviations of all the central part images are removed using the principal component analysis, the images are binarized. Morphological operations are performed to remove noise and facilitate character analysis. The profile histogram of the binarized image is used as a probability distribution for character spacing, and the Wasserstein distances among the profile histograms of all the central part images in the book are calculated to obtain a similarity matrix of the central part images. By performing an eigenvalue analysis using spectral clustering with the similarity matrix, we determine the number of clusters of the central part images, which indicates the number of chases simultaneously used in printing the book. When applying the proposed algorithm to the images of "Neungumgyeoung" Volume 4 (A.D. 1462), "Seokbosangjeol" Volume 6 (A.D. 1446), and "Worin cheongangjigok" Volume 1 (A.D. 1447), it was estimated that 3, 6, and 4 chases were used, respectively. To verify the reliability of the results, subjective classification of the core part based on the size or style of the characters was performed for "Neungumgyeoung" Volume 4, and it was classified into 3 groups, consistent with the results of the proposed algorithm. This proposed method can be utilized for the analysis of various metal movable chases.
AI Based Image Segmentation of Cultural Heritage Objects used for Multi-View Stereo 3D Reconstructions
(The Eurographics Association, 2023) Kutlu, Hasan; Brucker, Felix; Kallendrusch, Ben; Santos, Pedro; Fellner, Dieter W.; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Image segmentation (or masking) finds a very useful use case within 3D reconstruction of cultural heritage objects. The 3D reconstructions can be accelerated, reconstructing the object without any background noise. Conventional segmentation methods can calculate erroneous masks for certain objects and environments, which can lead to errors within the reconstruction: Parts of the 3D reconstruction may be missing or are incorrectly reconstructed, which contradicts adequate archiving. The automated iterative Multi-View Stereo (MVS) scanning process makes it necessary to obtain masks that reconstruct the object in the best possible way, regardless of the environment, the stabilizing mount, the color of the background and the object. In addition, it should not be necessary to tweak the best possible parameters for conventional masking procedures and to create masks manually. State-of-the-art artificial intelligence (AI) segmentation networks will be trained and applied to the MVS scans to verify the behavior of the associated 3D reconstructions and the automated iterative scanning process. In addition, a comparison between different AI segmentation networks and a comparison between conventional masking methods and AI segmentation networks is performed.
Inferring the Routes of Prehistoric Humans
(The Eurographics Association, 2023) Rahary, Adrien Ramanana; Lorre-Guidt, Mylène; Grégoire, Sophie; Cani, Marie-Paule; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Prehistoric research is in high demand for 3D simulation to help validate hypotheses and enrich knowledge. While previous multidisciplinary studies focused on the reconstruction of ecosystems around excavation sites, this work takes up a new issue: deducing and visualizing the routes and the time taken by the hominids to reach the places where they harvested their resources, based on field data about raw material sources, the paleoclimate, and the surrounding ecosystem. We rely on the energy consumption of a simplified version of a walking character to compute the most likely locomotion speed and the best routes between input sites of interest, while taking into account local slopes, ground types and the presence of vegetation along the way. We show that this approach allows to evaluate the duration of the typical journeys of Homo heidelbergensis staying at the Caune de l'Arago (France) to collect lithic raw materials and to hunt - allowing archaeologists to deepen their knowledge of economic and territorial practices to the Lower Paleolithic and the Acheulean culture, 500,000 years ago.
The NuragAI project: Artificial Intelligence-driven Image Analysis of Sardinia Landscape, Searching for Unknown Monuments
(The Eurographics Association, 2023) Palombini, Augusto; Baiocchi, Edoardo; Lanzino, Romeo; Malatesta, Saverio Giulio; Marini, Marco Raoul; Rosati, Paolo; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
The NuragAI project aims at using an AI approach for image analysis of satellite imagery, in order to identify the nuraghi monuments (typical monuments of the Sardinia region, Italy, lasting from 1.8 to 0.7 ky BC), exploring the landscape and attempting a systematic survey of the Sardinia island. At the moment, a preliminary training has been performed, with an exceptionally positive result, and an earlier experiment of use. The satisfactory results are leading the authors towards a systematic analysis of the satellite imagery of the whole region, in order to check new monuments and open new perspectives for the use of AI in archaeological research.
Developing a VR Game Featuring Optical Illusion Challenges to Support Cultural Heritage - A Progress Report
(The Eurographics Association, 2023) Nisiotis, Louis; Elia, Konstantinos; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
This paper presents the development state of a Virtual Reality game to support cultural heritage through puzzle-based optical illusion gaming challenges. It is an immersive escape room type of game focusing on the cultural heritage of Cyprus, where players must complete a series of optical illusion challenges to progress, while learning historical information about key landmarks, monuments and objects of significant heritage to the Cypriot community. The main project objective is to bring together various archaeological artefacts in a virtual space and providing an immersive gaming experience to its users/visitors. The paper highlights the need for investigating the topic of VR gaming to support cultural heritage. It presents the development progress to date, including information on the digitisation process, environment design, and the mechanics of the VR game. Some of the future directions of the project are also presented.
Interaction Design as Catalyzer of Creative and Interpretative Praxes in Museum Context
(The Eurographics Association, 2023) Veggi, Manuele; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
This paper argues that interaction design (IxD) can suggest valid solutions in rethinking museum mediation, focusing on conservation data as reference case study. Eliciting interest, it is possible to conceive meaningful and authentic experiences, aimed at activating a transformative dialogue with the visitors: with this approach, audiences are creatively engaged to reconsider their own expectations, behaviours and beliefs. To support this hypothesis, major attention will be paid to describe the set up and the results of a first proof of concept, conceived in the framework of the PERCEIVE Project and based on Les demoiselles à la rivière by Matisse, now exhibited at the Art Institute of Chicago.
Advanced Interactive Digital Storytelling in Virtual Reality presentation of Austrian-Hungarian fortresses around Trebinje
(The Eurographics Association, 2023) Rizvic, Selma; Boskovic, Dusanka; Mijatovic, Bojan; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
Interactive Digital Storytelling (IDS) is a way of communicating information in the metaverse [BH22]. Gameplay elements added to these applications turn them into serious games that can offer unique immersive experiences of historical objects and events to museum visitors and online. In this paper, we describe this "Advanced IDS" methodology on a use case of the Trebinje Fortresses VR project. Austrian Hungarian Fortresses around Trebinje, Bosnia, and Herzegovina, are in decay, hardly accessible, and completely neglected. They can be explored in Virtual Reality through an application that combines Interactive Digital Storytelling narrated by officers who used to inhabit them with gameplays where the user becomes a carrier pigeon and carries the messages from one fortress to another while avoiding eagle attacks. After the first mission is successfully completed the user gets a task to accomplish in a fortress underground, and, if successful, gets the opportunity to explore digitized museum exhibits found in the fortress. The application is installed in the Trebinje Museum. The quality of user experience will be proven through an evaluation study.
A Digital Data Curation-Based Photogrammetric Acquisition Methodology for Cultural Heritage, expanded with CIDOC CRM Compatibility: protocol BeA-PG
(The Eurographics Association, 2023) Lauro, Vittorio; Lombardo, Vincenzo; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
The increasing digitalization of cultural heritage assets and the exponential expansion of digital repositories have created new opportunities for archaeological documenting, preservation, and dissemination. Photogrammetry, a survey approach that uses photographic acquisitions to construct highly detailed 3D models of objects, buildings, and landscapes, is one of the most promising techniques in this field. The effectiveness of the photogrammetric process, however, has still limitations due to a lack of experience sharing and data interoperability, which have resulted from the singularities of the techniques in both process and data format. To address this issue, this paper proposes a novel workflow for the photogrammetric process, called BeA-PG, developed for the BeArchaeo archaeological project. BeA-PG implements two relevant features for improving the effectiveness: on the one hand, sharing is improved by widening the scope of the process representation from the specific archaeological context to the broader cultural heritage context, by adopting the Digital Data Curation process abstraction; on the other, it improves the interoperability by describing the process with a shared ontological vocabulary, namely the CIDOC-CRM model. The paper introduces the major features of the photogrammetric process, its description through the abstract flow of the Digital Data Curation process, and its encoding through the CIDOC-CRM vocabulary. We illustrate the BeA-PG workflow and its experimental application in the practical setting of the BeArchaeo project.
HerMeS - HERitage sMart social mEdia aSsistant: from Requirement Elicitation to Data Modelling for Feeding Artificial Intelligence Recommendation System
(The Eurographics Association, 2023) Bucciero, Alberto; Chirivì, Alessandra; Jaziri, Mohamed Ali; Muci, Irene; Orlandini, Andrea; Umbrico, Alessandro; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
In the last years, several mobile APPs have been developed within the cultural tourism domain to give new impetus to this sector which is booming both in Italy and worldwide. In the wake of the increasing importance of technologies based on artificial intelligence, even mobile applications for the use of cultural tourism heritage are increasingly taking advantage of these techniques. Machine learning strategies are increasingly used to recommend points of interest and itineraries that are compatible with the user's preferences, requirements and constraints. The quality and integrity of the data acquired become the starting point for training and implementing AI models. By levering well-structured data, these algorithms can offer valuable insights, personalised recommendations, and enhanced user interaction in the cultural tourism domain. The HerMeS APP that we present in this paper was designed starting from these premises. The application aims to provide a wide range of artificial intelligence-based features to enhance the enjoyment and exploration of cultural heritage, both tangible and intangible.
DataSpace-ISPC: a Semantic Platform for Heritage Science
(The Eurographics Association, 2023) Bucciero, Alberto; Chirivì, Alessandra; Anglada, Gerardo; Demetrescu, Emanuel; Fanini, Bruno; Paraciani, Nicolò; Bucciero, Alberto; Fanini, Bruno; Graf, Holger; Pescarin, Sofia; Rizvic, Selma
The contribution presents the DataSpace of the Institute of Heritage Science (ISPC) of the Italian National Council for Research. Cultural Heritage is well known to be a multidisciplinary domain where heterogeneous data, methods and techniques overlap, co-exist and collaborate to create evidence-based wisdom represented with local and (rarely) inter-operable standards. DataSpace aims to overcome the need to harmonise converging research perspectives, allow collaborative research, and debabelize the presentation of data. This horizontal perspective is based on a graph of knowledge, ruled by a semantic reference model, and acts as a glue to connect different digital resources created with different perspectives. The DataSpace, which is built on top of the Arches platform integrated and customised with original tools, has been validated with positive results against two case studies to cover the wider domain of Heritage Science. The contribution explains the activities and technological solutions adopted and goes into detail about the problems and limitations that arose with the implementation of the DataSpace offering a critical yet positive perspective on the use of the DataSpace-ISPC in production scenarios.

Browse

Browsing GCH 2023 - Eurographics Workshop on Graphics and Cultural Heritage by Issue Date

Results Per Page

Sort Options