A Natural Language Interface for the Visualization and Analysis of 3D Point Cloud Saliency Maps
Loading...
Date
2025
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Deep learning models for analyzing 3D point clouds have been extensively researched. However, there is limited research on explainable artificial intelligence techniques for such models. Approaches relying on saliency maps seek to attribute model outputs to their input; nevertheless, their practical use for extracting meaningful insights on model behavior depends on user knowledge, available visualization tools, and ad hoc data analytics pipelines. In this work, we propose a system that combines a natural language interface, backed by a large language model, with a traditional 3D point cloud visualization interface as an interactive technique for visualizing and analyzing 3D point cloud saliency maps. Our system can 1) answer general questions in the context of 3D point cloud saliency maps, 2) answer data analytics-related questions about the 3D point cloud being analyzed and its associated saliency map, and 3) generate additional visualizations that aid in the extraction of meaningful insights from the 3D point cloud data. We demonstrate the proposed system's workflow through a use case study involving a sample input to a 3D point cloud semantic segmentation model containing color, ground truth, predicted class, curvature, and attribution values. Our results indicate that including a natural language interface facilitates the exploration of the data and enhances the efficiency of the data analytics process, enhancing the value provided by the 3D point cloud saliency maps. We release our code as open source at https://github.com/JorgeFCS/llm-3DpointCloud-viz.
Description
CCS Concepts: Human-centered computing → Natural language interfaces; Graphical user interfaces; Visualization systems and tools; Geographic visualization
@inproceedings{10.2312:cgvc.20251217,
booktitle = {Computer Graphics and Visual Computing (CGVC)},
editor = {Sheng, Yun and Slingsby, Aidan},
title = {{A Natural Language Interface for the Visualization and Analysis of 3D Point Cloud Saliency Maps}},
author = {Ciprián-Sánchez, Jorge F. and Jobst, Adrian and Richter, Rico and Döllner, Jürgen},
year = {2025},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-293-6},
DOI = {10.2312/cgvc.20251217}
}