44-Issue 3

Permanent URI for this collection

https://diglib.eg.org/handle/10.2312/3607144

Browse

Browsing 44-Issue 3 by Subject "CCS Concepts: Human-centered computing → Information visualization"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Benchmarking Visual Language Models on Standardized Visualization Literacy Tests
    (The Eurographics Association and John Wiley & Sons Ltd., 2025) Pandey, Saugat; Ottley, Alvitta; Aigner, Wolfgang; Andrienko, Natalia; Wang, Bei
    The increasing integration of Visual Language Models (VLMs) into visualization systems demands a comprehensive understanding of their visual interpretation capabilities and constraints. While existing research has examined individual models, systematic comparisons of VLMs' visualization literacy remain unexplored. We bridge this gap through a rigorous, first-ofits- kind evaluation of four leading VLMs (GPT-4, Claude, Gemini, and Llama) using standardized assessments: the Visualization Literacy Assessment Test (VLAT) and Critical Thinking Assessment for Literacy in Visualizations (CALVI). Our methodology uniquely combines randomized trials with structured prompting techniques to control for order effects and response variability - a critical consideration overlooked in many VLM evaluations. Our analysis reveals that while specific models demonstrate competence in basic chart interpretation (Claude achieving 67.9% accuracy on VLAT), all models exhibit substantial difficulties in identifying misleading visualization elements (maximum 30.0% accuracy on CALVI). We uncover distinct performance patterns: strong capabilities in interpreting conventional charts like line charts (76-96% accuracy) and detecting hierarchical structures (80-100% accuracy), but consistent difficulties with data-dense visualizations involving multiple encodings (bubble charts: 18.6-61.4%) and anomaly detection (25-30% accuracy). Significantly, we observe distinct uncertainty management behavior across models, with Gemini displaying heightened caution (22.5% question omission) compared to others (7-8%). These findings provide crucial insights for the visualization community by establishing reliable VLM evaluation benchmarks, identifying areas where current models fall short, and highlighting the need for targeted improvements in VLM architectures for visualization tasks. To promote reproducibility, encourage further research, and facilitate benchmarking of future VLMs, our complete evaluation framework, including code, prompts, and analysis scripts, is available at https://github.com/washuvis/VisLit-VLM-Eval.
  • Thumbnail Image
    Item
    Player-Centric Shot Maps in Table Tennis
    (The Eurographics Association and John Wiley & Sons Ltd., 2025) Erades, Aymeric; Vuillemot, Romain; Aigner, Wolfgang; Andrienko, Natalia; Wang, Bei
    Shot maps are popular in many sports as they typically plot events and player positions in the way they are collected, using a pitch or a table as an absolute coordinate system. We introduce a variation of a table tennis shot map that shifts the point of view from the table to the player. This results in a new reference system to plot incoming balls relative to the player's position rather than on the table. This approach aligns with how table tennis tactical analysis is conducted, focusing on identifying empty spaces and weak spots around the players. We describe the motivation behind this work, built through close collaboration with two table tennis experts, and demonstrate how this approach aligns with the way they analyze games to reveal key tactical aspects. We also present the design rationale and the computer vision pipeline used to accurately collect data from broadcast videos. Our findings show that the technique enables capturing insights that were not visible with the absolute coordinate system, particularly in understanding regions that are reachable and those close to the pivot area of the player.