Search Results

Now showing 1 - 3 of 3
  • Item
    Visual Interpretation of Tagging: Advancing Understanding in Task-Oriented Dialogue Systems
    (The Eurographics Association, 2024) Zhou, Yazhuo; Xing, Yiwen; Abdul-Rahman, Alfie; Borgo, Rita; Hunter, David; Slingsby, Aidan
    In task-oriented dialogue systems, tagging tasks leverage Large Language Models (LLMs) to understand dialogue semantics. The specifics of how these models capture and utilize dialogue semantics for decision-making remain unclear. Unlike binary or multi-classification, tagging involves complex multi-to-multi relationships between features and predictions, complicating attribution analyses. To address these challenges, we introduce a novel interactive visualization system that enhances understanding of dialogue semantics through attribution analysis. Our system offers a multi-level and layer-wise visualization framework, revealing the evolution of attributions across layers and allowing users to interactively probe attributions. With a dual-view for streamlined comparisons, users can effectively compare different LLMs. We demonstrate our system's effectiveness with a common task-oriented dialogue task: slot filling. This tool aids NLP experts in understanding attributions, diagnosing models, and advancing dialogue understanding development by identifying potential sources of model hallucinations.
  • Item
    Reflections on the Evolution of the BookTracker Visualization Platform
    (The Eurographics Association, 2024) Xing, Yiwen; Dondi, Cristina; Borgo, Rita; Abdul-Rahman, Alfie; Hunter, David; Slingsby, Aidan
    Understanding the trade data of historical books is crucial for researchers investigating the distribution and provenance of Incunabula (books printed between 1450 and 1500). We incrementally developed BookTracker, a platform featuring multiple visualization and visual analytics applications to support these research efforts. This platform leverages data from the Material Evidence in Incunabula (MEI) database, which meticulously records detailed information on the provenance, ownership, and use of 15th-century printed books. BookTracker began with a focus on providing visualization and visual analytical solutions to effectively present each book provenance's chronological and geographical information. Through three years of collaborative work with domain experts, we continually explored the Material Evidence in Incunabula (MEI) data and discovered more possibilities for visualization to represent this rich information. Gradually, a suite of specialized visualization tools for specific analytical purposes was developed, including DanteSearchVis, DanteExploreVis, KURF2022, KURF2023, and OwnershipTracker. These tools now comprise the BookTracker platform, which has evolved to explore various features and aspects of the data. This paper details the evolution of BookTracker's design and development alongside domain experts, highlighting the reflections and lessons learned from its application in various research projects. We discuss this long-term collaborative visualization project, hoping to offer our experience as a case study for similar research in the future.
  • Item
    Visualizing Complex Data Decisions: Design Study for Ethical Factors in AI Clinical Decision Support Systems
    (The Eurographics Association, 2024) Surodina, Svitlana; Volkova, Daria; Abdul-Rahman, Alfie; Borgo, Rita; Hunter, David; Slingsby, Aidan
    Despite the proliferation of Artificial Intelligence (AI) technologies, their uptake in clinical settings has been lacking progress due to complexities of sociotechnical factors and intricacies of decision-making. Fairness and bias of predictive models, ethics and quality of training data, and corresponding compliance requirements become especially pressing while remaining fuzzy and implicit for various stakeholders who make the decisions. We present learnings and future directions from a design study with domain experts and propose a novel approach to encoding and collaborative reasoning on complex requirements for AI-Empowered Clinical Decision Support System (AI-CDSS) design based on Knowledge Graph (KG) representation. The insights will be useful to the community of visualization researchers who work on ethical AI-CDSS design and conduct design studies with clinical partners.