Visual Interpretation of Tagging: Advancing Understanding in Task-Oriented Dialogue Systems
No Thumbnail Available
Date
2024
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
In task-oriented dialogue systems, tagging tasks leverage Large Language Models (LLMs) to understand dialogue semantics. The specifics of how these models capture and utilize dialogue semantics for decision-making remain unclear. Unlike binary or multi-classification, tagging involves complex multi-to-multi relationships between features and predictions, complicating attribution analyses. To address these challenges, we introduce a novel interactive visualization system that enhances understanding of dialogue semantics through attribution analysis. Our system offers a multi-level and layer-wise visualization framework, revealing the evolution of attributions across layers and allowing users to interactively probe attributions. With a dual-view for streamlined comparisons, users can effectively compare different LLMs. We demonstrate our system's effectiveness with a common task-oriented dialogue task: slot filling. This tool aids NLP experts in understanding attributions, diagnosing models, and advancing dialogue understanding development by identifying potential sources of model hallucinations.
Description
CCS Concepts: Human-centered computing → Visual analytics
@inproceedings{10.2312:cgvc.20241236,
booktitle = {Computer Graphics and Visual Computing (CGVC)},
editor = {Hunter, David and Slingsby, Aidan},
title = {{Visual Interpretation of Tagging: Advancing Understanding in Task-Oriented Dialogue Systems}},
author = {Zhou, Yazhuo and Xing, Yiwen and Abdul-Rahman, Alfie and Borgo, Rita},
year = {2024},
publisher = {The Eurographics Association},
ISBN = {978-3-03868-249-3},
DOI = {10.2312/cgvc.20241236}
}