Zinat, Kazi TasnimYang, JinhuaGandhi, ArjunMitra, NisthaLiu, ZhichengBujack, RoxanaArchambault, DanielSchreck, Tobias2023-06-102023-06-1020231467-8659https://doi.org/10.1111/cgf.14821https://diglib.eg.org:443/handle/10.1111/cgf14821Real-world event sequences are often complex and heterogeneous, making it difficult to create meaningful visualizations using simple data aggregation and visual encoding techniques. Consequently, visualization researchers have developed numerous visual summarization techniques to generate concise overviews of sequential data. These techniques vary widely in terms of summary structures and contents, and currently there is a knowledge gap in understanding the effectiveness of these techniques. In this work, we present the design and results of an insight-based crowdsourcing experiment evaluating three existing visual summarization techniques: CoreFlow, SentenTree, and Sequence Synopsis. We compare the visual summaries generated by these techniques across three tasks, on six datasets, at six levels of granularity. We analyze the effects of these variables on summary quality as rated by participants and completion time of the experiment tasks. Our analysis shows that Sequence Synopsis produces the highest-quality visual summaries for all three tasks, but understanding Sequence Synopsis results also takes the longest time. We also find that the participants evaluate visual summary quality based on two aspects: content and interpretability. We discuss the implications of our findings on developing and evaluating new visual summarization techniques.CCS Concepts: Human-centered computing -> Visualization design and evaluation methods; Empirical studies in visualizationHuman centered computingVisualization design and evaluation methodsEmpirical studies in visualization10.1111/cgf.14821173-18513 pages