Sawada, ShokoToyoda, MasashiMadeiras Pereira, João and Raidou, Renata Georgia2019-06-022019-06-022019978-3-03868-088-8https://doi.org/10.2312/eurp.20191140https://diglib.eg.org:443/handle/10.2312/eurp20191140It is essential to assess the trustworthiness of the machine learning models when deploying them to real-world applications, such as healthcare and risk management, in which domain experts need to make critical decisions. We propose a visual analysis method for supporting domain experts to understand and improve a given machine learning model based on a model-agnostic interpretable explanation technique. Our visualization method provides a heat map matrix as an overview of the model explanation and helps efficient feature engineering and data cleaning. We demonstrate our visualization method on a text classification task.Humancentered computingHeat mapsVisual analyticsComputing methodologiesMachine learning10.2312/eurp.2019114037-39