Machado, AlisterBehrisch, MichaelTelea, AlexandruAigner, WolfgangAndrienko, NataliaWang, Bei2025-05-262025-05-2620251467-8659https://doi.org/10.1111/cgf.70101https://diglib.eg.org/handle/10.1111/cgf70101High-dimensional data analysis often uses dimensionality reduction (DR, also called projection) to map data patterns to human-digestible visual patterns in a 2D scatterplot. Yet, DR methods may fail to show true data patterns and/or create visual patterns that do not represent any data patterns. Projection Quality Metrics (PQMs) are used as objective measures to gauge the above process: the higher a projection's scores in PQMs, the more it is deemed faithful to the data it represents. We show that, while PQMs can be used as exclusion criteria - low values usually mean poor projections - the converse does not always hold. For this, we develop a technique to automatically generate projections that score similar or even higher PQM values than projections created by well-known techniques, but show different, often confusing, visual patterns. Our results show that accepted PQMs cannot be used as an exclusive way to tell whether a projection yields accurate and interpretable visual patterns - in this sense, PQMs play a role akin to that of summary statistics in exploratory data analysis. We also show that not all studied metrics can be fooled equally well, suggesting a ranking of metrics in their ability to reliably capture quality.Attribution 4.0 International LicenseCCS Concepts: Mathematics of computing → Dimensionality reduction; Computing methodologies → Machine learning; Humancentered computing → Information visualizationMathematics of computing → Dimensionality reductionComputing methodologies → Machine learningHumancentered computing → Information visualization10.1111/cgf.7010112 pages