Joos, LucasKeim, Daniel A.Fischer, Maximilian T.Schulz, Hans-JörgVillanova, Anna2025-05-262025-05-262025978-3-03868-283-72664-4487https://doi.org/10.2312/eurova.20251105https://diglib.eg.org/handle/10.2312/eurova20251105Systematic literature reviews (SLRs) are essential but labor-intensive due to high publication volumes and inefficient keywordbased filtering. To streamline this process, we evaluate Large Language Models (LLMs) for enhancing efficiency and accuracy in corpus filtration while minimizing manual effort. Our open-source tool LLMSurver presents a visual interface to utilize LLMs for literature filtration, evaluate the results, and refine queries in an interactive way. We assess the real-world performance of our approach in filtering over 8.3k articles during a recent survey construction, comparing results with human efforts. The findings show that recent LLM models can reduce filtering time from weeks to minutes. A consensus scheme ensures recall rates >98.8%, surpassing typical human error thresholds and improving selection accuracy. This work advances literature review methodologies and highlights the potential of responsible human-AI collaboration in academic research.Attribution 4.0 International LicenseCCS Concepts: Human-centered computing → Interactive systems and tools; Computing methodologies → Artificial intelligence; Applied computing → PublishingHuman centered computing → Interactive systems and toolsComputing methodologies → Artificial intelligenceApplied computing → Publishing10.2312/eurova.202511056 pages