ReLVis: Visual Analytics for Situational Awareness During Reinforcement Learning Experimentation

Abstract

Reinforcement learning (RL) is a branch of machine learning where an agent learns to maximize reward through trial and error. RL is challenging and data/compute intensive leading practitioners to become overwhelmed and make poor modeling decisions. Our contribution is a Visual Analytics tool designed to help data scientists maintain situation awareness during RL experimentation. Our tool allows users to understand which hyper-parameter values lead to better or worse outcomes, what behaviors are associated with high and low reward, and how behaviors evolve throughout training. We evaluated our tool through three uses cases using state of the art deep RL models demonstrating how our tool leads to RL situation awareness.