% ---------------------------------------------------------------------
% EG author guidelines plus sample file for EG publication using LaTeX2e input
% D.Fellner, v2.04, Dec 14, 2023

%\hypersetup{
%    hidelinks
%}

\usepackage{tikz}
\usepackage{amsmath}
\usetikzlibrary{shapes.geometric, positioning}

\title[Real-Time Angular Color Shift Compensation for OSVP]{Real-Time Angular Color Shift Compensation \newline for On-Set Virtual Production}

% for anonymous conference submission please enter your SUBMISSION ID
%\author[poster1009]{poster1009}

\author[Beck et al.]
{\parbox{\textwidth}{\centering C. Beck \orcid{0000-0003-4125-3605}, T. Schattkowsky \orcid{0000-0001-8513-4937}, S. Albertz}
        \\
{\parbox{\textwidth}{\centering Hamm-Lippstadt University of Applied Sciences, Germany\\
       }}
}
% ------------------------------------------------------------------------

% if the Editors-in-Chief have given you the data, you may uncomment
% the following five lines and insert it here
%
% \volume{36}   % the volume in which the issue will be published;
% \issue{1}     % the issue number of the publication
% \pStartPage{1}      % set starting page


%-------------------------------------------------------------------------
\begin{document}

\maketitle
%-------------------------------------------------------------------------
\begin{abstract}
On-set virtual production (OSVP) uses LED volumes to display real-time rendered backgrounds driven by camera pose and viewing direction. The camera-visible region, known as the inner frustum, is rendered at higher fidelity but is particularly affected by angular color shift at oblique viewing angles. Current calibration frameworks  focus only on static angle-independent compensation of errors in color rendition. In our approach, we use a robogoniometric setup to measure the far-field colorimetric behavior of LED panels and derive a lightweight, angle-dependent color profile for the LED wall that can be directly used for real-time angular color shift correction. \\

\begin{CCSXML}
<ccs2012>
   <concept>
       <concept_id>10003120.10003145.10003151.10011771</concept_id>
       <concept_desc>Human-centered computing~Visualization toolkits</concept_desc>
       <concept_significance>500</concept_significance>
       </concept>
   <concept>
       <concept_id>10010147.10010371.10010387</concept_id>
       <concept_desc>Computing methodologies~Graphics systems and interfaces</concept_desc>
       <concept_significance>500</concept_significance>
       </concept>
 </ccs2012>
\end{CCSXML}

\ccsdesc[500]{Human-centered computing~Visualization toolkits}
\ccsdesc[500]{Computing methodologies~Graphics systems and interfaces}
\printccsdesc   
\end{abstract}  
%-------------------------------------------------------------------------
\section{Introduction}
In on-set virtual production (OSVP), LED volumes are used on film sets to display dynamically rendered real-time scene backgrounds. The wall content is typically generated from the camera pose and viewing angle using Unreal Engine. The camera is freely movable and captures only a portion of the LED wall \cite{ves2023virtualproduction}. This region, referred to as the inner frustum, is therefore rendered at increased fidelity. Angular color shift \cite{ng_investigation_2025} here is particularly evident at certain camera viewing angles relative to the LED wall. This paper proposes an approach for dynamically compensating the angular color shift in real time directly within the rendering engine using a simplified far-field simulation model derived from goniometric measurements. 

\section{Related Works}

The challenge of color rendition in OSVP is multifaceted. \cite{Longetal2024} outline the color management principles required to match the directors intent with the final camera capture, noting the difficulty of managing the narrow-band spectral nature of LEDs. Current open-source solutions like OpenVPCal \cite{payne_openvpcal_2022} successfully effectively align LED panels to specific cameras using test patches. However, these frameworks typically generate static Color Correction Matrices (CCMs) or Look-up-Tables(LUT) based on on-axis measurements, leaving dynamic angular errors unaddressed. Characterizing angular color shift is well-established in display metrology. \cite{boher2016optical} demonstrated methods using Fourier optics to analyze the full color volume (Lab) of displays across viewing angles. While highly accurate, these methods are data-intensive. For simulation, \cite{jacobs_rayfiles_2015} proposed using near-field goniophotometry to create spectral ray files. They noted that while near-field models are essential for optical design, extended sources can be approximated as point sources in far-field conditions. Our work leverages this far-field approximation to reduce computational complexity for real-time application, similar to how \cite{yu_real-time_2025} utilized pre-computed LUTs to drive multispectral lighting in real-time. 
%Whether the angle-dependent colour drift differs for each individual primary colour requires further investigation, but it can be assumed that, with the exception of pixel errors per pixel board, a panel subpixel board exhibits similar ageing effects.

In terms of color rendition \cite{gudemann_color_2022} and metameric failures \cite{legendre_practical_2016} in OSVP settings, using RGBW panels with cold and/or warm white pixels are used for ceilings or backgrounds.  Multi-primary or RGBW light sources also consist of higher pixel pitches, resulting in higher light output. As these panels are often used for lighting from above, they are not primarily used as an inner frustum. Further investigation of the angular color shift for RGBW panels is likely to be necessary.
Aging phenomena are not considered in detail in our approach, as we assume that the effects of aging affect the whole panel or sub-pixel board equally. As manufacturers of receiver cards provide calibration data along new panels, the aging effects could be investigated and compensated by repeating the measurements for the individual pixels and correction of R,G,B primaries. 

\section{Methodology}
Angular color drift of a $50 \times 50$\,cm LED panel was characterized in the far field ($d = 8$\,m) using a robogoniometer, a spectroradiometer and ARRI ALEXA 35 camera recording in RAW. 
Measurements conditions were adopted from \cite{international_committee_for_display_metrology_information_2025}, incorporating a 20-minute warm-up, 1500\,cd/m$^2$ full white luminance, DCI-P3 gamut with $D_{65}$ white point. 
Spectral power distributions (SPDs) were captured at 1\,nm intervals as the panel was rotated in steps of 15$^\circ$ azimuth and 45$^\circ$ elevation. This initial characterization isolates full luminance; brightness-dependent color drift remains a subject for future investigation, assuming that the angular color show has it's maximum at full brightness.%Typical OSVP distances between camera and LED wall are approximately ($>2$\,m) to avoid moiré.
%the empirical SPDs were processed into an HLSL shader and applied to the inner frustum.
\input{img_coa}

To visualize and correct for color drift, we first transformed the spectral power distributions (SPDs) into a polar plot, mapping the correlated color temperature (CCT) for each data point (see Figure \ref{fig:polarCoA}). This angular color shift was then used to dynamically compensate for display inaccuracies in real time.
The angular color shift is addressed by a shader that is applied to the nDisplay's inner frustum render node. This process uses OpenColorIO for the purpose of achieving precise color management. 
Real-time camera tracking data – specifically positional coordinates and rotational quaternions relative to the the display mesh's normal are dynamically computed by the viewing angel $\theta$.
%We apply a custom HLSL shader to the nDisplay inner frustum, utilizing OpenColorIO for precise color management. Real-time physical camera tracking data—specifically positional coordinates and rotational quaternions—dynamically computes the viewing angle $\theta$ relative to the display normal. 
This view-dependent compensation is executed per-pixel via the transformation:
\begin{equation}
\mathbf{RGB}_{out} = (\mathbf{I} + \theta \mathbf{K}) \mathbf{RGB}_{in}
\label{eq:color_drift_ultracompact}
\end{equation}
where $\mathbf{I}$ is the identity matrix and $\mathbf{K}$ is the $3 \times 3$ coefficient matrix scaled by $\theta$.
\input{tikz_method}
While the focus of the research was on collecting data regarding angular color shift, it is essential that the compensation pipeline undergoes comparative color rendering evaluation in a production environment with different implementations, including but not limited to look-up tables or high-level shading languages as an outlook.

\section*{Acknowledgments}
We like to thank Frank Tappe and Jörg Meyer supporting the measurements. Grant Number: EFRE-20801830 

\bibliographystyle{ACM-Reference-Format}
\bibliography{EUROGRAPHICS_26_FEB.bib}

\end{document}

