
%-------------------------------------------------------------------------
\begin{figure}[tbh]
\centering
{\includegraphics[width=0.9\linewidth]{img/equirectangular-projection.jpg}}
 \caption{
\label{fig:equirectangular-projection} \textbf{Equirectangular projection (ERP).} Longitude (azimuth) maps to the horizontal axis and latitude (elevation) to the vertical, with horizontal wrap-around and the poles spanning the top and bottom edges.
}
\end{figure} 
%-------------------------------------------------------------------------

%---------------------------------------------------------------------------------
\section{Background} 
\label{sec:background}

Before reviewing state-of-the-art methods, we first establish a common background. We begin by introducing the most common representations adopted for omnidirectional image capture and processing~(\autoref{sec:background:360rep}), highlighting the dominance of the equirectangular format. Next, we motivate the reliance of many analysis and synthesis techniques on gravity-aligned images and summarize the methods used either to capture such data or to correct orientation in a preprocessing step~(\autoref{sec:background:gravity-alignment}). We then address the challenges of extracting geometric, structural, and visual information from incomplete and noisy $360^\circ$ panoramas and review the role of architectural priors, from early geometric reasoning to their integration in modern deep learning approaches~(\autoref{sec:background:priors}). Given the centrality of data for learning-based methods, we also provide an overview of the major publicly available datasets for training and evaluating indoor reconstruction and synthesis from $360^\circ$ imagery~(\autoref{sec:background:data}). 

%---------------------------------------------------------------------
\subsection{360$^\circ$ image capture and processing representation}
\label{sec:background:360rep}

A wide range of technologies can capture 3D indoor information, from mobile laser scanners to active depth sensors~\cite{Pintore:2020:SA3}. Image-based solutions are particularly important, as cameras offer a practical, affordable, and widely available solution, while also providing essential visual information for tasks such as navigation, localization, and as-built reconstruction~\cite{Pintore:2018:R3E}. However, conventional limited field-of-view cameras fall short for full-room reconstruction, requiring multi-view acquisition that increases capture effort~\cite{Godard:2017:UMD} and is often hindered indoors by clutter, narrow spaces, occlusions, and textureless or reflective surfaces, which complicate feature detection and surface reconstruction~\cite{Pintore:2020:SA3}. For these reasons, $360^\circ$ capture, providing the quickest and complete single-shot coverage, has now become the de facto standard solution~\cite {Firman:2016:RDP,DaSilveira:2022:3SG}. 
While panoramic imagery can still be produced by stitching conventional photo sequences~\cite{Cabral:2014:PPC}, spherical cameras have nowadays emerged as the standard solution thanks to their ease of use and reduced costs~\cite{Jokela:2019:HPU}. With a single shutter press, commodity devices internally process multiple fisheye views to deliver a seamless full-view image, effectively capturing the entire scene in one instant. Their design guarantees an (approximately) single center of projection and consistent horizontal resolution, which are difficult to obtain through casual stitching~\cite{Im:2016:ADS,Huang:2017:6VV}. From a geometric standpoint, a spherical camera can be modeled as a unit sphere without intrinsic parameters, with image formation determined solely by extrinsic ones~\cite{DaSilveira:2022:3SG}.

Because the sphere cannot be mapped onto a plane without distortion, representing panoramic captures as rectangular images necessarily requires selecting and applying projection transformations. Although some devices allow access to the original unstitched fisheye views, providing maximum achievable resolution, the most common standard is only to produce a $360^\circ \times 180^\circ$ \emph{full panoramic image} using an \emph{equirectangular projection (ERP)}~\cite{Eder:2020:TIM}. This projection maps longitude (azimuth) to the horizontal axis and latitude (elevation) to the vertical axis. As a result, meridians are represented as vertical straight lines with uniform spacing for equal angular intervals, while parallels appear as horizontal straight lines, also evenly spaced (\autoref{fig:equirectangular-projection}).  Such a mapping preserves neither areas nor angles: regions near the poles are increasingly stretched in the horizontal direction, resulting in shape distortions. Alternative projections are also employed to reduce such distortions. A notable example is the cube-map projection, which projects the sphere onto the six faces of a cube, each covering a $90^\circ \times 90^\circ$ field of view. This format is frequently used in viewing applications, including WebXR platforms and streaming services such as YouTube 360~\cite{Champel:2017:KFH}. Other representations, such as tangent image projections~\cite{Rey:2022:3HR,Eder:2020:TIM}, have also gained traction, as reviewed in recent surveys~\cite{DaSilveira:2022:3SG}.

Despite some limitations, ERPs are device-independent, widely supported by cameras and HMDs, and preserve full horizontal and near-complete vertical continuity. In gravity-aligned indoor capture, resolution loss and pole distortions have limited impact, as floors and ceilings contain relatively little information (\autoref{sec:background:gravity-alignment}). Thus, equirectangular images dominate dataset sharing (\autoref{sec:background:data}), processing (\autoref{sec:geostruct}), and exploration (\autoref{sec:interactive}).

%-------------------------------------------------------------------------
\begin{figure}[tb]
\centering
{\includegraphics[width=0.9\linewidth]{img/indoor-occlusion-and-reconstruction.jpg}}
 \caption{
\label{fig:background:indoor-occlusion-and-reconstriction} \textbf{Occlusion of the architectural boundary.} Areas occluded by furniture are in yellow, while those occluded by other walls are in red. Image adapted from Pintore et al.~\cite{Pintore:2021:D3R}.
}
\end{figure} 
%-------------------------------------------------------------------------
%-------------------------------------------------------------------------
\begin{figure*}[tbh]
\centering
{\includegraphics[width=0.9\linewidth]{img/indoor-architectural-priors-revised.jpg}}
 \caption{
\label{fig:background:indoor-architectural-priors} \textbf{Common indoor architectural priors.} From left to right: \emph{floor–wall boundaries}~\cite{Delage:2006:ADB}, \emph{cuboids}~\cite{Hedau:2009:RSL}, the \emph{Indoor World Model}~\cite{Lee:2009:GRF}, the \emph{Manhattan world}~\cite{Coughlan:1999:MWC}, the \emph{Atlanta world}~\cite{Schindler:2004:AWE}, the \emph{Vertical Walls}~\cite{Pintore:2018:3FP} and \emph{piece-wise planarity}~\cite{Furukawa:2009:RBI}.
}
\end{figure*} 
%-------------------------------------------------------------------------
%---------------------------------------------------------------------
\subsection{Gravity alignment}
\label{sec:background:gravity-alignment}

A common assumption in both reconstruction and exploration methods is that images are captured or synthesized with the camera’s up-vector aligned to the world’s vertical axis. In the case of equirectangular projection, this alignment ensures that horizontal structures in the scene are aligned with image rows. Such a condition provides several advantages. First, the regions of highest distortion in equirectangular panoramas are confined to the ceiling and floor, areas that typically contain less relevant detail. Second, vertically-aligned image processing can directly exploit structural features that are also aligned with gravity~\cite{Davidson:2020:3CA}. Finally, for rendering, this alignment maps head rotations to horizontal and vertical pixel shifts and enables stereopsis via horizontal disparities.

Gravity alignment is thus the standard setup for indoor capture, and nearly all publicly available 3D indoor datasets (\autoref{sec:background:data}) used for training and evaluation exhibit minimal orientation deviations~\cite{Pintore:2021:SDD}. In practice, alignment at capture time is easily achieved either by mounting the camera on a tripod placed on a level surface~\cite{Davidson:2020:3CA} or, in handheld scenarios, by relying on built-in Inertial Measurement Units (IMUs). Even when this condition is not satisfied, many preprocessing techniques for image-based gravity rectification exist (e.g.,~\cite{Xian:2019:UGC,Jung:2019:D3U,Davidson:2020:3CA}), thereby facilitating the application of gravity-oriented methods.

%---------------------------------------------------------------------
\subsection{Architectural and data-driven priors}
\label{sec:background:priors}

A single $360^\circ$ panorama captures the full appearance of a scene around a viewpoint, from which geometric, structural, and visual representations of the environment must be inferred. However, panoramas typically provide incomplete and noisy observations of the true underlying geometry, making reconstruction ill-posed. Textureless regions (e.g., bare walls), uneven angular sampling, and distortions in equirectangular images (\autoref{sec:background:360rep}) hinder feature detection and geometric reasoning. Transparent and reflective surfaces such as glass and mirrors introduce further complexity. Depth variation indoors is also more extreme than what is encountered in typical outdoor use cases, as images are taken in confined, narrow spaces, while depth ranges from nearby clutter to far-away walls. Most critically, furniture and objects often occlude large portions of the architectural layout, while concave room shapes cause severe self-occlusions, making much of the room boundary invisible to the viewer (\autoref{fig:background:indoor-occlusion-and-reconstriction}). Consequently, depth estimation and structural recovery require not only leveraging the wide contextual cues that panoramas provide but also incorporating strong, domain-specific priors stemming from the construction of architectural spaces~\cite{Pintore:2020:SA3}.

%-------------------------------------------------------------------------
\begin{table}[htb]
\centering
\small
\rowcolors{2}{tbloddrowcolor}{tblevnrowcolor}
\begin{tabular}{|cccp{0.35\linewidth}|}
\hline
\rowcolor{tblheadcolor} 
\textbf{Acronym} &                \textbf{Name} & \textbf{Reference} &                                        \textbf{Assumption} \\
\hline
     \textbf{FW} &          Floor–Wall & \cite{Delage:2006:ADB}    & Single flat floor with unconnected vertical walls; ceiling ignored \\
     \textbf{CB} &              Cuboid & \cite{Hedau:2009:RSL}     & Single cuboid-shaped room with six orthogonal faces \\
     \textbf{IW} &  Indoor World & \cite{Lee:2009:GRF}       & Single horizontal floor/ceiling, vertical walls meeting at 90$^\circ$  \\
     \textbf{MW} &     Manhattan World &  \cite{Coughlan:1999:MWC} & Horizontal floors/ceilings, vertical walls meeting at 90$^\circ$ \\
     \textbf{AW} &       Atlanta World & \cite{Schindler:2004:AWE} & Horizontal floors/ceilings, vertical walls not restricted to meeting at 90$^\circ$ \\
     \textbf{VW} &      Vertical Walls & \cite{Pintore:2018:3FP}  & Vertical walls, floors and ceilings may slope \\ 
     \textbf{PW} & Piecewise Planarity & \cite{Furukawa:2009:RBI}  & General polyhedral rooms with piecewise planar walls, floors, and ceilings \\ 
\hline
\end{tabular}
\caption{\textbf{Common geometric priors for indoor reconstruction.} See \autoref{fig:background:indoor-architectural-priors} for an illustration.}
\label{tab:background:architectural-priors}
\end{table}
%-------------------------------------------------------------------------

Beyond generic surface reconstruction assumptions such as smoothness, symmetry, or repetition~\cite{Berger:2017:SSR}, the indoor reconstruction literature has introduced explicit architectural priors. The most common ones are summarized in \autoref{tab:background:architectural-priors} and illustrated in \autoref{fig:background:indoor-architectural-priors}. As we will see in the following sections, while early approaches relied on restrictive models (e.g., \emph{floor–wall boundaries}~\cite{Delage:2006:ADB} or simple \emph{cuboids}~\cite{Hedau:2009:RSL}), modern solutions mostly adopt the Manhattan World prior~\cite{Coughlan:1999:MWC}, which aligns well with standard construction rules in residential and office buildings and, through orthogonality constraints, strongly reduces reconstruction ambiguity, offering ample spaces for method customization~\cite{Zou:2021:MRL}. Atlanta world solutions are also gaining strong traction, since they expand the applicability of the methods by relaxing right-angle restrictions, while still preserving many optimization possibilities~\cite{Pintore:2021:SDD}.

Classical reconstruction exploited such priors via geometric reasoning, extracting image features (corners, edges, uniform regions) and interpreting them as cues under the chosen prior~\cite{Pintore:2020:SA3}. The success of these methods, however, is highly sensitive to the number and quality of extracted features, and these methods often struggle in real-world scenarios due to clutter, non-cooperative materials, and deviations from idealized structural assumptions.  

As a result, research has shifted toward data-driven approaches~\cite{Zou:2019:3DM}, particularly deep learning~\cite{Zou:2021:MRL}, which can leverage large amounts of examples to learn implicit geometric and semantic cues directly from data. 
Deep models provide more robust solutions than purely algorithmic methods and allow relaxing many strict reconstruction assumptions. As we will see in the following sections, architectural priors, however, remain essential, often enabling lower-dimensional reasoning (e.g., extracting floor plans under vertical wall assumptions) or reducing network complexity (e.g., flattening features along the gravity axis).


%--------------------------------------------------
\begin{table*}[!htb]
\centering
\small
%
\rowcolors{2}{tbloddrowcolor}{tblevnrowcolor}
\begin{tabular}{|l|c|c|c|c|c|c|c|}
\hline
\rowcolor{tblheadcolor}
\textbf{Dataset} & \textbf{Method} & \textbf{\# Panos} & \textbf{Pano res.} & \textbf{Depth} & \textbf{Layout} & \textbf{Layout type} & \textbf{Other} \\
\hline
Matterport3D~\cite{Data:Matterport3D:2017} & Captured, Stitched & 10{,}800 & 2048$\times$4096 & (Yes) & — & (All) & S, P \\
MatterportLayout~\cite{Data:MatterportLayout:2019} & Captured, Stitched & 2{,}295 & 512$\times$1024 & — & Yes & Manhattan & (S, P) \\
Stanford2D-3D-S~\cite{Data:2D-3D-S:2017} & Captured, Stitched & 552 & 2048$\times$4096 & Yes & Yes & Cuboid & N, S, P \\
Gibson 3D scan database~\cite{Data:Gibson:2018} & Captured, Stiched & 45,196 & 1024$\times$2048 & (Yes) & (Yes) & (Atlanta) & P \\
3DSceneGraph~\cite{Data:3DSceneGraph:2019} & Captured, Rendered & 45,196 & 1024$\times$2048  & (Yes) & (Yes) & (Atlanta) & S, P \\
360MonoDepth/M3D~\cite{Data:360MonoDepthM3D:2022} & Captured, Stitched 
& 9{,}684 & 1024$\times$2048 & Yes & — & (All) & P \\
360MonoDepth/Replica~\cite{Data:360MonoDepthReplica:2022} & 3D Acquisition, Rendered
& 130 & 1024$\times$2048, 2048$\times$4096 & Yes & — & (All) & P \\
PanoContext~\cite{Data:PanoContext:2014} & Captured, Native 360 & 700 & 512$\times$1024 & — & Yes & Cuboid & I \\
Zillow Indoor (ZInD)~\cite{Data:ZInD:2021} & Captured, Native 360 & 71{,}474 & 1024$\times$2048 & — & Yes & Atlanta & I, O, P \\
ZInd-Tell~\cite{Data:ZInDTell:2024} & Captured, Native 360 & 71{,}474 & 1024$\times$2048 & — & Yes & Atlanta & I, O, P, L \\
Structured3D~\cite{Data:Structured3D:2020} & 3D Modeling, Rendered & 21{,}000 & 512$\times$1024 & Yes & Yes & Atlanta & N, A, I, S, P \\
PNVS~\cite{Data:PNVS:2021} & 3D Modeling, Rendered & 34{,}811 & 512$\times$1024 & Yes & Yes & Atlanta & P \\
SKI360~\cite{Data:SKI360:2020} & 3D Modeling, Rendered & 3,550 & 512$\times$1024 & Yes & Yes & Atlanta & S \\
% 360D~\cite{Zioulis:2018:ODD} & Mixed & Rendered (Matterport3D~\cite{Data:Matterport3D:2017} + 2D-3D-S~\cite{Data:2D-3D-S:2017} ) & 35{,}977 & 256$\times$512 & Yes & — & (All) & N \\
Pano3D~\cite{Data:Pano3D:2021} & Mixed, Rendered & 21{,}203 & 256$\times$512, 512$\times$1024 & Yes & — & (All) & P, N \\
%SUN360~\cite{Xiao:2012:RSV} & Real & Native & 67{,}583 & 9104$\times$4552 & — & Yes & Manhattan &  \\
FutureHouse~\cite{Data:FutureHouse:2022} & 3D Modeling, Rendered & 28{,}579 & 512$\times$1024 & Yes & (Yes) & (All) & N,M \\ 
PanoSCU~\cite{Data:PanoSCU:2025} & 3D Modeling, Rendered & 5{,}300 & 224$\times$928 & — & — & (All) & S,I,C \\ 
\hline
\end{tabular}
\caption{\textbf{Overview of most common large-scale publicly available 3D indoor datasets using panoramic images.} 
We list only datasets with substantial scale and impact for panoramic indoor scene analysis and synthesis. 
For each dataset, we report data type, source (real vs. synthetic), panorama generation method,  number of panoramic images and resolution in equirectangular format, supported layout types, and available additional information 
(I = object instances, N = normals, S = semantic segmentation, O = openings/doors/windows position, P = 3D poses, A = albedo, M = material and light; L = natural language description, C = change tracking.}
\label{tab:background:data}
\end{table*}
%----------------------------------------------------

%-------------------------------------------------------------
\subsection{Open research data}
\label{sec:background:data}

Publicly available datasets play a crucial role in advancing research on indoor reconstruction and exploration, as they not only provide standardized benchmarks for evaluating and comparing the results of different techniques but, with the current evolution towards learning methods, are also essential for providing learning material. Prior surveys on indoor reconstruction~\cite{Pintore:2020:SA3,Patil:2024:ADA} have covered generic datasets with different contents and coming from different sources (e.g., CAD models, point clouds). In this work, we restrict our analysis to those used for training and testing methods tailored for 360$^\circ$ data. These datasets, in addition to including panoramic images as equirectangular images, may also include a variety of other information, such as registered depth maps, layouts, 3D models, and semantic annotations, thereby supporting supervised training and testing for different tasks. Importantly, they also reflect different capture modalities, ranging from real 360$^\circ$ camera acquisitions to synthetic renderings of virtual environments, with the latter providing the more precise ground-truth data. The most widely used datasets in this domain are listed in~\autoref{tab:background:data}. 
We do not include in the table standalone smaller-scale datasets (<500 images) or data used for only a few specific publications (e.g.,~\cite{Pintore:2020:AI3,Data:I3D:2022}). Below, we summarize each dataset's key characteristics and intended usage.

\paragraph*{Matterport3D}~\cite{Data:Matterport3D:2017} This large-scale RGB-D dataset contains 10,800 panoramas from 90 properties, reconstructed from 194,400 RGB-D images captured with a Matterport Pro Camera. It provides raw data and annotations, including camera poses, textured meshes, floor plans, region labels, and object instances. Panoramas are sampled at human height, but spherical coverage is partial, with missing top and bottom regions often inpainted~\cite{Chang:2017:M3D}. Since original panoramic poses relative to the mesh were absent, lower resolution color and depth map pairs are commonly derived from textured meshes~\cite{Zioulis:2018:ODD}, a limitation later resolved in \emph{360MonoDepth/M3D}~\cite{Data:360MonoDepthM3D:2022,Rey:2022:3HR}. The dataset supports both single-image tasks (e.g., surface normal estimation, semantic labeling) and sparse multiview tasks (e.g., keypoint matching, overlap prediction), and has spawned multiple derived datasets.

\paragraph*{MatterportLayout}~\cite{Data:MatterportLayout:2019,Zou:2021:MRL} This dataset extends the \emph{Matterport3D} dataset with general Manhattan layout annotations for training and evaluation. It comprises 2,295 panoramas selected from the original \emph{Matterport3D} dataset and split into training, validation, and testing sets to facilitate direct comparisons of learning models. All the annotated layouts are aligned to a camera height of 1.6m.

\paragraph*{Stanford2D-3D-S}~\cite{Data:2D-3D-S:2017,Armeni:2016:3SP} This dataset provides large-scale indoor scans of public spaces such as offices, educational areas, lobbies, and hallways, captured with the same system as \emph{Matterport3D}. It includes 552 equirectangular images with depths, surface normals, semantic annotations, and camera poses, as well as cuboid layouts and annotated registered meshes and point clouds.

\paragraph*{GibsonV2 3D scan database}~\cite{Data:Gibson:2018,Xia:2018:GER} This dataset is similar in nature to \emph{Matterport3D} and \emph{Stanford2D-3D-S} but contains a greater variety of interiors (572 models, 1440 floors across households, offices, hotels, museums, hospitals, etc.). It provides reconstructed 3D meshes registered with panoramas. Several splits are provided, the largest containing 45,196 panoramas. Depth maps are not included but can be generated by ray-casting the 3D models. Originally intended for virtual simulation (e.g., robotics), it is also widely used for reconstruction and analysis tasks.

\paragraph*{3DSceneGraph}~\cite{Data:3DSceneGraph:2019,Armeni:2019:3SG}. This dataset augments the \emph{GibsonV2} scene collection with additional semantic information in the form of a 3D scene graph composed of four layers: building, room, object, and camera, further extending \emph{GibsonV2} towards semantic analysis tasks.

\paragraph*{360MonoDepth/M3D}~\cite{Data:360MonoDepthM3D:2022,Rey:2022:3HR} 
This \emph{Matterport3D} extension supports 360$^\circ$ monocular depth estimation with 9,684 panoramas (1024$\times$2048) from 90 buildings, subdivided into training, test, and validation sets. 360$^\circ$ structure-from-motion~\cite{Moulon:2016:OOM} was employed to align captures with scene meshes, yielding pixel-accurate ground-truth depths. Camera poses that co-register images taken within the same building are also provided, enabling tasks such as navigation, view synthesis, and VR.

\paragraph*{360MonoDepth/Replica}~\cite{Data:360MonoDepthReplica:2022,Rey:2022:3HR} This synthetic companion to \emph{360MonoDepth/M3D} adds 130 photorealistic equirectangular renders from 13 \emph{Replica}~\cite{Data:Replica:2019,Straub:2019:TRD} rooms, with co-registered ground-truth depths at 1024$\times$2048 and 2048$\times$4096. Images and co-registered ground truth depth were produced by the \emph{Replica360} renderer~\cite{Attal:2020:MR6}
The dataset's main purpose is to test generalization and evaluate depth estimation at the currently highest open-data resolutions.

\paragraph*{PanoContext}~\cite{Data:PanoContext:2014,Zhang:PAN:2014} The dataset contains 700 full-view panoramas for home environments from the \emph{SUN360} database~\cite{Xiao:2012:RSV} (no longer available for licensing reasons), including 418 bedrooms and 282 living rooms. Cuboid layouts are provided, as well as cuboid representations of interior objects. 

\paragraph*{Zillow Indoor (ZInD)}~\cite{Data:ZInD:2021,Cruz:2021:ZID} The dataset includes 67,448 panoramas from 1,575 unfurnished residential homes, with primary panoramas annotated for room layouts, openings, and labels, and secondary panoramas providing denser spatial coverage via semi-automatic localization. Primary views, chosen for full-room visibility and adjacent room co-visibility enable both single-image analysis and exploration in sparsely sampled environments.

\paragraph*{ZinD-Tell}~\cite{Data:ZInDTell:2024,Deb:2024:ZTT} This extension of the ZinD database augments the original data with floor plans and natural language descriptions of the indoor spaces. The objective of this multimodal representation is to establish semantic connections among natural language descriptions, panoramic images, rooms, and entire floors. Generative tasks are the main target. 

\paragraph*{Structured3D}~\cite{Data:Structured3D:2020,Zheng:2020:S3DE} This large-scale synthetic dataset contains 3.5K professionally designed 3D house models in the Atlanta Layout style. It provides over 21,000 panoramas with registered depth, normals, semantic segmentations, door/window annotations, and co-registered poses, for both empty and furnished rooms. The rich annotation sets make it a common choice for training and evaluating 360$^\circ$ solutions on a wide variety of tasks. 

\paragraph*{PNVS}~\cite{Data:PNVS:2021,Xu:2021:LGN} This photo-realistic dataset, built on \emph{Structured3D}, targets view synthesis with panoramas and displaced views rendered from translated cameras. It includes two subsets: small displacements (0.2–0.3 m, head movements) and large displacements (1–2 m, room-scale motion), with 13,080/1,791 and 17,661/2,279 train/validation images, respectively.

\paragraph*{Shanghaitech-Kujiale Indoor 360$^\circ$ (SKI360)}~\cite{Data:SKI360:2020,Jin:2020:GSB} This \emph{Structured3D}-based dataset includes 1,775 rooms captured both furnished and unfurnished. The 3,550 panoramas with color and depth are associated with architectural layouts represented by 3D corners, planes, and plane-plane intersection lines. Supported tasks include depth estimation, separation of furniture from architectural structure, empty-room synthesis, and layout recovery.

\paragraph*{Pano3D}~\cite{Data:Pano3D:2021,Albanis:2021:PHB} 
This synthetic dataset targets depth and normal estimation and cross-dataset evaluation. It re-renders 21,203 scenes from \emph{Matterport3D} point clouds~\cite {Data:Matterport3D:2017} and \emph{GibsonV2} meshes~\cite{Data:Gibson:2018,Xia:2018:GER} into 360$^\circ$ panoramas in equirectangular format containing color, depth, and normal maps. Splits are designed to test robustness under distribution shifts: \emph{Medium}, \emph{Tiny}, and \emph{Full} vary depth distribution and image count within residential scenes, \emph{Fullplus} adds non-residential contexts, and \emph{Filmic} variants of the splits simulate covariate color shifts.

\paragraph*{FutureHouse}~\cite{Data:FutureHouse:2022,Li:2022:PPI} 
This synthetic dataset is designed for inverse rendering applications. It contains 28,579 equirectangular views from 1,752 house-scale scenes rendered with \emph{Unreal Engine 4}. Per-pixel material annotations are provided, along with light sources and per-pixel HDR environment maps. 

\paragraph*{PanoSCU}~\cite{Data:PanoSCU:2025,Khan:2025:PSD}
This synthetic dataset supports change detection and understanding in single-shot panoramas. It contains 5,300 panoramas, covering only 360$^\circ$$\times$90$^\circ$ to match robotic acquisition with rotating standard FOV cameras, 132,222 panoramic object masks, and 8,275 panoramic captions, created by rendering 85 AI2-THOR~\cite{Kolve:2017:AI3} scenes. Each scenario randomly changes the position or state of 1–5 objects, categorized in 48 object types, with ground-truth change annotations, resulting in 4,150 rearrangement scenarios. It enables object detection, tracking, segmentation, and change understanding from a single panorama. The dataset is intended to support single-panorama object detection and tracking, segmentation, change understanding, and change reversal.

Several important observations arise from the analysis of the current landscape of available datasets. 

\paragraph*{Orientation} All collections provide (near) gravity-aligned panoramas or transformations that can produce near gravity-alignment. When gravity alignment is not perfect, previous research has shown that deviations are very small (e.g., sub-degree deviations for \emph{Stanford2D-3D-S}, \emph{Matterport3D}, and derived datasets~\cite{Pintore:2021:SDD}). Methods can, therefore, assume gravity alignment for training and testing data, or must introduce explicit augmentations when handling non-aligned data is required. 

\paragraph*{Resolution} Most collections are provided at relatively low resolutions (e.g., 512$\times$1024), and when higher resolutions are available, they often lack comprehensive annotations. As a result, dataset choice must be tailored to the intended task. For instance, while 512$\times$1024 panoramas captured from the center of typical rooms yield a few centimeters of spatial sampling that may suffice for floorplan or layout extraction, they may fall short for high-quality or immersive rendering~\cite{Rey:2022:3HR}. 

\paragraph*{Size and variety} Despite the existence of hundreds of millions of casually captured panoramas in domains such as real estate~\cite{Shabani:2021:ESF}, curated research datasets remain much smaller in scale and typically less diverse in terms of capture conditions and environments. This limitation stems not only from the challenges of aggregating user-generated data but also from the cost and complexity of creating annotation~\cite{Cruz:2021:ZID}. Synthetic datasets, either derived from re-rendering reconstructed models from dense captures~\cite{Albanis:2021:PHB,Rey:2022:3HR} or from fully modeled 3D environments~\cite{Zheng:2020:S3DE,Jin:2020:GSB,Xu:2021:LGN,Albanis:2021:PHB}, are emerging as a promising way to expand diversity and provide richer ground truth. However, even in these cases, the variety of environments, room types, and architectural styles is ultimately bounded by the available 3D sources. In this context, combining heterogeneous datasets for training and evaluation, though still uncommon (see~\autoref{sec:geostruct} and~\autoref{sec:interactive}), is an important area for research, particularly for testing robustness and transfer learning across domains~\cite{Albanis:2021:PHB,Rey:2022:3HR}. Complementary strategies such as semi- and self-supervised learning may further enhance downstream performance by leveraging vast amounts of unlabeled data in addition to the limited annotated collections~\cite{Zhuang:2023:SPDET,Li:2025:SIS}.