Lens correction
Condition on each feature’s projected screen-space location to account for spatially varying optical aberrations.
Spatially aware SIGGRAPH ASIA 2026 / RESEARCH PROJECT
ControlGSConditioning Neural Gaussians for
Downstream-Processing–Aware XR Rendering
Optimize the image that reaches the eye.
1 University of Rochester2 Reality Labs, Meta
ControlGS in actionPROJECT TEASER · SOUND AVAILABLE
From Gaussian primitives to the post-optics image.Download video ↓
3conditioned processing stages
20resolution–power evaluation settings
2benchmark datasets
End-to-endquality measured after the optics
01 / OVERVIEW
Rendering doesn’t end
at the framebuffer.
Lens correction, resampling, display mapping, and physical optics all change what an XR user sees. ControlGS brings that path into the training objective.
Abstract
02 / METHOD
One scene.
Conditions that adapt.
Learn a condition-injection module for a neural Gaussian decoder, then adjust its injection strength to match the downstream state at render time.
Anti-aliasing resampling
Adjust the condition strength with the effective display sampling rate as the viewing state changes.
Resolution awareDisplay mapping
Condition Gaussian decoding on the display power target to improve quality under a power constraint.
Power aware03 / RESULTS
Better images.
After the entire pipeline.
Evaluation on Mip-NeRF 360 and NeRF Synthetic, averaged across scenes and 20 resolution–power settings. The results below use the Scaffold-GS backbone.
ControlGS vs. SS-Scaffold-GS
+0.37 dB over baseline
+0.028 over baseline
−0.030 below baseline
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|
Values reproduced from Table 1 of the paper. Higher PSNR/SSIM and lower LPIPS are better. See the paper for the evaluation protocol, perceptual metrics, and runtime/storage trade-offs.
Explore how individual conditions change the rendering +
04 / RESOURCES
Take a closer look.
PDF ↗
Paper + supplement
The complete method, experiments, and additional results.
MP4 ↓Teaser video
A walkthrough of downstream-aware XR Gaussian rendering.
Cite ControlGS
@inproceedings{lin2026controlgs,
title = {ControlGS: Conditioning Neural Gaussians for
Downstream-Processing--Aware XR Rendering},
author = {Lin, Weikai and Zhao, Junjie and Marshall, Carl and
Kondguli, Sushant and Zhu, Yuhao},
booktitle = {SIGGRAPH Asia 2026 Conference Papers},
year = {2026}
}