MetaSapiens: Real-Time Neural Rendering with Efficiency-Aware Pruning and Accelerated Foveated Rendering

Weikai Lin¹, Yu Feng², Yuhao Zhu¹,

¹ University of Rochester ² Shanghai Jiao Tong University

Video Comparisons

The first row shows a comparison between the Dense Model (Mini-Splatting-D [Fang 2024]) and our method, which includes Our-L1 (Pruned Version) and FR (Foveated Rendering). We also display the L2~L4 used in FR in the second row. All videos are fixed to 90 FPS for comparison.

- Note:

FPS in the parentheses is measured on mobile device - Jetson Xavier.
FR fixes the user's gaze at the center of the red circle.
Use Play All Videos botton to synchronize videos.
Videos here are downsampled and compressed. High quality version (1.2GB) can be downloaded here.

Mini-D (12 FPS)

Our-L1 (60 FPS)

Our-FR (85 FPS)

Our-L2

Our-L3

Our-L4

Method Overview

Our Foveated Rendering. In addition to introducing Computation-Aware Pruning, we further enhance rendering efficiency by employing Multi-Versioning-based Foveated Rendering (left figure), leveraging human perceptual characteristics. A user study conducted in VR verifies the final quality. The right figure compares the Quality-FPS trade-off, measured using the objective metric PSNR and Jetson Xavier, between our method and other approaches.