GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learning

key contributions

Three Core Breakthroughs

GS-Playground addresses three fundamental bottlenecks that have prevented vision-centric robot learning from scaling.

General-Purpose Embodied Simulation Platform

Custom parallel physics engine with full-scene native support for all robot morphologies, cross-platform, superior contact stability.

Parallel embodied simulation preview: multi-quadruped rollout on structured terrain

CPU/GPU Win/Linux/Mac Multi-Modal Sensors

CONTRIBUTION 02

Memory-Efficient Batch 3DGS Rendering

10,000 FPS photorealistic rendering across 2,048 parallel scenes on a single GPU. 90%+ Gaussian compression, PSNR drop <0.05.

10K FPS RLGK Sync SpeedySplat

CONTRIBUTION 03

Automated "Sim-Ready" Real2Sim Workflow

Single RGB image → fully simulation-ready digital twin with 3DGS, mesh, 6D pose, in under 5 minutes. Zero manual modeling.

Real2Sim workflow preview: lab setup bridging captured scenes into simulation-ready assets

Auto Segmentation 3DGS Assets Digital Twin

simulator comparison

Simulators	Physics Engine	Batch Physics	VRAM Usage	Integrated Batch IK	Batch Renderer	3DGS Env. Num.	Dynamic 3DGS Scene	3DGS Render FPS
MuJoCo/MJX	Brax/MJX	CPU/GPU	💰💰	✕	Madrona	—	—	—
IsaacLab	PhysX5	GPU	💰💰💰💰	✓	omni.RTX	—	—	—
ManiSkill	PhysX5	GPU	💰💰💰	✓	Vulkan SBR	—	—	—
Genesis	Taichi	GPU	💰💰	✓	Madrona	—	—	—

DISCOVERSE	MuJoCo	—	—	✕	—	1 ~ 4	✓	~650
GSWorld	PhysX5	—	—	✕	—	1	✓	—
GaussGym	PhysX4	GPU	💰	✕	GSplat	Up To 2048	✕	—

GS-Playground	Self-Dev.	CPU/GPU	—	✓	BatchSplat	Up To 2048	✓	~10k

// Note: (1) Batch Physics indicates supported hardware. (2) VRAM Usage uses more 💰 for higher VRAM (headless: physics only). (3) Batch Render Fidelity uses more eye icons for higher fidelity. (4) 3DGS Render FPS tested at 640×480 on RTX 4090 + i9-14900K. (5) Startup Speed uses more lightning icons for faster startup. (6) Physics Cross Platform: W/L/M.

Contribution 01

General-Purpose Parallel Physics Engine

Full-scene native support — all robots, all tasks, all platforms.

A ground-up, cross-platform (Windows, Linux, macOS) parallel physics engine supporting both GPU and CPU backends. Uses a velocity-impulse formulation with Constraint Island parallelization for linear performance scaling. Provides high-fidelity physical dynamics and comprehensive sensor integration—RGB cameras, LiDAR, force/contact sensors—across diverse robot embodiments: quadrupeds, humanoids, manipulators.

// N=50 humanoids · single environment · AMD 9950X + RTX 5090

GS-Playground 1,015 FPS

MuJoCo 32 FPS

MjWarp 1.71 FPS ↯ collapse

32×

vs MuJoCo

~600×

vs MjWarp

3+

platforms

CPU/GPU Win/Linux/macOS MJCF LiDAR+Force

stability benchmark — mujoco vs gs-playground

MUJOCO

GSP

Grasp stability

MUJOCO

GSP

Interaction stability

MUJOCO

GSP

Large-compensation motion stability

complex-scene stability benchmark

Complex Multi-Body Scene — Stable Equilibrium

rendering fidelity comparison

real2sim2real simulator — sim vs real

SIM

REAL

SIM

REAL

SIM

REAL

Contribution 02

Memory-Efficient Batch 3DGS Renderer

10,000 FPS — photorealistic rendering at unprecedented scale.

A specialized point-pruning strategy (inspired by PUP 3DGS and SpeedySplat) reduces Gaussians by over 90% while keeping the PSNR drop below 0.05—imperceptible to visuomotor policies. Rigid-Link Gaussian Kinematics (RLGK) binds Gaussian clusters to rigid bodies for zero-overhead, artifact-free dynamic rendering. The result: 2,048 scenes simultaneously at 10,000 FPS on a single RTX 4090.

10K

FPS @ 640×480

2048

Parallel scenes

90%

Gaussian pruned

RLGK — Rigid-Link Gaussian Kinematics

Binds 3D Gaussian clusters to physics rigid bodies. Visual state updates incur zero additional overhead—enabling perfect physics-render synchronization at any speed.

Batch-3DGS RLGK Zero-Overhead Sync SpeedySplat Pruning PSNR drop <0.05

render speed vs. isaac sim

Contribution 03

Automated "Sim-Ready" Real2Sim Pipeline

Photo → simulation in <5 minutes. Zero manual modeling.

Users input a single RGB image and receive a complete, simulation-ready digital twin— including 3DGS scene representation, object meshes, and sub-millimeter pose estimates— in under five minutes. The pipeline handles object detection, segmentation, background inpainting, 3DGS reconstruction, and scale-consistent pose alignment automatically.

INPUT

Single RGB image

01

Object detection — Grounding DINO

02

Segmentation + Background inpainting — SAM2 + LaMa ~25s

03

3DGS + Mesh reconstruction — SAM-3D + AnySplat ~18s

04

Sub-millimeter pose alignment + 3DGS pruning

OUTPUT

3DGS + Mesh + 6D Pose — Sim-ready in <5 minutes

Grounding DINO SAM-3D AnySplat LaMa Inpainting

policy experiments

Seamless Sim2Real Transfer

Rigid-Link Gaussian Kinematics (RLGK) synchronizes the visual and physical states with zero overhead— closing perceptual and physical reality gaps simultaneously. Train in simulation. Deploy directly. No fine-tuning required.

10min

Quadruped (Go2)
locomotion convergence

1,024 parallel envs

6hr

Humanoid (G1)
walking policy training

2,048 parallel envs

Zero

Visual navigation
zero-shot deployment

Go2 → real cone search

90%

Arm grasping
zero-shot success rate

Airbot Play · no special setup

Experiment 01

Locomotion

Unitree Go2 quadruped (10 min, 1024 envs) and Unitree G1 humanoid (6 hr, 2048 envs). State-based policies trained in simulation and deployed directly on hardware.

Direct sim-to-real deployment

Experiment 02

Manipulation

Block-grasping with Airbot Play arm. RGB + proprioception → 6-DOF joint actions. 3DGS digital twin trained with domain randomization.

90% zero-shot success rate

Experiment 03

Navigation

Visual navigation with Unitree Go2—hierarchical RL with egocentric RGB input. Policy trained in GS-Playground deployed zero-shot on real robot.

Zero-shot real-world navigation

Acknowledgments

Joint Contributors

Yufei Jia^*, Heng Zhang^*, Ziheng Zhang^*, Junzhe Wu^*, Mingrui Yu^*
Zifan Wang, Dixuan Jiang, Zheng Li, Chenyu Cao, Zhuoyuan Yu, Xun Yang, Haizhou Ge
Yuchi Zhang, Jiayuan Zhang, Zhenbiao Huang, Tianle Liu, Shenyu Chen, Jiacheng Wang, Bin Xie
Xuran Yao, Xiwa Deng, Guangyu Wang, Jinzhi Zhang, Lei Hao, Zhixing Chen, Yuxiang Chen
Anqi Wang, Hongyun Tian, Yiyi Yan, Zhanxiang Cao, Yizhou Jiang, Hanyang Shao, Yue Li, Lu Shi
Bokui Chen, Wei Sui, Hanqing Cui, Yusen Qin, Ruqi Huang
Lei Han^†, Tiancai Wang^†, Guyue Zhou^†