A research team at ShanghaiTech University has published a framework in Visual Intelligence that covers the entire workflow of Gaussian Splatting, from capturing footage to rendering it on a screen or a headset. The paper, released on 27 March 2026, targets the two problems that have kept volumetric video largely confined to laboratories: the sheer size of the data and a toolchain that has never been standardised across research groups.
What Gaussian Splatting actually does
3D Gaussian Splatting builds a scene out of millions of small points, each carrying colour, position and shape information, rather than the polygons used in conventional 3D graphics. The technique renders in real time and with a level of visual fidelity that has made it attractive for virtual and augmented reality. Extending it into four dimensions — adding time to the three spatial axes — allows a moving person or object to be reconstructed as a navigable 3D recording, viewable from any angle rather than from a fixed camera position. That extension is what underpins volumetric video.
As Professor Jingyi Yu of ShanghaiTech University put it, the approach lets viewers explore a scene freely and “narrows the gap between digital and physical realities.”
Why storage has been the real obstacle
Static Gaussian Splatting scenes are already data-heavy. A dynamic sequence multiplies that weight by every frame captured over time, which pushes storage, transmission and interactive playback requirements far beyond what most pipelines were designed to handle. Dr Lan Xu, also from ShanghaiTech’s School of Information Science and Technology, described storage cost as the most pressing issue precisely because the temporal dimension inflates data volume so quickly.
Alongside the storage problem, the team points to a second, less technical but equally limiting issue: existing compression and reconstruction methods have been developed independently, using different code bases, data formats and loaders. That fragmentation makes it hard to compare results, reproduce experiments, or combine techniques from different research groups into a single working system.
An integrated pipeline, plus a public dataset
The framework addresses both problems in one structure. It standardises data preprocessing and loading across a range of 3D and 4D Gaussian Splatting reconstruction methods, and adds a general-purpose compression module compatible with the outputs of those different methods, reducing storage requirements while preserving visual quality. A cross-platform rendering plugin then handles playback on desktop, mobile and XR devices, aiming at consistent free-viewpoint interaction regardless of the device used.
To support the research community, the team also built a dense capture rig of 81 synchronised RGB cameras and used it to record more than 130 sequences of human motion, including interactions involving changes in topology — bodies merging with objects or with each other. The system captures timecode-aligned footage at 3840 x 2160 resolution and 30 frames per second, giving other researchers a shared, high-quality dataset to test reconstruction and compression methods against.
What this means for real-world projects
For anyone designing an XR experience around a real person or a real object in motion, this is precisely the layer of technical infrastructure that usually decides whether a project is feasible in the first place, not just how it looks. Volumetric video is attractive for museums, training simulations and public engagement precisely because it lets visitors walk around a subject rather than watch it from a fixed angle. The catch has always been practical: capturing a few minutes of high-fidelity dynamic footage can generate storage and bandwidth demands that make deployment on standard hardware, or delivery to a museum kiosk or a training headset, genuinely difficult.
A published, peer-reviewed pipeline that standardises acquisition, compression and rendering does not remove that cost entirely, but it changes the conversation with a client. Today, a studio proposing a volumetric video installation has to explain, case by case, how storage and playback will be handled, often stitching together tools from different research papers that were never meant to work together. A common framework, even one still maturing in an academic context, gives that conversation a reference point: a documented method, a compression approach compatible with multiple reconstruction techniques, and a rendering plugin already built for XR devices rather than adapted after the fact.
There is a caveat worth stating plainly. This is a research publication, not a commercial product, and the paper does not report deployment timelines, licensing terms or performance benchmarks on consumer-grade hardware. Turning an academic pipeline into something reliable enough for a public installation running eight hours a day, on hardware a museum or a training centre can actually afford, is a separate body of work. What matters for now is the direction: the fragmentation that has made volumetric video a bespoke, high-risk undertaking for every new project is being addressed as a shared infrastructure problem rather than left to each team to solve on its own.
For studios working in cultural heritage and training in Italy, where budgets for immersive installations are rarely unlimited, this kind of groundwork is worth tracking closely. A project that once required custom engineering for capture, compression and playback may, over the coming years, require integration work instead — a meaningfully different, and cheaper, kind of effort. It does not make volumetric video simple. It makes the cost of trying it more predictable, which is often the detail that determines whether a public commissioning body approves a proposal at all.