Gaussian splatting is a volume rendering technique that deals with the direct rendering of volume data without converting the data into surface or line primitives. The technique was originally introduced as splatting by Lee Westover in the early 1990s
With advancements in computer graphics, newer methods such as 3D Gaussian splatting and 3D Temporal Gaussian splatting have been developed to offer real-time radiance field rendering and dynamic scene rendering respectively.
Gaussian splatting is a technique used in computer graphics for rendering 3D scenes in real-time. It involves representing a scene using 3D Gaussians, which are essentially small, smooth, and continuous distributions of points in space. Here’s a brief overview of how it works:
- Sparse Point Cloud: The process starts with a set of images or videos of a scene, from which a sparse point cloud is generated using methods like Structure from Motion (SfM).
- 3D Gaussians: Each point in the point cloud is converted into a 3D Gaussian. These Gaussians are defined by parameters such as position, covariance (which determines the shape), color, and opacity.
- Optimization: The parameters of these Gaussians are optimized using techniques like stochastic gradient descent to minimize the difference between the rendered image and the actual images.
- Rasterization: Finally, the Gaussians are projected onto a 2D plane (the screen) using a visibility-aware rendering algorithm. This step involves blending the Gaussians to create the final image.
The result is a high-quality, photorealistic rendering of the scene that can be done in real-time. This method is particularly useful for applications like virtual reality, augmented reality, and real-time simulations.
3D Temporal Gaussian splatting
Extending 3D Gaussian splatting to dynamic scenes, 3D Temporal Gaussian splatting incorporates a time component, allowing for real-time rendering of dynamic scenes with high resolutions. It represents and renders dynamic scenes by modeling complex motions while maintaining efficiency. The method uses a HexPlane to connect adjacent Gaussians, providing an accurate representation of position and shape deformations. By utilizing only a single set of canonical 3D Gaussians and predictive analytics, it models how they move over different timestamps.
It is sometimes referred to as “4D Gaussian splatting”; however, this naming convention implies the use of 4D Gaussian primitives (parameterized by a 4×4 mean and a 4×4 covariance matrix). Most work in this area still employs 3D Gaussian primitives, applying temporal constraints as an extra parameter of optimization.
Achievements of this technique include real-time rendering on dynamic scenes with high resolutions, while maintaining quality. It showcases potential applications for future developments in film and other media, although there are current limitations regarding the length of motion captured.
