In this blog post, I want to write about something that was one of the most popular research papers in the 3D world a few years ago: 3D Gaussian Splatting (3DGS), published at SIGGRAPH 2023. If you’re like me, you’re always on the lookout for new tools and techniques to make your workflow faster, more efficient, or just plain cooler. So, what’s the deal with this fancy-sounding term?
3D Gaussian Splatting is a relatively new technique for creating 3D representations of real-world objects or scenes. Unlike traditional methods like photogrammetry, which rely on stitching together photos to create a 3D model, Gaussian Splatting uses a mathematical approach to represent 3D data.
Imagine you’re looking at a point cloud (a bunch of dots in 3D space). Instead of just leaving those dots as they are, Gaussian Splatting “splats” them into little 3D blobs (technically called Gaussians) that overlap and blend together. These blobs are optimized to represent the shape, color, and texture of the object or scene in a view-dependent way that’s both efficient and visually stunning.
A few technical words: Think of a 1D Gaussian as the classic bell curve—a smooth, symmetrical hill that peaks in the middle. Now, imagine stretching that idea into 3D space. Instead of a flat curve, you get a 3D ‘blob’ (like a squishy ball or a stretched-out sphere) that has a center point, can be scaled bigger or smaller, rotated in any direction, and has fuzzy, faded edges instead of sharp ones. These blobs are called 3D Gaussians.
Okay, let’s compare this to photogrammetry, which most of us are familiar with. In photogrammetry, you take a bunch of photos of an object or scene from different angles and then software (like RealityCapture or 3DF Zephyr) analyzes those photos and reconstructs a 3D model by matching points between images. Finally, the result is a dense mesh with textures, which you can tweak and clean up in your favorite 3D software like Autodesk Maya, which is my favorite.
On the other hand, in 3D Gaussian Splatting instead of creating a mesh, it works directly with a point cloud. Each point is replaced with a 3D Gaussian “blob” that has properties like position, size, rotation, and color. These blobs are optimized to represent the scene in a way that’s lightweight and visually accurate.
The key difference? Photogrammetry gives you a mesh, while Gaussian Splatting gives you a cloud of blobs that look like a 3D model when rendered. It’s like the difference between building a sculpture out of clay (photogrammetry) and creating one out of glowing, overlapping orbs (Gaussian Splatting).
So, why is 3D Gaussian Splatting getting so much attention? Here are some reasons it’s worth exploring:
- First, Gaussian Splatting can be faster than photogrammetry because it skips the mesh generation step. You go straight from a point cloud to a renderable representation.
- Second, the results can look incredibly realistic, especially for complex scenes with lots of detail. The overlapping blobs create smooth transitions and natural-looking textures.
- Third, since it’s based on math (not a fixed mesh), you can easily tweak the representation. Want to change the density or resolution? No problem—just adjust the parameters.
- Fourth, Gaussian Splatting can produce smaller file sizes compared to dense meshes, which is great for real-time applications like games or VR.
Of course, because nothing’s perfect, 3D Gaussian Splatting isn’t a magic bullet. So, there are some downsides like: There isn’t traditional Mesh. If you’re used to working with meshes (like in Maya or Blender), Gaussian Splatting might feel a bit alien. You can’t directly edit the “blobs” like you would a polygon model. While it’s great for visualization, it might not be the best choice if you need a clean, editable model for animation or 3D printing. Also, you should consider that this technique is still pretty new and evolving, so the tools and workflows aren’t as mature as photogrammetry. You might need to experiment a bit to get the results you want. The last one in my view is hardware requirements. Rendering Gaussian Splats can be computationally intensive, especially for large or complex scenes.
When Should You Use It? As a 3D artist, you might want to try 3D Gaussian Splatting if:
- You’re working on a project where realism is key, and you don’t need a traditional mesh.
- You’re dealing with complex scenes that would be challenging to model manually, especially when these detailed scenes can be used as background for traditional 3D objects in the foreground like in this example.
- You’re exploring real-time applications like VR or AR, where lightweight assets are a must.
On the other hand, stick with photogrammetry if:
- You need a clean, editable mesh for animation, 3D printing, or further refinement.
- You’re already comfortable with the workflow and don’t want to learn something new (we’ve all been there).
In conclusion, it’s not going to replace photogrammetry or traditional modeling anytime soon, but it offers a fresh way to create stunning, realistic representations of the world around us. So, should you dive in and start splatting? If you’re curious and love experimenting with new tech, even if it doesn’t become your go-to method, it’s always good to have another trick up your sleeve.
The scene below was trained in Postshot with about 200 photos, and then the result was imported into the Unreal Engine.
