Recreating real-world landscapes and buildings captured with a smartphone or camera as realistic 3D spaces that make you feel as though you were actually there. One technology attracting attention for making this possible is 3D Gaussian Splatting (hereafter, 3DGS).

Until now, turning real-world objects into 3D data has mainly relied on a production method known as 3D modeling.

This involves creating the shapes of buildings, cars, people, and other objects on a computer, then applying colors and surface properties to them. It is widely used in fields such as games, movies, and architectural visualization.

3DGS, on the other hand, uses a different approach from traditional 3D production to reproduce real-world spaces as 3D data.

So what exactly is different?

To understand that difference, let us first look at how traditional 3D production works.

Traditional 3D Production: Building 3D Objects from Vertices and Faces

One of the most widely used representation methods in traditional 3D production is the mesh.

A mesh represents a three-dimensional object by combining vertices, edges, and faces.

For example, to create a cube, eight vertices are placed, connected by edges, and then used to form faces that make up the three-dimensional shape.

Complex buildings, people, cars, and other objects are also basically created by combining large numbers of these small faces.

Colors and surface qualities are then added to the 3D object by applying images called textures and settings called materials.

For example, a car may be given the metallic appearance of its body, a building may be given the texture of its walls, and a person may be given the appearance of skin and clothing.

In this way, the basic idea behind traditional 3D production is to first create the shape and then give its surface color and texture.

One major advantage of meshes is that they allow shapes to be clearly defined.

Vertices can be moved to change the form, individual components can be edited separately, and details can be added as needed. This makes meshes well suited to applications where shapes need to be freely edited, such as game characters, automotive design, and architectural design.

However, reproducing a real-world scene with photographic quality requires a considerable amount of work.

It is not enough to create the basic shape of a building. Windows, wall patterns, trees, the ground, small decorations, and other details also need to be reproduced. Their colors and surface properties must also be adjusted.

In other words, creating realistic 3D data from a real-world scene requires both the shape and the appearance to be carefully created.

3DGS: Reconstructing Real-World Spaces with Large Numbers of Gaussians

3DGS represents 3D spaces using a completely different concept.

In 3DGS, large numbers of small elements called 3D Gaussians, which have spatial extent, are placed throughout the scene.

Each Gaussian contains information such as its position, size, orientation, color, and opacity.

These Gaussians are projected into the image from the camera's viewpoint and composited together to generate the final image.

This may be a little difficult to imagine.

If a mesh is a method of “building a three-dimensional object by combining vertices and faces,” then 3DGS can be thought of as a method of “reconstructing a real-world scene by placing large numbers of small, light-like particles throughout the space.”

Of course, actual Gaussians are not simply particles of light. They are data elements containing information such as position, size, color, and opacity.

A major characteristic of 3DGS is that it can represent a 3D space with a photographic appearance without having to model the surfaces of objects in detail using polygons.

For example, leaves, grass, fine architectural decorations, and complex surface irregularities would traditionally require detailed geometry and textures when using meshes. With 3DGS, these details can be reproduced visually through collections of Gaussians.

3DGS is also not simply a technology for displaying static images.

Using the Gaussians placed in 3D space, it can generate images from various viewpoints as the camera position and direction change.

In other words, users can move through the space and change their viewpoint while maintaining a realistic, photographic appearance.

The original 3DGS paper published in 2023 demonstrated that high-quality images from novel viewpoints could be generated in real time.

What Is the Difference Between Meshes and 3DGS?

Based on what we have covered so far, the main difference between meshes and 3DGS lies in how they represent 3D space.

What is important here is that this is not a question of which approach is superior.

The two methods are fundamentally different approaches to representing three-dimensional space.

Meshes are well suited to clearly defining geometry and freely editing or modifying shapes.

3DGS, on the other hand, is good at reproducing real-world scenes and objects with a photographic appearance.

Consider, for example, creating a 3D representation of an existing historic building.

When using a mesh, the shape of the building is first created, and textures are then applied to surfaces such as windows, walls, and roofs.

With 3DGS, images captured from the actual building can be used to reconstruct a space that includes its exterior appearance, detailed decorations, and surrounding environment.

Even when digitizing the same building in 3D, the approaches used to reach the result are very different.

What Will 3DGS Change?

One expectation surrounding the emergence of 3DGS is that it will expand the range of options available for converting real-world spaces into 3D data.

Until now, creating realistic 3D spaces has often required specialized modeling work and detailed texture production.

With 3DGS, however, photographic-quality 3D scenes can be generated from images captured in real-world spaces.

This creates the possibility of digitizing more places and objects in 3D and using them in digital environments.

Potential applications include:

Real estate: Recreate buildings and interiors as realistic 3D spaces that can be viewed remotely from freely chosen viewpoints.
Architecture and civil engineering: Digitize construction sites and existing facilities in 3D so that site conditions can be checked and shared remotely.
Cultural heritage: Digitally preserve historic buildings and exhibits with an appearance close to their real-world condition.
Video production: Capture real locations, convert them into 3D, and use them as background assets for visual production.
XR: Bring real-world spaces into virtual environments to create more realistic digital experiences.

What these applications have in common is that the ability to freely edit the object's shape is not necessarily required.

For example, in a real-estate virtual tour, the ability to change the shape of a wall is less important than accurately conveying the actual size and atmosphere of a room and the arrangement of its furniture.

Similarly, in the digital preservation of cultural heritage, there is value in recording the object as close as possible to its real-world appearance, including detailed decorations and changes caused by aging.

3DGS is a technology that responds to the need to digitize real-world spaces while preserving an appearance as close as possible to reality.

Will 3DGS Replace Traditional 3D Production?

If 3DGS becomes widely adopted, will traditional mesh-based 3D production no longer be necessary?

The answer is no.

3DGS and meshes each have different strengths, so they are likely to continue being used according to the purpose of each project.

Again, consider the example of digitizing a building.

If the goal is to realistically reproduce the exterior or interior and allow people to explore it freely from a remote location, 3DGS can be a strong option.

On the other hand, if the goal is to move walls, change window positions, or modify the building's design, a mesh is better suited because its geometry can be clearly edited.

3DGS also has limitations, including the difficulty of freely editing geometry and restrictions on using the data directly for applications such as physical simulation.

For this reason, research is also progressing on approaches that combine 3DGS with meshes in order to take advantage of the strengths of both.

In the future, rather than choosing only one method, it may become increasingly important to combine multiple forms of 3D representation depending on the purpose.

Conclusion: 3DGS Is a New Approach to Digitizing Real-World Spaces in 3D

In traditional 3D production, three-dimensional forms are created using meshes made from vertices and faces, and colors and surface properties are then applied to reproduce real-world objects and spaces.

3DGS, by contrast, uses large numbers of Gaussians to reproduce real-world spaces with a photographic appearance.

The two methods represent 3D space differently and each has its own strengths.

Meshes are strong when it comes to editing and modifying geometry, while 3DGS is particularly effective at reproducing the appearance of the real world.

The emergence of 3DGS has introduced a new approach to digitizing real-world spaces, a task that previously often required specialized production work.

Buildings, streetscapes, cultural heritage sites, factories, stores, and many other things around us may become easier to preserve and use as realistic 3D spaces.

3DGS is more than simply a new 3D representation technology.

It has the potential to change the way we bring the real world into digital space.

Understanding 3DGS, Part 1: How Is 3DGS Different from Traditional 3D Modeling?