Capturing Content Using Photogrammetry
Our goal is to capture the real world into a virtual 3D model at the highest possible resolution.
This is a step-by-step tutorial on how the content in the Love Bridge demo was made. (You can also skip to the end to see the live demo)
Camera setup we used for the photogrammetry:
- Canon EOS 6D Mark II
- Canon 20 mm 2.8 wide-angle lens
We shot the photos at:
- F/16 - for good depth of field
In this tutorial, we use about 200 photos for a segment of the bridge picture above. In the below screenshot you can see the camera positions in relation to the model.
There are many great software packages for turning photos into 3D models. This time we used RealityCapture.
We have worked very hard for many years to make a pipeline where you don't have to worry about triangle or texture budgets at all. You can use as many triangles and as high-resolution textures as you need for creating the perfect looking content and we promise it will still load fast, run at a high frame rate and stay within the memory budget on the device.
Thus, we choose RealityCapture parameters to output the maximum amount of detail possible (and let Umbra take care of optimizing the mesh and textures).
The photos are 26Mpix each so we tune up the alignment settings to make sure we use the available features.
High Detail reconstruction can take a lot longer and often Normal Detail is sufficient, but this time we're aiming for the maximum amount of detail so we choose High Detail.
"Adaptive" or "fixed texel size" work best - we don't care about maximum texture count - the more the better! Normally you might have to worry about how much GPU memory budget you have for the textures but thanks to the optimization pipeline all that will be handled automatically and we can use as many high-resolution textures as we need.
We export the resulting mesh and textures as .OBJ into its own folder. "Save mesh by parts" helps manage file size as .OBJs can grow to be several gigabytes. This time the exported model is about 12 GB in .OBJ format including textures.
For this step, you need an account at umbra.io. Luckily there's a trial version available you can test for free.
Content can be uploaded to Umbra using various different tools. For large files, the command line interface(CLI) is the most efficient. You can download the CLI tool as part of the SDK package available on the download page once you have logged in.
umbra.exe umbrafy --api-key <your key from umbra.io> --input-obj <folder where you exported the model> --scene-name "Locks" --feature-size 0.001291 --content-profile scanned --import-zIsUp true --import-right-handed true
The feature-size parameter is the most important and defines the output resolution of the data i.e. what is the smallest detail in the model. When processing photogrammetry models this usually equals to texel size in the input model. From RealityCapture we can see the texel size was 0.001291 for this model so we use it as the Umbra feature-size parameter as well.
After you launch the process you can monitor the state of the processing from the umbra.io portal and once it's done, the content is immediately available for streaming to the web, mobile apps, Unity or through the C++ SDK.