[Photogrammetry] How to modify models Reality Capture version + 3DF Zephyr version

This is lilea, the dragon who is photogrammetric every day.
In this article, I will show you how to modify a model generated by photogrammetry.

There are two major approaches to modifying a model.

1. Read in the model modified by external software and regenerate the textures.
2. Doing it within the photogrammetry software

This article mainly describes method 1.
The first half of the article describes the method using Reality Capture, and
the second half describes the method using 3DF Zephyr.

For method 2,
The method to modify within Reality Capture is briefly introduced at the end of the Reality Capture section.
The method to modify within 3DF Zephyr can be found in this article by noria.

Reality Capture Edition

1. exporting the model

Here is the generated model.
There is a hole in the model due to a thin member.

Reconstruction > Export > Model to export the model.

The export destination must be a single-byte path.
If you specify a double-byte path, you will get the error [Exporting Textured and Colored Mesh – FBX SDK : Export failed.

When exporting, a file named .rcInfo is also generated.
This file is required for loading and should be placed in the same hierarchy as the exported file. 2.

2) Correction with DCC tools

Correct the broken mesh with your favorite software.
The following is a case of using Cinema4D’s Fill Hole tool.

The same principle can be used to fill holes in Reality Capture (see below).

3) Import the modified model

Import the modified model into the same project as when exporting.
At this time, also place the .rcinfo file in the same hierarchy as the model to be imported.

Reconstruction > Export > Model

4. generate UVs

This step is not necessary if you have already generated UVs with DCC tools.
If you generate them in Reality Capture, follow the steps below.

Reconstruction > Tools > Unwrap

A settings window will appear in the 1Ds window.
Here, you can specify the size and number of pieces to be used when generating the texture.

Large triangle texture resolution: The higher the value, the more UV area will be allocated for large polygons.
Maximum texture count: The number of textures to be generated.

Click the Unwrap button to generate the UV map.
Scene > Scene Render > Sweet to check the map status. 5.

5. regenerate the texture

Press the Texture button to start generation.

If you press the Settings button at the bottom right of the Texture button, the Unwrap settings screen will also appear.
In this case, the UV map has already been generated in the previous step, so the settings made here will be ignored. 6.

6. export the model and complete

Once the texture is generated, export the model to complete the process.

How to fill in the holes using Reality Capture alone

It is also possible to fill holes in the model using Reality Capture alone, without using external software.

However, this is only possible if the hole is flat.
If the hole is complex in shape, it may not be filled as expected.

Select the selection tool:
Reconstruction > Selection > Losso

Select the edges of the hole.

Hold Shift to add a selection,
Ctrl to delete a selection.

The selection is shown below.

Now Tools > Filter Selectiion will delete the mesh.

Next, Tools > Clean Model.

The hole will be closed.

The rest is done by generating textures as usual.

However, the mesh will not be very clean, so it is better to use a DCC tool if you want to make accurate modifications.

3DF Zephyr version

1. exporting the model

Here is the generated model.

As with Reality Capture, there are holes in the model.
(The position of the holes is roughly the same even if the software is different.)

Export from the context menu.

When exporting, leave the [Local Display Reference System] checkbox unchecked.

If it is checked, the model will not be displayed in the same way as it was when it was exported.

If this is turned on, the coordinates will be out of alignment when the modified model is imported.
(white is the imported model)

2) Correct with DCC tools

Fix the broken parts of the model with your favorite software.

At this time, polygons should be triangulated.
3DF Zephyr cannot generate textures correctly if polygonal polygons are present.

NG (polygonal polygons)

OK ( Triangular polygons)

3. adjust UVs

Since this is the situation as it is…

Adjust the UVs as shown below. 4.

4. load the modified model and generate textures

Load the modified model into the same project as when it was exported.

A texture generation dialog box will appear, so set the texture size, etc. and execute. 6.

6) Export the model and complete the process.

Once the texture is generated, export the model to complete.

Optimization

This workflow is not limited to modification, but can also be used to generate textures for full-scratch models.
Since meshes generated by photogrammetry tend to be high-polygon, it may be a good idea not to use this mesh depending on the content to be used.

By creating a model using the generated mesh as a guide and generating textures for that model, you can create a photorealistic and lightweight model.

The model used in this explanation is a chair designed by Itsuko Hasegawa that was left behind in the old Metropolitan Civic Center.
The model that has been modified can be viewed in AR here.

Written by Ryu Fujiwara (Ryu lilea )