[Photogrammetry] How to modify models Reality Capture version + 3DF Zephyr version

This is lilea, the dragon who is photogrammetric every day.
In this article, I will show you how to modify a model generated by photogrammetry.

There are two major approaches to modifying a model.

  1. Read in the model modified by external software and regenerate the textures.
  2. Doing it within the photogrammetry software

This article mainly describes method 1.
The first half of the article describes the method using Reality Capture, and
the second half describes the method using 3DF Zephyr.

For method 2,
The method to modify within Reality Capture is briefly introduced at the end of the Reality Capture section.
The method to modify within 3DF Zephyr can be found in this article by noria.

Reality Capture Edition

1. exporting the model

Here is the generated model.
There is a hole in the model due to a thin member.
How to update modified model in Reality Capture and 3DF Zephry

Reconstruction > Export > Model to export the model.
How to update modified model in Reality Capture and 3DF Zephry

The export destination must be a single-byte path.
If you specify a double-byte path, you will get the error [Exporting Textured and Colored Mesh – FBX SDK : Export failed.

When exporting, a file named .rcInfo is also generated.
This file is required for loading and should be placed in the same hierarchy as the exported file. 2.

2) Correction with DCC tools

Correct the broken mesh with your favorite software.
The following is a case of using Cinema4D’s Fill Hole tool.
How to update modified model in Reality Capture and 3DF Zephry

The same principle can be used to fill holes in Reality Capture (see below).

3) Import the modified model

Import the modified model into the same project as when exporting.
At this time, also place the .rcinfo file in the same hierarchy as the model to be imported.

Reconstruction > Export > Model
How to update modified model in Reality Capture and 3DF Zephry

4. generate UVs

This step is not necessary if you have already generated UVs with DCC tools.
If you generate them in Reality Capture, follow the steps below.

Reconstruction > Tools > Unwrap
How to update modified model in Reality Capture and 3DF Zephry

A settings window will appear in the 1Ds window.
Here, you can specify the size and number of pieces to be used when generating the texture.

Large triangle texture resolution: The higher the value, the more UV area will be allocated for large polygons.
Maximum texture count: The number of textures to be generated.

Click the Unwrap button to generate the UV map.
Scene > Scene Render > Sweet to check the map status. 5.
How to update modified model in Reality Capture and 3DF Zephry

5. regenerate the texture

Press the Texture button to start generation.
How to update modified model in Reality Capture and 3DF Zephry

If you press the Settings button at the bottom right of the Texture button, the Unwrap settings screen will also appear.
In this case, the UV map has already been generated in the previous step, so the settings made here will be ignored. 6.

6. export the model and complete

Once the texture is generated, export the model to complete the process.
How to update modified model in Reality Capture and 3DF Zephry

How to fill in the holes using Reality Capture alone

It is also possible to fill holes in the model using Reality Capture alone, without using external software.

However, this is only possible if the hole is flat.
If the hole is complex in shape, it may not be filled as expected.
How to update modified model in Reality Capture and 3DF Zephry

Select the selection tool:
Reconstruction > Selection > Losso
How to update modified model in Reality Capture and 3DF Zephry

Select the edges of the hole.
How to update modified model in Reality Capture and 3DF Zephry

Hold Shift to add a selection,
Ctrl to delete a selection.

The selection is shown below.
How to update modified model in Reality Capture and 3DF Zephry

Now Tools > Filter Selectiion will delete the mesh.
How to update modified model in Reality Capture and 3DF Zephry

How to update modified model in Reality Capture and 3DF Zephry

Next, Tools > Clean Model.
How to update modified model in Reality Capture and 3DF Zephry

The hole will be closed.
How to update modified model in Reality Capture and 3DF Zephry

The rest is done by generating textures as usual.
How to update modified model in Reality Capture and 3DF Zephry

However, the mesh will not be very clean, so it is better to use a DCC tool if you want to make accurate modifications.

3DF Zephyr version

1. exporting the model

Here is the generated model.
How to update modified model in Reality Capture and 3DF Zephry

As with Reality Capture, there are holes in the model.
(The position of the holes is roughly the same even if the software is different.)

Export from the context menu.
How to update modified model in Reality Capture and 3DF Zephry

When exporting, leave the [Local Display Reference System] checkbox unchecked.
How to update modified model in Reality Capture and 3DF Zephry

If it is checked, the model will not be displayed in the same way as it was when it was exported.
How to update modified model in Reality Capture and 3DF Zephry

If this is turned on, the coordinates will be out of alignment when the modified model is imported.
(white is the imported model)

2) Correct with DCC tools

Fix the broken parts of the model with your favorite software.
How to update modified model in Reality Capture and 3DF Zephry

At this time, polygons should be triangulated.
3DF Zephyr cannot generate textures correctly if polygonal polygons are present.

NG (polygonal polygons)
How to update modified model in Reality Capture and 3DF Zephry

OK ( Triangular polygons)
How to update modified model in Reality Capture and 3DF Zephry

3. adjust UVs

Since this is the situation as it is…
How to update modified model in Reality Capture and 3DF Zephry

Adjust the UVs as shown below. 4.
How to update modified model in Reality Capture and 3DF Zephry

4. load the modified model and generate textures

Load the modified model into the same project as when it was exported.
How to update modified model in Reality Capture and 3DF Zephry

A texture generation dialog box will appear, so set the texture size, etc. and execute. 6.
How to update modified model in Reality Capture and 3DF Zephry

6) Export the model and complete the process.

Once the texture is generated, export the model to complete.
How to update modified model in Reality Capture and 3DF Zephry

Optimization

This workflow is not limited to modification, but can also be used to generate textures for full-scratch models.
Since meshes generated by photogrammetry tend to be high-polygon, it may be a good idea not to use this mesh depending on the content to be used.

By creating a model using the generated mesh as a guide and generating textures for that model, you can create a photorealistic and lightweight model.

The model used in this explanation is a chair designed by Itsuko Hasegawa that was left behind in the old Metropolitan Civic Center.
The model that has been modified can be viewed in AR here.

Written by Ryu Fujiwara (Ryu lilea )