8photos photogrammetry

Starting Photogrammetry with just 8 photographs

In this article, I will introduce tips for photogrammetry to advance your skills.

Introduction

Everyone, are you photogrammetring? My name is noria901.

There are many articles about photogrammetry in STYLY Magazine. If you want to learn how to use photogrammetry, there are introduction articles like Introduction to photogrammetry – taking photo & generating 3D model, How to take photos for successful Photogrammetry, and Photogrammetry information – Summary. In addition, the 銭洗弁天VR article [Photogrammetry] Architectural digital archive has more advanced tips, and is perfect for mid-level learners.

If you have read this articles, you many notice a certain fact.Photogrammetry takes time and money. Also, you may experience failures from unknown causes. This may be because you were too challenging. If so, maybe you should start with smaller steps at a time. Yes, you should do photogrammetry without any possibility of failure. This means that you should take as few time as possible, use as few money as possible, and keep your project understandable. Ok, so what should we start with?

Start with 8 photos

How many pictures do you need to start photogrammetry? The answer is 2. This is the minimum amount needed to start a Metashape alignment. If you are using 3DF Zephyr, you need 3. Now, let’s see what the models look like when using the minimum amount of pictures.

写真3枚でフォトグラメトリ

Hmm… hard to comment on.

As you see, a model is created, but it’s hard to see what the original object was. We cannot use this for artworks.

The question becomes, what is the minimum amount of pictures needed to create a reliable mode? The answer is 8. Ok, let’s see a sample model created from 8 pictures.

写真8枚でフォトグラメトリ

We can now see that the model is a coaster on top of a flyer.

As you see, we can recognize the model. But you can also see that the back is extremly hollow. We can see that 8 is the number to create a reliable model from 1 direction.

Of course, it depends on the object if the model can be created from 8 pictures. If the original object is large, it tends to need more, but small objects can be created with 8. Let’s look at different sized sample models created from 8 pictures.

Photogrammetring small objects

The procedure of photogrammetring should be as below.

  1. Decide what you want to take photos of
  2. Photographing
  3. Creating a model using a photogrammetry tool

An important point is to image how to take the photo according to the model. In other words, to have a policy.

Photographing policy

No matter what you choose to photogrammetry, you need to follow the basic rules of photogrammetry. For example, you need to take photos with 60% overlap. If you think of how you take photos, you could probably imagine how you should take photos with 60% overlap by moving parallel to the object.

平行に撮影する

Photographing with 60% overlap

Well then, if you are taking a round object, how should you take photos of it? 60% overlap of a round object… is that even possible? Imagine taking the photo from a high angle. The near side should look larger, and the far side will look smaller.

円形の物

Taking a round object from a high angle

In order for a photogrammetry tool to recognize a object, the feature points need to be in sight with enough resolution. So, the near side information is much easier for the tool to recognize. For example, let’s say that the object is recognizable for 90 degrees from the picture center. Then, we need to take photos with 60% overlap of the side within the 90 degrees. Which means we need a photo for each 90°×60%=54°, so we need a total of 360°÷54°=7 pictures. In order to create a good looking texture map, add a photo from direct above, resulting in a total of 8 pictures. Yes, the amount of pictures needed to create a reliable model from one side is 8 pictures.

写真8枚で十分

If you just need one side to be reliable, you only need 8 pictures

Models and Sample creations

Here, I want to introduce some models and sample creations using this way of thinking.

This is a manhole lid.

マンホールの蓋

Existing anywhere.

There are no limitations to manhole lid desgins by JIS standards, so there are more thousands of different manhole lids in Japan. If you take a walk, you can see them anywhere, and even if they have the same design, the look will differ according to wear. Worn out ones are sometimes swapped for new ones, and they are well suited to use as photogrammetry models since they are never scarce. Also, people only see manhole lids from one side even in everyday life. There is no need to create a back side, so for photogrammetry, you only need to create one side, making it an even more perfect model to use for 8 picture photogrammetry.

This is a photogrammetry sample created following the previous policy, taking photos of the manhole lid from 7 directions, and 1 from above. It is a small enough model, and there is no back mesh, so even if you decrease the polygons to 5000 when creating the texture map, it is still recognizable enough.

写真8枚でマンホールの蓋

The rough surface is recognizable too if you have 8 pictures

Now we have the created model with the policy to take 8 pictures. The result created by photogrammetry is this. Our policy for this tutorial is pretty hard, but it is suited to find out how much detail we can get, and if we can recognize the result as a manhole lid. Please check it out in VR.

How to shorten time using a photogrammetry tool

In ”Manhole cover maniac”, there are about 30 manhole lids in the scene, but I took photos of 67 lids before I completed this scene. Meaning, I used the photogrammetry tool at least 67 times to create a model. But, you don’t have to be afraid of the time it is going to take. Since I used only 8 photos for each model, the modelling process didn’t take that much time. But if you get better at handling the tool, you can shorten the processing time even more.

If you are using 3DF Zephyr, first create a low density point cloud before modellingBoundary box setting is what you call Region for metashape. The boundary box setting makes it so that no high density point cloud is created outside the box, so the processing time gets shorter with it. I experimented how much time is reduced using 3DF Zephyr. The computer I used is GPD Pocket2 (CPU:m3-7y30, RAM:8GB), which has almost the same machine power as a MacBook 2017.

3DF Zephyr With boundary box setting Without boundary box setting

Create low density point cloud
Category: Close range
Preset: Default

1 minute and 1 second

1 minute and 39 seconds

(Boundary box settings included)

Create high density point cloud
Category: Close range
Preset: Default

2 minutes and 1 second 1 minute and 8 seconds

Mesh
Category: Close range
Preset: Default

5 minutes and 15 seconds 3 minutes and 30 seconds

Mesh with texture

テクスチャサイズ:8192×8192、
Max vertices: 5000、Max textures: 1

4 minutes and 51 seconds 54 seconds

Total time

13 minutes and 6 seconds 7 minutes and 18 seconds
Created model
境界ボックスあり

Exported entirely

境界ボックスあり

Inside of the boundary box is exported

Even with the boundary box settings included, the result was pretty fast.

By the way, I experienced some failures as I made so many models. Let me introduce some of them.

Failure 1:Alignment is not recognized

Alignment is to sort out the photos by the angle they were taken from. If you only take photos from one side and the alignment fails, you should suspect that the photos aren’t taken properly. Meaning, the object being out of focus, or camera shake is highly suspectable. The next likely case is failure due to light reflection. Since the manhole lids are made of iron, there is some amount of reflection. In the sample below, both camera shake and light reflection is occurring, and the photo was a failure. It’s hard to recover such a photo, and it is quicker to take another photo properly. Manhole lids do not run away.

アライメント失敗1

Alignment failure because of camera shake and light reflection

Failure 2:After alignment, something is wrong

The alignment succeeded, but the photo is misplaced from the proper place. When I tried to create a mesh with this data, a really weird looking manhole lid got created. This is an example of failure due to light reflection. The day I took the photo was misty and wet. Not only the manhole lid, but the whole area is reflecting light, resulting in alignment misplacement. Lesson: Do not take photogrammetry pictures on a rainy day.

アライメントがおかしい

Photo alignment is misplaced

Failure 3:The mesh was not created properly

Since the alignment was a success, I created the mesh, but the result was not what I was expecting. This is another case where reflected light caused a failure. If the positions are not recognized properly due to light reflections, this may happen. But you can also see that the manhole lid shape is somewhat recognizable. If the light reflection is small as this, if you choose a cloudy day to take your photos, you can prevent such failures. Or if you can afford it, you can use a single-lens reflex camera with a PL filter.

メッシュがおかしい

Failure due to light reflection

We looked at small things first. Next, let’s try large scenes.

Photogrammetring a huge scene

If you have read so far, you should be able to imagine what we need to photogrammetry a large scene. Yes, we need a photographing policy suited for large scenes.

Photographing Policy

As I mentioned previously, 8 pictures can only reassemble one side of the target. So, let’s try something as simple as a house wall. If you are targeting a flat object, you need to take 60% overlap photographs in the parallel direction. Let’s see how much area of a wall we can photogrammetry with 8 pictures. This depends on how much space you can take in each photo. Not only does the lens count, but physical circumstances such as not being able to take distance from the wall due to it being a corridor can also be an obstacle. (A 360° camera is extremely helpful for closed areas such as corridors, but this option is rather expensive and I will forget about it for now). This doesn’t mean all you need to do is take a wide angle photo. The texture map resolution, and details such as the roughness of the manhole lid will become worse. Let’s see some examples.

The first example is a large gate. I took this photo walking across a footbridge. I wasn’t able to reproduce the sky from the photo, but the gate looks rather solid. I took the photo while walking sideways, so the text on the gate got created nicely.

リコー通り

A gate at Rikodori, Numazu taken from a footbridge

The second is a shop in a street. The upper parts such as the billboard was not created since the photos lacked information, but you can feel the atmosphere of the shop.

昭和レインボー

A retro shop found in Numazu shopping street

Of course, as your target becomes larger, there will be more disturbance elements, resulting in failures. Let’s see some failed examples.

Failure 4:The wall becomes distorted

This is a photogrammetry of the front of a building. As you see, it looks like a ruin. If you look at the original picture, the sky is clear blue, and the wall is single colored, resulting in extremely few focus points. Another problem is that in the photo there is glass that changes color according to the angle, which is a public enemy to photogrammetry. If the wall is affected by whiteout due to good weather, you could lower the camera exposure (EV), or quicken the shutter speed to prevent whiteout from occurring. You can also see that there is a large hole in the roof. I focused on taking the whole picture, so there are sides that are not visible in the pictures such as under the eaves. If you want to reconstruct the whole building, we need to change our policy. Architectures have many surfaces, and you should think that you will need at least 8 pictures for each surface.

建物の正面をフォトグラメ撮ってみた

It does look like a building, but we cannot use this

Conclusion

I made a summary of what we learned about the 8 picture policy photogrammetry.

  • It can capture 1 side of the photogrammetry target
  • We only take 8 photos, so the photographing doesn’t take time
  • We only take 8 photos, so the modelling does not need massive computer power
  • Only 1 side is created, so the texture size and polygon number will keep low
  • The photos are limited, so it is easy to find out the reason of failures
  • We can try and create a model frequently and easily, so you will learn about the photogrammetry software quickly

By the way, in games or animations, creators will not try to construct a manhole lid from 8 photographs and 5000 polygons. Of course not, it’s just a ordinary manhole lid. You might feel, isn’t a manhole lid just good enough with texture? If so, the manhole lid may not be suited for you. Please try to find a object that you feel this must be perfect for photogrammetry! Find an object that you feel satisfied with, image what policy you are going to take, and take the photographs. If you fail in the alignment step, renew your policy. And again. If you focus on how many surfaces you need to create from the target, you will gain hints for how many photographs you will need.

How was this tutorial for photogrammetring with 8 photos? I hope you realized how you can photogrammetry without using too much time, too much money, and can understand the process. All you need to do now is start photogrammetring!

If you consider using a photogrammetry model in STYLY, you will probably be curious if there will be unneeded backgrounds or garbage data included, and if the whole model is fit enough. There may be, but you can edit the created model using the photogrammetry tool. In my next article, I will introduce how you can edit the model and make it look even better.