In this article, I will introduce tips for photogrammetry to advance your skills.
- 1 Introduction
- 2 Photogrammetring small objects
- 3 Photogrammetring a huge scene
- 4 Conclusion
Everyone, are you photogrammetring? My name is noria901.
There are many articles about photogrammetry in STYLY Magazine. If you want to learn how to use photogrammetry, there are introduction articles like Introduction to photogrammetry – taking photo & generating 3D model, How to take photos for successful Photogrammetry, and Photogrammetry information – Summary. In addition, the 銭洗弁天VR article [Photogrammetry] Architectural digital archive has more advanced tips, and is perfect for mid-level learners.
If you have read this articles, you many notice a certain fact.Photogrammetry takes time and money. Also, you may experience failures from unknown causes. This may be because you were too challenging. If so, maybe you should start with smaller steps at a time. Yes, you should do photogrammetry without any possibility of failure. This means that you should take as few time as possible, use as few money as possible, and keep your project understandable. Ok, so what should we start with?
Start with 8 photos
How many pictures do you need to start photogrammetry? The answer is 2. This is the minimum amount needed to start a Metashape alignment. If you are using 3DF Zephyr, you need 3. Now, let’s see what the models look like when using the minimum amount of pictures.
As you see, a model is created, but it’s hard to see what the original object was. We cannot use this for artworks.
The question becomes, what is the minimum amount of pictures needed to create a reliable mode? The answer is 8. Ok, let’s see a sample model created from 8 pictures.
As you see, we can recognize the model. But you can also see that the back is extremly hollow. We can see that 8 is the number to create a reliable model from 1 direction.
Of course, it depends on the object if the model can be created from 8 pictures. If the original object is large, it tends to need more, but small objects can be created with 8. Let’s look at different sized sample models created from 8 pictures.
Photogrammetring small objects
The procedure of photogrammetring should be as below.
- Decide what you want to take photos of
- Creating a model using a photogrammetry tool
An important point is to image how to take the photo according to the model. In other words, to have a policy.
No matter what you choose to photogrammetry, you need to follow the basic rules of photogrammetry. For example, you need to take photos with 60% overlap. If you think of how you take photos, you could probably imagine how you should take photos with 60% overlap by moving parallel to the object.
Well then, if you are taking a round object, how should you take photos of it? 60% overlap of a round object… is that even possible? Imagine taking the photo from a high angle. The near side should look larger, and the far side will look smaller.
In order for a photogrammetry tool to recognize a object, the feature points need to be in sight with enough resolution. So, the near side information is much easier for the tool to recognize. For example, let’s say that the object is recognizable for 90 degrees from the picture center. Then, we need to take photos with 60% overlap of the side within the 90 degrees. Which means we need a photo for each 90°×60%＝54°, so we need a total of 360°÷54°＝7 pictures. In order to create a good looking texture map, add a photo from direct above, resulting in a total of 8 pictures. Yes, the amount of pictures needed to create a reliable model from one side is 8 pictures.
Models and Sample creations
Here, I want to introduce some models and sample creations using this way of thinking.
This is a manhole lid.
There are no limitations to manhole lid desgins by JIS standards, so there are more thousands of different manhole lids in Japan. If you take a walk, you can see them anywhere, and even if they have the same design, the look will differ according to wear. Worn out ones are sometimes swapped for new ones, and they are well suited to use as photogrammetry models since they are never scarce. Also, people only see manhole lids from one side even in everyday life. There is no need to create a back side, so for photogrammetry, you only need to create one side, making it an even more perfect model to use for 8 picture photogrammetry.
This is a photogrammetry sample created following the previous policy, taking photos of the manhole lid from 7 directions, and 1 from above. It is a small enough model, and there is no back mesh, so even if you decrease the polygons to 5000 when creating the texture map, it is still recognizable enough.
Now we have the created model with the policy to take 8 pictures. The result created by photogrammetry is this. Our policy for this tutorial is pretty hard, but it is suited to find out how much detail we can get, and if we can recognize the result as a manhole lid. Please check it out in VR.
How to shorten time using a photogrammetry tool
In ”Manhole cover maniac”, there are about 30 manhole lids in the scene, but I took photos of 67 lids before I completed this scene. Meaning, I used the photogrammetry tool at least 67 times to create a model. But, you don’t have to be afraid of the time it is going to take. Since I used only 8 photos for each model, the modelling process didn’t take that much time. But if you get better at handling the tool, you can shorten the processing time even more.
If you are using 3DF Zephyr, first create a low density point cloud before modellingBoundary box setting is what you call Region for metashape. The boundary box setting makes it so that no high density point cloud is created outside the box, so the processing time gets shorter with it. I experimented how much time is reduced using 3DF Zephyr. The computer I used is GPD Pocket2 (CPU:m3-7y30, RAM:8GB), which has almost the same machine power as a MacBook 2017.
|3DF Zephyr||With boundary box setting||Without boundary box setting|
Create low density point cloud
|1 minute and 1 second||
1 minute and 39 seconds
(Boundary box settings included)
Create high density point cloud
|2 minutes and 1 second||1 minute and 8 seconds|
|5 minutes and 15 seconds||3 minutes and 30 seconds|
Mesh with texture
|4 minutes and 51 seconds||54 seconds|
|13 minutes and 6 seconds||7 minutes and 18 seconds|
Even with the boundary box settings included, the result was pretty fast.
By the way, I experienced some failures as I made so many models. Let me introduce some of them.
Failure 1：Alignment is not recognized
Alignment is to sort out the photos by the angle they were taken from. If you only take photos from one side and the alignment fails, you should suspect that the photos aren’t taken properly. Meaning, the object being out of focus, or camera shake is highly suspectable. The next likely case is failure due to light reflection. Since the manhole lids are made of iron, there is some amount of reflection. In the sample below, both camera shake and light reflection is occurring, and the photo was a failure. It’s hard to recover such a photo, and it is quicker to take another photo properly. Manhole lids do not run away.
Failure 2：After alignment, something is wrong
The alignment succeeded, but the photo is misplaced from the proper place. When I tried to create a mesh with this data, a really weird looking manhole lid got created. This is an example of failure due to light reflection. The day I took the photo was misty and wet. Not only the manhole lid, but the whole area is reflecting light, resulting in alignment misplacement. Lesson: Do not take photogrammetry pictures on a rainy day.
Failure 3：The mesh was not created properly
Since the alignment was a success, I created the mesh, but the result was not what I was expecting. This is another case where reflected light caused a failure. If the positions are not recognized properly due to light reflections, this may happen. But you can also see that the manhole lid shape is somewhat recognizable. If the light reflection is small as this, if you choose a cloudy day to take your photos, you can prevent such failures. Or if you can afford it, you can use a single-lens reflex camera with a PL filter.
We looked at small things first. Next, let’s try large scenes.
Photogrammetring a huge scene
If you have read so far, you should be able to imagine what we need to photogrammetry a large scene. Yes, we need a photographing policy suited for large scenes.
As I mentioned previously, 8 pictures can only reassemble one side of the target. So, let’s try something as simple as a house wall. If you are targeting a flat object, you need to take 60% overlap photographs in the parallel direction. Let’s see how much area of a wall we can photogrammetry with 8 pictures. This depends on how much space you can take in each photo. Not only does the lens count, but physical circumstances such as not being able to take distance from the wall due to it being a corridor can also be an obstacle. (A 360° camera is extremely helpful for closed areas such as corridors, but this option is rather expensive and I will forget about it for now). This doesn’t mean all you need to do is take a wide angle photo. The texture map resolution, and details such as the roughness of the manhole lid will become worse. Let’s see some examples.
The first example is a large gate. I took this photo walking across a footbridge. I wasn’t able to reproduce the sky from the photo, but the gate looks rather solid. I took the photo while walking sideways, so the text on the gate got created nicely.
The second is a shop in a street. The upper parts such as the billboard was not created since the photos lacked information, but you can feel the atmosphere of the shop.
Of course, as your target becomes larger, there will be more disturbance elements, resulting in failures. Let’s see some failed examples.
Failure 4：The wall becomes distorted
This is a photogrammetry of the front of a building. As you see, it looks like a ruin. If you look at the original picture, the sky is clear blue, and the wall is single colored, resulting in extremely few focus points. Another problem is that in the photo there is glass that changes color according to the angle, which is a public enemy to photogrammetry. If the wall is affected by whiteout due to good weather, you could lower the camera exposure (EV), or quicken the shutter speed to prevent whiteout from occurring. You can also see that there is a large hole in the roof. I focused on taking the whole picture, so there are sides that are not visible in the pictures such as under the eaves. If you want to reconstruct the whole building, we need to change our policy. Architectures have many surfaces, and you should think that you will need at least 8 pictures for each surface.
I made a summary of what we learned about the 8 picture policy photogrammetry.
- It can capture 1 side of the photogrammetry target
- We only take 8 photos, so the photographing doesn’t take time
- We only take 8 photos, so the modelling does not need massive computer power
- Only 1 side is created, so the texture size and polygon number will keep low
- The photos are limited, so it is easy to find out the reason of failures
- We can try and create a model frequently and easily, so you will learn about the photogrammetry software quickly
By the way, in games or animations, creators will not try to construct a manhole lid from 8 photographs and 5000 polygons. Of course not, it’s just a ordinary manhole lid. You might feel, isn’t a manhole lid just good enough with texture? If so, the manhole lid may not be suited for you. Please try to find a object that you feel this must be perfect for photogrammetry! Find an object that you feel satisfied with, image what policy you are going to take, and take the photographs. If you fail in the alignment step, renew your policy. And again. If you focus on how many surfaces you need to create from the target, you will gain hints for how many photographs you will need.
How was this tutorial for photogrammetring with 8 photos? I hope you realized how you can photogrammetry without using too much time, too much money, and can understand the process. All you need to do now is start photogrammetring!
If you consider using a photogrammetry model in STYLY, you will probably be curious if there will be unneeded backgrounds or garbage data included, and if the whole model is fit enough. There may be, but you can edit the created model using the photogrammetry tool. In my next article, I will introduce how you can edit the model and make it look even better.