Surface object almost always have the issue that some parts of mesh(always in some type of square shape) lost all detail. By using the control point I found that those error area were in photos and aligned.
When I set DDF(detail decimation factor) to different value, the area of detail lacking move to another part of the mesh.
DDF:1
DDF:2
My reconstruction settings
My camera poses
I thought it was cause by maximal vertex count but it turns out in the same problem.