Relief carvings etc, as Shadowtails example are indeed impressive works of art, and the software does a fantastic job of extracting the depth from them. However I don’t think it shows the true ability of the software, and could give the impression it is about 2D depth mapping and not full 3D scene/object recreation.
The end result needs to be technologically impressive as well as visually complex and interesting.
Using someone else’s work is definitely no go from the start.
We definitely will be able to create our own data.
To ensure the software is not being held up with strange issues, knowing the conditions of how the data was captured is important. Full Exif data etc.
Finding a versatile and impressive subject/s that will capture the essence of what the software is capable of shouldn’t be too hard with a bit of brainstorming. I do not believe a small dataset of say 36 images will be enough to stress systems realistically to gather the data we require for a benchmark. Nor will produce a adequately high quality result to represent what is possible. It is very impressive what can be done with a few images, it is even more impressive what can be done with a larger number taken carefully.
Gotz’s point of closed circles makes sense, and having a full scene would be nice.
With regards to over complicating :) It is indeed a good idea to be able to walk before you can run.
That said, I feel if a job is worth doing, it’s worth doing properly. If rushed and poorly thought out you don’t achieve what you set out too, and end up with sub par project that is lacking in many areas. A good balance is important. If you bite more than you can chew there is the risk things do not get completed.
I’ll have a ponder over the next few days. Keep the suggestions coming 