Hello there,
I am trying RealityCapture(RC) as I want to use its SDK for a web/phone-based application. Until now I have been using Polycam and it has been great. But it is only for iOS and they don’t provide with an SDK. Also, for what I understand, seems like RC should have a very similar or even better performance than Polycam. I am still quite noobie on RC and I have been struggling to get the same results. However, I am getting the next problem:
As I have different models of people I have taken pictures of, I upload these pictures into RC and execute the process to Align Images and then Create Model. For most of my models, this works well:
Using the this alignment settings, RC is capable of recognizing and align the images taken.
Up to here, all is good. However, I have this one model that I can’t manage to make it work. And It works in Polycam:
But I can’t manage to make it work in RC. Which is weird, because I am using the same settings that I use on the other models…
Can anybody help me with this? I am quite lost… My only theory here is that RC doesn’t work well with dark colors (If I remove the pictures used for the current alignment, then others take over. But RC never creates a full-body complete model)
1 Like
Hi, are you using images or video? It seems that the images are not taken in one moment, so it could be problematic for RC to align images, as persons could move during capturing. in cases like this are mostly used camera rigs around the person which take all images in the same time. How many images were captured in your second try? What is the background behind the person? In Polycam are used same images as in RC?
Hello @OndrejTrhan ,
I am using images. For both models, I used Polycam. They have a “video” feature in which you press the recording button and the system takes pictures one after the other (so not a real video). I took the model of the girl with Polycam’s “video” feature. After recording, they show the pictures taken during this process. My process here was to download these images and try it in RC. For this, Polycam allows for RAW image download.
There is no plain background. The photos were taking in a room with things around. Even Polycam tryies to reproduce these objects with no success. There are 202 images for the model of the girl and 136 for the model of the boy. Both models are successfully reproduced by Polycam. RC can only reproduce the male model. Pictures are the same for RC and Polycam.
Would it help if I try to manually change the date and time of the files? Please let me know if you have further questions.
Hi PabloGCBCN, is it possible to share those data with us for some tests? I think changing time and date won’t have influence on the process. Do those images have also georeference information?
We could arrange a meeting in discord and work together from my pc, but I can’t give you the dataset because I cannot break the privacy of the model by disclosing all her images. I don’t know if you dispose of any legal document that your company can do to ensure the confidentiality of these images. If you do, I could speak to the model and see if she agrees to deliver you her images under this confidentiality agreement. Please let me know if any of these options could work for you.
Hi, I noticed that you have set Image downscale factor in your alignment setting to 2. Can you change it to 1? Also, you can try to increase the max features per mpx to 30 000, max features per image to 60 000 and preselector features to 30 000.
Also, what could help here is using some CPs placed behind/around the model, which could be used in the alignment process and also for scaling.
Hello there,
I tried with the settings you recommended.
The results were better. However, still not acceptable or even near to Polycam results>
In a real case scenario, I woulnd’t be able to use CPs. We are trying to come up with our own app. The idea would be to have a mobile platform for scanning, like Polycam, and add our own developed features to it. With Polycam, you just take the pictures and the software is capable of generate something acceptable, without CPs. So right now I am trying to implement the same interaction, that’s why I am trying to avoid CPs.
How many images are taken? Polycam can use also other sensors to align the images, but that is just my guessing. Also, are the images captured in regular grid around model?
Hi there,
I have used the Lidar sensor in Polycam. That’s the only “Special” sensor that I am aware of. But I did not use it for the female model that I am having problems with.
Images are taken following the next pattern:
I forgot to answer the amount of pictures. I made 155 pictures for this model
155 images should be enough. What is not ideal for RC is the pattern of taking images, as it should follow the shape of the captured object and also it would be good to take images in circles, not spiral.
Also, as the images are not taken in same time it could be problematic. In Polycam could be also used gyroscopes, which could help in the alignment with known position (?it is the same in RS). After import images into RC are you able to see the georeferencing sign? Also, the clothes are not ideal for RC, as they are black and grey. It is possible that RC is not able to found enough features for alignment.
Hello there,
Sorry for the delay with my answer. I am seing that the .heif files exported from polycam seem to contain metadata.
Most of this numbers don’t mean anything to me. However, I would be interested in knowing what data can be used by RC to improve the aligment of the pictures. Does it help if I introduce a file with metadata containing the gyroscope data? Would georeferencing help to scan a single person? Are there any other sensors coming from a phone/tablet that you think could be used to improve this scanning?
Thanks for your time and help 
Hi, it doesn’t seem that there are some position or orientation information. But I can be wrong.
Position and orientation information could be helpful, but it is possible they won’t be so precise from the phone.
I am not aware of other sensors from iPhone which could use RC.
Have you also improved your capturing way?
Hello there,
Sorry for my late reply once again, I am retaking this thread again. The capturing here feels irrelevant to me. Not because it is not important, but because Polycam is capable to generate a proper model with these same images. Right now my objective is trying to reach the same result using Reality Capture.
I also went though the CLI commands that I can send to the IDE, but I can’t seem to find out how to tell the SW that my images contain additional metadata. Is there a way I could do this? My objective: Receive a bunch of pictures through my server, call Reality Capture, and return a 3D model in FBX format.
Hi Pablo,
the capturing is relevant here, as it depends for RC how the images were captured. Also, what is the overlap and image orientation.
I understand that other app was able to create a model from the data, but those data were captured by that application. As I wrote before, there can be used some information which are not able to use in RC.
Also, it would be nice to see the images and test them (if I remember there were some restrictions for it).
All needed metadata should be readable from image files. Which metadata do you want to use? Aren’t the information loaded in RC?
