Hello, Benjamin,
First, I have to say that I really feel for your plight. It’s very painful – excruciating even – to find you’re unable to get your work done for reasons that really aren’t your fault. I’ve worked in visual effects for film for many years and there’s a feeling in the TD community that you spend your time ‘tricking software into working’. That’s no fun.
Since you’re on deadline with this, I know your time is short. Please humor me by reading my capsule discussion first, not just my concrete suggestions after. Götz – you and I have talked about this before, so you can feel free to skip ahead. 
The real root of your problem, I think, is that RC is not allowing you to find high quality points in the feature search state. RC obviously puts great emphasis on speed – to the point that they don’t allow you to set the feature search parameters at all. Whether this is a good marketing move for them I won’t expand on here (I think it’s dubious, actually – I’m biased).
The result is that RC feature search fails under challenging situations. Wishgranter can rely on the old trope “just shoot it right and you’ll have no problems”, but you and I understand that in difficult capture situations like your cave or my Egyptology locations, you often have very difficult access and no possibility for reshooting.
As such, RC is particularly brittle when it comes to scenes without true lighting consistency, and I think your input photo set falls into that group. Because of that, RC can’t establish tight enough correspondences to “deliver this scene in its entirety into a common scale and coordinate system”.
This is even more true if you’re using manual control points. I think the user has no business competing with SIFT anyhow – even with good ‘subpixel’ point selection you end up introducing noise error even if you can supply tie points the system couldn’t compute.
My practical suggestion for getting a result with the least effort is:
- Create models for both side A and B separately, if there’s any overlap between the two. Export high resolution meshes for both sides, and then align the two via scaling ICP in CloudCompare, MeshLab, GeoMagic, et cetera. Since the two models will be in two separate coordinate systems, you need to use SICP, e.g.: assuming CloudCompare for now, turn on scaling here: http://www.cloudcompare.org/doc/wiki/index.php?title=ICP