There is no a “correct” value, as each measurement could be different.
The camera priors have the higher priority as the constraints (but I not not aware on the values for those), so if the camera prior poses are not so precise, the constraints could be different to the original one. But you can try to set its accuracy to 0.
It really differ on the case and it is basically the playing with the values.
The best option is to avoid using camera priors at all. With properly measured GCPs the orientation of the model will be proper.
As you think, that the camera orientation is right, then you can set the accuracy to lower values and the hardness to bigger and opposite for position accuracy in the Alignment settings:
