Ok, I started a clean project and just imported a few LSP files and one image. It seem that it happens once I add a photo that has the same control point as the one in an LSP. If I disable the control point the issue goes away. See photos below.
Aligned data after adding a control point to an LSP and one image. See missing points that the LSP with the CP is facing.
Here is the result after disabling the control point.
Here is the LSP as requested.