Deprojection fails within the Panini Projection

I’m developing an application that requires wide fields of view. Here is a snapshot of the scene at 140 Horizontal FOV:


As shown in the image, I’m able to click in the image and use the mouse X/Y to deproject the screen coordinates and spawn a sphere in the world at that location (red dot at cursor). In the default projection (Rectilinear?) the deproject (DeprojectScreenPositionToWorld) works well.

However, as you can see the wide fields of view result in large distortion at the boundaries so we are using the Panini Projection (r.Upscale.Panini.D=1). Here is the same scene at 140 HFoV with Panini


The distortion is virtually eliminated but as shown in the image, when I click in the image and spawn the sphere the deprojection doesn’t seem to be reflecting the change in projection (red dot not at cursor). Has anyone witnessed this? The DeprojectScreenPositionToWorld method has no reference to the panini console variable (although I’m not certain it should). The method relies on the projection data that was generated and stored in the SceneView but its not clear if that data reflects the panini post processing or not.

I’m not sure how to proceed here. Its not clear to me how/where this should be fixed. I’d be willing to hack the DeprojectScreenPositionToWorld method to include panini support if that is appropriate but I’m not yet convinced that is the right place (I’m also not certain how to fix). As I understand it, the panini projection is just one way the projection can be altered via post processing. Seems a more general solution would allow post processing effects to wire into the deprojection. Please advise.