So, to any future people out there who want to use this idea, I just found the workaround I needed. The solution was to use 2D scene captures instead of cubemaps, and rotate the camera and the mesh based on the player’s relative location.
