I still get questions occasionally so I’m writing this up hoping others will find it useful.
This is a high level explanation of how it works;
- We capture the scene from multiple angles to render targets (front, left, right, top, bottom)
- We display those render targets onto the sides of a cube that has been distorted into the shape of a sphere.
- We put a camera in the middle of the cube-sphere which is what the player actually sees out of
Here is the capture array:
Then you have a spherized cube which you apply the render targets to (textures here are for debug, in-game each side would be displaying a render target):
The camera in the center of the cube-sphere is what the player actually sees from. How you choose to actually set this up so that the player can see out of it is up to you. In my case, I made the cube-sphere a separate actor and then attached it to the player camera at runtime. So essentially the player was running around with the sphere-cube stuck to it. I don’t recommend this as the cube-sphere can clip through the level geometry, but it worked well enough at the time.
The position of the camera determines the “FOV”, the closer the camera is to the “front” side, the narrower the FOV will be.
Here’s how I modeled a “cube side” in Blender:
Some things you’ll want to keep in mind:
- You will encounter problems you need to figure out how to work around. Unreal was not designed to be used like this.
- Set your render targets to clamped. This will avoid seams from texture sampling.
- Do your exposure and color grading in the main camera, not the render targets.
- Most screenspace effects will have varying degrees of visible seams, there’s no way to avoid this as this is essentially just a collection of isolated render targets, the frame buffers are not merged so screenspace effects have no information from the other render targets. The worst example of this is screenspace reflections, which look extremely bad…
- When I tried this in UE5 there was extreme blurring every time the camera moved. I’m not sure if this is due to a change in some default settings or an actual issue with UE5. I suspect it’s either caused by the temporal super-resolution or motion blur, but I’m not sure.
- I wouldn’t expect this to work with Lumen with screen traces enabled. If you disable screen traces, it might work… maybe…
- Should go without saying but this method comes with a high performance cost. You can mitigate this somewhat by ensuring your render target resolution is set to something reasonably low.
- Again this should be obvious, but make sure your cube-sphere has enough subdivisions, the quality of the curvature is directly determined by the amount of geometry.