How to do fisheye lens in UE4?

I REEAALY want fisheye lens/spherical view, at least to an extent, I mean I don’t need a full on spherical view but I do need curvature on straight edges based on the PoV.
When I search for this, I get a video of someone testing an effect in UDK, or posts where either no one answered or their suggestion is to use an image and do a domain material or whatever thing and see if that works but no one actually showing/posting a definite way to do it without having to tinker to find a solution themselves.
In Unity5, there is a provided script that you can attach to your camera do fisheye lens; the script is small which means the effect isn’t too complicated to do, and it’s as easy as Drag&Dropping into the camera.

So, can someone tell me a simple way to do fisheye lens in UE4?

bump to front

Have you tried increasing the camera’s FoV to ~120 or so?

HTH

Is that sarcasm and a joke, or are you actually suggesting that?
Because if you’re serious then search for images of “fisheye lens” to get an idea of what I’m looking for.

2 Likes

No sarcasm, I know what a fisheye lens is, Just the only way I can think of to get that look with one camera is to increase the FoV to an extreme.
The only alternative is to setup multiple camera’s and then blend them together.

HTH

I have not seen any finished solution so I guess you have to create PostProcessing material manually and add it as a blendable in PostProcessing.

I think this tutorial might help to start working in right direction

That led to a problem that another user was having here which prompted another user to post at the bottom of a simpler setup to achieve very similar results.

However a problem persists with both, the second opting to just cover the biggest problem with black, which is that it creates a sphere/oval distortion area but leaves the outer area of that shape with blurred colors of the shape’s edges to the end of the viewport. My other problem is that no matter which settings I change, the distortion field is off; by that I mean it’s suppose to work from the center of the screen and pinch it towards/from a circle but the problem is the center isn’t exactly the center of my viewport.
I’d expect the point directly between the top left and bottom right of the 3d view box to be the center, but the effect treats a different area as the center, like the bottom-right point, or a bit towards the right from the middle.

The solution for both of these that I need is to make these filters work from the center of my viewport, and extend the resulting distortion to have the edge(s) of the the sphere/oval be outside the viewport area.

  1. Bigger FoV
  2. Some refraction (Don’t overdo it)
  3. Add an overlay on the edges or add some vingette

I’m hoping that you actually tried that and aren’t just making a random suggestion for me to go test.
That IS suppose to be a complete solution on it’s own (mostly) and not an addition/fix for the solution I currently have, right?

If it’s suppose to work as is, how to add refraction to a post-process material-function?

If that’s suppose to fix my current material, what is any of that suppose to do?
I said the view is off center and I didn’t want to simply cover up the problem area.

You could do refraction as a custom algorithm. Messing around with Screen Position and Screen Color to achieve some good results , combined with the Fresnel node.

Is that a set of things I can drag and drop or are you just suggesting a theory that “might” work and not something you already did?

I would like to know how to do this for game development purposes. I saw it in cancelled Sonic game from 1997 in December named Sonic X-Treme(on Sega Saturn)(it was also used for a first time in Super Mario Odyssey on Nintendo Switch).

It’s not quite fisheye as it’s more cylindrical than spherical, but UE4 does now support Panini Projection out of the box thanks to the UT project (R.I.P.). Nothing is really required to enable it other than running the command on that page.

If you need something much more closer to a real fisheye, and you’re much smarter than I am, then someone has an opensource project on github called Blinky that adds a bunch of different projection methods to Quake. The author has also put together some very interesting visualizations about how projection works as well as done a presenation (part 1, part 2) on it with ANOTHER developer who seems to have livestreamed much of his effort implementing the effect in Super Mario 64. It’s also been partially implemented by someone in Godot.

Unfortunately it is very expensive, requiring rendering from anywhere from 3-6 different views depending on the projection mode you choose. (maybe 5 if you only want fisheye) and probably more unfortunately, there are noticeable seams between the projections due to certain effects, especially camera facing billboards and screenspace effects. Sadly I don’t know of any better way to accomplish this.

After my last post I got to thinking and came up with a really hacky way to do this, I basically just spherified a cube, chopped it up into 5 sides, and assigned a render target to each one. Then I attached it to the player camera (so the camera is basically just looking at a spherified cube with a bunch of render targets on it). Worked better than expected.

You can get away with as few as 3 render targets if you don’t need the distortion to be particularly intense.

2 Likes

How did you do you do that? Also is lens disortion and fisheye lens the same thing? How to make Classic Sonic spindashing by holding Ctrl keyboard button because he just keep standing still,but he’s walking/running normally anyway.

So what if you could make it more spherical? Anyway which fisheye lens Unreal Engine 4 tutorial should I follow in Google searchbar?

Hi Arkiras,

how did you spherize the cube? While the process of doing so in a 3d app is pretty simple, the “problem” is the smoothing in my opinion. It depends on the amount of geometry, UNLESS one is using not a mesh, but a perfect primitive. So, which way did you go? And how does it work in UE4?

Gamerz31w: You shouldn’t follow ANY of the tutorials out there. NONE I’ve found so far do it correctly. Spent a few hours yesterday to do it correctly as a post process. It works “brilliantly” in terms of being a correct fisheye lens, BUT since it’s a post process it pixelates the center of the image horribly so that it is visually not pleasing.

The stuff you see out there with using a gradient, falloff, etc. is hocus pocus. It distorts the image. But not in the slightest way like a fisheye would do. And all these tutorials don’t care about image ratio when deforming the UV space for distortion. As a photographer all these approaches make your eyes bleed. Tip: Vector length and Tangent are your friends, as is screen resolution (for the ratio) and screen space. Keep in mind the f… up way UE4 handles angles. You need to divide the actual angle you want to use by 360° in order to make it work correctly (Mathematical eyes are bleeding, too).

So the only proper way to do so is to record the image correctly, like Arkiras did. Though I doubt that his method works with post process settings, or does it?

Please share you thoughts! : )

Cheers
Tom

Indeed this can be an issue, but the solution is obvious: Just use more geometry. That’s what I did and I find it doesn’t actually require as much geo as I initially thought it would, the mesh I used in that video has 512 triangles per side.

I planned on doing a tutorial for setting this all up, I just haven’t gotten to it.

Really depends on what you’re doing. Auto exposure works fine for example, so does most color grading. You just need to make sure your render targets are set to output HDR and that you’re doing the post effect in the main camera, not the render targets.

Anything more advanced though and you’re going to probably have a rough time getting it to work. The primary problem is that the main camera doesn’t have access to the gbuffers of the render targets, I suspect you could set it up so that you could get them but I’m not sure how and haven’t really tried to invest any effort into figuring it out.

Most (all?) built-in screenspace effects will fail to some degree for this reason and I don’t know of any way to fix this without modifying the engine. The degree at which they fail depends on the effect, for example exponential heightfog will work fine, volumetric fog will work “okay” (at strong densities you get seams from the temporal reprojection) and screenspace reflections just won’t ever look even remotely correct at all.

It’s a bit strange that they haven’t included this natively in UE4, next to Ortho and people need to build even stranger hacks to get close to a fish eye. Not sure why this is so difficult to implement.

I imagine the issue is just making it function with the rest of the engine and/or in a performant manner. Perhaps there will be renewed interest in it now that Epic is pushing into virtual production and cinematic vfx.