Pixels

General Feedback & Requests
1

Why bother rendering in polygons at all… You could render pixels directly on the screen… By snapshotting 3d geometry per frame… Fake everything… For ultra realism… Who cares if its accurate??? One frame equals one snapshot… Apply fake post… Done…

I’m pretty sure faking it would be cheaper than realism… And look almost as good… Probably save a huge amount of frames…

And your applying the fake post to the pixels instead of polygons… Massive speedup…

Basically… Apply lighting and effects in 2d rather than 3d…

An algorithm that looks almost real versus an algorithm with massive compute that is realistic is definitely the better approach…

In other words you need to look at photography… And how 2d images appear real… Instead of trying to create things in 3d…

In other words… Snapshot your 3d…which is only data…draw your snapshot on the screen… And apply fake per pixel 2d based realistic lighting and post…

So in other words… 3d lighting and post effects go bye bye… And you do that stuff in 2d per frame…

2d lighting and post… Massive fps speed up versus per poly…

So you could get the shape of an object… And snapshot it… And apply lighting in 2d…

Or if you don’t want to do it that way… Have 2d lighting and post as an option in project settings for people on older cards or who want a massive speedup…

For 2d… All you have to do… Is get the light radius and intensity… And apply that to the snapshot silhoute… Easy peasy… Incredi speedup…

You guys are looking at things in 3d…because that’s the way games have gone… But at the end of the day it’s a 2d image… All you’ve got to do is separate the snapshot… Ito it’s separate parts and apply fake 2d lighting and post… If your doing that in 3d the performance cost is humongous versus doing it in 2d…

And its the same with characters and combat and particles… All of that is just data… All you need to do is correlate the data with the correct pixel and cut out the rendering process…so in other terms… You take the data directly to 2d pixel fake lighting shadows and post… And cut out the rendering stuff… Drawing directly to 2d
…

Have it as an option… 3d rendering versus 2d rendering…

I bet if you studied photography a bit you could have the 2d rendering almost as good as 3d rendering…

Doesn’t have to be realistic as long as it looks real… And a simple algorithm for lighting based on 2d scene depth versus realistic per poly lighting and shadows would be an insane speed up…

This would definitely work…

Everyone knows lighting and shadows are the biggest performance cost…

Calculate the lighting based on pixels in 2d versus per poly lighting… Its so obvious…

2

It already does that, that’s what post processing means, it’s done after it renders the 3D objects. Things like bloom work off the existing image to add a glow to the bright pixels in the image. UE4 also has screen space reflections, screen space GI, and screen space contact shadows which all work off the rendered image so that it can do those things faster although they aren’t going to be as accurate like you mentioned.

3

I know a lot of post process is done in the final image… What I’m talking about is using the same approach for lighting and shadows… As it stands… Light is calculated from a source and mapped onto objects in 3d during the rendering process… What I’m talking about is skipping that entirely and applying fake lighting on a per pixel basis instead… In 2d rather than in 3d…

In other words… You wouldn’t need light maps at all… Because the lighting is handled in 2d…

Like fake lighting a character in Photoshop… Versus raytracing a high poly…

4

That’s what screen space GI and contact shadows do. They only work off the image and not the scene. The problem is that you can’t do that for all of the lighting because things that affect the lighting will be off-screen. The new Lumen lighting system in UE5 actually uses screen space GI primarily but for the parts of the image that it can’t cover it uses a different GI calculation method that isn’t screen space.

5

If you know all the positions of the lights and their intensities you can calculate that information in 2d without light maps… You can totally skip that step…

6

It’s the difference between lighting a character in Photoshop versus raytracing a character in keyshot…

The Photoshop image looks amazing and is faked on a per pixel basis…

And the keyshot image takes ages to render on a per poly basis…

7

You could render most of the lighting that way if all of the 3D objects are within view, by just having the lighting positions. Any surfaces that aren’t visible like the underside of something or something in front of another object would cause problems though. There wouldn’t be a lot of games where that kind of situation would work though.

I don’t know what you mean by lighting a character in Photoshop. While you can do some limited 3D rendering in Photoshop it’s not faked, it uses the same kind of 3D rendering techniques that any other 3D tool uses, it’s not rendering realistic lighting in post.

8

Photoshop is a 2d application… And that’s besides the point… The point is… You can skip the middleman completely… And cut out light maps altogether… And lighting and shadows are the most expensive aspects of games in general… And that stuff can definitely in this modern day and age be faked on a 2d per pixel basis probably almost as accurately as Ray tracing with a humongous performance boost…

Its not done that way man… Why do you think we have light maps??? Its done on a per poly basis… Which has to take every vertex and normal into account… I’m talking about skipping that completely… And totally faking it with an algorithm that is based on per pixel lighting…

You could probably use machine learning to figure out how to light a 2d image realistically just as well as you could accurately light a 3d scene…

That’s the point…

Whoops missed what you were saying
…

Your still taking the data… You’ve got the 3d scene in data… You know where everything is… Even everything off screen… You are just using a per pixel lighting technique versus per poly…

9

Yes, Photoshop is a 2D software, but it has some 3D features: Essential 3D concepts and tools in Photoshop

You can’t do lighting the way you describe if objects are off-screen that need to cast shadows on things in the image. If the image would realistically be affected by anything that’s off-screen it wouldn’t have any way of doing that simply based on the pixels of the rendered image.

10

By the way, the existing ray tracing technique does trace from each pixel back to the light sources if that’s what you’re thinking.

11

You totally can do off screen objects… Your not listening to me… You have the data… You have all the light locations and all the object locations… You are just lighting it per pixel instead of per poly…

Let me put it to you like this…

Take a photo of your bedroom…

Would it be harder to calculate the light and shadows in the photo or the actual light and shadows in the room…

12

I know how raytracing works thanks very much… And what I’m talking about is totally different…

Basically you are lighting a 2d image with totally fake per pixel lighting…

Versus realistic real-time per poly rendered lighting…

Think about the performance gains???

13

Basically right now in the games industry…

Every model has a light map… Which is used to calculate the light in real time… Every polygon and vertex is taken into account as well as the distance from every light in the scene and the intensity and colour of the light… And they use an algorithm to determine how much influence that particular light has on that particular polygon… In real time…

This is totally different…

You snapshot the 3d scene (which is just data) then you correlate the 3d scene with screen pixels… Then you use an algorithm to light the 2d image accordingly on a per pixel basis… No light maps required… All those calculations per poly and vertex and light intensity are replaced by an algorithm that calculates light per pixel… There’s a lot less pixels on a screen than polygons… And they only have one value (rgb)

Basically… Just taking a screen grab of your 3d scene every frame and fake lighting it… Instead of doing millions of calculations to figure out how that light would actually look in 3d…

14

The existing rendering methods kind of do what you say, but it seems you think there’s some way to cut out part of it and I don’t think that’s the case. They’re already very optimized, and to get a fully dynamic lighting system there’s always a downside unless you’re using ray tracing.

Existing games right now do not always use lightmaps, like open world games often don’t because they need dynamic time of day, or they’re just too big and it would be a lot of data.

15

Man… All models have light maps… And it’s usually laid out as uv 0 or uv 1…and it’s been that way for a very long time… Because when the light is being calculated they need to know how the light is going to interact on a per poly basis… There’s even an option to layout your light maps in ue4…

That’s kind of the point…

You are literally cutting out the middleman…

You don’t render light in real time realistically hitting every polygon in the scene…

You render a snapshot with extremely cheap but almost as good lighting…

Performance boost bro…

16

What I’m talking about… Is kinda like using a real-time cubemap… Its totally fake lighting… But the difference is it works per pixel instead of per poly… So there’s very little performance cost…

17

You could literally in unreal engine code a post process material that did exactly what I’m talking about…

The problem is…

The underlying engine relies on calculating the light per poly…

18

No, not all models have lightmaps. For example in UE4 skeletal meshes and movable meshes do not use them because lightmaps are static. Most of the time the direct shadows are fully dynamic and aren’t baked into lightmaps either. Any game that uses a dynamic Time of Day system can’t use them at all since the lighting must be fully dynamic (like GTA V)

UE4 does have a probe based dynamic lighting system with Volumetric Lightmaps; after the static lighting is baked it can render a 3D grid of probes that sample the lighting throughout the scene that can then be used to provide faked indirect lighting for dynamic meshes like characters. But in that situation you still need the baked lighting in the first place.

19

Couldn’t you render the world as Unlit and apply your lighting in the Post Process Material?