Pixel Tracking Methods: Optical Flow Methods

Development Rendering
22

hmmm I aimed my render target at some moving objects and the screen is black.

Does the deferred shader not have the GBuffer.velocity as the base pass pixel shader? In BasePassPixelShader.usf, there is a comment that says “2d velocity, includes camera an object motion” and then below it gives the following code:

#if WRITES_VELOCITY_TO_GBUFFER_USE_POS_INTERPOLATOR
	float2 Velocity = Calculate2DVelocity(BasePassInterpolants.VelocityScreenPosition, BasePassInterpolants.VelocityPrevScreenPosition);
#else
	float2 Velocity = Calculate2DVelocity(MaterialParameters.ScreenPosition, BasePassInterpolants.VelocityPrevScreenPosition);
#endif

	// Make sure not to touch 0,0 which is clear color
	GBuffer.Velocity = float4(EncodeVelocityToTexture(Velocity), 0, 0) * BasePassInterpolants.VelocityPrevScreenPosition.z;

If they are not the same velocities, is there a way to get access to GBuffer.Velocity in the base pass?

23

What is the whole code of your custom node right now?
Just trying to make sure that the scene texture as plugged in isn’t involved.

I am honestly not exactly sure at this point as it gets a bit outside of my coding experience, especially in terms of all the specifics involved here.

Brian did mention that they planned to re order some render passes in the future to make this sort of thing cleaner to properly expose but he seemed to think this should be possible now in the custom node as long as the project setting is enabled and scenetexture is hooked up somewhere in the material.

24

Code.PNG
Code.PNG707×325 20.9 KB

MaterialFloat2 UV = MaterialFloat2(ScreenAlignedPosition(Parameters.ScreenPosition).xy);
FScreenSpaceData ScreenSpaceData = GetScreenSpaceData(UV, false);
return ScreenSpaceData.GBuffer.Velocity.xy;

25

bump! GrohBot, did you manage to finally get access to the previous frame’s world position?

26

Bump X2! This would be really awesome to be able to do. :slight_smile:

27

Bump X3 - I’d be really interested in being able to do this…

28

bump x4 - also something I’m after

29

Also interested.

I looked over the UE4 code (DeferredShadingCommon.usf, BasePassPixelShader.usf) and the GBuffer.Velocity should be there, but accessing it on the material from a custom node always returns a black image. Accessing other buffers like Metallic and WorldNormal work as expected from any custom node.
I have blur set to 1 on my postprocess volume, and “Accurate velocities from vertex deformation” enabled on project settings (Rendering section, optimizations).

Working example:


MaterialFloat2 UV = MaterialFloat2(ScreenAlignedPosition(Parameters.ScreenPosition).xy);
FGBufferData GBData = GetGBufferData(UV, false);
return GBData.Metallic;

Does not work:


MaterialFloat2 UV = MaterialFloat2(ScreenAlignedPosition(Parameters.ScreenPosition).xy);
FGBufferData GBData = GetGBufferData(UV, false);
return GBData.Velocity;

Setup is the same as GrohBot’s.

So, the Velocity GBuffer is not being written to. Any ideas on why is that happening? I see forcing it should be easy by modifying some lines, but client uses launcher build. Thank you.

30

I am also interests in extrcat motion vector. but I don’t know how.
I search for ue4 source code for gbuffer velocity.
Hope it can help you guys.

ue4_Velocity.png
ue4_Velocity.png1198×527 51.8 KB

31

visulize velocity

ue4_velocity.png
ue4_velocity.png1032×569 510 KB

I can use this cmd to visulize velocity, Is it help a bit?

Cmd: VisualizeTexture velocity

32

That yellowish color means 0.5 red, 0.5 green (0 in both x and y).

Does the color change when blur is visible in the scene?
Also, does the material node setup posted before work for you? It should show the shame velocity image as you see with that command.

33

That might be a bit of help I guess, before the command “VisualizeTexture GBufferVelocity RGB UV0” showed the motion vectors, so I guess now it would be “VisualizeTexture velocity RGB UV0”, that should make it fullscreen, modify the code of the visualizetexture command and remove the info text in the center of the screen. But still, I don’t think you can export this pass in high bitdepth so its kinda useless.

Oh also, when I did “VisualizeTexture GBufferVelocity RGB UV0” the texture resolution was like 1280x600 so it didn’t work with every aspect ratio, and when having different FOV it would be also mismatched.

34

Yes, of course. this is color changes while obj moves per-pixel.

I have try your custom code in shader, but shows error. how could you do that?

pp_Velocity_Error.png
pp_Velocity_Error.png1576×804 203 KB

35

Try maybe with PostProcess0 instead of SceneColor in your SceneTexture bloc

36

I tried that but when I enter the PP volume, my whole unreal engine crashes. If i set the return GBData.Velocity; then the screen is just black image.

37

Try with GrohBot’s code:



MaterialFloat2 UV = MaterialFloat2(ScreenAlignedPosition(Parameters.ScreenPosition).xy);
FScreenSpaceData ScreenSpaceData = GetScreenSpaceData(UV, false);
return ScreenSpaceData.GBuffer.Velocity.xy;
38

Same Here, Just black, no matter mul a larger number or normalize it, just black.

39

Could some Epic dev enlighten us? Is the velocity exposed through the GBuffer? Where should we be looking at if we wanted to expose velocity to the GBuffer, and, ultimately, to a render target?

40

I found some codes might relate with.
Please found and open DeferredShadingCommon.usf at \Engine\Shaders\

in line 344：

// Velocity for motion blur (only used when WRITES_VELOCITY_TO_GBUFFER is enabled)
float4 Velocity;

in line 448:

#if WRITES_VELOCITY_TO_GBUFFER
OutGBufferVelocity = GBuffer.Velocity;
#else
OutGBufferVelocity = 0;
#endif

So where is this WRITES_VELOCITY_TO_GBUFFER? how to enable it?

And,there’s more Gbuffer.velocity in these usf shader files.

Hope these can helps a little bit more.

41

I found Definition.usf

line 154:
#define GBUFFER_HAS_VELOCITY 0

Seems epic disable in purpose.
So I change is 0 value to 1.
and still waiting UE compile all shaders.
maybe it’s hopeful.
Than I use the GrohBot’s code, but still gets black.