MediaTexture doesn't work with RGBtoHSV or world displacement

We are trying to use a media texture MP4 file to drive a media texture with animated world displacement.

The way we used to do it in 4.16 is have a split texture, with RGB on one side and HSV color gradient in another, this way we could split the depth into 360 steps of hue, and use the V channel for a 2bit mask.

In the latest preview (4) the RGBtoHSV node errors out with “(Node If) If input A must be of type float.” and won’t compile.

Even if we drop the RGBtoHSV and just grab a single color channel and pipe that directly into world displacement. Nothing happens, the tessellation is active in wire view but no displacement occurs.

Does that mean the change to having to use sampler type: External kills being able to use mediatextures for anything other than emissive and base color now?

This totally kills our project.

The problem is that the sampler’s white pin outputs a float4, but the RgbToHsv function expects a float3. One simple way to fix this is to insert a MakeFloat3 node and wire it up to the RGB components of the sampler.

No change in error message

Did you recompile the BP?

Yes.

The input video is an 8bit h264 1080p video @ 30 fps

I cannot repro you problem in 4.18 (final release). Perhaps something is different in the Preview version.

This is will final release.

Also you are plugging it into the emissive, we are using the output of the RGBtoHSV in world displacement.

I just made a clean 4.18 project and set up this material. It works fine plugged into base color or emissive, but gives the error on world displacement

here is it working fine if plugged into emissive.

I was able to repro WorldPositionOffset not working. This seems to be a shader compiler bug. I will reassign this ticket to the Tools Team.

Is there a publicly trackable ticket? We really want to get from 4.16 to 4.18 but this bug is holding us back at the moment.

I added UE-51763, which should be accessible at Unreal Engine Issues and Bug Tracker (UE-51762) soon

This should be fixed in CL# 3733463. The problem was that the media texture only supports per-pixel attributes, but not does not support per-vertex attributes, which is a requirement for WorldPositionOffset to work.

Our external texture implementation did not return a correct error code, so the shader compiler spit out a completely misleading error message instead. The fix adds appropriate error messages to external textures. Please note that this will not make WorldPositionOffset work with media textures, but only that an appropriate error message will be printed by the shader compiler.

I will merge this change into 4.18 Hotfix 1.

Is there any plans to allow media textures to work woth worldpositionoffset again or allow the use of the other texture sampler modes?

We experience an similar problem here (4.18.1):

Material Function for dissolve effect:

https://i.imgur.com/0xxvpN6.png

And the Material BP:

https://i.imgur.com/JxLkNyU.png

When we create the Material BP it works fine but once we add some UV changing node (rotation for mask for example) this error occurs. Any idea?

It’s currently tracked as a critical bug for 4.19 in UE-52612, but I’m not sure if a fix will make it in.

External texture samplers do not currently work with materials that require vertex shaders. It’s currently tracked as a critical bug (UE-52612).

I cant seem to find that bug on the tracker, was it taken down?

Sorry, there was a typo in my link. I updated it. Your link is the correct one, thanks!

I hope the regression is fixed in 4.19.

6-DOF 360 video is starting to become viable with cameras like the Kandao Obsidian outputting 360 depth maps. Being able to drive vertex shaders with h264/h265 video makes it viable. Hopefully when oculus go/gearvr finally gets on the vulkan train it can work on mobile as well.

example (built in unity Kandao Obsidian 6DoF Demo - YouTube)

I looking at the 4.19 preview it looks like it’s still broken. I guess we are transitioning to unity