Debugging USF (Unreal Shader Files)

[ambershee]

Hello!

I’m looking for information on how people go about debugging USF shaders - including when they are not working, but also if they are not compiling (where can one get information on why they are failing?).

Edit: I’ve found this document, which is a good start!

Shader Development

[ambershee]

Thanks - all handy information. Hopefully someone’ll have a decent idea of how to debug it with something like the VS tools.

[ambershee]

Thanks - that’s more or less what I’ve been up to, but having a decent tool to work with would be infinitely more helpful

[Drayan]

For my part, I was able to capture frame with RenderDoc and with Nvidia nSight. It work very well, both. (With a little preferences for Nvidia nSight, because it’s pluged in VS).

But I can’t have HLSL code out of my shaders. (I modified the flags of ShaderFormatD3D to have DEBUG_SHADERS 1, it should therefore compile shaders with debug info, but I got the shaders in ASM…)
If someone have a solution to have the debug symbols, it would be great !

[anonymous_user_69853c21]

Sorry, I don’t have a solution.
I’ve tried to capture debug data with pix and with renderdoc, but without success. Actually, with renderdoc I can run the editor lobby (where I need to select a project) and nothing more - on project select editor closes without any errors. Renderdoc logs have no error info too. So how do you using it?

Upd.: I packaged a game and now can run renderdoc with it. But I don’t know - packaging happens in release mode, so maybe where’s some data loss? Another sad side - in order to debug shader I need to build a project what takes 15 min on my machine. Oh, and no hlsl too, just asm, but it’s better than nothing.

[Drayan]

Hello,

Okay, maybe I’ll make a tutorial for this; I found a way to have HLSL code in nvidia nsight, with renderdoc, it doesn’t work (even if it should…).
Actually, UE4 strip debug information from the compiled shaders in the UE4 shader compression process. I found how to add debug information in the shader, but I didn’t see that the after the compilation, the shaders was compressed, and farewell debug informations.

I have another problem now, but it’s very probably because my hardware doesn’t allow it, more than a problem with nsight or UE4 -> I can’t make breakpoint in shaders. But well, I can have the output of UE4 (there’s actually two way to have them), and I can have some information in the pipeline with nsight. It’s enough for now, but I’ll probably check on the nvidia website if the breakpoint problem come from my hardware (I don’t even meet the UE4 minimum shame), or if the compilation process omit some informations.

EDIT :
I can confirm the problem come from my configuration, as stated on the nvidia website.

@NVidia

You cannot debug a shader with a local debugging configuration. You must configure your system for remote debugging to debug a shader.

So I’ll try later, if I need, to remote debug my UE4 project on another computer, from this one.

Anyway, if someone need it, you should now be able, with a remote configuration, to set breakpoint in a shaders.

PS : I’m not english speaker/writer. So, if you found some big mistakes, please, let me know.

[Drayan]

@DanielW;82571

I haven’t tried source in Nsight for a while, but some things you can try:

• Remove the call to D3DStripShader in D3D11ShaderCompiler.cpp. You have to recompile ShaderCompileWorker Win64 Development to propagate the change. And then add a space or something to common.usf to force shaders to recompile (if you are working on a global shader just edit it a bit).
• In UE4 we compile the shaders in a different app (ShaderCompileWorker) which allows making use of all your cores. However tools like NSight don’t know about this other app and they only hook the compile calls in the main executable. If you open BaseEngine.ini and set bAllowCompilingThroughWorkers=False, any future shaders will be compiled from the main executable. Of course, this will make compiling super slow.

Thanks,

That’s exactly what I’m doing, but I didn’t add a space in common.usf. This could explain why I only got my Vertex Factory source in nsight and not the others ?
And I can try for removing the compiling with worker. (Already did too, actually). But, as you said, it’s very slow. And (I guess) as I’m working on a VF, every times I make a change to my VF it need to recompile all the shaders (around 4000 shaders…).

With the worker, it takes something like 10 minutes on my computer. So without, it will be long…

In annex, sometimes my VF compile in less than a seconds (it just recompile the VF ?), sometimes it recompile the whole… I don’t really know why.

[Drayan]

Yes, that’s make sense, thanks for your input .

Actually, I WAS looking at Local VF. Until I understand what it make. But well, I’ll change my ShouldCache methods.

Thanks again !

[Temaran]

Hello!

I do not mean to necro this thread, but I thought my small tutorial on this topic might be useful for people who find this thread but are still wondering what tools can be used and how.
Here is the link:

A new, community-hosted Unreal Engine Wiki

I hope this will be useful to someone

Best regards,
Temaran

[MarkEntingh]

I’m new to developing shaders using USF, but I would say, try creating a shader first using shadertoy.com and translate the GLSL code over to HLSL once you are satisfied with the results that you created on shadertoy.