[New plugin] Shader debugging made easy with RenderDoc for UE4.

Hello everyone!

I thought I’d share what I’ve been working on for the last couple of weeks.
One thing I learned from my tag buffer adventure was that debugging certain parts of UE4 can really be a pain, and one notable area is the rendering pipeline and shaders.
All tools I tried either I could not get to work (RenderDoc) or they just crashed (nSight & VS GPU debugging tools). As a pleasant surprise though, the main developer of
RenderDoc, BaldurK (baldurk (Baldur Karlsson) · GitHub BIG PROPS to this guy!!) contacted me, curious of why I couldn’t get it to work. With his help, I managed to sort out what was wrong and made a wiki entry to explain how others could get it to work
using the same techniques. I’m a big fan of renderdoc, especially since it went open source, and as it was kind of awkward setting it up with UE4, plus there were some things that didn’t mesh
that well with the editor, I decided to try to make it easy for everyone by making a set of plugins that integrates renderdoc with UE4 in a more user friendly manner.

I say a set of plugins since it is actually two of them. The first one is simply a loader that injects the proper DLL as early as possible to be able to catch device creation etc.
This would not have been needed if the same plugin could get callbacks for all different load stages, but this is currently, as far as I know, not supported.

The plugin has a few features:

  • It detects your current installation of renderdoc, so you are not bound to the version I use, and can update at will as long as the API is compatible.
    If you don’t have an installation, and have just downloaded a zip, built it yourself etc. It will prompt for a path. Or you could set the path yourself in your project game.ini under [RenderDoc] BinaryPath=PATHGOESHERE

  • It adds capture buttons to all viewports of the editor. This means you no longer have to cycle swap chains to get to the element you want. When you press the capture button (or its hotkey) it will capture the window the viewport
    is attached to, and then launch renderdoc with that capture as argument. It also opens a connection to renderdoc so you can perform subsequent captures etc.

  • I’ve added a small options menu button next to the capture button. This small window lets you set several properties that control how the plugin operates:

The settings are:

Capture callstacks

Save the call stack for every draw event in addition to the event itself. This is useful when you need additional information to solve your particular problem.

Capture all resources:

Capture all resources, including those that are not referenced by the current frame.

Save all initial states

Save the initial status of all resources, even if we think that they will be overwritten in this frame.

In addition, when UE4 compiles it’s shaders, it strips them of all of their symbols and other meta-data, presumably to make them more compact.
This makes them very hard to debug though as you can no longer see the variable names, attributes etc.
I’m working on a pull request that adds support to optionally not strip the data by sending a flag to the compiler. I don’t want to make this the default behavior though as that might break some peoples project (deploy) that is why it will default to off, but I highly recommend using this when debugging.

I am not currently done with this feature, but you can follow my progress on the AutoCompileShaders branch of the repository.

After a capture is complete, the plugin simply launches the RenderDoc UI. After this point, it is just like any other RenderDoc debugging session. If you are new to RenderDoc, I would suggest checking out these video tutorials:
RenderDoc - An overview - YouTube

To install the plugin, simply clone this git-repo to your project/plugins/ folder:
GitHub - Temaran/UE4RenderDocPlugin: A plugin that adds easy access to Renderdoc inside UE4</title

You can also add the plugin as an engine plugin by adding it to your engine plugin folder, although I recommend using the project plugin route as I feel it is cleaner.

The plugin also requires recent code from RenderDoc’s github (GitHub - baldurk/renderdoc: RenderDoc is a stand-alone graphics debugging tool.) which isn’t in the latest stable release.
You can find a beta release here: http://renderdoc.org/builds or build it yourself from source if you prefer.

2 Likes

I completed the feature that prevents shader meta-data stripping.
I put it on a branch in the main repo:

I made a pull request for it here:
https://github.com/EpicGames/UnrealEngine/pull/457

But until then if you’d like to try it you can get the fork/branch here:
https://github.com/Temaran/UnrealEngine/tree/RenderDocPluginChanges

I hope someone other than me finds this useful :smiley:

Best regards,
/Temaran

1 Like

Excellent work, Temaran!

Having a graphics debugger such as RenderDoc semi-attached to the UE4 Editor is a life-saver for anyone trying to hack/mod the core of the rendering subsystem.

I would just like to add that RenderDoc 0.21 beta (or any of the more recent nightly builds) are necessary for the plugin to work properly.
The current stable version (v0.20) won’t work since it exports a different API (e.g., there is no ‘RENDERDOC_GetAPIVersion’).

Once again, kudos to you!
Fantastic job.

Actually, forget my previous remark; I just read it through your initial post more carefully and noticed that you already pointed the necessity for the beta/versioned build. =)

Thats insanely helpful! Thanks a lot man :slight_smile:

Awesome to hear that other people think it’s useful too :smiley:

/Temaran

Hey man,

we just integrated your Plugin into our Build and it works like a charm. However, we found a bug caused by the Plugin.
When you start and uncoocked game, the plugin crashes the Slate UI, so you cant start a game anymore.

We fixed it which looks like this:

I also have a .diff file for you with the changes. Would be cool if you can update your Plugin on GitHub :slight_smile:

This looks brilliant! Good idea.

Hello everyone!

I noticed that 4.6 is out, so I updated the code to work with the new version :slight_smile:
The changes are in the r/UE4.6_v0.1.

Special thanks to Daedalus51 for noticing the bug! I’ve included those changes into the new branch as well.

Best regards,
Temaran

Hi Temaran,

Did you made renderdoc works with usf shader?
How did you made renderdoc display wireframe mesh just like explained on RenderDoc video tutorial?

I couldn’t seem to make renderdoc works like the video, it only displays the final texture without information of the vertices, only attributes, not position and normals, is this expected?

I’m using UE 4.5 with renderdoc plugin from r/0.1 and renderdoc 0.21

Thanks for the plugin anyway!

My trouble is that there are assemble codes but .usf code or hlsl code.Are there someone know how to get debug info form ue4.sln and tools?

Hello,

I can’t open it on the 4.18. Is it me or it’s not working in the new version ?

Thx !

from the git page
ANNOUNCEMENT: At the start of the year, I created a pull request to make this plugin a part of the engine, and looking at the 4.17 preview, it seems to be merged. I might still work on experimental stuff in this repo, but it is of course easier to just use the integrated plugin when 4.17 is released if they decide to integrate it into the release. :slight_smile:

/Temaran

its a built in plugin now, just enable it from plugin list in the editor

Hello Ixicalibur

Thank you for your answer. I try to enable it in the 4.18 version but nothig append :confused:
I have no additionnal button in the top right viewport

Thank you :slight_smile:

not sure why its not working for you, but just to confirm, it does work in 4.18.3

Hi Ixicalibur,

After enabling the plugin and restarting the editor I have no additional button in the top right viewport either (4.18.3 and 4.19)

Thx

not sure how to help you, all i did was install renderdoc application, restart pc, load ue4 enable plugin and the restart ue4. i did it just to see if it worked, i’ve never used it before. im using a compiled version of ue4 with gameworks in it, not the launcher one.

Make sure to download the app from here: https://renderdoc.org/

Once it’s installed you need to point UE4 to the executable location, in the project settings. You can search for Render Doc in the project settings to find where to plug in that path.

I’m not sure how it used to work, but I had to do this process to get it running in 4.18 and later.