FSlateApplication::TakeScreenshot and -NullRHI?

Development Programming & Scripting
1

Dear community,

The context

We are a Machine Learning research lab using UE4 to generate a database of simple physical scenes. The objective is to train AI algorithms to learn the physical laws only by observation, as infants do.

For each frame of a scene we capture a screenshot, as well as depth and object masks, as PNG images. Here is an example of such a scene :

screenshot
depth
masks

The code works pretty well, and we now need to scale up to the industrial level. Indeed we need to generate many thousands of scenes. The idea is to run multiple instances of the game in parallel on a headless cluster by SSH. We are not interested by the realtime rendering, only by the captured PNGs.

The question

The screenshots are taken using FSlateApplication::TakeScreenshot (see our code here). But this is not working anymore when the game is launched with the -NullRHI option.

  • Are NullRHI and FSlateApplication::TakeScreenshot incompatible?
  • Is there another way than -NullRHI to launch the game without rendering it on screen?
  • Is there a better way than FSlateApplication to take a screenshot of the scene, that would work with -NullRHI?

Many thanks,

Mathieu

2

This is late, but we are having the same issue and Google brought me here.

I kind of have a solution for Linux but not for Windows.

  • If your shell does not have the DISPLAY environment variable, UE4 will instead render the viewport to an off-screen buffer. No further configuration is required. See this answer.
    • We manage this in our project by deleting that variable when starting the UE4 subprocess from Python (code)
  • To prevent UE4 from rendering the viewport at all (even to an off screen buffer), we are essentially using the ShowFlag.Rendering console command. It’s a bit hackish, but it works well enough for us.

See our code here

Using both of these methods, we can not have a viewport or rendering, but can still render screenshots (using a different method for taking screenshots - we don’t take screenshots from the viewport’s perspective)