With the Unreal Engine Movie Render Queue plugin, there are multiple options for command-line rendering content with Unreal Engine 4.27 and 5.x. The goal of this tutorial will introduce these concepts in an understandable and lightweight way. These commands can be applied to command batch scripts or any render manager command line job.
https://dev.epicgames.com/community/learning/tutorials/nZ2e/command-line-rendering-with-unreal-engine-movie-render-queue
1 Like
Please be aware that I’ve included a changelist to the tutorial so changes and revisions can be tracked.
I encountered a problem when outputting an EXR sequence frame using the command line. The object ID channel is not working properly.
Super useful, thank you very much! I ran into an issue worth mentioning - If you try to create a command line render as part of another process - in my case procedurally building a level and level sequence with specific camera and animation data - Unreal will terminate the whole thing as soon as the initial setup is complete. It doesn’t know to wait around for the render. I assume this is related to how Unreal handles the rendering as a separate process. My solution was to create a second command line call at the end of my setup code. It opens a second instance of unreal, completes the render, and the first instance waits for it to complete. There’s probably a better way to do it, but this is relatively easy and works. Thanks again!
1 Like
Sorry for the delay as I just noticed this. Can you share what version you are running of the engine and a simple reproduction case so I try as well?
Thanks for that, I hadn’t considered that usecase and it is outstanding that you shared back to the community. There are always a million ways to approach things, but these are great breadcrumbs! Grateful.
It might be the same issue.
The article mentions 4.27 and later versions, but I have been able to run the command-line rendering described in the article on 4.26 and it works. When I render with object ID included in the Movie Render Queue settings to output .exr, the resulting .exr file doesn’t have Cryptomatte applied.
However, when I render the same Render Queue directly from the UE4 Editor, Cryptomatte is applied correctly.
Is it not possible to include elements in command-line rendering? Or could it be that the functionality doesn’t work correctly because of the older version?
I’m having trouble getting this to work on UE 4.27 while specifying Executors.
Trying with the Python MoviePipelineExampleRuntimeExecutor return conflicting information. The Command
"Path\To\UE4Editor-Cmd.exe"^
"Path\To\DEV_427.uproject"^
Main^
-game^
-MoviePipelineLocalExecutorClass=/Script/MovieRenderPipelineCore.MoviePipelinePythonHostExecutor^
-ExecutorPythonClass=/Engine/PythonTypes.MoviePipelineExampleRuntimeExecutor^
-LevelSequence="/Game/Sequence_1.Sequence_1"^
-windowed -StdOut -allowStdOutLogVerbosity -Unattended
fails with
[2023.05.08-23.25.24:247][ 1]LogMovieRenderPipeline: Loaded explicitly specified Movie Pipeline Executor /Script/MovieRenderPipelineCore.MoviePipelinePythonHostExecutor.
[2023.05.08-23.25.24:247][ 1]LogMovieRenderPipeline: Using default Movie Pipeline . See '-MoviePipelineClass' if you need to override this.
[2023.05.08-23.25.24:250][ 1]LogUObjectGlobals: Warning: Failed to load '/Engine/PythonTypes': Can't find file.
[2023.05.08-23.25.24:250][ 1]LogUObjectGlobals: Warning: Failed to load '/Engine/PythonTypes': Can't find file.
[2023.05.08-23.25.24:251][ 1]LogUObjectGlobals: Warning: Failed to load '/Engine/PythonTypes': Can't find file.
[2023.05.08-23.25.24:251][ 1]LogUObjectGlobals: Warning: Failed to find object 'Class /Engine/PythonTypes.MoviePipelineExampleRuntimeExecutor'
[2023.05.08-23.25.24:251][ 1]LogMovieRenderPipeline: Loaded executor from Python class: /Engine/PythonTypes.MoviePipelineExampleRuntimeExecutor
[2023.05.08-23.25.24:251][ 1]LogOutputDevice: Warning:
Script Stack (0 frames):
[2023.05.08-23.25.24:251][ 1]LogWindows: Windows GetLastError: Der Vorgang wurde erfolgreich beendet. (0)
[2023.05.08-23.25.27:820][ 1]LogWindows: Error: === Critical error: ===
[2023.05.08-23.25.27:820][ 1]LogWindows: Error:
[2023.05.08-23.25.27:820][ 1]LogWindows: Error: Fatal error: [File:D:/Build/++UE4/Sync/Engine/Plugins/MovieScene/MovieRenderPipeline/Source/MovieRenderPipelineCore/Private/MovieRenderPipelineCommandLine.cpp] [Line: 341]
[2023.05.08-23.25.27:820][ 1]LogWindows: Error: No class set in Python Host Executor. This does nothing without setting a class via "unreal.get_default_object(unreal.MoviePipelinePythonHostExecutor).executor_class = self.get_class(), or passing -ExecutorPythonClass=...".
[2023.05.08-23.25.27:820][ 1]LogWindows: Error:
[2023.05.08-23.25.27:820][ 1]LogWindows: Error:
[2023.05.08-23.25.27:820][ 1]LogWindows: Error: [Callstack] 0x00007ff8322bcb69 KERNELBASE.dll!UnknownFunction []
[2023.05.08-23.25.27:820][ 1]LogWindows: Error: [Callstack] 0x00007fff728d0216 UE4Editor-Core.dll!UnknownFunction []
[2023.05.08-23.25.27:820][ 1]LogWindows: Error: [Callstack] 0x00007fff728d4218 UE4Editor-Core.dll!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007fff725f242d UE4Editor-Core.dll!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007fff4459f31b UE4Editor-MovieRenderPipelineCore.dll!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007fff44594cde UE4Editor-MovieRenderPipelineCore.dll!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007fff631b92e0 UE4Editor-Engine.dll!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007fff631e7de9 UE4Editor-Engine.dll!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007fff628cf97d UE4Editor-Engine.dll!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007fff6269f56c UE4Editor-Engine.dll!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007ff6a0bc87a0 UE4Editor-Cmd.exe!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007ff6a0be0fcc UE4Editor-Cmd.exe!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007ff6a0be10ba UE4Editor-Cmd.exe!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007ff6a0be40dd UE4Editor-Cmd.exe!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007ff6a0bf5984 UE4Editor-Cmd.exe!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007ff6a0bf78be UE4Editor-Cmd.exe!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007ff8347e7604 KERNEL32.dll!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error: [Callstack] 0x00007ff834a026a1 ntdll.dll!UnknownFunction []
[2023.05.08-23.25.27:821][ 1]LogWindows: Error:
I’d be fine with implementing my own executor in C++ but I’m having trouble to figure out how to get it to load from my plugin. The executor inherits from UMoviePipelineLinearExecutorBase and for now is a copy of UMoviePipelineInProcessExecutor, which works fine in the otherwise same command.
[2023.05.08-23.33.48:726][ 1]LogUObjectGlobals: Warning: Failed to load '/Script/DeadlineInterface': Can't find file.
[2023.05.08-23.33.48:726][ 1]LogUObjectGlobals: Warning: Failed to find object 'Class /Script/DeadlineInterface.MyMoviePipelineDeadlineExecutor'
[2023.05.08-23.33.48:726][ 1]LogOutputDevice: Warning:
Any pointers would be greatly appreciated
I suspect you need 4.27 as the release notes mention the following:
The Object ID render pass has been improved, adding support for grouping ID’s in a variety of ways. You can now group your objects based on actor name, material, folder, and more.
1 Like
It seems like this post may help you as it appears there is a scope issue here to me. I am curious if it isn’t being init. This is outside of the scope of my writeup here but these posts may help you.
I’m curious if you have seen this site. It has a nice write-up.
Hello,
thanks for all theses information it has been very useful !
I’m trying to automate multiple render process in command line with python, and I want to change the scene before rendering so I need to access the projects assets and spawn them, modify the components etc. But when I try to access unreal.EditorAssetLibrary it tells me that unreal as no module like this.
Maybe it’s not the way to do it ? if anyone has an idea
Thanks for this useful tutorial.
We have a lot of render jobs which we can already generate via Unreal Python. We would now like to use Unreal Python to automatically save a custom Movie Render Queue as an Asset so that we could use the method you outlined in [Straightforward: Rendering Movie Pipeline Queues].
The tutorial shows how to save MRQ as Asset manually via UI but is that step also possible with Unreal Python (UE 5.2.1)?
EDIT: We have now a working Python solution by duplicating an existing empty MRQ asset and using copy_from() function to copy data from current pipeline_queue.
Our new issue is the same as posted Mar 23/May 8: The Cryptomatte masks are not properly generated for EXR files. The cryptomatte manifest string is empty instead of containing the JSON information. ActorHitProxyMask00/ActorHitProxyMask01/ActorHitProxyMask02 are included but are empty. Using Object Id type Layer.
When using Unreal Editor to render MRQ asset instead of command line the EXR Cryptomatte is OK. Using Unreal 5.2.1.
$ exrheader commandline_render.exr | grep cryptomatte
cryptomatte/3fd1687/conversion (type string): "uint32_to_float32"
cryptomatte/3fd1687/hash (type string): "MurmurHash3_32"
cryptomatte/3fd1687/manifest (type string): ""
cryptomatte/3fd1687/name (type string): "ActorHitProxyMask"
Hi all,
I am pulling this information out of a colleague’s reply, but I think it is relevant to the challenges shared here. This isn’t a place where I can maintain a support thread, but I appreciate folks supporting others and feel this is relevant to those tackling the topic.
TLDR:
Unfortunately, the ObjectId pass is based on the internal “Hitproxy” pass which is used by the editor when clicking the mouse in the viewport to determine which object you’ve clicked on, so it requires the full editor. Because of this, it doesn’t work in “-game” mode - as a workaround, we suggest moving your render farm to use the full editor instead of -game. This will enable support for the ObjectID pass and you will get more consistency with renders that artists are doing using Render (Local), as it is the exact same code path as when they are using the user interface. It will increase the load times by a small amount but the repeatability (and more tested code paths) are worth it.
Details:
I have provided some sample code below on how to render queue assets using the full editor (but launched from a command line). It is a two-part solution - the first part is a “bootstrap” Python script that is run automatically after the editor finishes loading. This Python script does some additional work (waiting for the asset registry) and then it tells the editor to render with a custom Executor. The custom executor does some additional work that is normally done internally (when using -game mode) such as loading the queue asset specified on the Command Line. After doing this custom work it then creates a UMoviePipelinePIEExecutor and runs that - this is the same executor that is normally used when pressing Render (Local).
To get the Python script to run the console command after the editor launches you can use -execcmds=“pyCustomRenderBootstrap.py” and that Python file will be run on the first frame after the editor opens. Unfortunately at this point the Asset Registry is not fully loaded, so attempts to load a particular queue/sequence will fail, so your script will need to wait for the registry to finish and then proceed. Below is an example of CustomRenderBootstrap.py that registers a hook to check every frame if the registry is finished, and then once it is, proceeds. You will need to place this in your project’s Content/Python/ folder as CustomRenderBootstrap.py.
import unreal
import MyCustomEditorRenderExecutor
"""
This is a bootstrapping script that is executed when the editor starts in a
mode where it should connect read the command line and automatically render a job specified on the command line without artist intervention. It simply
calls render_queue_with_executor with a custom executor which then spawns
normal PIE executors - MyCustomEditorRenderExecutor is effectively a 'wrapper' around a PIE executor.
USAGE: UnrealEditor-Cmd.exe C:/Path/To/Project.uproject -execcmds="py CustomRenderBootstrap.py" -MoviePipelineConfig="/Game/Path/To/YourQueueAsset.YourQueueAsset"
The editor should launch, then automatically load the map specified by the first job in YourQueueAsset, then render it in PIE, then load the map for the next job, etc. Finally it will quit the editor on finish.
"""
tick_handle = None
custom_executor = None
def initialize_render_job():
print('Initialize render job')
# Create an instance of our custom executor
global custom_executor
custom_executor = MyCustomEditorRenderExecutor.MoviePipelineMyCustomEditorRenderExecutor()
# Listen for the executor to be finished so we can request editor shutdown
custom_executor.on_executor_finished_delegate.add_callable_unique(on_custom_executor_finished)
# Now tell our custom executor to render which will load the queue asset and then create PIE executor instances.
subsystem = unreal.get_editor_subsystem(unreal.MoviePipelineQueueSubsystem)
subsystem.render_queue_with_executor_instance(custom_executor)
def on_custom_executor_finished(executor, success):
# Unfortunately the success bool isn't very useful at this time (errors report success)
# so we can't do much with it here, but if you really need it you can get the correct
# information from the individual job work callbacks on the PIE Executor and then you can
# bubble that information up with another delegate, etc.
unreal.log("Custom Executor Finished. Quitting editor now! Success: " + str(success))
unreal.SystemLibrary.quit_editor()
def wait_for_asset_registry(delta_seconds):
asset_registry = unreal.AssetRegistryHelpers.get_asset_registry()
if asset_registry.is_loading_assets():
unreal.log_warning("Still loading...")
pass
else:
global tick_handle
unreal.unregister_slate_pre_tick_callback(tick_handle)
initialize_render_job()
# The asset registry may not be fully loaded by the time this is called, so we
# will wait until it has finished parsing all of the assets in the project
# before we move on, otherwise attempts to look assets up may fail
# unexpectedly. This registers an OnTick callback and it will get called once
# per frame. Once the registry reports that we're loaded then we'll start the
# render job and unregister from this callback so that we only try to start
# rendering once!
tick_handle = unreal.register_slate_pre_tick_callback(wait_for_asset_registry)
The second part of solving this issue is a custom executor. This custom executor is responsible for using the editor version of commands to load the appropriate map, then once loaded, it invokes the PIE Executor (which is what is run when you render with Render (Local)). I have attached the file as an attachment here due to its length.
In your /Content/Python folder add a file named “init_unreal.py” and inside of it add “import MyCustomEditorRenderExecutor” to the top. This is required because the MyCustomEditorRenderExecutor Python module contains a UClass implemented inside of Python, so it needs to be imported on startup for Unreal to recognize it and be able to create instances of it.
Once you do that you can drop the -MoviePipelineLocalExecutorClass, -MoviePipelineConfig, and -MoviePipelineClass arguments from the command arguments. I believe -Unattended still works as expected and -RenderOffscreen works as well (if you already have it working with -game mode)
Final usage would look something like this:
UnrealEditor-Cmd.exe C:/Path/To/Project.uproject -execcmds=“py CustomRenderBootstrap.py” -MoviePipelineConfig=“/Game/Path/To/YourQueueAsset.YourQueueAsset”
2 Likes
Hey Mike,
Thank you for the detailed answer, this is very useful.
I am still wondering if we will ever have a solution instead of a workaround?
Do developers know this issue?
The engine continues to evolve and the Product team manages the priorities. They are aware of the topic. Thanks for letting me know it’s been helpful. Helps me know what can help the community.
Is it possible to open a level sequence before starting render?
I have a weird case where I need to open level a sequence (e.g. how you open it in the editor), so it will load assets and this is somehow helps to make sure everything is loaded properly on render.
I’ve tried to insert to both execute_delayed and _post_init.py but I get the same error AttributeError: module 'unreal' has no attribute 'EditorAssetLibrary'. I assume EditorAssetLibrary is not loaded when you start Unreal from cmd or something.
print("looking for an asset")
a = unreal.EditorAssetLibrary.load_asset("/Game/Resources/Spawnable")
print("Found asset", a)
sel = [a]
# is also None starting code from commandline
aes = unreal.get_editor_subsystem(unreal.AssetEditorSubsystem)
aes.open_editor_for_assets(sel)
Hi, I’m looking for solution of remote rendering on UE. Is there a posibilty of sucessful local render farm? As a slow learner, I wish I could find a step by step tutorial to figure out the workflow. Thank you.
I’m specifying LS and config and it’s finding them correctly. However, the config doesn’t seem to know what the LS sequence is for file naming purposes. Anyone know how to fix this? I haven’t tried with a MoviePipelineExampleRuntimeExecutor yet.