I´m having quite a problem here.
I´m working in one environment for Virtual Production, I optimized the scene and everything is working great in both editor and during play mode.
I´m arriving to 50fps or even 60fps in editor, and more than 100fps during play.
This same scene, when Unreal is already closed and I open the scene with the switchboard, suffers a drastical fps dropdown, It goes from a range of 47fps 50fps to a range of 6fps to 10 fps
I was trying to understand why the result is different in the two cases when the scene is the same.
As you can see Postprocesing goes from 3.29 to 69.79, and TSP (Super Resolution) jumps from 2.60 to 13.82
In real time this scene was rendering a single frame in 21.55 miliseconds, but in switchboard under nDisplay it takes more than double, 126.35 miliseconds
The render settings are different in Editor and Switchboard? Someone have a clue about what can be happening?
I was trying to optimize more the scene in editor, but it seems that I´m missing something, because it seems that is not affecting to the performance in Switchboard.
I can understand your frustration, this problem may happen because of a couple of hardware and software reasons, in my opinion. So I will ignore the hardware reasons as I’m confident that you’re already sure that it’s not the problem, such as slow walls or confused nodes.
As you mentioned, you ran a smooth optimization before you display the environment. So, I don’t think it is an optimization problem. However, double-check textures mipmapping, light, and shader complexity, culling, merge actors, and proxy meshes,…
In Switchboard Settings, make sure that your multiuser session is updated probably for every node.
Also, print screen for the script editor while launching nDisplay, Let me know if it works for you, or not. We can troubleshoot again and again and again…
I was able to solve it, seems that the resolution of the inner frustrum in nDisplay was different than in the editor, and it made that there was a big drop of fps
Also, I added the postprocess and improve the quality of the inner frustrum to get an approximation of the fps similar to what I get in the Switchboard, this help me a lot to improve the performance
Sorry to revive a somewhat dead post but I have been fighting this issue for like 6months and just happened upon this post. When you say your resolutions didn’t match what were you comparing them too? The viewport size? The editor size?
I’m facing the same issue, and unfortunately, the suggested solutions don’t seem to be working for me.
I’m working on a stereo render and have activated the frame sequential feature. While the stereo effect is working as expected, there’s a significant drop in fps. In the editor, the application runs smoothly at 120fps, but with Switchboard, it drops to 22fps (with frame sequential) and around 50fps (in mono, without frame sequential).
I tried updating the Default Frame sizes, ensuring they match the viewport sizes, but it hasn’t made any difference.
Any suggestions would be greatly appreciated. Has anyone else encountered a similar problem? Thanks!
We recently had a similar issue with 120 fps stable in editor on our workstation and numbers similar to what you report (25-50 fps) when running the ndisplay application on the target machine without the editor.
For us it was 2 settings that caused the drop in FPS. Disabling “CustomDepth with TemporalAA Jitter” and switching the AA method away from TSR brought FPS back up to what we expected.
I have to admit though that we were running on older Quadro Cards (Pascal), so not sure if it was a problem with Unreal or the older hardware for us.
Can confirm this, we’re also running on P6000s and TSR kills performance quite a bit.
However, we also noticed that we’re scaling quite badly with numbers of nodes, so synchronization effects can also lead to frame drops.
Good point about CustomDepth with TemporalAA Jitter, I’ll check this setting for us as well.
Switching the Anti-Aliasing method and disabling ‘CustomDepth with TemporalAA Jitter’ has doubled the FPS! Thank you very much. We now have 40 FPS in stereo.
FYI, we have a Quadro RTX 5000, so I believe we shouldn’t have any fps problems.
Even though 60 fps in stereo is what we want to achieve, this is still a significant step forward.
Yes, sync is quite slow and just running on multiple nodes causes FPS to drop quite a bit. Have you ever investigated why?
I took a quick look at some profiling data (running on Linux/Vulkan with ethernet barrier as Vulkan does not implement the nv sync) and for the random application I chose it claims rendering times from 12-15ms and then everything stalls and waits for the custom present that has a total time of ~28ms per frame according to the profiling … that seems very slow to me.
I took a very quick, superficial glance at the code and as expected it just seems to send a message to the sync service and just wait for a reply that all are finished. I would expect a simple network back and forth to be faster than this. Especially considering our cluster is running a 100 GBit/s network with a switch just for it and nothing else.
I would like to know why the CustomPresent/Sync is that slow. Would be interesting to see if this is the same on Windows/DirectX and if using the Nvidia sync mode would speed things up.
We did a lot of benchmarks/debugging like 3 years ago before they pulled parts of our Vulkan implementation into the engine: Back then it was orders of magnitute slower because the syncing threads didn’t have a large enough local cache, so each (internal) alloc ran into an OS barrier - as the threads were all pretty much synced, this completely killed performance.
Unfortunately we didn’t actually investigate further, as since then performance was always “just good enough”.
I’ve been closely following this: https://github.com/KhronosGroup/Vulkan-Docs/pull/1364 Vulkan discussion, that might help with accurate presentation timings, but it seems still way in the future.
We unfortunately don’t have a large enough Windows cluster to do comparative tests, but I would be really curious to see the difference as well with the NV Barriers.
Apart from performance, what latency do you actually get from Tracking System to Screens? Vsync completely kills it for us, adding like 50ms to the already high system latency.
I’ll keep you updated if we ever get around to doing some more benchmarking! It would also be great to somehow connect the handful of people using nDisplay on Linux Clusters.
We have not yet looked at turning VSync off, but Unreal is adding a lot of latency for us for some reason, at least as far as we can tell.
Running an ART system with ARTTRACK6/M cameras at 120Hz internal sync on the controller we are seeing around 50ms to wall as far as we can tell (that is by just listening for the UDP dataframes and printing them on the wall), while in Unreal latency is a lot worse with 150+ ms. We looked at old UE4.22 vs UE5.1 applications and both suffer from the latency, a bit worse with UE5 since FPS is about half with the newer ndisplay. We are using VRPN for Unreal 4.22 and switched to the Unreal DTrack Plugin in UE5.
Latency is definitely noticable. That is just rough estimates though by taking a video and counting frames between the target moving and the wall reacting.
We played around a bit more with the profiling data and found out that if we switch the ndisplay sync from ethernet to none, the FPS does actually not go up.
According to the profiling the GPU time is about 4.5 ms. With ethernet sync barrier it spends around 20ms in the custom present, with no sync it spends around 20 ms in the game thread in a “FrameTime” function.
Not sure what it is actually going on there, but something seems to be killing performance. We will have to try turning VSync off to see if that changes anything.
As far as I know there isn’t. Doubt this can be nicely displayed as well, but you can record traces on each node, and then open the recordings on one machine. Helped us track down quite a few issues a few years ago.
In that case, I can also recommend using a custom source and adding your own scoped traces into nDisplay/Livelink code.
