Need help eliminating tracking delay in ndisplay

I am currently building a virtual production set and I have been running into an issue with ndisplay. I am using a optirack tracking system and have tracking working perfectly in the editor, tracking seems very snappy and I see no delay when I’m in editor but as soon as I run the scene in Ndisplay I have about a half second delay in motion, it does seem to be related to scene complexity so I’m assuming I might be running into some performance issues but its strange to me that in the editor I have no problems. My output resolution is a bit large at 5760x1536. I am using a 4090 in a threadripper system so I cant imagine some of my very basic scenes should be taxing on the system and in editor everything runs very smooth. Any help would be greatly appreciated.

What are your frame timings when running in nDisplay? Are they the same as standalone/editor?