So I found the culprit: leaving FrameRateLimit at it’s default setting of 240 was creating all the lag.
By setting FrameRateLimit in the GameUserSettings to 0 or 144 completely fixed the problem. No idea why but yeah.
Manually setting it to 240 or 180 didnt fix the issue. If anyone knows why this would be bugging out my project, please let me know. Hopefully the dev team will noticed this as well