I have a crowd alembic as a test of only 50 agents from Houdini. No materials applied yet.
single Directional light source with Ray Traced Shadows enabled.
Stat Unit shows Game Draw threads at about 12ms and my GPU also about the same at 12ms.
When I playback in Sequencer my GPU jumps to over 45 ms and frame rate drops from almost 80fps to an unacceptable 20 fps.
Disabling shadows entirely has almost no effect on performance.