Bad render thread performance - UpdatePrimitiveTransform is very slow


I also posted this question on AnswerHub but it’s not getting a lot of love.

I have complex moving Actors that are made of ~30 UStaticMeshComponents. Those components are welded together using auto-weld. Most of them don’t have any kind of collision, they don’t have shadows, they’re low on polycount etc. They just happen to be numerous. The Actors are moved through physics impulses.

When I take a look at the profiler, I have around 0.1ms of UpdatePrimitiveTransform time in the render thread, per component. This is huge and it breaks performance in my game - 10 actors already means 3ms of render time. It also get some more time in the game thread, although a bit less.
This is on a solid rig : Core i7 3770K, GTX 970.

How can I make the performance more acceptable here ? It’s not like 300 moving meshes is a big deal in 2015. Games like Kerbal Space Program or Besiege do it with physics constraints on top of it and they hardly require a high-end computer. I guess I did something very wrong, but what ?