Using world space in rigidbody node in Animation BP fixes its scale to 1

I was trying to use world space simulation to simulate some freely swinging limbs in scaled meshes, but when I try to do so, the scale of the simulated limbs always get fixed to 1.

It works normally in the normal scale of 1
2023-01-15 20-03-23 medium

When the mesh is scaled up, the limb scale stays at 1, so it looks small in comparison.
2023-01-15 20-02-12 big2

When the mesh is scaled down, the limb scale also stays at 1, so it looks large in comparison.
2023-01-15 20-02-46 small

This is the setup in the animation blueprint, nothing fancy, just attaching the rigidbody node before the final output pose node.

Anyway I can fix this?