If they have a collision, it can be expensive… If the collision is disabled, then problems will arise when the number is already thousands.
Not sure, but you can try using World Position Offset (or something like that) in the material to change the position visually, it will probably be much faster than Set Actor Location.
But in this case you won’t be able to use collision anymore.