Now there seems to be a slowdown when using temporal samples in AA or when using higher framerate in sequencer.
A solution that seems to be working (at least in my quick tests) is using Chaos Cache Manager that allows to bake the simulation and then play it at any speed in sequencer with its “Star Time” parameter.