Thanks for the answer Jant.
I think the problem was in the Ball torque- and inertia settings… All good now.
I updated project now from 5.0 to 5.1 and disabled Async completely, but the bug still happened, so I decided to start from the clean table completely. My previous Ball Physics settings worked nicely in 4.26, but at the same time I updated project to 5.0 I started to utilize Async, so I assumed the problem could be there instead of ball settings.