I dont see any padding in that last shot…are you using no padding and 64px rez? And yeah…100 bounces wont do a lot BUT they also wont really add to the baking time either…so it kinda doesnt really matter.
Might be worth though trying the scene in 4.19 if you can.