You cannot set the physics delta time to 0, that will indeed make everything slow down. Setting it to 1/90 or 1/120 should work. Sphere cast is a tad more expensive than raycast but it should be more precise for your use case.
You should try visualizing the collisions the physics engine sees and check them out. Use these commands at the console:
p.chaos.debugdraw.enabled 1
p.chaos.solver.debugdrawshapes 1