High velocity objects without CCD?


I have a few problems regarding the Physics in my game containing a physical ball I am using. The ball can reach a High Velocity that lets it pass through objects which is extremely annoying. I then went ahead to enable CCD to negate the tunneling effect, which results in unexpected physical behaviour. The ball is not passing through anything anymore but it can’t take ramps anymore at high velocity without unexpected results. Before without CCD the ball could manage ramps without any problem (unless way to fast it would pass through) and behaved as expected.
So I am a bit stuck at the moment. I want the ball to behave correctly and be able to take ramps etc, but I also want it to not be able to pass through objects at high velocity. I tried Sub-Stepping and adjusting Physics Delta Time before, aswell as numerous playing around with Physics Settings but nothing has helped so far. Does anyone know what I can do/try? I am getting out of ideas.