CharacterMovement weird reaction to physics objects

Hi guys weird problem thats kinda bugging me.

I’m using a CharacterMovement blueprint with Gravity set to 0 so that I have like a “fly cam” control.

Now the problem is whenever I run into an object with “Simulate Physics” enabled. My character just starts dropping at a constant velocity (even if i try to fly back up it still keeps dropping) until it hits the floor, then I can fly back up without dropping again.

I thought it was depenetrating at first but after setting it all to 0 it still happens.

Any help is greatly appreciated!