if its just the camera doing this then based on what you have in your blueprint component section, my guess is your issue may be related to not having a spring arm. By having the camera locked in relation to your capsule component you can potentially move it passed walls and floors. The Spring arm acts like a variable distance arm that when it detects the camera attempting to clip, it will bring it forward. Heres the documentation page on it. Using Spring Arm Components | Unreal Engine Documentation
There are also a couple visualization modes you can try to narrow down your issue. You can go to show in the scene view and enable show>collision to visualize the collision geometry, you can also change the view mode from lit (the normal view) to player collision (hold ctrl+alt over the options to get more info) and it will show you the scene as it is via collide-able objects.
