It has to be done in the Editor, thus making a mesh in 3D app and bringing it into UE4 isn’t an option at this moment.
Is there a way to extrude BSP brush the way I need it and then use it in am Actor BP as collision shape (having it invisible in-game as any regular collision shape would be) ? If so, how would I do that?
If you want to convert the BSP to a blocking volume you can just use the Actor category in the Details Panel of the selected BSP and under Convert Actor > Select Blocking Volume. This will simply make the BSP a blocking volume like you get when you drag in one from the Modes window.