Hi GrindingMetal,
Did you add collision to the converted BSP static mesh? Currently when a BSP is converted it doesn’t have collision. It must be added manually in the static mesh editor.
We have that logged as JIRA UE-30892.
Let me know if that works for you.
Cheers,
TJ