Edit: didn’t see your post, but yeah it might be worth looking into custom mesh generation techniques and try to piece together some type of runtime BSP system from the current engine code. From the comments in the FBSPOps class, it looks like some of that code ranges back to 1999, so the performance difference from then to now might have opened up more possibilities.
The FBSPOps class seems to do what you want, but it’s under the Editor and UnrealEd scope so I’m not sure if you’d be able to access it at runtime. Also, there may be certain calculations happening somewhere else to determine intersections between n brushes in order to generate a new mesh, so having those calculations done at runtime may cause performance issues.
If you’re pretty comfortable with C++, look through the source code for that class and see if there’s anything in there that you could use.