World Position Offset as a displacement replacement solution?

Hi all, so I am trying to find the best solution for getting Displacement functionality back to UE5. I am working with a scene where I have a single family house BIM model from ArchiCAD. I am trying to make displacement effect on the bricks. For the gravel I used Modelling tool in UE (increasing mesh polygons and attaching displacement modification). The problem with that method on the walls is that there is a limit to a resolution where that solution works. For walls it just does not work. I then need to split is into tiny fragments and even then, there might be still UVW issue when trying all of them to work with eachother in that regards.

So I looked into WPO option. It is working almost fine. I can increase the number of polygons, but currently I have an issue, where the meshes are being treated separately to the point where I don’t have any displacement effect joining between two sides of the same brick wall, as you can see in the picture. Does anyone know how to deal with it in WPO?