There is a DX9 limit to the number of texture samplers a material can have, and unreal will generate a material instance of your landscape material for each landscape component. If you paint up a bunch of layers with a lot of textures, you may be exceeding the texture count for that landscape component and then it turns to default grid. Try putting specular/roughness into the alpha of the diffuse map, and put height into the normal map alpha, if possible, to keep each layer to a of two textures, then you should get to paint four layers. If you’re really careful and make special compromises with your texture usage, you can get up to 6 or 7 layers per component without hitting the cap and turning to the default grid texture. Note that on the Trello roadmap, Epic plans to increase the texture cap to 32 for DX11.
For the black layer border problem, make sure that at least one layer is using Alpha blend instead of Height blend. Usually this should be the base layer that will look good next to any layers. - You might have to paint this layer over the landscape first.