As far as I know, the only way to accomplish this inside of a material would be to use tessellation with a displacement map. I would not recommend that approach.
Under the foliage editor → Paint → Show Painting settings, there is a Landscape Layer box which will only paint in the Layer specified.