Hello,
I am currently working on some software that uses the rendering system to create ground truth data for semantic segmentation (Microsoft AirSim).
This is done using the CustomStencil and a color palette (as well as an API to set the stencils), as done in photo 1.
Now, the problem I am having with this, is that I want to have a segmentation label for large foliage, while I want grass etc to be a part of the ground label. When using the foliage tool in Unreal, all of it is put under a single ‘InstancedFoliageActor_0’ mesh, and I cannot find a way to split it up. Is this somehow possible?
Another way to split it up would be to somehow figure out the ‘height’ of the foliage, if the above is impossible, does anyone have a suggestion on how I could do this?
I will also be only sampling images from above, so the difference between two ‘depth’ images could be used, one for the distance between the camera and the foliage hit, and another from the camera to the ground (ignoring foliage, if this is possible). The first is already implemented in AirSim, (Camera Relative World Position -> VectorLength), but I am unable to figure out a way to ignore foliage (e.g. get world coordinates of custom stencil hit, ignoring foliage mesh).
Any help would be appreciated, thanks.