Hello Bored Engineer,
the effect is done before anti-aliasing, so i was wrong in the first place. The cut out into another objects happens if the blur kernel to strong in edge areas, so the blurred pixels are reaching to far into another surface/object.
I did manage to clean up edges vertical (at some point) but cant´t fix it on the left and right. The question is, how much is possible ? I don´t know how much can be done over postprocessing. This effect here is
on the object which looks like a much better solution : UE4 - Morphscape Cinematic on Vimeo
But i wanted to find a way to use it with post processing and at some point, it worked good. I wouldn´t even mind if some edges are cut out, i really enjoy this effect. But on the other hand, i want to archieve paint style in realtime in motion.
One thing that would be really exiting is to use flow map on postprocess. Think about it, that would blur Pixels in directions and even follow the surface. But all my tests about that failed. If there would be one extra masking channel , where you could define pixel flow maps on the objects material and then blurring it on screen. That would look nice i guess.
Anyway, here is the screenshot explaining what i want to avoid :