And finally…this is it.
I found a combination that seemed to work both in X and Y. As far as I have seen, this seems to be the best capsule pushing element (for this problem IMHO) yet.
Simply place this in your third person character blueprint.
Leave X and Z alone in the two nodes. Set the left one to to 1 on the Y, and the right node to -1 on the Y. Set sweep off on the left one, sweep on on the right one.
I don’t fully understand how this works (something to do with collision calculation while moving?), but I’m not complaining. 
I’m just glad this problem is finally solved.