Paper2d character bouncing off plane

Hello!
Me and a few other students at my university are making a multiplayer 2d shooter using ue4.

We’re having a problem where the Paper2d characters we’re using are bouncing off their plane when jumping on eachother. They then fall down and are destroyed by the killZ.

We’ve tried every solution we can think of to this, including locking their axis on both the sprite and capsule component, locking the axis in the project settings and in the construction script.
I also briefly tried allowing the characters to step on eachother instead (didn’t prevent it either, and this is not a feature we want).

This only shows up in the packaged version of the game, when playing in the editor everything works just fine.

I should also say we have a base Paper2dCharacter blueprint which all our characters inherit from, setting the locked axis there makes no difference.

Thanks in advance!