Thank you very much for trying to help me! Your game looks amazing! I hope you can manage to complete it and it’ll find success! 
I like your implementation a lot. I think it would take quite a lot of modification of other mechanics I have implemented in my game, and I think that’s just too much work for what I’m trying to achieve. The line trace in front of my character is used for other mechancis, and I couldn’t really afford changing that trace without it breaking a lot of other things.
Fortunately my issue got solved. I’m not 100% sure why though, but you did lead me down the right path. You assumed that ground friction was messing with the velocity, and I think you were right. That must happen right after I copy the velocity of the pawn. So what I did was turn off gravity of the actor when the pawn is in the push/pull state. This seems to stop the ground friction from being calculated, even if there is a collision with the ground. I also found another solution, which would be to lift the actor ever so slightly above the ground when moving, this also avoids any ground friction.
Thank you for helping me, I truly appreciate it!