I tried changing the cast to the individual characters (children) and it didnt work. I even tried resetting the movement mode on character switch and it didnt solve it either. I decided to restrict the switch only when all characters are idle to avoid the issue. Not the best solution but it works for now :(.
Here is the end result, if you are interested:
If anyone is interested in getting the code to do this, you can download the template and take a look at how I solved it - https://forums.unrealengine.com/showthread.php?125187-FREE-ThirdPerson-Action-Platformer