All Player Start does is to spawn a pawn from the pawn class you indicated in your GameMode. If you don’t have a GameMode or no indication of the pawn class to use, UE4 will spawn a Default Pawn.
If you look at the World Outliner while your are playing, you will see the Default Pawn spawned in the level.
If you want to control how the spawned pawn behaves, the only way to do it is to create your own Pawn blueprint class, override the GameMode and tell UE4 to spawn your own pawn. Then you can do anything you want with it, including changing its collisions and other properties.
You can do this very simply. Right click in the Browser, Create a new Blueprint, choose Pawn as base class or, better, Character. Call it MyPawn.
Then right click again in the Browser, Create a new Blueprint, choose GameMode. Call it MyGameMode. Edit MyGameMode to set the pawn class you created earlier, MyPawn, as Default Pawn class.
Last but not least, go to the World Details tab and override the GameMode choosing the one you just created.