If you want a hacky yet simple workaround, you could create a custom default pawn like you are doing currently, but edit it’s blueprint begin play code to immediately call destroy actor on self. The default pawn will still spawn, but it will not persist.
If you just want the pawn to never get spawned at all, you could override the SpawnDefaultPawnFor function in GameMode that is exposed to blueprints like so:
Just be sure to leave the return value empty and viola, nothing spawns.
Hope this helps you with an awesome project!
-Spiris
-Edited for clarity