Update: As of UE 5, SteamVR no longer starts automatically.
Original answer:
You can disable it starting by default by editing “Engine\Plugins\Runtime\Steam\SteamVR\SteamVR.uplugin” and changing "EnabledByDefault": true
to "EnabledByDefault": false
.
Note that you have to do this every time you update the engine.