I managed to find the solution to this based on help from Unity people.
By adding this to your Extra Tags for manifest node:
xmlns:tools="http://schemas.android.com/tools"
(ignore the trailing “>” at the end in this picture: )
You can use the “tools” attributes in your RequirementsManifestOverride.txt.
This allows you to add attributes such as: tools:node=“remove”.
Then add this in your RequirementsManifestOverride.txt:
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" tools:node="remove" />
<uses-permission android:name="android.permission.ACCESS_MEDIA_LOCATION" tools:node="remove" />
This worked in removing the permissions in my case.
Since I could find little helpful information on this myself.
I’m leaving this here in case it helps someone else!