After some further testing, it would appear that the island settings does work at the time the players spawn, but after teleporting or respawning them with verse code, they change back to default island settings.
Turns out the spawn island was too far away from the play area, and teleporting a very large distance (1,000,000 units in this instance) seems to strip island settings from the player. After moving the spawn island much much closer to play area, all seems to be working again.
Incidentally, switch devices also didn’t seem to work if they are a very long distance from 0,0,0.
ps. The respawn immunity time island setting doesn’t appear to work still, but according to discord that is a known bug so I’ll chalk it down to that.