The arrow doesn’t do anything- it’s just to see which way the box is pointing as you place it. Purely convenience.
Make sure you’re only adjusting the Z rotation:
