I think it’s because you started with a plane and extruded it, right?
That will give you stretched UVs.
If you want the UVs to be ok from the get go, you need to model a cube.
Another solution is to use world aligned materials