Amending my answer with with what I think is a complete solution:
I was able to eliminate all the Metal-related “nullable” errors and build the dev-ar-kit branch successfully by using XCode 9 beta 2, which is still available for download (as of the time I’m writing this) here:
Originally I downloaded XCode 9 beta 4, the latest version from Apple and the only one easily available on their website. But using beta 4 gave me the exact same errors mentioned here. (Beta 3 seems to cause the same problems.)
To successfully built the AR Sample project, I deleted XCode beta 4 and the old dev-ar-kit repo from my Mac, freshly downloaded and set up beta 2, and then got a new clone of dev-ar-kit and followed all the build instructions here:
https://github.com/EpicGames/UnrealEngine/tree/dev-ar-kit
Hope this helps. Leaving my previous answer below in case it’s helpful for other Mac / XCode newbies… but ultimately I got the same errors when trying to build AR Sample until I reverted to XCode 9 beta 2.
======================
[Previous answer]
I got a similar error (below) the first time I tried to build in XCode. However, I realized that I had just selected the ShaderCompilerWorker scheme file in the left column (the file browser) and then hit Command-B (Product > Build).
When I instead selected the scheme using Product > Scheme > ShaderCompilerWorker (per @Asmtechnology 's instructions) , then hit Command-B / Product > Build, the build succeeded.
Being an XCode n00b (usually use Windows), I didn’t know how to select a specific scheme for building. You have to use the Product menu as described above. You will know it’s right when you click into the Report Navigator and it says you’re building “ShaderCompileWorker” instead of “UE4”:

Hope this helps. Here’s the error I originally got:
Showing Recent Issues ERROR: UBT
ERROR: Failed to produce item:
/Users/bryanc/Downloads/UnrealEngine-dev-ar-kit/Engine/Binaries/Mac/UE4Editor.app/Contents/MacOS/UE4Editor-MetalRHI.dylib