This issue seems to be specific to 4.21
Solution is to go to Project Settings > Platforms > Android SDK and set SDK API Level and NDK API Level both to ‘latest’.
You also need to have CodeWorksforAndroid-1R7u1-windows installed. If you have android studio or anything else that uses android, don’t put updates on the same folder. UE needs the exact versions in 1R7u1