After hours of fiddling around I managed to get it working for UE 5.2.1 using these settings:
==Android==
Min SDK : 29
Target SDK : 32
==Android SDK==
C:/Users/win/AppData/Local/Android/Sdk
C:/Users/win/AppData/Local/Android/Sdk/ndk/25.1.8937393
C:/Program Files/Android/Android Studio/jre
android-32
android-29
====Android Studio====
==SDK Platforms==
Android 10.0+ (API Level 30)
Android 8.0
==SDK Tools== (make sure to select ‘Show Package details’ at the bottom)
Android SDK : 30.0.3
NDK : 25.1
Android SDK cmd-line tools: 10.0 and 8.0 (120.o might be sufficient tho)
Hope this helps someone.