After a full day on this issue, I’ve found out that you can build a C++ project for arm64 only for iOS, and have no issues with getting the app to build, submit to iTunes Connect, distribute through TestFlight, and run on device.
From what I can tell from Apple’s documentation, it is okay to just support arm64. And, at that point you can also drop support for opengles and just go with Metal (all arm64 devices support Metal).
Devices with arm64 are: the (but not the 5C) or newer, or newer, and or newer. (Source: Device Compatibility)