Can I use C++ to access a MySQL server?

Standard UE4 APIs does not support any SQL protocols direclly, only some online subsystems which is Amazon, Facebook, SteamWorks etc. (to access this github link you need to be active subscriber and have github account tied with Unreal profile)

https://github.com/EpicGames/UnrealEngine/tree/6147965f2f00f9241542c3db41c57f9f74f76b08/Engine/Source/Runtime/Online

Other then hat you could use external liberies, problem is with UBT it little more complicated to include those to project

But considering it’s Microsoft you might try to include sql header file and try to use sql libs, maybe they will work ou of the box