C++ is C++ in UE4 so if you want to use anything outside UE4 APIs (and keep in never gonna able to support everything there is in the world) you simply use normal C++ libaries as you would usally do in C++ or code your own support. Here is tutorial how to add library to your code:
Because UE4 don’t use defult VS build manager and has it own, it’s done diffrently then some outside tutorials how to use specific libaries do, so you always need to do it based on this tutorial. Most C++ liberies contains dll files (or source code to build them) and include files, so it usally should fit in to that tutorial, just palce things where they should be.
Now there other issue you will have, MySQL library free licence is GPL. which UE4 EULA prohibits to use with UE4 because GPL force software that use GPL library to also be GPL licenced which would make UE4 free redistributable without any money benefits to Epic. In order to use MySQL in commercial software (in this case also includes free closed source software) you need to buy commercial licence from Oracle:
http://www.mysql.com/about/legal/licensing/oem/
I not sure if non-distributed software (which you would place just in your server) is effected by any of those licences. There tricks to avoid that, use of HTTP server as a database interface or some other medium that sperates MySQL software with UE4. Alternatively you can use SQLite, which UE4 has incomplete module:
I does not build by default… i’m not even sure if it builds at all as it seems it didn’t been touched for long time, but it might be good start point if you plan to use SQLite