I ended up creating a separate project which compiles and places the libs in the appropriate places, then defined a Build.cs file with Type set to External.
I then modified my includepaths of my game project and pointed it at the source directory of the library. You can now debug the library and step through the code. Only recompilation has to be done using the other project.