It was a problem of having CRT (C runtime) dependencies.
To recap, I was using VCPKG to generate the binaries and the include files. Then paste it to the ThirdParty folder and create a new library module, setting everything up like the documentation suggested.
Then during runtime, the Plugin would crash having a silent exception.
Using Dependency Walker helped me figuring out what C runtime libraries where needed. So I included them.