If you want one plugin to be distributed to systems of both kinds, you cannot link directly against either DLL. That means, your plugin shouldn’t mention the DLLs (or their link libraries) at all in the build scripts.
Instead, your plugin needs to use manual dynamic library loading.
You can use LoadLibrary() to load the appropriate library for the platform you’re on. You then use GetProcAddress() to get the function pointers to call.
You can also get an error from that call that you can return as appropriate to the user, meaning your plugin can still “load” into the game, but it will return some “no support library present” error at runtime.
Using Run-Time Dynamic Linking - Win32 apps | Microsoft Learn