Well I’ll be darned! I just reconstructed the relevant parts in a clean project in order to be able to upload a small and clean character and animation blueprint and to my huge surprise it is actually working fine. (which is strange because I thought I was really thorough in disabling possible conflicts)
It Works!! 
*(To those people with the same problem, my last screenshot, the one under the client/server windows, is the solution :D)
Thank you so much Rekenq. You’ve been a incredible help and your willingness to go even further and test some stuff yourself was absolutely great 
I’m quite sure I’d be fighting this problem for many hours if it wasn’t for your help. So thanks again man, you’re awesome!
I’m sorry but it turns out that this wasn’t the solution after all.
The reason why I thought it worked was because without the GoogleVR plugin enabled, it does work on windows builds and I forgot to turn that on last night in the new project.
But now I’ve ruled everything out.
Even with all other VR plugins disabled, when GoogleVR is enabled, then the clientside motion jitters(between 0 and the proper HMDvalue), while at the same time replicating correctly to the server without jittering.
For my test I’m using a Windowsbuild + Androidbuild and it does not matter who is server or client. In both cases the client will jitter while replicating correctly to the server.
With 3 Windowsbuilds, the two clients will jitter in their own window while replicating correctly to the server and each other.
I think that there is nothing more I can try and that this is a bug in the GoogleVR plugin.
How should I proceed?
Could/should this thread be converted into a Bug-report?
