First of all, even if marmoset is using physical based shading…they have to take care about way less things than Epic, because Toolbag features small scene renders without functionality compared to UE4 which is a game engine.
Of course they have to approximate differently so save computing power. Also…Toolbag has a forward renderer which, for example makes it a lot easier to properly implement translucent shading.
However, it is still possible to provide results that come very close to Toolbag2 with UE4.
Some of the things needed are already explained in posts above, but for the sake of completion, I will just quickly summarize what I did to achieve good results. Since I am at work right now, I cannot provide an example for you, but if you are interested I could add one as soon as I am back home in the evening
So, what makes stuff in Toolbag look like it does? (thats the first question you need to answer) Mainly, Toolbag uses Image Based Lighting to a) make a nice ambient light environment (fake GI) and b) provide reflections based on that image.
So what you should do first is: get in a nice HDR cubemap (sometimes you can get lucky and find the same one online that you specified in toolbag) which you can put into the scene. You also might want to have that image mapped onto your skydome, so you should also import the jpg file that is mostly with it as well so you can map it on the skydome (hdrs dont really work well for that).
Next thing is that you should decide if you want it static or dynamic. If you want it static, add a skylight to your scene, if not, chose the ambient cubemap slot in the post volume (note that you can also do it dynamicaly with the skylight, …basically just dont bake it…but you will loose the nice shadowing from it when you do not bake it) If you go for the skylight, you can switch from captured scene to use a cubemap which would then be your HDR image.
If you now put your asset into that scene, you will have almost the same lighting situation as in Toolbag.
Then you can add additional lights that match sources and directions found in the HDR image. Toolbag does something like this as well.
Now lets get to the interesting part…shaders:
First of all, that marmoset does not use metallic is not really true. Toolbag has the option to switch to whatever workflow you prefer. So if you want to use metallic, just switch under reflection from specular to metallic, and blub…you will need a metallic mask instead of a spec map, and color bleeding for metal reflection is defined by their color in the albedo/diffuse/base color/whatever map.
Then you can use a gloss map under microsurface…that means that black is rough and white is glossy. However, you can also say that you want to use roughness (where black is glossy and 1 is rough), so you switch the slot to that and enable sRGB and bämm…looks the same as before if you did it right^^
What that means is, just put all these maps into unreal as you used to (you just need that metallness map) take your glossmap and put a 1-x node after it and use it as a mask for the lerp to get more control over it. Also, you need to uncheck sRGB in the texture properties as mentioned by someone else here.
That should actually get you quite close the Toolbag2. The rest will just be tweaking and improving things.
BTW: If you are working like that, then you dont need reflection capture actors because the ambient cubemap is also your reflection. And if you go for dynamic, they dont work anyways since they read from the lightmap data which is obviously not there if you go for dynamic lighting