So after hours of tweaking different things I finally got the result I wanted. Thank you for the help!
I have another small question, my project crashed and I had to load autosave and tweak RVT setup a bit, but after that my meshes looks very weird, they are very bright in shadows after light baking. It is especially noticable sincei am using RVT setup
You can compare those meshes with ones before light baking:
You can also see a lot better blending with landscape with RVT setup
edit: the issue is skylight, for some reason when I set it to “Movable” everything is alright