So I decided to put there as many lights as needed, switching to normal shadowmap instead of virtual, which gave me around 40fps and only made the shadows sharper, and now I can run it in windowed mode on 75-80fps with the look that I wanted.
