This solution worked for me: Is Layer panel bugged in UE5? - #15 by chandanGalore
Bad part is that I had to restart the level from scratch, and it is not open world anymore. For my goals it was fine, but I’m still figuring it out how to make this work on an open world level