Sorry for the delayed response, but nonetheless I’ll reply to old unanswered posts. This is possible with a caveat.
You can set up revealing fog of room-by-room by placing actors with MapRevealer components, which represent square shapes or circle shapes of fog revealed areas. They can be turned on or off to show or hide the room and icons within. However, this works best if the player does not also reveal fog of around himself. If he does also reveal fog around himself, that area can intersect with a hidden room and he’ll be able to look inside while on the other side of the wall.