Technically, yes. It is possible to join multiple rooms at once. However, platform limitations, such as concurrent socket connection restrictions, can impact this. This is because each room opens a Websocket and a UDP socket.
https://dev.epicgames.com/community/learning/knowledge-base/2Y9kx/epic-online-services-with-eos-voice-are-players-able-to-be-in-more-than-one-room-at-a-time-and-choose-which-room-they-transmit-to