Quick sanity check: Make sure your SpawnedRooms array is a UPROPERTY. Otherwise the spawned room actors will get garbage collected next time the GC runs (the GC will think that nothing references your room actors and destroys them).
If your array is already a UPROPERTY, then please post the callstack for the crash, so we can take a look at what exactly is crashing, thanks!