I’m making a spectator mode in VR for a project I’m working on. For the sake of this post imagine the base game being a standard RTS with a decent world scale (i.e. units can be km apart). What I’m developing is a VR mode where the game is represented like a table-top game and spectators can interact with both their view of the world and the units in the game. By interacting with the view I mean rotation, translation and scaling of the RTS world (including clipping the terrain to remain within the virtual table bounds). Much like the way we interact with google maps on an iPad. By interacting with units I’m talking about clicking on units to bring up unit information, mostly as location pinned UMG elements. I would also to expect some UMG elements embedded in the RTS world that need to scale appropriately as to always be legible (like a compass or labels for territory borders).
The VR/spectator mode will have UMG scoreboards/UI of its own floating around and, though not something I’m looking at now, might eventually have other spectators sharing the VR space where you can point at what’s going on, etc. I don’t have a vive yet, but I don’t want to rule out the possibility of room-scale, ‘walk around the table’ view changes.
I have a clear vision of where I want to go, but of course I’m more fuzzy on how to handle the implementation in the most elegant way within UE4. In many ways it would be ideal to have two separate UWorlds, one for the RTS and one for the VR room, but I know rendering the two together (essentially embedding the RTS within the VR room) isn’t an out of the box feature.
I imagine solving this might have solutions simular to the ideas thrown around in this thread about multiple UWorlds: Multiple Concurrent UWorlds
I’ve thought of a couple techniques:
- Some render-to-texture, alpha compositing scenario where I render the RTS to a target and just embed a quad in the VR room. This seems to have many drawbacks: potential depth/overlap discontinuities in 3D, I can’t natively hit/line trace from one scene into the other. Although I’m not really worried about these dynamic elements now, should I ever want lights from the VR world to influence the RTS map or have particle effects from the RTS interact with the VR world this option also inhibits those interactions.
- Manually construct a map visualization each time as an VR scene actor(s). Have the interactions in the RTS world happen within their own frame of reference, without any rendering, and then more “manually” build a minimal scene using something akin to a grid/screen aligned mesh that dynamically represents terrain. Place representations of RTS elements as meshes in VR coordinate space. This preserves hit-tracing, lighting, and other interaction properties, but is just plain a lot more work as I have to manually scene cull & translate actors across the worlds per frame. However, it might be easier than the next potential solution when only trying to render a small subset of the entire simulated world.
- Have the tabletop game be a single all encompassing actor in the VR coordinates with a high degree of scaling applied. Essentially, the RTS take place in its own native coordinate-system (everything is relative to the root actor) maybe with a spectator camera within that space. Another similar implementation would be along the lines of embedding the RTS camera as a node in the scene graph of the VR world. This seems like it would be easiest to work with in game logic terms. The RTS happens at sensible coordinate scales and so does the VR world and I can hit-trace and all that other stuff. However, I’m worried that such huge scaling will cause floating point error issues for rendering/hit tracing, etc. I’m also not entirely sure the mechanics of this embedding; how and what do I make use of USceneComponent/FScene/Cameras/etc to make this both easy to work with and performant (again this is a VR project).
- Can this be done with some malicious misuse of the world composition system that is focused on scale embedding instead of 2D topographic placement?
I plan to go off and experiment with some or all of these ideas, but I figure it was worthy of a post as others are likely considering similar scenarios and there a lot of people on here way more versed with the guts of the engine. I know I’m digging into a potentially tricky problem, I’d appreciate any guidance or notification of any alarm bells I set off when describing what I’m trying to accomplish.
TL;DR: Ideally I have an RTS world in its own coordinates that gets visualized as a table-top in VR with its own (cm scaled) coordinates. Hit tracing seems like a must as does UMG understanding where to pin UI elements, bonus if other factors (lighting, particles) can bridge both worlds.