Download

There is real interest in VR within the academia.

Hi,

I delivered my thesis recently and I have been invited to present my work on several meetings and conferences, one of which is the international INCF conference ongoing now. There was massive interest in using VR for education and research, but mostly education. The people who were presented with my demo was amazed at the power of UE4 for creating educational applications. I was right away offer funding for future development and integration. There is real interest in VR as a tool for other things than fun and games, but the academic sphere just dont know about VR yet.

Some backstory about me and my project is in order. I have developed a VR based visualizer using UE4 and the HTC Vive offering volumetric and geometric rendering of scientific data sets. This is related to another forum thread I have here on the unreal forums: Your thoughts on and comments to Volume Rendering in Unreal Engine 4. - Rendering - Unreal Engine Forums
The source code can be found here: GitHub - NoobsDeSroobs/VRVizualizer: A virtual reality visualization system for scientific data.</
A quick tutorial video of the core features:
ForDemo - YouTube
A quick summary of an slightly earlier version:
Quick VR Visualiser Demo Summary
If anyone wants to read the thesis with much more detail it can be found attached to this post or via this link: Dropbox - MasterThesisFinalVersion.pdf - Simplify your life

During my research I learned a lot. Lets break it down.

VR can be useful for both education and research, but different demands need to be met for each
Expense and availability is the main factors that hinder mass adoption for education. Universities will not give each student a vive or a rift, but they might be willing to offer GearVR cases that the students can use. The teacher can handle a vive or rift and through it interact with the virtual world, but the students can only look. For higher education and more specialized tasks a few vives can be purchased for common use by students. Custom static experiences are acceptable and useful for education so UE4 is great for this.
Research is another story. Researchers need proper readers and writers of scientific data formats, they need complex input options and they need a means of cooperation. Multi user support and maybe tablet integration for non-immersed participants might be useful. High accuracy is important for extended research, which made the HTC Vive the favorite, but the mobility of the Oculus Rift was a very good alternative for demonstrations, conferences and meetings where mobility was key.

VR aids information flow across knowledge levels
The students and the teachers felt they understood the data better and what each party said much better when VR could help them bridge the gap of knowledge between them. The students did not have to imagine what the teacher was saying and try to make sense of it and the teacher could just show the student what they were talking about. This needs more research, but the initial response was promising.

VR becomes more useful as interleaved complexity increases
“Interleaved complexity” is a term I coined for this thesis. It refers to spatial complexity in 3D where the model overlap and interleave with itself. Look up 3D flow field, streak lines, genomes or something similar to see what I mean. Many parts have a lot of depth data, but also relative data. Sections overlap each other and are often interleaved with different parts of itself and other sections. VR benefits those who need to understand spatial data, but much more so those who need to understand interleaved data sets because the usefulness of VR seems to grow with the interleaved complexity of the models.

VR uses the brain’s natural power to control the neck and to understand stereoscopic images
This allows for many of the otherwise hard to understand aspects of complex models simple. The brain naturally understands what it sees and you are less prone to misunderstand the data. Further, due to the humans ability to control the neck and get a spatial sense of where it is, users are much less likely to get lost inside complex data sets where you travel through the models. Another benefit of the natural control humans have over their neck is the speed and ease with which the users can do complex and accurate camera placements. This is a major selling point.

The focus on ensuring 90 FPS puts entirely new restrictions on algorithm design
Standard techniques and priorities can not be used when designing tools for VR. Breaking things down into steps where the VR part is only the final link might be recommended. For example, if you are working on a data set using already existing powerful monocular 2D tools you can do most of the work in this tool as normal and then export to a VR application. This provides the best of both worlds, but the export and import process must be as smooth as possible.

Game Engines are good for experiences where the interactivity is in focus
Experiences, games and the like is perfect examples at this. More general visualization tools they are not. They are usually designed and optimized for a different purpose so creating the same functionality in a game engine is often more work. You can change the engine to fit your needs, but please keep this in mind when you analyze your options for your project.

Gear can be cumbersome and tiring
As it is, most users will tire after a while. Perspiration and the pressure against the skin becomes uncomfortable. This will be reduced over time, but I thought I should mention it if any designers of physical devices read this.

VR needs powerful, useful and standardized input devices
As it currently stands we only have up to 8 buttons per controller + a trackpad/thumbstick. This is not enough for most powerful toolkits used in education and research today. This must be fixed, somehow, or a simplified control schema must be applied to best suit the task at hand. Using gestures, the 3rd dimension and voice commands to reduce the context enough so that the features can be properly mapped to the controllers is important. HOWEVER, the controllers must still be simple enough to be used by relatively new people and common features like movement of the user, grabbing etc. must be unified across these tasks if possible.

Communication between immersed and non-immersed people is hard
People inside VR and outside VR have issues communication with each other, which is a problem since research and education is a cooperative activity. I have no idea how to fix this, so I leave it to you. :stuck_out_tongue:

Multiuser is a must
To continue the cooperation theme, both professors and researchers said that internet based multiuser must be implemented for a proper toolkit. Methods of sharing data, cooperating across the world by meeting in VR etc. are all features they said they would have to demand from a proper tool, should they adopt it.

User movement
We can either use teleportation or continuous movement. Since they only need to move once between the data sets and not continuously as you would in games, teleportation was preferred. Continuous movement used a lot of buttons, cause nausea and was harder to control (Either too slow or too fast). When designing the teleportation mechanic you can either use an arc or a straight line. Using the “laser pointer” approach allows for massive jumps, is easier to control and cheaper to execute. Some users preferred this method. The arc was much more preferred because it gave sufficient distances per jump, was much more accurate on longer distances and allowed movement between floors. You have to find what works best for you.

Sparse scenes were best
Dark and sparse scenes with little but the model being used was preferred. A bright screen tired the eyes and unnecessary objects were distracting, according to feedback. It is important to provide an experience that can be used for a long time and as efficient as possible.

Avoid using 3D widgets
Having 2D planes in 3D with menus on them works fine for a few things, but they have to be scaled up to be usable and they tend to obscure other menus or parts of the model you are looking at. If placed against a wall, this is not such an issue.

There must be some reference that matches the real world
When immersed you need something that matched the real world even when you move and rotate. Examples are gravity should point down, static object MUST be visible to avoid dizziness and fear and interactions using your hands should be close to handling object in VR using your controller.

Finally I want to repeat that there is a lot of interest in education and research. The problem is that VR is just not known to them so reach out to universities and other research organisations. Reach out to schools. Talk to them, show them a demo you prepared and ask if it is of interest. (I shouldnt have to say this, but: Make sure that the demo is a good demo, so put some work into it. If you cant showcase the power of VR you are doing more harm than good) Collaborate with them and maybe get them to fund the development. For example, in my case, I got my system to be tested and used by experts for cutting edge research and education. I have been approached by researchers and professors across the world in Pedagogy, Geology, Neurology, Epigenetics and more. My online repo has been starred and followed by 8 people. I was invited to showcase my work at INCF Oslo 2017, which is a leading conference, albeit for a very narrow field, where people who are at the forefront of their research lined up to speak with me. I have been offered funding up to at least 80k dollars for a properly integrated VR mode/tool. All of this without doing any marketing. All I did was ask people what they wanted, made a few good demos and produced a few videos to show them. The rest came from word of mouth, google searches, links posted on forums and sent around by mail.

Get out there, show them what can be done, spread the word, research their use cases and create a demo. Spark the interest. You can use my work, linked above, and modify it and improve it. Maybe even submit a pull request. Feel free to ask questions and I will try to address them when I can to the best of my ability. I hope this was helpful and I look forward to seeing what you can produce!

Very useful post, thanks! Wish vr would have more mainstream reaction too. Looks like big gaming press lost their momentum about vr, hope I’m wrong.

Thank you very much for sharing your work and conclusions! Hopefully someone can benefit from your research and equally contribute (at least for the development of VR and UE)

very cool and a nice read. I would imagine more kinds of measurement-tools would be useful. And perhaps the possibility to add notes in locations, for others to read or comment at. just some thoughts :slight_smile: great work!

Great write up.

The thing about the bright screen being tiring is very good to know.

Hey Magnus,
Thanks for sharing this. I’m working on my own thesis now that involves a bit of VR (peripheral) and it’s interesting to see your ambitious project.
I’ve downloaded your thesis and will check it outl
Great work.

Hi there, thanks for sharing. I’m interested in the academic potentials of VR and am actively working towards this. Brilliant thesis, and the open source approach you are using is fantastic!