VR for Animals

Hi there,

I’m quite new to UE and would like to ask you guys for some advices and guidelines.
I’m currently working on integration of UE into our system of virtual reality for animals (fruit-flies, zebra-fishes). Currently the setup consists of two main parts: tracking device (multiple cameras which track the position of an animal in 3d on every frame) and the engine (based on OpenSceneGraph), which receives the tracking data and adjusts the displayed environment according to the animal’s position and rotation.

My goal is to replace OpenSceneGraph with modern engine (I’ve chosen UE since the significant part of our system is also C++). The biggest question for me now is what is the best way to send coordinates and rotation of an animal to Unreal Engine’s actor? Currently the communication runs via ROS, but we would like to get rid of it and run something based on “Server-Send events”.
As far as I understand VRPN might be an option. I’ve found couple of plugins for UE around the web, but the latest version of UE they support is 4.10-4.12. And this makes me wonder why.
We’ve been also thinking about adjusting OptiTrack or similar hardware plugins to our needs, but they appear to be too complex for our case of just one tracked animal.

Most of the guidelines on server-send events are related to multiplayer games, that is why i’m a bit confused.
Any help, advices, guidelines are highly appreciated.

This is a really cool application! I would be curious to see how it looks like and how your animals react to it. :slight_smile:

You could stream your tracking data into UE4 using a TCP (or UPD) stream. It should be relatively easy to setup.

See here: A new, community-hosted Unreal Engine Wiki - Announcements - Epic Developer Community Forums

Hi, Marco. Thank you for your reply and the link. So far we’ve decided to stream the data via VRPN. It works quite stable, at least with sample test data.

If you’re interested in the application you can check out the paper which describes previous successful experiments on animal’s spatial behavior in VR (current setup, based on OpenSceneGraph). here is the link: Straw Lab: Publications (the one from 2017). In addition, on the homepage of this web-site there are some fancy videos and other interesting stuff. Glad you’re interested. Cheers.