Networking with Server that is not running UE4

Hello Everyone,

For my thesis I am trying to use UE4 as UI and to render images, but my own codebase (c++) should be separate from that. I have experience in networking and programming, but I am struggling to find the best way to do this.

I am building a pipeline for localization and control of a catheter in brain surgery. I have two problems I would like to solve with UE4:

  1. During development, access to live patient and x-ray is obviously not possible. I aim to simulate the x-ray images with UE4 to be then processed by my own application.
  2. UE4 would be great for visualisation of the data (and control) my own application produces: Based on the x-ray images I create a datastructure that represents the arteries in the brain. Ultimately we will control the catheters orientation during brain surgery. While path / motion planning should still be in my own application, visualisation and interaction would be much nicer with a UE4 front end.

I was thinking about networking, because that would allow to run UE4 on a standard computer, while my own application runs on something else entirely (surgical machine). I like the modular approach, because then we could switch out the simulated images with live images pretty easily.

For the control I was thinking about RPC, sending commands like “move x-ray left/right from my application to UE4”, commands like “analyze images”, switching image source (basically the whole UI part) from UE4 to my own application.

For the data itself I was thinking of just using files transferred over network (rendered images, data structure of paths).

I am struggling to find information, and I lack the experience with UE4 to know what the best way to proceed would be. Almost all content I find does not apply, since it concerns game development where both sides run UE4.

Do you think this is the right approach? Am I missing something completely? How would I go about sending RPC’s to UE4 without relying on the standard UE4 Client-Server model where the server also runs UE4? Reverse engineering the packets? Or changing RPC’s altogether?

Thanks in advance!

Okay, I think the best way to go forward would be to integrate SteamSockets into my application and enable the SteamSockets plugin for UE4. And then use this to message between UE4 and my application.