For my thesis I am trying to use UE4 as UI and to render images, but my own codebase (c++) should be separate from that. I have experience in networking and programming, but I am struggling to find the best way to do this.
I am building a pipeline for localization and control of a catheter in brain surgery. I have two problems I would like to solve with UE4:
- During development, access to live patient and x-ray is obviously not possible. I aim to simulate the x-ray images with UE4 to be then processed by my own application.
- UE4 would be great for visualisation of the data (and control) my own application produces: Based on the x-ray images I create a datastructure that represents the arteries in the brain. Ultimately we will control the catheters orientation during brain surgery. While path / motion planning should still be in my own application, visualisation and interaction would be much nicer with a UE4 front end.
I was thinking about networking, because that would allow to run UE4 on a standard computer, while my own application runs on something else entirely (surgical machine). I like the modular approach, because then we could switch out the simulated images with live images pretty easily.
For the control I was thinking about RPC, sending commands like “move x-ray left/right from my application to UE4”, commands like “analyze images”, switching image source (basically the whole UI part) from UE4 to my own application.
For the data itself I was thinking of just using files transferred over network (rendered images, data structure of paths).
I am struggling to find information, and I lack the experience with UE4 to know what the best way to proceed would be. Almost all content I find does not apply, since it concerns game development where both sides run UE4.
Do you think this is the right approach? Am I missing something completely? How would I go about sending RPC’s to UE4 without relying on the standard UE4 Client-Server model where the server also runs UE4? Reverse engineering the packets? Or changing RPC’s altogether?
Thanks in advance!