[Feature Request] OSC, MIDI, and Syphon

Hello devs,
I came here on behalf of some of my fellow live visual artist developer friends, casually called VJs. We often work in many different programming environments with various tools to try to accomplish our visions whether simple generative graphics or projection mapping. In recent years game engines that offered real time rendering have provided an interesting tool for us to work with. In example Unity engine has allowed interactive projection mapping such as these examples:

WORKSHOP 3D INTERACTIVE PROJECTION MAPPING - UNITY 3D on Vimeo NuFormer, 3D Mocap Mapping - NuFormer/Motek, the Netherlands - March 2012 - Short Version (20 min) - YouTube
Well after seeing the Unreal 4 demo a lot of us were joking around wouldn’t it be nice to get some native support for OSC ( http://opensoundcontrol.org/ ), MIDI, and Syphon ( http://syphon.v002.info/ ) for Unreal. One said he would look at the Unreal plugin API but I wanted to see if anyone suggested it. I searched the forums and didn’t get any results so I figure I would post a request. The real time rendering capabilites, among everything else, could provide a wonderful new tool for a lot of us. I personally have my background in using a programming language called Quartz Composer which is losing support from its developers and becoming more outdated. I would be very interested in Unreal 4 becoming the new tool for me to use.

Hi there,

I would also like to put forward a comment about adding OSC and Syphon support.

I work in the field of live visuals and projection mapping, creating animated content that can be played live. There is an increasing demand for this kind of work with brands and festivals (Vivid festival over here in Sydney being the biggest in the Southern Hemisphere), especially when it comes down to audience participation and interactive experiences. Unity has provided a great base for this so far but it would be amazing to see the Unreal 4 Engine be capable of this too, and go further with with it’s implementation - especially as OSC is such a versatile tool.

To see this implementation would open up doors to some fantastic creative opportunities.

Also very interested in this, would love to have even better support to integrate music on a more advanced level in the game play for example.

We are interested in features like this too! We have an experimental plugin called ‘LiveEditor’ which uses the portmidi library (which is part of our distribution) to allow you to tweak parameters using a MIDI controller. We have not had a chance to polish this up to the level of a supported feature, and we would also like to support OSC instead of MIDI. But if you are feeling brave you could have a go at playing with it!

Hey James, that sounds ace! I’ve still barely even scratched the surface yet with UE4 but I’d love to have a go at using the LiveEditor plugin. Is it documented at all so far or still in it’s raw form?

Nope, no documentation I’m afraid, you are rather on your own at the moment! Sorry.

Native osc would be a game changer : D Im on the same side right now … using unity with uniosc and syphon. I am and many others would jump stright to ue when there will be a solution for this. : ) Dream would be to have an option to assign an osc data value to any slider/component/ with a right click : P Oh and a syphon node in the shader editor : D

JamesG: Are there any possibilities for you guys to do a small tutor or what ever to show how to hook up a component or float to LiveEditor.
Im using vdmx on a mac and it’s able to send midi and osc. So basicly i could convert the osc data from ei kinect / leap etc in vdmx and send out midi to ue.

These are the assets i use in unity would love to see something like this in ue.



Yes OSC support would be great! and syphon would be incredible!!! a plug in might be possible.

Hello someone could use the MIDI protocol in some way within the Unreal 4? Could you post some code examples?

Open OSC - yes please! - I’d consider paying a developer to implement this as a blueprint node. The great thing about open OSC is that you can pretty much use it for anything. If there is anyone interested in taking on a project like this please reply here - maybe we could get some funding to you.

midi tc would be nice if implemented in UE4
I could sync game or 3d events from an external audio sequencer



To be able to right click on any slider in the interface and to bring up an option assign OSC control would be amazing, as with the ability send a syphon output from a camera in Unreal Engine to projection mapping software (such as madmapper) would have a lot more live visual artists wanting to use Unreal Engine.
I really hope this is implemented at some point.
Olly from 1000 Errors

Still…midi should not be completely dropped. It might still be useful for some projects. For example, a game that handles midi data is Synthesia.

I want to see this stay alive. Anyone doing any work with the LiveEditor that JamesG mentioned?

+1 recent convert from Quartz Composer (QC), was attracted due to the similarities in the visual scripting/coding with Blueprint. I would hope to do some interactive audio visual games and of course MIDI, OSC are key.

Syphon is just a game changer especially as I work with a lot of Augmented reality. Please include these functions into a release of Unreal . :slight_smile:

I just published the first version of UE4OSC. It is a free, open-source plugin that allows Unreal Engine 4 to communicate send and receive OSC messages.

It has a page on the Unreal Engine Wiki, and is on GitHub.

It is has basic functionality at this point in time, but I will continue to expand it based on my own process and any feature requests I get.

Hope Epic won’t drop Midi. I have some projects in plan, that require connecting my midi keyboard to PC.

I released my OSC plugin here:

It receives and sends OSC messages in Blueprints. It manages float, integer, bool and string parameters.

Right now, you must compile it yourself though…

Hi Monsieurgustav. This is great!!! Could you please give us a little rundown on how to compile it?

Thanks a lot!

ok i was able to compile it

can you please post a sample blueprint