MIDI in Unreal

Jun 25, 2021.Knowledge
Article written by Anna L.

MIDI Device Support

This Unreal plugin adds the ability to send and receive MIDI (Musical Instrument Digital Interface) protocol messages. This is most frequently used to communicate between Unreal and external hardware such as MIDI keyboards. However, because MIDI is a data protocol, a user could use data parsed from messages to drive non-audio related parameters, as well.

Currently, Unreal only supports streamed MIDI data. It does not allow for reading MIDI files.

What Is MIDI?

MIDI is a data protocol designed for communicating between different audio hardware devices, via MIDI messages. A MIDI message will start with a byte containing the type of message it is (a “status” byte), followed by data bytes. For example, one of the most common MIDI messages is a “Note On” message, which consists of a status byte that both signifies it as a Note On message and communicates the channel it is intended for (a value from 0 - 15), followed by a byte of pitch data, and then a byte of velocity data. Because the most significant bit of each data byte is used to signify that it is a data byte, both the pitch and velocity data can only contain a value between 0 and 127.

The MIDI protocol also specifies how to interpret bytes of data, such as how to translate a pitch value of 0 to 127 into a unit such as Hz. While Unreal does not directly translate MIDI data this way, which allows users to potentially utilize the MIDI messages to drive behavior other than audio, our Sound Utilities plugin includes several functions that can be used to do data conversions like this.

Workflow

The general workflow for getting MIDI Output working in engine is:

  • Find the ID of the device you want to send the MIDI signal to. You can do this either by selecting the default MIDI device, iterating through all available MIDI devices, or by searching for a specific MIDI device by name. The relevant Blueprint functions will be “Get Default MIDI Output Device ID,” “Find MIDI Devices," and “Get MIDI Output Device ID by Name.”
  • Create your MIDI Output UObject, via the Blueprint function “Create MIDIDevice Output Controller.” This function takes an ID, and returns either a reference to the MIDI Output, or a null reference if it was unsuccessful in connecting to the MIDI Device.
  • Note: It’s important to make sure you cache your MIDI Output Controller into a variable. Otherwise, your MIDI Controller is at risk of being unexpectedly garbage collected.
  • Send over your MIDI commands! These are going to be the functions under MIDI Device Manager that act on a MIDIDevice Output Controller. Generally, the functions people care most about are Send MIDINote On, Send MIDINote Off, and Send MIDIPitch Bend.
  • If you’re more experienced with MIDI protocols, you also have the option of constructing raw MIDI Event data via Send MIDIEvent.

A basic Blueprint implementation for MIDI Output will look a bit like this, though likely with more elaborate instructions for what MIDI data to send over:

For MIDI Input, a common workflow might look like the following:

  • Get your MIDI Input Device set up and cached in a variable: this will look almost identical to the first two steps in setting up MIDI Output listed above, with the exception that the function names will contain “Input” instead of “Output”
  • Register to delegates for the MIDI Events you care about, through actions such as “Assign To On MIDI Note On” on your MIDI Input Device
  • Use the resulting data from these MIDI Events to drive parameters in your game, such as the pitch or velocity of a playing sound. Some of the functions in the Audio Utilities Plugin can be helpful in translating MIDI’s [0 - 127] integral data points into units more readily interpreted by the audio engine, such as frequency and volume scalars.
  • For more advanced users, you may want to experiment with interpreting MIDI data such as Control Change messages, or with creating a polyphonic music system by utilizing multiple Audio Components and keeping track of which notes are active.

A basic MIDI Input system implementation might look like this, although likely with more complex uses for the incoming MIDI data:

Hi, I have a project in Unreal using 2 MIDI controllers to control various things. I would love to be able to get my devices to give me feedback based on input actions, things like buttons lighting up when they are pressed and turning off when pressed again. I am using UE5 and my devices are APCMini Mk2 and MIDIMix.

I am using MIDI input for sending data from the device to the engine so I thought using the midi output might be a way to do this. I created the output device following the documentation and what is written above, then sending midi note on events but I’m not sure this is the right way to make the buttons light up.

Any help on this would be really appreciated

@AdamMc
I know the instructions above were designed for UE4. The UE5 instructions here shed a little more light on how it plays together. I know the MIDI Events are a little tricky, just because I believe the int signal sent back correlates to which key is selected.
The only problem I can see arising is the separate banks / limited keys, as midi controllers usually contain anywhere from 25 - 88+ keys, but the numbers should be correlated to the correct octave.
That being said, do you have a current implementation that you could share? With beta features like this, it can be hard to determine what will happen without trying.
I hope this can help a bit, and let me know if it does!
-Zen