Media Framework Roadmap

Development Programming & Scripting C++
, , ,
82

Hello,
About a month later, can you provide/confirm us the ETA about landing in master? We are looking for having adaptive streaming support through HLS into Unreal Engine in a cross-platform fashion, and using libvlc to do that should perfectly do the trick.

Moreover, will you provide a statically-linked build with libVLC ?

Edit: I don’t see in the list of supported extensions .m3u8 but I see .ts, so will be HLS supported ?

83

Its already July 7th you said this was coming in 4.13, will 4.13 come out before the end of the month :smiley:

84

No, we’re branching end of July. There may be a preview by the end of the month though.

85

July 19th

HLS will be supported on Android, PS4, and possibly macOS/iOS. I probably won’t have enough time to implement it for Windows for 4.13.

I’m still working on the VlcMedia plug-in as well, but it’s currently broken, and I can’t figure out what. I may not have enough time to finish it for 4.13.

No, never. It’s against VideoLAN’s license.

Update: I was able to get VlcMedia working again, and I also implemented audio playback support for it today.

86

Hi there I’m also eagerly awaiting this update to the media framework and hoping the performance improvements will enable me to use movie textures in my VR game.

Will this new version be available in the official compiled and released version of UE or would I have to compile it myself to get my hands on it early?

Cheers

87

Yes, the Media Framework is really just two modules: Media (the API) and MediaAssets (implementation of various assets in the Engine).

The official build will also include a number of player plug-ins, which perform the actual decoding of media:

  • AndroidMedia (Android)
  • AvfMedia (macOS, iOS)
  • MfMedia (XboxOne, Windows 8+) not finished yet, but we’re pushing hard
  • PS4Media (PlayStation 4)
  • WmfMedia (Windows Vista+)

Inofficial plug-ins:

Starting in September, I will also look into:

  • Google VPx
  • HLS support on Windows 8+
  • YouTube support
  • Video capture hardware
  • Webcams
88

CANT FREAKING WAIT!!! :smiley: Keep up the hard work

89

Licensees can now access the Media Framework 2.0 preview in the Dev-Sequencer stream.

Integration into GitHub Master for general UE4 subscribers is scheduled for July 19th.

90

Where can I checkout the 2.0 preview, do I have to go to Github?

91

No, only licensees have access to our internal Perforce streams.
Like I said, the GitHub update will come next Tuesday.

92

Latest code has been merged to GitHub Master. This stuff is coming in pretty hot, and we haven’t tested everything yet. A bunch of stuff may still be broken. If you have time, please check it out and report any problems. We will continue to fix issues until 4.13 comes out.

Here are some instructions for troubleshooting:

General
Most player plug-ins provide detailed logging of potential problems. To enable it, add the following to your project’s Engine.ini:



[Core.Log]
LogMedia=Verbose
LogMediaAssets=Verbose
LogAndroidMedia=Verbose
LogAvfMedia=Verbose
LogMfMedia=Verbose
LogPS4Media=Verbose
LogWmfMedia=Verbose

Please do not submit bug reports without verbose logging enabled, because I won’t be able to tell what’s going on!

If a media source fails to open or play, check the Output Log for additional details:

forummf1.png
forummf1.png936×75 47.6 KB

Another good place to check for additional information is the MediaPlayer editor’s Info tab:

forummf1.png
forummf1.png936×75 53.4 KB

If Verbose logging is enabled for the MediaAssets log category, this information is also added to the Output Log. Otherwise, please include this information manually with bug reports!

PlayStation 4
The PS4Media player plug-in is disabled by default and must be enabled in the project’s Plugin Settings.

The video decoder on PS4 consumes a lot of Onion memory, especially with HLS. You may run out of memory when trying to play more than one video at the same time, because we allocate only 200 MB by default. Currently, these default settings are not exposed in the UI (scheduled for 4.14), but you can modify them by hand.

In PS4Engine.ini



[SystemSettings]
r.GarlicHeapSizeMB=2600
r.OnionHeapSizeMB=200

In your game’s *.Target.cs:



public override void SetupGlobalEnvironment(
            	TargetInfo Target,
            	ref LinkEnvironmentConfiguration OutLinkEnvironmentConfiguration,
            	ref CPPEnvironmentConfiguration OutCPPEnvironmentConfiguration)
{
            	// ...

            	if (Target.Platform == UnrealTargetPlatform.PS4)
            	{
OutCPPEnvironmentConfiguration.Definitions
.Add("GARLIC_HEAP_SIZE= (2900ULL * 1024 * 1024)");

	OutCPPEnvironmentConfiguration.Definitions
.Add("ONION_HEAP_SIZE= (100ULL * 1024 * 1024)");
            	}

            	// ...
}

Windows
The Windows Media Foundation (WMF) based player plug-in WmfMedia has a number of limitations for MP4 containers. See here for details: MPEG-4 File Source - Win32 apps | Microsoft Docs
XBoxOne
The MfMedia player plug-in is disabled by default and must be enabled in the project’s Plugin settings.

It is possible to compile and test the MfMedia plug-in on Windows 7 and up if the Engine is compiled against Windows 7 (by default we compile against Windows Vista). This can be accomplished by changing the following code in the SetUpCompileEnvironment() function in UEBuildWindows.cs:



// Windows Vista or higher required
InBuildTarget.GlobalCompileEnvironment.Config.Definitions.Add("_WIN32_WINNT=0x0601");	InBuildTarget.GlobalCompileEnvironment.Config.Definitions.Add("WINVER=0x0601");
93

Known Issues & Limitations

Android

  • Audio currently plays via the OS and cannot be piped through UE4’s sound system

Audio

  • There is currently no synchronization of audio samples, so playback may get out of sync. Workaround: use shorter videos for now.
  • Audio is currently always mixed down to 16-bit stereo 44.1kHz, regardless of the input format. We are planning to support multi-channel audio in the near future.

Editor

  • The Playlist user interface in the MediaPlayer editor is very limited right now and only shows the currently opened MediaPlaylist asset, if any. Workaround: Edit the playlist contents by double-clicking on the MediaPlaylist asset open its own editor.
  • The Stats tab in the MediaPlayer editor is not implemented yet

iOS / macOS

  • Not yet finished

PlayStation 4

  • Playlists are currently broken (UE-33481)

Sequencer

  • Media playback will not render out properly when recording cinematics from Sequencer. We know this is important, and we have it scheduled for 4.14.

Windows

  • The media player state machine is still fragile and may break if multiple control commands are issued simultaneously (UE-20209). Workaround: insert short delays between commands, so that the player has time to process them.
  • Some videos with multiple audio tracks fail to open. We are still investigating.
    QuickTime movie (.mov) support on Windows 7 and later is very erratic. This format should not be used right now. We are investigating.

XboxOne

  • Not yet finished
94

Upgrade Notes

Please note that, if you have existing Media Framework content in your project, you have to update your content. Your project will still load, but playback will no longer work with existing 4.12 content. The workflow is slightly different now. In particular:

  • Media Player assets no longer store a URL for the media. Instead there are now new asset types called ‘media sources’ that describe the media.
  • File Media Source assets are used for media files stored on your device or in local network file shared
  • Stream Media Source assets are used for streaming media over the internet
  • Media Playlist assets are used to combine multiple media sources and play them one after another. Playlists will automatically cycle back to the beginning.
  • Media Texture and Media Sound Wave assets no longer hold a reference to the Media Player. Instead the Media Player now references the texture and the sound assets.
  • There is no longer a many-to-one relationship between Media Texture / Media Sound Wave and Media Player assets. Every Media Player now references at most one of each.
  • There is no longer an option for automatically starting media playback in Media Player assets. Instead you must open a media source via Blueprint. For convenience, Media Player assets have a Play On Open option that will begin playback after a media source opened successfully. If you do not set this option, you must initiate playback via Blueprint
  • The Loop option will loop the currently opened media source. Please note that, while looping is enabled, playlists do not advance.
95

Hi, haven’t tried media framework, but just noticed something when building master branch:

In “WmfMedia.Build.cs”, “Settings” module might need to be wrap inside “Editor” target:



			if (Target.Type == TargetRules.TargetType.Editor)
			{
				DynamicallyLoadedModuleNames.AddRange(
					new string] {
					"Settings",
				}
				);

				PrivateIncludePathModuleNames.AddRange(
					new string] {
					"Settings",
				}
				);
			}
96

Is media playback or video playback working on the mac yet or not?

97

Hi Sir @gmpreussner,
For the first point, now we are able to load an movie from file location in a package game? or I misunderstood?, and second, it´s means now we are able to play a movie from an FTP server with link? or something like that?
because I like to much that features!

Cheers

98

Good catch! Fixed.

99

yes

VlcMedia is currently the only player plug-in that supports FTP.

100

not yet. we’re trying to get it done next week, hopefully.

101

Awesome :smiley: So hopefully it will be working by the first week in August if all goes well? Will you update this forum with a post of all the changes and fixes and stuff when you get it working?