Rtsp media

anonymous_user_9e8de314 · February 24, 2015, 2:47am

I have started to work on a media plugin that supports rtsp media decoding. But it turns out that the Media Player ignores any path other thatn a local system path!!!

Any change that can be changed?

anonymous_user_9e8de314 · February 24, 2015, 6:57pm

In MediaPlayerCustomization.cpp you guys only check for SupportedFormats, but it would be nice to have a check for URI scheme support so you could open a rtsp stream like this

rtsp://192.168.1.200/stream1/

or a file

file:///c:/mymedia.m4v

Anyone working on something like this? Or am I on my own?

I have a standalone example working with live media streaming over rtsp from camera using live555 and cuda H264 decoder. Now I’m trying to create a plugin for Unreal, but it looks like I would have to do some serious hacking!!

anonymous_user_9e8de314 · February 25, 2015, 10:56pm

Woohoo got live H264 720P (1280*720) rtsp streaming working from my GoPro4 Currently have about 300ms latency, need to look a bit further into why unreal have that high latency in texture rendering. Sample application have about 160ms latency!

anonymous_user_9e8de314 · February 25, 2015, 11:02pm

After looking a bit I found this:

In MediaSampleBuffer.h

// IMediaSink interface

void ProcessMediaSample( const void* Buffer, uint32 BufferSize, FTimespan Duration, FTimespan Time ) override
{
	TArray&lt;uint8&gt;* Sample = new TArray&lt;uint8&gt;();
	Sample-&gt;AddUninitialized(BufferSize);
	FMemory::Memcpy(Sample-&gt;GetData(), Buffer, BufferSize);

	CurrentSample = MakeShareable(Sample);
	CurrentSampleTime = Time;
}

Looks like you are allocating memory for each media sample! Can’t we just allocate once and then do memcpy? Or allocate a pool of memory?

gmpreussner · February 27, 2015, 11:19pm

Hi ,

Good job on getting the plug-in work!

You are right, we are currently only checking by file extension to determine the player plug-in. I will add support for URL schemes as well.

I’m not sure where your high latency stems from. It is certainly not the memory allocation as there is only one allocation per video frame, and the size of the allocated memory doesn’t matter. I think you may have to do some profiling in order to determine what’s taking so long.

As for the allocation itself, we allocate a new block per frame, because we don’t know how long that data will be used and how many consumers there are. For example, if you have the Editor up, the frame may be shared between the Texture Editor, the Media Player Editor and the thumbnail renderer in the Content Browser. Instead of copying the data for each one, they all access the same buffer through a shared pointer, and when the last consumer is done using it, the buffer will self-destruct.

Are you going to be at GDC next week? If yes, I’d love to chat with you in person. If not, please feel free to continue responding with any questions or progress updates here. If you need any help, I’ll be around. I’ll see what I can do about the URL schemes

EDIT:

I forgot to add that, if you’re not on 4.7 yet, you may want to upgrade. A bunch of issues with the MediaPlayerCustomization were fixed, particularly the handling of file paths and URLs. You should be able to use any path or URL now. It would be even better if you could sync to the Master branch every once in a while as there are still a lot of changes going into the Media Framework. More stuff will be added after GDC.

anonymous_user_9e8de314 · February 28, 2015, 12:11am

I’m synced with the master branch so should have the latest and greatest.

Yes I 'll be at GDC next week, but only got Expo Pass this year. Would love to meet so I can show you the setup and also tell you a little about what we are trying to build.

gmpreussner · February 28, 2015, 12:41am

Ok sounds good. The expo pass should give you access to the Epic Games booth, right? Just swing by and ask for Max

gmpreussner · March 2, 2015, 8:42pm

,

I checked in some changes that should allow you to add support for RTSP (CL# 2466800, GitHub Commit).

Rather than adding more APIs for URI schemes, I have instead removed the assumptions about media URLs containing file extensions. This is an API breaking change. Your plug-in must now implement the IMediaPlayerFactory::SupportsUrl() method. In there you can perform custom checks, such as Url.StartsWith(TEXT(“rtsp://”)). I also renamed the old GetSupportedFormats() method to GetSupportedFileTypes() in order to make clear that this function is used only for file based media containers that have a file extension.

Let me know if you run into any other problems or suggestions, thanks!

anonymous_user_9e8de314 · March 3, 2015, 4:01am

Thanks Max,

I updated my plugin, but I still have issues in the editor. It looks like the code in

void FMediaPlayerCustomization::HandleUrlPickerPathPicked( const FString& PickedPath )

is calling ConvertRelativePathToFull that converts the rtsp://… path to something strange.

BTW: I’ll try to catch you Wednesday morning at your booth to see if we can fix the texture update delay.

gmpreussner · March 9, 2015, 10:59pm

This should now be fixed in CL# 2473173 (GitHub Commit).

I1ariusz · May 19, 2015, 2:55pm

Hello , could you spare some informations as to how you have implemented RTSP plugin? What kind of libraries do your recommend?

I`m also interested in making such extension. I understandt that it is not applicable to existing WMF player.

anonymous_user_edacb118 · June 7, 2015, 9:10am

hi ,thank you very much,This is a great contribution, why I do it in step, but I do not show the video flow of RTSP?

anonymous_user_cc50648b · September 5, 2016, 3:50pm

This is exactly what I am trying to do! Trying to get the cuda parser (created with cuvidCreateVideoParser) to consume h264 data that is streamed from VLC over rtsp using the live555 library. I send the data to the parser and it doesn’t do anything (at least if I send the raw UDP to it I at least get the callbacks working). I would love to know how you managed it.