Hello Evil_Fischi,
Firstly I would make sure all your videos are referencing the correct file path location from your Unreal Projects/YourProject/Content/Movies folder. They might all be in there, but be sure within the project that the file directory is referencing that location.
In regards to what is and is not expected to work, the Media Frame work as mentioned in our documentation is both Engine and Slate agnostic, which means it can be used in pretty much any application not just the game engine or the Editor. This also means that any restrictions in place for mobile for various set ups will apply to the media player as well.
If you would like you can provide me with a test project or steps to follow so I can try and reproduce the issue you are experiencing, on my end. Let me know if you have further questions or need additional assistance.
Cheers,