When rendering a 4K monoscopic video with media framework on Gear VR, the video quality drops dramatically as soon as any other object starts being rendered on top of the video. Even a debug message being printed on screen will trigger the quality drop, and the quality will stay low even after the message has disappeared. Any other kind of overlay will also do it, for example, leaving the Default Virtual Joysticks enabled (default for Mobile projects).
Here are the steps to reproduce the bug:
- Using UE 4.16, create a new project with the following parameters: C++, Basic Code, Mobile/Tablet, Maximum Quality, No Starter Content;
- Create a new Empty Level;
- Create a new MediaPlayer asset (check to create both additional assets: SoundWave and MediaTexture);
- Create a Material from the MediaTexture;
- Edit this newly created Material: Check the “Two sided” option; Connect the Texture Sample output to the “Emissive Color” input of the final material node; Save and close it;
- (Optional) Import a high-poly Sphere static mesh with equirectangular UVs;
- Add a Sphere (or the imported one) to the Level, reset its Location to default (0, 0, 0);
- Apply the Material (created from MediaTexture) to the Sphere;
- (Optional) Add the SoundWave Asset to the Level, reset its Location to default (0, 0, 0);
- Add a Player Start to the Level and reset its Location to default (0, 0, 0);
- Place a high resolution (4K) 360º monoscopic video file inside the /Content/Movies folder;
- Add a new FileMediaSource asset and set its “File Path” to the video file from the previous step;
- Open the Level Blueprint and set the Event Graph like the following: On BeginPlay, Open Source using the MediaPlayer and the FileMediaSource; On InputTouch Pressed, Print String (any string), making sure “Print to Screen” is checked;
- Save all the assets and the level;
- Edit the Project Settings:
- In Maps & Modes, set the created Level as the “Game Default Map”;
- In Packaging, add the created Level to the “List of maps to include in a packaged build”;
- In Engine - Input, clear the “Default Touch Interface” (should display “None”);
- In Platforms - Android, press the “Configure Now” button, set both Minimum and Target SDK Versions to 21 and enable “Configure the AndroidManifest for deployment to GearVR”;
- In Engine - Rendering, disable “Mobile HDR” (This should prompt an engine restart to apply the settings. Do so right away);
- Package the project as Android ASTC;
- Install the package in a Samsung GearVR-compatible phone (used Galaxy S7 in my tests) and run it;
- You should be able to watch the video in the intended quality.
- Touch the Gear VR touchpad (or the phone’s screen if running with Developer mode on), and a message will be printed on the screen while the video will instantly have its quality lowered, becoming much more pixelated and/or with a lot of aliasing.
If you repeat the steps above using UE 4.14 or 4.13 instead, the video quality will remain the same (instead, what may happen in this case is an upside-down video being rendered until the first touch, which is a known bug, already fixed in recent versions).
The following repos contain two projects created using the above steps. The first one uses UE 4.14, in which the bug does not occur, and the other uses UE 4.16, in which it does. Important to note that this bug was not present in 4.13, but in 4.15 it already was.
This bug happens regardless of video codec (tested with H.264 and HEVC) or resolution (tested 4096x2048 3840x2048, among others).
Researching online, this issue seems to be very similar (if not the same root cause) to the following recounts:
Finally, here are two images to better illustrate the problem:
UE4.14 without Bug (expected video quality):
UE4.16 with Bug (poor video quality):