[SUPPORT] Stereo Panoramic Player

[UNAmedia]

[Update 2020-03-10]

Version 1.1.0 is out. See the ChangeLog for details. Added a new rendering mode (more efficient and supports out-of-the-box the rendering of 3D meshes). Simplified the management of custom UMG hotspots. This version is backward compatible.

[Update 2019-05-07]

The plugin is available in the Marketplace: https://unrealengine.com/marketplace...noramic-player

[Original]

Hi,

we just submitted "Stereo Panoramic Player", a plugin to quickly integrate an interactive, customizable player for 360° panoramic images and videos, supporting both mono and stereo media sources. It will be available as soon as Epic will approve it. Below an introduction.

Description
Stereo Panoramic Player is a runtime plugin to show and interact with 360° panoramic images and videos, supporting both mono and stereo sources. It offers both an high-level solution (Panoramic Director), to quickly assemble an interactive panoramic virtual tour, and a low-level solution (Panoramic Sphere) to allow a programmatic customization of the user experience. It’s compatible with all the major platforms and it’s actively tested on PC, VR (Steam and Oculus), iOS and Android.

Documentation: https://www.unamedia.com/ue4-stereo-...ic-player/api/

The Panoramic Director actor allows you to define, directly in-editor, a complete panoramic virtual tour: for each imported panoramic source (image and/or video) you can define custom hotspots to move between different media; the actor will then take care of everything at run-time, detecting when the user is interacting with an hotspot and managing the transition between the panoramic media.

The Panoramic Sphere actor gives you complete control of the panoramic experience: you can programmatically control it to render panoramic media and detect interactions with the panoramic sources. It can render images and videos, supporting both mono and stereo formats.

Technical details
Features:

• rendering of 360° panoramic images (mono and stereo);
• playback of 360° panoramic videos (mono and stereo);
• support for Equirectangular projection;
• support for Over-Under stereoscopic layout;
• overlayed UMG widgets for user interaction;
• interactive areas through color-coded masks;
• graphics effects for transitions and blending with the 3D environment;
• rendering of 3D objects inside the panoramic sphere;
• high-level actor (class APanoramicDirector) to quickly assemble a panoramic virtual tour directly in the editor;
• low-level actor (class APanoramicSphere) for programmatic and complete access to all the supported features;
• all the features available from both Blueprints and C++.

Code Modules:

• StereoPanoramicPlayer - Runtime

Network Replicated: No

Supported Development Platforms: Windows, Mac.
Supported Target Build Platforms: Windows (32/64-bits), Mac, iOS, Android, VR (SteamVR/Vive, Oculus).

Documentation: https://www.unamedia.com/ue4-stereo-...ic-player/api/

Example Project:

• Source code: https://github.com/UNAmedia/ue4-ster...ic-player-demo
• Pre-built binaries: https://github.com/UNAmedia/ue4-ster...-demo/releases

[Chisomo]

This is exactly what I need. Is it available yet

[UNAmedia]

This is exactly what I need. Is it available yet

Hi! Thanks for asking. Epic approved the plugin few days ago, we are finalizing some PR materials and we'll made the plugin available in few days!

[UNAmedia]

Hi everyone! The plugin is officially available in the marketplace: https://unrealengine.com/marketplace...noramic-player . Write us for any question or support request.

In the next weeks we'll make available a reference sample UE4 project compatible with PC, VR, iOS and Android. It will depend on the plugin and will showcase some common use-cases.

[surferHalo]

Hi, I got some question before purchase.

Currently I've built my own CAVE system which is kind like the NDisplay, what's different is that my CAVE system is based on the Unreal GameFramework, utilize the Unreal Replicate system(not like the NDisplay doing some OutBand replicate). In a nutshell, my CAVE system is a traditional multiplayer system, every machine instances which built up the CAVE cluster is just a traditional multiplay client.

What I want to do with your plugin is can I treat it just like a spherical skybox, then every client (just simple camera facing at different direction) of my CAVE cluster can see the paranomic sphere at different side.I can do the network replicate work like sync the Panoramic video play in different Cluster Client.

All I just want to make sure is, your plugin should not take control of my PlayerController, PlayerCameraManager, and especailly my Pawn. It should just be a simple Sphere and place it at some place in my control, then I can control my pawn and camera in the exact place to watch the around paranoma.

So the PanoramicDirector actor in your plugin should not work, its just like Sequencer's CameraCuts taking control of my local PlayerController or Pawn, the PanoramicSphere actor, which you say give me complete control, does it meet my needs? If it is taking control nothing, just be it a simple sphere playing its own paranomic image, then it should no problem.

[UNAmedia]

Hi, I got some question before purchase.

Currently I've built my own CAVE system which is kind like the NDisplay, what's different is that my CAVE system is based on the Unreal GameFramework, utilize the Unreal Replicate system(not like the NDisplay doing some OutBand replicate). In a nutshell, my CAVE system is a traditional multiplayer system, every machine instances which built up the CAVE cluster is just a traditional multiplay client.

What I want to do with your plugin is can I treat it just like a spherical skybox, then every client (just simple camera facing at different direction) of my CAVE cluster can see the paranomic sphere at different side.I can do the network replicate work like sync the Panoramic video play in different Cluster Client.

All I just want to make sure is, your plugin should not take control of my PlayerController, PlayerCameraManager, and especailly my Pawn. It should just be a simple Sphere and place it at some place in my control, then I can control my pawn and camera in the exact place to watch the around paranoma.

So the PanoramicDirector actor in your plugin should not work, its just like Sequencer's CameraCuts taking control of my local PlayerController or Pawn, the PanoramicSphere actor, which you say give me complete control, does it meet my needs? If it is taking control nothing, just be it a simple sphere playing its own paranomic image, then it should no problem.

Hi! Thanks for asking.

Neither the PanoramicSphere nor PanoramicDirector take control of the player Pawn or used camera, so you should be able to use both of them in your user-case.

Using the PanoramicDirector in your use-case could be a little tricky, but you would end with more functionalities out-of-the box. You can use the APanoramicDirector::OnStageChanged delegate on the "authoritative" machine to detect interactions, and then manually "synchronize" the change of Stage in the remote PanoramicDirector instances calling on them APanoramicDirector::MoveOrExit(). You'll likely have to disable automatic management of inputs on non authoritative machines (see APanoramicDirector::InteractionComponent; and UPanoramicInteractionComponent::PushCustomWorldTargetDirection() and UPanoramicInteractionComponent::TryToInteractManually() to manually inject inputs).

In alternative, the PanoramicSphere class should work without any issue. It's a "simple" (non-replicated) actor that doesn't interact directly with the player. It has low level methods (eg APanoramicSphere::UVMap()) to allow you to manually manage interactions with the sphere. As long as you can synchronize its state between different machines, you should be able to use it for your use-case.

If you decide to try the plugin, let me know of your experience and if you need any further help and assistance. This is the first release of the plugin, we tried to expose all the basic functionalities to manage the most frequent use-cases and if there's something we missed we are very interested to know and fix it!

Best regards.

[surferHalo]

@UNAmedia Thanks for your reply, I will test it out.
I got an engine crash when I play the video source directly with the PanoramicExperienceDataAsset, below is the crash report:
https://drive.google.com/open?id=1Ts...ykzZBpQTQm2fNO

I can play the video in MediaTexture assigned to the a normal sphere mesh, but the Video sync is not always accurate correct in frame time, especially when video played after one or two minutes, I know its because the multiplay is not framesync. But may I request a function that you expose a function can effciently set the video to a exact frame or time, and play from there. Then I can sync the video every five or ten seconds, fake it nicely synchronize. I can sync the video time with the MediaPlayer's Seek function, but it seems being a very expensive call, it stuttered and make the video sync worse in production enviroment(two different computer play the same video).
By the way, I found this useful blog: https://medium.com/@invicticide/accu...k-87a3f9262594

Best regards, thanks for this brilliant plugin.

[UNAmedia]

@UNAmedia Thanks for your reply, I will test it out.
I got an engine crash when I play the video source directly with the PanoramicExperienceDataAsset, below is the crash report:
https://drive.google.com/open?id=1Ts...ykzZBpQTQm2fNO

Hi! I downloaded the crash log and we'll investigate it. Thanks for reporting it! It seems an issue with the asynchronous loading of an UPanoramicStage asset while navigating between stages. If we'll need further info, I'll write you a PM.

I can play the video in MediaTexture assigned to the a normal sphere mesh, but the Video sync is not always accurate correct in frame time, especially when video played after one or two minutes, I know its because the multiplay is not framesync. But may I request a function that you expose a function can effciently set the video to a exact frame or time, and play from there. Then I can sync the video every five or ten seconds, fake it nicely synchronize. I can sync the video time with the MediaPlayer's Seek function, but it seems being a very expensive call, it stuttered and make the video sync worse in production enviroment(two different computer play the same video).
By the way, I found this useful blog: https://medium.com/@invicticide/accu...k-87a3f9262594

Best regards, thanks for this brilliant plugin.

You should be able to solve it with the current API. If you are using APanoramicDirector:

• the UMediaPlayer object used by APanoramicDirector is set in the property APanoramicDirector::MediaPlayer.
• we call UMediaPlayer::Play() soon before we notify the change of stage event;
• you can register to the delegate APanoramicDirector::OnExperienceStarted to be notified when such change of stage happens;
• from there (or later, when you'll sync the client with the server), you can call UMediaPlayer::Seek() to set the playback position to whatever you want (check also UMediaPlayer::SupportsSeeking()).

Best regards!

[UNAmedia]

@UNAmedia Thanks for your reply, I will test it out.
I got an engine crash when I play the video source directly with the PanoramicExperienceDataAsset, below is the crash report:
https://drive.google.com/open?id=1Ts...ykzZBpQTQm2fNO

Hi! I think we tracked the source of the crash, it seems caused by a call to APanoramicDirector::Play() followed immediately by a call to APanoramicDirector::End(). This causes a check() to fail because stage data are not loaded in time before the call to End(). In the next days we'll submit a fix for this issue. In the meantime, if you can avoid this quick Play/End sequence, the crash should not happen. If this is not the case, please let me know.

I can play the video in MediaTexture assigned to the a normal sphere mesh, but the Video sync is not always accurate correct in frame time, especially when video played after one or two minutes, I know its because the multiplay is not framesync. But may I request a function that you expose a function can effciently set the video to a exact frame or time, and play from there. Then I can sync the video every five or ten seconds, fake it nicely synchronize. I can sync the video time with the MediaPlayer's Seek function, but it seems being a very expensive call, it stuttered and make the video sync worse in production enviroment(two different computer play the same video).
By the way, I found this useful blog: https://medium.com/@invicticide/accu...k-87a3f9262594

I misread part of your previous message. Unfortunately we rely on the standard UE4 media framework for media playback and providing more advanced playback solutions is beyond the scope of this plugin. Anyway you could try pre-caching the media resource, loading it in memory before playing it. This should reduce the syncing issues and make the Seek() calls cheaper to be called, as they can work directly in memory. The pre-caching is a property of the UFileMediaSource asset. See also https://docs.unrealengine.com/en-us/...ework/Overview (search "Precache") and https://answers.unrealengine.com/que...fer-video.html .

Best regards.

[surferHalo]

Aha! Thanks for your tip, I solve it using the precache, and also I found a way that automatic sync the media play: Using sequencer's Media Track and set sequncer replicate playback(should use external MediaPlayer and cache ahead serveral seconds, will stutter in client if no cache).

[UNAmedia]

Aha! Thanks for your tip, I solve it using the precache, and also I found a way that automatic sync the media play: Using sequencer's Media Track and set sequncer replicate playback(should use external MediaPlayer and cache ahead serveral seconds, will stutter in client if no cache).

Great! And thanks for sharing the "Sequencer" tip for replicate playback!

[surferHalo]

Can not compile for a custom source build, not a big problem, seems you should add explicit category specifier.

1>------ Build started: Project: CAVESystemDev, Configuration: Development x64 ------ 1>Using 'git status' to determine working set for adaptive non-unity build (E:\Epic Games\UnrealGraniteTrueSky422). 1>Creating makefile for CAVESystemDev (.uproject file is newer) 1>Parsing headers for CAVESystemDev 1> Running UnrealHeaderTool "E:\Temp\CAVESystemDev\CAVESystemDev.uproject" "E:\Temp\CAVESystemDev\Intermediate\Build\Win64\CAVESystemDev\Development\CAVESystemDev.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicPosition.h(35) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicPosition.h(39) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(36) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(40) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(47) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(51) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(55) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(61) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(104) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(108) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(112) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(117) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(130) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(147) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(151) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(155) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(159) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(163) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(174) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(180) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicStage.h(184) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicDirector.h(145) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicDirector.h(149) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicExperience.h(27) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicExperience.h(31) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicPawn.h(38) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicPawn.h(41) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicSphere.h(28) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Public/PanoramicSphere.h(32) : LogCompile: Error: An explicit Category specifier is required for any property exposed to the editor or Blueprints in an Engine module. 1>E:/Epic Games/UnrealGraniteTrueSky422/Engine/Plugins/Marketplace/StereoPanoramicPlayer/Source/StereoPanoramicPlayer/Private/ProceduralSphereComponent.h(22) : LogCompile: Error: An explicit Category specifier is required for Blueprint accessible functions in an Engine module. 1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.MakeFile.Targets(44,5): error MSB3075: The command ""E:\Epic Games\UnrealGraniteTrueSky422\Engine\Build\BatchFiles\Build.bat" CAVESystemDev Win64 Development -Project="E:\Temp\CAVESystemDev\CAVESystemDev.uproject" -WaitMutex -FromMsBuild" exited with code 5. Please verify that you have sufficient rights to run this command. 1>Done building project "CAVESystemDev.vcxproj" -- FAILED. ========== Build: 0 succeeded, 1 failed, 2 up-to-date, 0 skipped ==========

[UNAmedia]

Hi! We just submitted v1.0.1 to Epic for review:

• fixed a possible crash when calling method APanoramicDirector::BeginPlay() followed immediately by APanoramicDirector::EndPlay();
• fixed missing "Category" declarations on some UPROPERTY fields.

[UNAmedia]

Hi! We just submitted v1.0.1 to Epic for review:

• fixed a possible crash when calling method APanoramicDirector::BeginPlay() followed immediately by APanoramicDirector::EndPlay();
• fixed missing "Category" declarations on some UPROPERTY fields.

The update is available on the marketplace. You can update it directly from the Epic Launcher.

[WHHwxiaop]

Hi,can provide the Example Project as soon as possible?