Announcement

Collapse
No announcement yet.

4.20 Virtual Camera Project and Plugin

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    4.20 Virtual Camera Project and Plugin

    I've been experimenting with the Virtual Camera project and plugin shipped with 4.20, and figured it might be helpful for those of us working with it to have a central location for sharing information about how it works. Documentation for the Virtual Camera plugin lives here.

    Here's what I've discovered so far:
    • The easiest way to test it that I've found so far is to install Unremote2 on an ARKit-capable iPad, and connect it to the IP address of the PC running the VirtualCameraSample scene. Works pretty well, but is sensitive to network latency.
      • If your performance is poor, check the following:
        • Is your play-in-editor window set to the same resolution as your iPad? Performance will suffer if it's having to scale dynamically.
        • Are you facing a blank wall? ARKit relies on visible geometry to track movements reliably. If you're using it in a room with blank walls, adding markers to the walls can help.
    • Virtual Camera assets and code live in the Virtual Camera plugin. To view these assets, enable Show Engine Content and Show Plugin Content in your content browser's view options.
    • It seems generally to be a good idea to lock rotation on your iPad when using the virtual camera.
    UIClick image for larger version  Name:	VirtualCameraUI.JPG Views:	1 Size:	217.7 KB ID:	1512104
    • UI Elements, Top Row:
      • Hide/Show UI
        • Click image for larger version  Name:	VirtualCamer_ShowUI.JPG Views:	1 Size:	1.0 KB ID:	1512114
        • You can set user options to determine which elements remain displayed when UI is turned off.
      • Focus mode: Manual, Target, Automatic
        • Click image for larger version  Name:	VirtualCamera_FocusMode.JPG Views:	1 Size:	2.3 KB ID:	1512115
        • Be aware that targeted focusing only works while the focus UI is displayed.
        • When using Manual focus, you can dial in a desired focus distance or tap an object in scene to set focus distance.
        • To set a focus target, have the focus UI displayed and tap the target object to set it as the tracked focus object.
        • When using auto-focus, you can move the focus reticle to set your desired focus point.
      • Options
        • Click image for larger version  Name:	VirtualCamera_OptionsButton.JPG Views:	1 Size:	1.3 KB ID:	1512116
        • General Settings
          • Click image for larger version  Name:	VirtualCamera_GeneralSettings.JPG Views:	1 Size:	25.8 KB ID:	1512105
          • Input Source
            • ARKit uses the iPad Pro's ARKit implementation to track the camera position.
            • Vive tracking requires the Steam Controller plugin and Steam VR plugins to be enabled.
          • Playback
          • Global Boom
            • Not yet clear what this describes.
        • Film Format / Aspect Ratio
          • Click image for larger version  Name:	VirtualCamera_FilmFormat.JPG Views:	1 Size:	27.7 KB ID:	1512106
          • Film Format
            • Update the CineCamera Component of the Virtual Camera Pawn to add new camera types.
          • Matte
          • Matte Opacity
        • Focus
          • Click image for larger version  Name:	VirtualCamera_Focus.JPG Views:	1 Size:	31.3 KB ID:	1512107
          • Focus Method
            • Identical to the MTA buttons on the main UI.
          • Focus Distance
          • Focus Visualization
        • Stabilization
          • Click image for larger version  Name:	VirtualCamera_Stabilization.JPG Views:	1 Size:	28.0 KB ID:	1512108
          • Stabilize Movement
          • Stabilize Rotation
        • Axis Locking
          • Click image for larger version  Name:	VirtualCamera_AxisLocking.JPG Views:	1 Size:	35.1 KB ID:	1512109
          • Truck
          • Dolly
          • Boom
          • Tilt
          • Pan
          • Dutch
        • Motion Scale
          • Click image for larger version  Name:	VirtualCamera_MotionScale.JPG Views:	1 Size:	35.5 KB ID:	1512110
          • Truck
          • Dolly
            • Truck and Dolly can be locked in sync with each other using the link button between them.
          • Boom
        • View Freeze
          • Click image for larger version  Name:	VirtualCamera_ViewFreeze.JPG Views:	1 Size:	32.6 KB ID:	1512111
          • Allows you to freeze the view to move to a new physical location for easier tracking.
        • Visibility
          • Click image for larger version  Name:	VirtualCamera_Visibility.JPG Views:	1 Size:	33.3 KB ID:	1512112
        • Presets
          • Click image for larger version  Name:	VirtualCamera_Presets.JPG Views:	1 Size:	31.9 KB ID:	1512113


    .
    Last edited by KevODoom; 08-13-2018, 06:18 PM.

    #2
    • UI, Middle Row
      • Focal Length
        • Click image for larger version

Name:	VirtualCamera_FocalLength.JPG
Views:	1
Size:	10.2 KB
ID:	1512121
      • Focus Distance and Lens Aperture
        • Click image for larger version

Name:	VirtualCamera_FocusAndAperture.JPG
Views:	1
Size:	12.7 KB
ID:	1512122

    Comment


      #3
      • UI, Bottom Row
        • Record
          • Click image for larger version  Name:	VirtualCamera_RecordMenu.JPG Views:	1 Size:	6.1 KB ID:	1512130
          • Screenshot Menu
            • Allows user to manage screenshots
          • Screenshot Button
            • Takes a screenshot
          • Main Record Button
            • Starts and stops a camera movement recording
        • Joystick Docking
          • Click image for larger version  Name:	VirtualCamera_Docking.JPG Views:	2 Size:	1.7 KB ID:	1512132
          • If your virtual joysticks aren't working, ensure that you've set bAlwaysShowTouchInterface=True in your DefaultInput.ini
        • Performances
          • Click image for larger version  Name:	VirtualCamera_Performances.JPG Views:	1 Size:	7.6 KB ID:	1512133
          • Allows selection of existing recorded or generated Sequences in the scene for playback
        • Waypoints
          • Click image for larger version  Name:	VirtualCamera_Waypoints.JPG Views:	1 Size:	5.8 KB ID:	1512134
          • Waypoints Menu
            • Displays a list of saved waypoints
          • Home
            • Teleports the view to the home waypoint
          • Main Waypoint
            • Saves the current view location as a waypoint
      Attached Files
      Last edited by KevODoom; 08-10-2018, 10:44 PM.

      Comment


        #4
        i tired using the example project with my ipad 12.9in
        at first it was working with rotation
        but after following the https://docs.unrealengine.com/en-us/...alCameraPlugin
        instructions I can't get UE4 to react to the ipad
        any thoughts?

        Comment


          #5
          Originally posted by YuuJin View Post
          i tired using the example project with my ipad 12.9in
          at first it was working with rotation
          but after following the https://docs.unrealengine.com/en-us/...alCameraPlugin
          instructions I can't get UE4 to react to the ipad
          any thoughts?
          This happened to me as well.

          Commenting out this block in DefaultEngine.ini restored movement:

          [RemoteSession]
          +Channels=(Name=FRemoteSessionFrameBufferChannel,Mode=Write)
          +Channels=(Name=FRemoteSessionInputChannel,Mode=Read)
          +Channels=(Name=FRemoteSessionARCameraChannel,Mode=Read)
          +Channels=(Name=FRemoteSessionXRTrackingChannel,Mode=Read)

          Camera rotation did work briefly with these RemoteSession channels set up, but then failed on a subsequent run and couldn't be used again without commenting the block out.
          Anybody else have thoughts on this?

          Comment


            #6
            I to had this problem. Removed it and now have movement back through the ARKit function. Weirdly however my vive tracker know longer works through the IPad. It works fine through the PIE window but as sson as I connect via the remote app the camera changes. Any ideas on this? or how its integration works. I wonder if when the remote connection is made its pulling a different Tracker ID
            Last edited by JTilla; 08-19-2018, 11:20 AM.

            Comment


              #7
              Originally posted by KevODoom View Post

              This happened to me as well.

              Commenting out this block in DefaultEngine.ini restored movement:

              [RemoteSession]
              +Channels=(Name=FRemoteSessionFrameBufferChannel,Mode=Write)
              +Channels=(Name=FRemoteSessionInputChannel,Mode=Read)
              +Channels=(Name=FRemoteSessionARCameraChannel,Mode=Read)
              +Channels=(Name=FRemoteSessionXRTrackingChannel,Mode=Read)

              Camera rotation did work briefly with these RemoteSession channels set up, but then failed on a subsequent run and couldn't be used again without commenting the block out.
              Anybody else have thoughts on this?
              A response from Andrew Grant indicates the following:
              Andrew Grant

              Epic Games, Inc.
              These are no longer needed, in an earlier version it was necessary for a project to specify its required channels but now the virtual plugin sets them at runtime.

              I imagine the problem is the ARCameraChannel which is both experimental and nothing to do with Virtual Camera
              So it looks as though the correct response is not to specify these RemoteSession channels.

              Comment


                #8
                Hey guys,
                Ive managed to get this to work and connect, ive tried pasting the remote session and removing it from the .ini
                But my gyroscope doesnt seem to work at all

                Is there something I could be missing?
                And does it require Xcode from a mac OS to run

                Cheers!

                Comment


                  #9
                  Hi Stylerm,
                  You shouldn't need a Mac or Xcode for the virtual camera plugin - just the Unreal Remote 2 app from the app store.
                  The RemoteSession channels specification should not be included in your DefaultEngine.ini - it will interfere with your camera rotation.
                  Not sure why your gyroscope wouldn't be responding if it's all set up correctly - I imagine you've tested it with their example project already. Be aware that it is pretty sensitive to WiFi congestion, so if your network's not allowing solid data throughput, it can take a while for the iOS device and the desktop session to get in sync with each other. Hmm.

                  Comment


                    #10
                    Could be potentially due to the wi-fi congestion, but I have tried it without the channels and it gives me the same issue.
                    Ive also tried both sample project and my own and neither seem to pick up gyroscope. I can interact with controls and record, also use camera functions. But the gyroscope movements arent picked up

                    Thanks for the response!

                    Comment


                      #11
                      I just got a new ipad pro (11 inch, IOS 12.1) yesterday and for the life of me cant get the Unreal Remote 2 app to work at all. I only get a black screen for about 10 seconds then the app minimises, no screen to enter IP addresses etc. Is the 11inch somehow incompatible with the app? anyone else successfully run it?

                      Comment


                        #12
                        Hey, has anyone resolved the issue where the movement from the Ipad is not being received by Unreal?
                        i tried with and without the changes to the .ini files.
                        The joystick navigation data is being sent through fine from the Ipad though.

                        I am on 4.20. Using ipad pro, version 2 or 3.
                        Thanks!

                        Comment


                          #13
                          Hey guys,

                          Does anybody have an idea how to remove preview of camera controls on the pc editor window (PIE) and retain them on tablet window at the same time?

                          Thanks!

                          Comment


                            #14
                            Hello! i was able to connect but it works for 1 second and then freeze. I'm using an ipad pro with everything updated.

                            Comment

                            Working...
                            X