4.20 Virtual Camera Project and Plugin

Development Platform & Builds XR Development
1

I’ve been experimenting with the Virtual Camera project and plugin shipped with 4.20, and figured it might be helpful for those of us working with it to have a central location for sharing information about how it works. Documentation for the Virtual Camera plugin lives here.

Here’s what I’ve discovered so far:

  • The easiest way to test it that I’ve found so far is to install Unremote2 on an ARKit-capable iPad, and connect it to the IP address of the PC running the VirtualCameraSample scene. Works pretty well, but is sensitive to network latency.

    • If your performance is poor, check the following:

      • Is your play-in-editor window set to the same resolution as your iPad? Performance will suffer if it’s having to scale dynamically.

      • Are you facing a blank wall? ARKit relies on visible geometry to track movements reliably. If you’re using it in a room with blank walls, adding markers to the walls can help.

  • Virtual Camera assets and code live in the Virtual Camera plugin. To view these assets, enable Show Engine Content and Show Plugin Content in your content browser’s view options.

  • It seems generally to be a good idea to lock rotation on your iPad when using the virtual camera.

UI

VirtualCameraUI.JPG
VirtualCameraUI.JPG1293×755 186 KB

  • UI Elements, Top Row:

    • Hide/Show UI

      • VirtualCamer_ShowUI.JPG

      • You can set user options to determine which elements remain displayed when UI is turned off.

    • Focus mode: Manual, Target, Automatic

      • VirtualCamera_FocusMode.JPG

      • Be aware that targeted focusing only works while the focus UI is displayed.

      • When using Manual focus, you can dial in a desired focus distance or tap an object in scene to set focus distance.

      • To set a focus target, have the focus UI displayed and tap the target object to set it as the tracked focus object.

      • When using auto-focus, you can move the focus reticle to set your desired focus point.

    • Options

      • VirtualCamera_OptionsButton.JPG

      • General Settings

        • VirtualCamera_GeneralSettings.JPG

        • Input Source

          • ARKit uses the iPad Pro’s ARKit implementation to track the camera position.

          • Vive tracking requires the Steam Controller plugin and Steam VR plugins to be enabled.

        • Playback

        • Global Boom

          • Not yet clear what this describes.

      • Film Format / Aspect Ratio

        • VirtualCamera_FilmFormat.JPG

        • Film Format

          • Update the CineCamera Component of the Virtual Camera Pawn to add new camera types.

        • Matte

        • Matte Opacity

      • Focus

        • VirtualCamera_Focus.JPG

        • Focus Method

          • Identical to the MTA buttons on the main UI.

        • Focus Distance

        • Focus Visualization

      • Stabilization

        • VirtualCamera_Stabilization.JPG

        • Stabilize Movement

        • Stabilize Rotation

      • Axis Locking

        • VirtualCamera_AxisLocking.JPG

        • Truck

        • Dolly

        • Boom

        • Tilt

        • Pan

        • Dutch

      • Motion Scale

        • VirtualCamera_MotionScale.JPG

        • Truck

        • Dolly

          • Truck and Dolly can be locked in sync with each other using the link button between them.

        • Boom

      • View Freeze

        • VirtualCamera_ViewFreeze.JPG

        • Allows you to freeze the view to move to a new physical location for easier tracking.

      • Visibility

        • VirtualCamera_Visibility.JPG

      • Presets

        • VirtualCamera_Presets.JPG

.

2

  • UI, Middle Row

    • Focal Length

      • VirtualCamera_FocalLength.JPG

    • Focus Distance and Lens Aperture

      • VirtualCamera_FocusAndAperture.JPG

3

  • UI, Bottom Row

    • Record

      • VirtualCamera_RecordMenu.JPG

      • Screenshot Menu

        • Allows user to manage screenshots

      • Screenshot Button

        • Takes a screenshot

      • Main Record Button

        • Starts and stops a camera movement recording

    • Joystick Docking

      • VirtualCamera_Docking.JPG

      • If your virtual joysticks aren’t working, ensure that you’ve set bAlwaysShowTouchInterface=True in your DefaultInput.ini

    • Performances

      • VirtualCamera_Performances.JPG

      • Allows selection of existing recorded or generated Sequences in the scene for playback

    • Waypoints

      • VirtualCamera_Waypoints.JPG

      • Waypoints Menu

        • Displays a list of saved waypoints

      • Home

        • Teleports the view to the home waypoint

      • Main Waypoint

        • Saves the current view location as a waypoint

4

i tired using the example project with my ipad 12.9in
at first it was working with rotation
but after following the Virtual Cameras | Unreal Engine Documentation
instructions I can’t get UE4 to react to the ipad
any thoughts?

5

This happened to me as well.

Commenting out this block in DefaultEngine.ini restored movement:

[RemoteSession]
+Channels=(Name=FRemoteSessionFrameBufferChannel,Mode=Write)
+Channels=(Name=FRemoteSessionInputChannel,Mode=Read)
+Channels=(Name=FRemoteSessionARCameraChannel,Mode=Read)
+Channels=(Name=FRemoteSessionXRTrackingChannel,Mode=Read)

Camera rotation did work briefly with these RemoteSession channels set up, but then failed on a subsequent run and couldn’t be used again without commenting the block out.
Anybody else have thoughts on this?

6

I to had this problem. Removed it and now have movement back through the ARKit function. Weirdly however my vive tracker know longer works through the IPad. It works fine through the PIE window but as sson as I connect via the remote app the camera changes. Any ideas on this? or how its integration works. I wonder if when the remote connection is made its pulling a different Tracker ID

7

A response from Andrew Grant indicates the following: [TABLE=“cellpadding: 5”]

		[TABLE]

[TR]
[TD=“width: 130”]
Andrew Grant

					Epic Games, Inc.

[/TD]
[/TR]

		[TABLE="cellpadding: 8"]

[TR]
These are no longer needed, in an earlier version it was necessary for a project to specify its required channels but now the virtual plugin sets them at runtime.

					I imagine the problem is the ARCameraChannel which is both experimental and nothing to do with Virtual Camera
				


	[/TR]

So it looks as though the correct response is not to specify these RemoteSession channels.

8

Hey guys,
Ive managed to get this to work and connect, ive tried pasting the remote session and removing it from the .ini
But my gyroscope doesnt seem to work at all

Is there something I could be missing?
And does it require Xcode from a mac OS to run

Cheers!

9

Hi Stylerm,
You shouldn’t need a Mac or Xcode for the virtual camera plugin - just the Unreal Remote 2 app from the app store.
The RemoteSession channels specification should not be included in your DefaultEngine.ini - it will interfere with your camera rotation.
Not sure why your gyroscope wouldn’t be responding if it’s all set up correctly - I imagine you’ve tested it with their example project already. Be aware that it is pretty sensitive to WiFi congestion, so if your network’s not allowing solid data throughput, it can take a while for the iOS device and the desktop session to get in sync with each other. Hmm.

10

Could be potentially due to the wi-fi congestion, but I have tried it without the channels and it gives me the same issue.
Ive also tried both sample project and my own and neither seem to pick up gyroscope. I can interact with controls and record, also use camera functions. But the gyroscope movements arent picked up

Thanks for the response!

11

I just got a new ipad pro (11 inch, IOS 12.1) yesterday and for the life of me cant get the Unreal Remote 2 app to work at all. I only get a black screen for about 10 seconds then the app minimises, no screen to enter IP addresses etc. Is the 11inch somehow incompatible with the app? anyone else successfully run it?

12

Hey, has anyone resolved the issue where the movement from the Ipad is not being received by Unreal?
i tried with and without the changes to the .ini files.
The joystick navigation data is being sent through fine from the Ipad though.

I am on 4.20. Using ipad pro, version 2 or 3.
Thanks!

13

Hey guys,

Does anybody have an idea how to remove preview of camera controls on the pc editor window (PIE) and retain them on tablet window at the same time?

Thanks!

14

Hello! i was able to connect but it works for 1 second and then freeze. I’m using an ipad pro with everything updated.

15

Try testing on a WiFi network that has zero traffic, just your PC and iPhone. See if you have better luck.
Keep in mind the amount of data being passed back and forth.

16

I was having the same issue and about ready to give up.
Removed all of:

[RemoteSession]
+Channels=(Name=FRemoteSessionFrameBufferChannel,Mode=Write)
+Channels=(Name=FRemoteSessionInputChannel,Mode=Read)
+Channels=(Name=FRemoteSessionARCameraChannel,Mode=Read)
+Channels=(Name=FRemoteSessionXRTrackingChannel,Mode=Read)

And it worked fine after that

17

Hey guys,

check your apple device for arkit compatibility:

https://developer.apple.com/library/archive/documentation/DeviceInformation/Reference/iOSDeviceCompatibility/DeviceCompatibilityMatrix/DeviceCompatibilityMatrix.html

Cheers,
Kuba

18

Can we still using vive tracker for the location of Ipad pro?
I saw the latest official Virtual Camera Plugin document removed the Vive tracker words.
Now only shows Vicon or Optitrack.
And I found up there is still a vive tracker unrecognized with unreal remote 2 issue,

If they really give up on steamvr tracking for the virtual camera, that’s really sad…

19

You can force using the tracker through blueprint and use the “custom” option instead of Arkit in settings

20

May i ask for a bit moredetail how you solved that? Thanks in advance

Another question is, if somebody figured out, why the steamvr tracking is freezing as soon the ipad unreal remote2 app connects to unreal?