Download

Scene Capture out of synce

Hello. I’m using Unreal for robot simulation and trying to employ several SceneCapture2D components as virtual cameras on the robot. To test the idea, I created a new class “UCamera” which inherits from USceneComponent. In UCamera::BeginPlay I create an internal USceneCaptureComponent2D instance like so:


bool UCamera::CreateSceneCapturer()
{
  sceneCapturer_ = NewObject<USceneCaptureComponent2D>();

  if (!sceneCapturer_)
  {
    UE_LOG(LogTemp, Warning, TEXT("Error creating scene capturer"));
    return false;
  }

  sceneCapturer_->AttachTo(this);
  sceneCapturer_->FOVAngle = 90;
  sceneCapturer_->bCaptureEveryFrame = true;
  sceneCapturer_->TextureTarget = renderTarget_;
  sceneCapturer_->CaptureSource = ESceneCaptureSource::SCS_FinalColorLDR;
  sceneCapturer_->PostProcessSettings.bOverride_AntiAliasingMethod = true;
  sceneCapturer_->PostProcessSettings.AntiAliasingMethod = EAntiAliasingMethod::AAM_TemporalAA;
  sceneCapturer_->RegisterComponentWithWorld(this->GetWorld());

  return true;
}

I then create an instance of UCamera in the Unreal editor and attached it directly to the FirstPersonCamera found in the FirstPersonShooter demo project. The structure of the FirstPersonCharacter components is as follows:

Capture.PNG

On each call to UCamera::TickComponent I call UpdateContent on my internal USceneCaptureComponent2D and visualize the results in an external program. Here is a video of the results (my scene capture results on top, the game’s main camera results on the bottom):

http://postimg.org/image/5rsrs55rt

As you can see, my biggest problem is that my camera is out of sync with the mesh and other camera. I don’t know if my camera is capturing before the other components have moved or is physically located in the wrong position, but it always seems to be one step ahead of everything else. I’ve tried playing with tick groups with no luck. I’ve also seen other post suggesting using a SpringArm to make the Scene Capture camera lag behind. This gets the two cameras closer to being in sync, but is never exactly fixed to the robot. Obviously if I want to use this for a perception pipeline, I can’t have my cameras sloshing around. I need them to be fixed to the robot.

Any ideas on what I might be doing wrong or how I could otherwise resolve this issue?

Many thanks!
Mike