Add orthographic projection support to USceneCaptureComponent2D

General Feedback & Requests
1

This feature has already been requested before, among other improvements to the scene capture functionality, but I think this single feature would be really easy to implement, and it could be really useful for some people.

You can quickly check why it would be useful if you have some kind of VR headset. For instance, if you have a Gear VR:

  • Create a new project using the 2D Side Scroller template.
  • In the project settings, configure the Android platform to target the Gear VR. (Minimum SDK Version = Target SDK Version = 19. Configure the AndroidManifest for deployment to GearVR = true.)
  • Package the project for Android (ETC2).
  • Install the project on your phone; run it; connect your phone to the Gear VR; put on the Gear VR.
  • Look around.

The only way to make Paper 2D really useful in the VR era would be by allowing to capture the scene into a texture using an orthographic projection. Then you could draw that texture on a virtual cinema screen.

It seems likely that Google Daydream will let any game be playable both on the device’s screen and on the VR headset. I’m sure most gamers would prefer playing a 2D game on a virtual cinema screen rather than on a phone or tablet screen.

2

yeah I’m still waiting for this feature as well
here’s the answerhub post from 2014:

that post already contains an implementation but not all of us can have the luxury of making and maintaining changes to the full-source engine

3

It’s already done and coming in 4.13:

cl 2982619
5/18/2016
Scene capture 2d improvements

  • Orthographic projection supported
  • Opacity is now captured in alpha, allows partial rendering in a scene capture and compositing into another scene later
  • Various GBuffer attributes are now available to be captured, including depth
  • Changed Blueprint capture function to CaptureScene, which happens immediately (was previously deferred), allowing multiple captures with different parameters
4

yay finally!

does this fix UE-28468?

5

meh this made UE-28468 worse :mad:

before I could at least turn the component off via BP (deactivate or set activate false), but now even that fails :frowning:
@DanielW since my workaround was killed, do you know of some other workaround? UE-28468 is backlogged :frowning:

6

I don’t know why PP materials fail to work when calling CaptureScene, it would take some investigation.

Workaround idea - if you are just capturing normals you can do that with CaptureSource in 4.13. These ones are supported:

7

@DanielW I’m not just capturing normals, I’m capturing depth from a topdown view and then applying a postprocess.
this severely hurts my in-editor performance, as the BP deactivate of the component only works in-game now

using ‘SceneDepth in R’ does not work for me, as the only option to get my postprocess materials to work is to use the FinalColor LDR option (every other option completely ignores my blendables)
:frowning:
should I report it in an existing AH post? the severity of UE-28468 just climbed greatly for me with this regression

[edit 1]
I wasn’t calling CaptureScene btw, I was just using ‘capture every frame’ (turning it on and off). I’ll try with Capture Scene and report back

[edit 2]
calling Capture Scene doesn’t work either, as it ignores blendables as well.
the only way to get blendables to work is with CaptureEveryFrame enabled.
so yeah, still an issue and a much worse one now :frowning:

8

@DanielW I tried debugging it a bit

with CaptureEveryFrame Disabled and calling CaptureScene from my BP, I breakpoint into this function:


void FScene::UpdateSceneCaptureContents(USceneCaptureComponent2D* CaptureComponent)

the line where it calls


FSceneRenderer* SceneRenderer = CreateSceneRendererForSceneCapture(

and ended up in Material.cpp, here:


void UMaterialInterface::OverrideBlendableSettings(class FSceneView& View, float Weight) const
{
	check(Weight > 0.0f && Weight <= 1.0f);

	FFinalPostProcessSettings& Dest = View.FinalPostProcessSettings;

	const UMaterial* Base = GetMaterial();

	//	should we use UMaterial::GetDefaultMaterial(Domain) instead of skipping the material

	if(!Base || Base->MaterialDomain != MD_PostProcess || !View.State)
	{
		***return;***
	}

it returns in the above statement…
Base is not null
Base->MaterialDomain is MD_PostProcess
but
View.State is null
:eek:

but well, it almost seems like the code was designed to not support blendables… :confused: :eek:
CreateSceneRendererForSceneCapture() is calling this:


ViewInitOptions.SceneViewStateInterface = SceneCaptureComponent->GetViewState();

which looks like this:


FSceneViewStateInterface* USceneCaptureComponent::GetViewState()
{
	FSceneViewStateInterface* ViewStateInterface = ViewState.GetReference();
	if (bCaptureEveryFrame && ViewStateInterface == NULL)
	{
		ViewState.Allocate();
		ViewStateInterface = ViewState.GetReference();
	}
	else if (!bCaptureEveryFrame && ViewStateInterface)
	{
		ViewState.Destroy();
		ViewStateInterface = NULL;
	}
	return ViewStateInterface;
}

it looks explicitly designed to only create the state if bCaptureEveryFrame is true (otherwise just destroys it).

so how in the world am I ever going to have a working blendable with bCaptureEveryFrame disabled,
if blendables require a valid View State,
and a View State is only ever allowed when bCaptureEveryFrame is true?
:confused: :confused: :confused:

9

It can be made to work with some effort. PP materials are only working with a view state because they are pooled in the view state. It needs to either skip pooling or use a pool stored somewhere else when no view state.

One other workaround idea - if you are rendering a small subset of things into your scene capture instead of the whole scene (through ShowOnlyActors), then you can actually override their material to one that outputs depth / normal / whatever in BP before you call CaptureScene, and then restore their material after.

10

your other workaround would not be the greatest either, as it would kill the bCaptureEveryFrame functionality (which I also need as an option)

you could also keep the ViewState alive until the next update (and force such a next update later)
I actually tried this today and it worked, but I only made it at work (at home I focus on gameplay so I strictly stick to precompiled engine releases). and my code wasn’t really elegant. but I know it can be made to work.
any other ideas? or can this regression cause the issue to be bumped from the backlog?

11

@DanielW I updated my AH post with this info. let me know if you’d be interested in the code that I used (at work) to hackfix the issue, I’d really appreciate it if there’d at least be a workaround :confused: