DOF Compensation Workflows

Development Cinematics & Media Virtual Production
, ,
1

We have begun a research project at our company test and roll out standard workflows for using Live FIZ data across all of our stages using DCS LDT-V2s

LDT-V2

We have some questions/feedback regarding working with depth of field in engine.

In our experience, creative stakeholders like DPs and VP supervisors will consistently give notes on DOF when it is enabled and we’ve encountered two issues with attempting to do what they’ve asked when DOF compensation is enabled.

First, they want more Bokeh, and the DOF gain setting maxes out at 4, sometimes it works and we get more bokeh they are looking for, but sometimes not enough. What we’ve found is more effective is applying an offset to the aperture lowering it, that allows us the ability to go beyond this limitation of 4 that the gain setting has. The question is why does this setting max out at 4?

Second, DOF compensation is designed to have a falloff from the background to the objects that are right next to the wall. This is an desired effect because prior to DOF Compensation the entire frustum would defocus at the same level with no falloff which is not what creatives expect, nor is accurate which is what DOF compensation fixes. However, then you will get a note that they creatively want the object that is right next to the wall to have some defocus (can happen when on tight lenses and whatever object that is right next to the wall is taking up the majority of the frame) and you can’t do that with DOF compensation on by design, cranking DOF gain and/or lowering the aperture will increase the bokeh in the background and not what’s next to the wall. This is physically correct, but more than once is not creatively what we are being asked to do.

Our solution in this instance is that we have to disable DOF compensation going back to the inaccurate flat double defocus, and apply aperture offsets to get the bokeh amount that feels right.

I have attempted to see if editing the Compensation LUT file could be a solution to this issue, allowing for just a bit of defocus at the wall that falls off rather than completely disabling DOF compensation, however, my edits have not been successful.

Can we get an explanation of the layout of this LUT file so we can understand how to edit it/make our own?

[Attachment Removed]

2

Matthew I have been sharing what you are up to with one of the developers and here is some info that I hope helps.

The gain limit is a relic of the original implementation of the compensation LUT, which were encoded with 8 bit grayscale textures, and setting the gain too high would cause a saturation and flatten the DOF to a maximum. We have since rewrote the LUTs to be 32 bit floating points, so there shouldn’t be anything preventing the limit from being increased.

The code expects from the LUT:

  • LUT is 1D 256 pixel long texture, R32F pixel format
  • LUT’s pixel coordinate is a ratio of object CoC (so the object being blurred, we grab the original CoC radius the standard DoF shader has computed) to something called the ‘in-focus CoC radius’, which is essentially the CoC radius an object must have to appear sharp on the wall. Most of the time, this radius is computed to be at the wall, so we take all the camera parameters, calculate the amount of blur that would be at the wall normally, and then pass that blur amount into the shader so that objects that have that blur amount (e.g. objects at the wall) appear sharp
  • The value encoded in each of the LUT’s pixels is the amount to remove from the object’s CoC radius when computing its final, rendered DoF blur. The pixel value is scaled by the ‘in-focus CoC radius’ to compute the amount to offset. So, putting a value of 1.0 into a pixel means that an object’s CoC radius will be reduced by the ‘in-focus CoC radius’, essentially reducing the blur by the amount needed to make objects at the wall sharp
  • ‘in-focus CoC radius’ can be shifted to be further away or more in front of the wall with ‘Distance to Wall Offset’, to control at what distance you want objects to be sharp
  • The amount of blur removed from objects is always clamped so that objects can’t get ‘more sharp’ than perfectly sharp; in-engine, having a CoC that is negative transforms an object from background DoF blur to foreground DoF blur, so while in theory we could allow the LUT to encode values that would make objects on the wall blur as if they were in the foreground, we opted not to support that as foreground blur is a whole different issue

Some examples to potentially help make things a bit more clear:

A LUT that is a constant color will result in all objects having their blur reduced by a fixed amount, proportional to the ‘in-focus CoC radius’. So, if you have a LUT where all pixels have a value of 1.0, and you set the in-focus CoC radius to match the CoC at the wall (which happens by default), then all objects will have their blur reduced by the amount of blur that would be at the wall, so objects at the wall are sharp, and objects further out all have the same reduction in blur amount. This replicates the original version of DoF compensation, when a constant amount was taken off all object blurs so that objects at the wall are sharp.

The LUTs included in the engine are all S curves, starting with a pixel value of 1.0 in the first pixel, decreasing for objects with larger and larger CoCs. This generally means that the further away objects are from the in-focus CoC radius (the wall), the less we reduce their blurs. The position of the S curve in the LUT generally dictates where the inflection point from large corrections to small corrections occurs. There are three LUTs included, a default one, one called a ‘heavy’ correction because it moves the S curve inflection point further away, resulting in more objects near the wall having the constant wall blur offset removed, and a ‘light’ correction, that moves the inflection point closer, so objects nearer to the wall are compensated for less, making them look blurrier.

An example LUT accomplishing more blur at the wall would shift the first few pixels of the default LUT down from 1.0 to something like 0.5, so that only half the blur at the in-focus CoC would be removed, resulting in objects still having some blur on them at or near the wall

In DisplayClusterConfigurationTypes_ICVFX.h, change the UIMax and ClampMax values on

DepthOfFieldGain UProperty:

[Image Removed]

simply setting that value using C++ to a value greater than 4 should work.

Let me know if this helps and if you are able to use VS to update for now. Will look at increasing this amount for 5.8

[Attachment Removed]

3

Thank you Kevin for your detailed reponse.

Based upon your explanation I was able to create some new LUTS that allow us to add a bit more blur at the wall while preserving the rest of the falloff curve.

I’ve also passed along the clamp values to our developers to update our engine with a larger value.

With these two adjustments, I feel that we’ll be able to leave DOF compensation on while also having the creative control we are looking for with DOF Gain and the Luts, without resorting to turning off DOF Compensation and/or applying Aperture offsets.

[Attachment Removed]