The 4.3 release introduced a lot of exciting new tech. I’m pretty excited to start using imposters, and see what kind of performance gains we can get from using them. Are there any tutorials, documentation, examples of creating and implementing imposters? I’ve discovered the imposter Material Function and Blueprint in the Art Tools package, but have failed miserably at setting up anything successfully. I have rendered out a sprite sheet with base color of a mesh using the new texture render Blueprint, but I’m not sure what to do next.
I have documentation written and submitted to some people to help clean it up for review. I will post it here for now. Sorry it should be up soon!
Basic Usage:
The RenderToTexture Blueprint works by using a bare-bones GameMode setup. There is a companion asset called the RenderToTexture_Game which references the RenderToTexture_Pawn. The Pawn is a very basic blueprint that simply has a Camera Component set to orthographic and a few event graph nodes to setup the rendering console commands.
The first step to using these tools is to create a new blank level and assign the RenderToTexture_GameMode as the GameMode in the World Settings:
Blueprint’/Engine/ArtTools/RenderToTexture/Blueprints/RenderToTexture_Game.RenderToTexture_Game’
You must then Place the RenderToTexture_LevelBP into the level. It is located at:
Blueprint’/Engine/ArtTools/RenderToTexture/Blueprints/RenderToTexture_LevelBP.RenderToTexture_LevelBP’
This method currently saves .bmp format images. It is currently necessary to open and resave as .tga using an image processing tool. In practice that is not a big deal since in most cases you will want to render at a large res and downsample using an image processing pass later.
Note: In UE4.3, there is a minor bug with the blueprint that requires you to place it at world location 0,0,0. If it is moved outside of 0,0,0, the camera will not align properly with the target.
You must also place a playerstart in the level that has 0 for all rotation values.
Note: In 4.3 you must also go to Editor-> Preferences → Play (or click the small down arrow next to the “Play” button and choose advanced settings). Then you must type in a size that is a power of 2 (or at least square if you are planning to downsample later) for the “New Window Size” setting. I suggest you use 512x512 to keep the window small as that will not affect quality.
The setting to choose which type of basic function will be performed is located under the ‘Default’ Group up top. Below you can see Property Groups for the various functions:
Here you can see the options for “Render Type” expanded. Which option is selected here determines which function gets performed and which settings below are relevant.
The only Property Group besides Default that is relevant to all functions is the “Capture Settings” group. This group allows you to select which buffer targets you wish to save. For most cases BaseColor and Normal are sufficient. To render out opacity masks the “Decal Mask” option works well.
The resolution multiplier will multiply the settings you entered into the “New Window Size” in the Editor Preferences. So if you entered 512x512 , a multiplier of 2 would result in a 1024x1024 texture size. A multiplier of 4 would result in a 2048x2048 texture size. Depending on the desired results, you may want to render textures as high as 4096 or 8192 in order to downsample them in Adobe Photoshop. In those cases it is usually suggested that you add a post process volume and set the Anti Aliasing setting to “none” to get a more crisp result.
- Render a Tiling material to textures
Note: In UE4.3, this particular function is missing the setting to specify the material target. This is easily worked around by placing the following mesh at 0,0,0 in the world, and setting the Scale to match the number specified in “Sheet Size” on the Blueprint (Defaults to 1000):
StaticMesh’/Engine/ArtTools/RenderToTexture/Meshes/S_1_Unit_Plane.S_1_Unit_Plane’
Then simply apply the desired material and it will work.
You must also enable the ability to export buffer visualization targets in the editor. That can be found by clicking the upper-left Down-Arrow button in the perspective viewport and selecting “High Resolution Screenshot”
That will bring up a small window of the same name. You must check the box named “Include Buffer Visualization Targets”
You must actually leave this window up. It is ok to just click anywhere on the editor again, this window will move behind the editor and should not interfere. If you accidentally close the window, the buffer images will not render. You will need this box checked and the window open for all of the render functions as well.
Now the only remaining step is to Play the game in a new Window, press ~ or tab to bring up the console and type “rendertextures” and hit enter.
You should get a popup message indicating that images were saved, the message is actually a clickable link to the folder location. If you are not able to click the link in time, you can find the saved textures here:
\YourProject\ Saved\Screenshots\Windows\
- Render 3D imposter sprites
Imposters are sprites that use flipbook style textures to store a view of a static mesh from every possible view - or at least the number of views you can comfortably fit onto your texture sheet. The result is a sprite that matches the original mesh very well in terms of the material and lighting; however, there will be some popping between frames. That means they are not perfect for all applications. When you need to render LOTS of objects that won’t move slowly and closely past the camera, it can be very useful.
Before you can create an imposter sprite, you need to bake out a series of imposter textures.
Imposter Static Mesh: This is the static mesh you want to make an imposter for.
Imposter Material Array: This is an array so that you can specify materials for each index for meshes that have more than one material ID.
Note: You need to apply a Tangent->World transform on the normalmap input for the materials in the imposter material array . The reason for that is we want to represent the world normal as viewed from the direction the camera would be, instead of all using the same camera which this top-down method forces. You can simple use 0,0,1 transformed Tangent->World if there is no normalmap texture.
Frames around Z: This setting determines how many frames around Z your imposter will store. Higher numbers result in less ‘popping’ in the final result during motion but will either cause less texture resolution or require you to use very larger texture sizes.
Mesh Scale: This allows you to scale down the mesh slightly if you have any issues with edge bleeding in the final imposter sprite material. This is only necessary in rare cases where the mesh very closely matches its sphere bounds causing the meshes to get close to the edge of each grid cell. Only adjust when necessary as the smaller the scale, the more resolution that is lost.
Aspect Ratio 1 by: This setting allows you to create nonuniform textures. At the default 1, the imposter grid will be square. If the setting is 2, the aspect ratio will be 2:1; if the setting is 4 it will be 4:1 etc. It is suggested to keep the aspect ratio square for “Full 3D Imposters” and to only modify it when “Single Rotation Axis” mode is used. Note: I only realized upon typing this doc how incorrect this setting name is. Expect a more friendly name for this setting soon.
Imposter Enum: This is a drop down that allows you to choose between two different types of Imposter baking: “Full 3D Imposters” and “Single Rotation” axis. Full 3D imposters get represented from all angles, including above and below. “Single Rotation Axis” only rotates the mesh around Z which is ideal for distant LOD for things that are grounded on the horizon such as Trees, large buildings or general skybox elements. Single Rotation axis allows for better use of texture memory as there aren’t as many necessary to store.
Single Rotation Axis rotation: This setting is a start rotation setting that is only used when the single rotation axis option is selected. It allows you to orient the mesh how you want relative to the camera.
After you have configured your imposter with the desired number of frames, rendering the imposter textures is just like the original “tiling material” step.
Select the desired buffer targets under “Capture Settings”; Make sure the “Include buffer visualization” button is checked and the “High resolution screenshot” window is open; Play the game in a New Window and type “rendertextures” at the console and hit enter.
Now you should have a series of rendered imposter images. To make use of these textures, you will need to create a material that uses the ImposterUVs material function:
MaterialFunction’/Engine/ArtTools/RenderToTexture/MaterialFunctions/ImposterUVs.ImposterUVs’
This function sets up the UVs, Normals and WorldPositionOffset for your material. There are a few parameters you need to define to match your Imposter setup:
FramesX (S): This should be the number of frames on the X axis for your imposter. For example if you specified 16 rotations around Z there will be 16 frames on the X axis.
Frames Y (S): This should be the number of frames on the Y axis. For square aspect ratios this should always match the frame Y. If you entered an aspect ratio of 2:1, then the the Y frames would be half of the X frames.
Fixed Z (B): This is a Boolean option that should only be set to True when using an imposter with “Single Rotation Axis”. This makes the calculations much cheaper.
SpriteWidth (S): This is how wide your resulting sprite will be.
SpriteHeight (S): This is how Tall your resulting sprite will be.
Position (V3): Defaults to AbsoluteWorldPosition and should be left untouched in most cases. Can be used if you want to trick the function into displaying the imposter at a weird angle or if you want to use custom data to represent locations.
VertexShaderZrotation (S): This is a debug only setting. What this feature does is break up the sprite’s Z rotation into a set of discreet steps rather than smoothly interpolating across the entire range. This prevents popping from looking worse due to how imposters are baked. With smooth sprite interpolation the imposter appears to overshoot its rotation at each frame transition. This fixes that when the value equals 1. You can set to 0 to see what it looks like disabled. Mostly noticeable from above an imposter mesh.
Rotation (S): The imposter sprites can be rotated around the specified axis. This allows you to randomly rotate various copies of the same imposter sprite in the world so that they do not look the same.
Rotation Axis (V3): This setting is currently disabled. The rotation axis always defaults to 0,0,1 or around Z. There were some issues when trying to support any rotation axis but this support may be added in the future.
Normals (V3): This should be your normalmap texture. What this does is rotate the normal to match your specified Rotation value.
Note: You must also click your material settings and uncheck the “Tangent Space Normal Map” option:
Function Output Pins:
TransformedNormals:
Contains transformed normalmap, connec to material Normal input.
WorldPositionOffset: contains WPO for camera facing sprite, connect to World Position Offset input.
UVs: This is the UVs for the Imposter textures. Any Imposter textures you place in the material should have this output as the UVs. That includes the normalmap which will then appear to loop back into the function in the “Normals” input.
Phase2UVs: This is an experimental feature that was not fully implemented. It is the start of the ability to have two or more texture lookups that are one phase ahead and/or behind and fade/dissolve between them to reduce popping.
Example Full Imposters:
Thanks for posting this, it was really helpful; however I’ve still got a few issues that you may be able to help with.
-
After following all the steps I don’t get anywhere near the same quality of results as in your example image. There are severe filtering issues which are noticeable even when the imposter is fairly small on screen. It looks like texels are being selected from the wrong tiles in the texture. I’ve zoomed in close in the image below to make it more obvious.
-
The imposter’s lighting doesn’t really match the mesh. The normals seem like they may be inverted. (I’ve ensured the normals have a tangent-to-world transform).
-
What is the best way to draw the imposter in game? In the image below the material is simply applied to a plane. I tried using a Billboard Material Component, but the quad becomes strangely skewed and is not oriented correctly in the z-axis (and rotating the component/actor did nothing). How are the imposters drawn in your example?
-
Why does the Imposter Material array only accept Material Instances? Why not normal Materials too?
-
The exported bmp files seem to be unreadable by everything expect MS Paint! They won’t import directly into Unreal, and won’t load with Gimp or DX Texture Tool, so I have to first load with Paint and re-save them in a different format before I can do anything further with them.
Thanks in advance for your help.
I apologize one of the steps I listed was incorrect and I forgot a step!
In the material to be captured, the transform should be: Tangent->Local
Also, I neglected to include another vital step for importing the normal map. You must change the compression method to be TC_Default instead of TC_Normalmap. Then you must manually remap the range to be -1 to 1 inside of the imposter material. To do that simply use the constantbiasscale node with -0.5 as the bias and 2.0 as the scale. The reason for this is that the imposter normal stores backwards facing normals which the engine assumes are incorrect and normalizes back to face the camera.
Both of those mistakes add up to quite a bit of lighting error, sorry about that!
For quality comparison, the example imposter texture was rendered as 8192 and downsized to 4096x4096 in photoshop. That step alone provides a fairly substantial quality improvement. Imposters can be 2048 or smaller but you must understand that is where the quality becomes questionable up close, or you sacrifice # of frames which introduces more popping.
For the geometry, simply find the mesh: StaticMesh’/Engine/ArtTools/RenderToTexture/Meshes/S_1_Unit_Plane.S_1_Unit_Plane’
Then either scale the mesh to be small, or subtract AbsoluteWorldPosition and add ObjectPosition in the WorldPositionOffset shader. But generally, if your projected sprite size is much bigger than 1.0, you can ignore those steps since the mesh will be small enough to not interfere much with the projection.
And yes you currently need to resave the .bmp images. It is rather strange you couldn’t open them in an image editing program. I typically use photoshop for this. I usually render at very large sizes (8192) with antialiasing disabled via post processing volume. That makes it easy to do color-fill to replace the background colors in photoshop before downsampling to get a super clean image.
Thanks for the info, I’ll give that a try.
No worries. As far as I know you may be the first person outside of epic to give this a shot. I really appreciate you taking it for a spin and reporting back anything that isn’t working right. It will really help to make sure others don’t have to run into the same issues. Thanks!
I’ve made those additional changes but its not fixed the issues. I’m still getting quite broken results. Could you share your test level so that I can see exactly how it’s setup? That’s probably going to be the easiest way to spot what’s different in my setup. Cheers.
Hi Ryan, Are the procedural texture creator, Unwrap vertex setup, shadow projection tools. Also are imposter sprites able to cast shadows (dynamic of course as baked wouldn’t work) if you capture the alpha of the 3d object in the bake then surly there is no reason baked shadow projections couldn’t be done, Also can you do some explanation like you have above for these tools as well. Cheers Kingbadger3d
Hi Ryan, Are the procedural texture creator, Unwrap vertex setup, shadow projection tools. Also are imposter sprites able to cast shadows (dynamic of course as baked wouldn’t work) if you capture the alpha of the 3d object in the bake then surly there is no reason baked shadow projections couldn’t be done, Also can you do some explanation like you have above for these tools as well. Cheers Kingbadger3d
I also would love for you to provide the level files we saw you demo in the twitch stream when you first showed the tools. It would just make life easier
The procedural texture making material you saw in the stream will still be a few versions out. The brick thing you saw was basically one big swiss-army material with lots of parameters. The “tool” captures screenshots of all the view buffers with a camera that is aligned with 0-1 UVs. So you could bake a material down fairly easily. And yes the unwrap and shadowmap tools are there now. Your post reminded me that I should have also included some examples on how you can use the depth texture on a “WorldPositionBehindTranslucency” shader to create a pseudo decal using a regular mesh…
We have documentation written for these elements that should be out tomorrow or Thursday.
And yes the imposters will cast a shadow. Currently though the shadow will fade away on one axis when the sprite geometry is parallel to the light vector. You can fix this by actually making your sprite a cross plane and projecting one of the planes 90 degrees to the camera facing one. So you will have an invisible sprite right in the middle that is parallel to the view direction, giving a solid 3d looking shadow. I actually had to do this exact thing on the recent Landscape demo pine trees. The distant LOD is a sprite that has a cross plane that you can’t actually see unless you disable the vertex shader.
It is probably something simple. Yes we definitely want to make a content example map for this. It won’t be out for 4.4 though I am sorry to say.
I am a bit puzzled by that not working. In your image, it looks like the imposter is broken up into several bits. Are you still seeing that? Using a regular mesh with a vertex shader should have fixed that part.
If your lighting is still messed up, can we verify the following steps?
Inside material on MESH TO CAPTURE TO IMPOSTER SPRITE:
make sure a Tangent->Local transform is used
Make sure Tangent Space Normal = FALSE
On imported normalmap texture:
Normalmap as sRGB=FALSE
Normalmap is set to TC_Default not TC_Normalmap
Inside imposter sprite material:
normalmap is constantbiasscaled into -1to1 range by setting bias=-0.5 scale=2.
Make sure Tangent Space Normal = FALSE
Ok, I went though your check list and found I was missing the ‘Tangent Space Normal = FALSE’ for the material used during capturing. I had set it for the final material used on the imposter sprite, but not the capture material (I may be that I simply misunderstood, but from you original instructions it seemed to imply it was just the sprite material that required the flag). Anyway, with that fixed the lighting looks much better.
The remaining problems are the odd issue which you described as the sprite being “broken up into several bits”, and some discrepancy in the imposter’s orientation. It’s much improved after your vertex shader suggestions, but the artefacts are still noticeable in certain situations, particularly when the sprite pops between frames. Sometimes it looks like its using two sprite tiles at once. You’ll also notice the orientation of the imposter doesn’t quite match the real mesh in the image below.
This could be something missing with the vertex shader. I’ve used the S_1_Unit_Plane.S_1_Unit_Plane mesh, but can you what you said before “Then either scale the mesh to be small, or subtract AbsoluteWorldPosition and add ObjectPosition in the WorldPositionOffset shader”. If I’m understanding you correctly, I did this it by inserting the calculation between the WorldPositionOffset pins of the ImposterUVs function and my sprite material (see image below).
][2]][2]
In the material graph image you can also see some artefacts in the thumbnail images of the texture samplers. Is that also a symptom of the same problem we see on the sprite?
Finally, how to you scale and rotate the sprite in game, since only the location on the actor’s transform works when using the vertex shader changes described above. Are you limited to using the ImposterUVs function inputs to do this?
Thanks again for your help with this, I’ll get it working eventually!
Ok I know exactly that this is. Hook up ActorPositionWS to the “Position” input on the function.
The issue here is that even when the source mesh is very small, it is evaluating the Position for each vertex instead of for a single position for all verts. So the material was blending between different angles when the verts crossed the angle threshold at different times.
As far as not matching perfectly, there will always be a little bit of error when you are between two frames. This method also cannot account for perspective stretching when you get very close to the imposters.
A good test to see how closely they match is to try and line up something like the bottom base element to be flat, and see how close the imposter is. Keep in mind that the imposter is basically evaluated for the view angle to the center of the imposter, it can’t do change the perspective of just the bottom piece. So the bottom piece will appear flat when your view is flat relative to the center of the imposter. Make sense?
If you look carefully in my example shot above, you can see how the imposters closest to the camera on the far right have a bit of a ‘skewed’ appearance since a real 3D mesh would have been affected by perspective which isn’t possible.
Yeah using Actor Position fixed the sampling artefacts, cheers!
I suspect I still have a setup issue somewhere though. I’m seeing a lot more popping compared to in your 4.3 livestream demo. Here’s a video of the problem: imposters_1.m4v. I’m using a sprite sheet captured at 4096*, down sampled to 2048, with 16 frames around Z. Any ideas what the issue might be?
There’s also an issue with culling. I had to manually set the scale on the mesh actor to get the bounds to the right size, but that will only work for the sphere bounds. Since the mesh is a plane the box bounds will not scale in Z, which seems to be why the sprites get incorrectly culled sometimes. Is this expected? Should I just oversize the sphere bounds?
*(I get a driver crash trying to render the normals at 8192. I don’t appear to be running out of memory on my 2GB GTX 660Ti)
Try rotating the mesh 90 degrees on either X or Y. You are right, the boundscale needs to be increased. I typically was dealing with multiple sprite meshes merged together which was automatically fixing the boundscale thing (and specifying pivots as well using UVs).
In the imposter example I showed, that was using 24 frames around Z and I rendered a 8192 downsampled to 4096 in engine. The amount of popping you are seeing is roughly what you should expect given that there are 360 degrees broken up by 16 steps. This means that every imposter texture will “pop” by 22.5 degrees.
This is the main reason I think the single rotation axis option will be more useful in more cases. It is much easier to reduce the popping to where it is not noticeable. If your camera is going to get close to these objects, perhaps another LOD solution would be better suited to these assets. This technique truly pay off when you need tons of objects that cannot be easily LODd.
I started working on a setup to ‘dissolve’ between imposter transitions sort of like the LOD in speedtree. With a bit more work it can help reduce the popping much more bit it will add to the shader cost since it requires 2 more texture lookups. That technique is always in a state of fading with the closest horizontal and vertical neighbor frame.
It is unfortunate that you get the driver crash rendering 8192. I use a 680 GTX which is also 2gb. What is your system memory? Would you mind posting the crashlog if there is one?
I’ve tried with 24 frames but even then the popping is still greater than i noticed in your demo. I suspect there is still a problem with the set up but not sure what.
The camera will not be close to these objects, but the popping is noticeable even at significant distances. I’ve just been using your example as a benchmark for what is achievable with the system, and in comparison to that the quality is not as good.
At the moment I’m just evaluating the feature, and it may well be that using simplified meshes or other LOD methods is going to be more appropriate.
My system memory is 32GB, Here’s the crash log.