Download

Projection Mapping technique to projects video files on objects?

Wow looks very interesting, it may work, looks bit complicated with setup, but if you can share project file I can look for reference, I only have question about video-file or maybe loop image sequence to play? Can you make quick demo file with playing video?

I’ll see what I can do. Been meaning to share this for a while, but haven’t had a lot of time.

Looks like you used “Transform to Z Vector”. that will only work as long as the camera is never tilted sideways at all since it derives the XZ vectors assuming an up vector of 0,0,1 to start. That node was meant to create a tangent basis from vertex normals on terrain and things like that.

If you do a full transform using the camera X, Y and Z vectors it will work anywhere. The bottom half of this image (stopping at the texture sample, the top part is only needed if you are doing a shadow depth comparison and the texture is a shadow depth map):

In the above case “Object Position” is simply the center of the object, and will determine the center of the projection coordinates. Use whatever there.

Object Radius can be changed to equal the size of the texture you want to project.

There is actually a function in Engine\Content (not exposed to the right click search menu) called “BoundingSphere_LightTransform” that I made a while ago to create shadowmaps for unlit materials, but it will likely be replaced by something closer to the above graph which is much more robust. Details on what its used for here:

http://shaderbits.com/blog/custom-per-object-shadowmaps-using-blueprints

Thanks Ryan - that’s how I got it working properly in the end. Took a while to figure out, though… :o

I’m now trying to figure out a way to turn it into a slick “import camera > choose projection image > choose geometry to project onto” workflow. Will open up some DMP techniques in Unreal.

Many thanks

I’ve got it packed up in some kind of transferrable form.

Step 1:

Download the following .zip file, and unpack it into the ‘Content’ folder of your project.

DOWNLOAD

It contains 3 .uassets:

>‘decal2’ material
>‘projection_decal2’ blueprint
>‘video’ media object

Step 2:

1 - drag the blueprint into the level

2 - choose the camera actor you want to project from

3 - choose the texture you want to project

4 - offset the decal bounding box to enclose the objects to are try to project onto

Step 3:

I added a blank media texture to the decal material to check that it hooked up okay. It won’t do anything without some blueprint shenanigans, but it looks like a viable workflow.

Bit ‘bare bones’ for now, but let me know if it works, and if there is anything I should do to improve it.

2 Likes

Hello Scribbler…

Ive downloaded your zip file…
This Blueprint has a great deal of potential!
Ive been searching for a way of incorporating a matte painting approach into Unreal.

Using an camera to render a shot, bring this image into Photoshop to paint over, then re-project this back.

The blueprint works great with the default texture associated with it, but I’m unable to swap textures.
Adding a new texture in the “Projection Image” area does nothing.

Ive opened the blueprint itself in an attempt to replace it there, but cant find where its located.

any ideas?

Thanks!

Jeff

hello there, excellent solution :slight_smile:, wonder why a special tool does not exist in the engine, it can have so many uses.
Have used your blueprints for a personal test project and here is some feedback:

comments:

  1. used it with a video texture, it works just fine with some tweaks.

  2. not really sure what i changed in the blueprints to get the new texture working, simply setting as a camera texture did not work.
    Maybe the video texture induces extra lag in user feedback so it got me confused.

issues:

1. texture does not update when moving camera or setting camera field of view.

Need to manually reselect camera from the Projection camera variable in the projecteddecal options. How can we have this auto-refreshed when camera moves?
This is very crucial for precise/interactive texture positioning, or any realtime usage, in example the player has a projector/flashlight of any kind that lightsup stuff while moving.

  1. scaling from the offset actually distorts (not only crops) projected texture. this does not happen with my video texture but with the original texture you provided.

I think that is all for now. Thank you for sharing this, we really appreciate.

Is this still NOT part of UE? This is something that should have been there for years already! In regular 3D apps this is used all the time for VFX, and it´s a big omission

Thank you @Scribbler and @RyanB for your awesome work! I used this script it in my Bachelors Thesis, without your groundwork it would not have been possible for me!

I improved your script quite a bit, I added the capability to cast shadows, and improved the usability.

So first the shadows: The projection only shows on the first layer of objects it hits, all objects behind don’t recieve any light, just like a real life projection would behave.
The shadows are projected from the camera, so that if you view your scene through the eyes of the projection camera, you’re going to see no shadows.

Now to the improved usability: I set up the Blueprint in a way, that you only need to drop in the Projection Blueprint and specify your media texture. Thats it!

There are a few more options that you’ll often use though, First the “X Scale” in the “Offset” Variable. This changes how far your texture will be projected.
The second one is the “Edge Bleed”. This determines how hard the shadow is blended with the projected image where both areas meet.
The last one is the “Field of View” on the Camera Sub-component, which adjusts the size of the projection.

I hope this helps someone as much as your work helped me! If you have any questions, just leave me a message! Download it here: GitHub - ChristopherRemde/Projection_shadow: This script is able to project a texture in Unreal Engine

Just click on “Clone or Download”, download and unzip, then open the project and drop a file into the Media Player, et voila!

Thank you @Dementium4ever ! Great work on the script! I have just one question. The projection is bleeding/smearing outside of the screen area. How can I get rid of this? Any ideas?

That’s actually a bug on the shader I still have to fix. Maybe I’ll upload a patched version today or tomorrow.

​​​​​​Edit: A thing I missed out on was that my script is currently only made for 16:9 content. But I’ll fix that too!

@JsCholo : I think I fixed the bug! Sorry that it took so long, was a bit of a busy week. I updated the Github Repo, just download it again: GitHub - ChristopherRemde/Projection_shadow: This script is able to project a texture in Unreal Engine

The shader/blueprint should also now automatically adjust to the texture aspect ration. For that to happen with a video texture, you have to start the media player attached to the video texture **first **, and then set the media texture in the projection blueprint. A normal image based texture should always adjust automatically.

Would be great if you could give me feedback if that has solved your problem!

@Dementium4ever I love what you have done! I have a couple of request is there a way to have a plane with a material that is transparent but only effected by the projecting texture? Also is there a way to make a object have a material that is in the projection area that is not effected by the projection texture?

Hi Virtualbread, I’m not sure if I understand the first question, do you simply want that the Projection works on objects with transparent materials? E.g. when the Projection hits the transparent object, it displays the color of the Projection, but the transpareny of the original object material?

For the second question: just switch off the “Recieve Decals” property of the desired object in the Rendering tab.

Hello, Dementium4ever

This script has a great deal of potential.
As for its usability for Matte Painting Camera projection,… Currently the camera projects a texture onto the scene. But how would one go about using the same camera to render from?
It doesn’t show up in the Camera list. When I use the Blueprint “Projector_Final” inside the Level sequencer to render a single frame, It yields blurry results.

Any idea how to Render from the same camera?

Thanks!

Hi JSGP,

sorry for the long silence, but I added a screenshot feature to the script! Your usecase is quite interesting.
Just redownload the new Version from Github. For the feature to work its very important to start **Unreal Engine
as Administrator! **

Just press on the Screenshot button to take a screenshot, the other settings (Screenshot Res and Screenshot Path) should be
self-explanatory.

Great project, Dementium4ever!

I have one issue. I can successfully project video files during preview, but as soon as I package (Windows 64bit) none of the projections show up. This holds true for the vanilla project file with default “no_input_texture” and with my modified project that is set up to play mp4 video clips.

Am I missing a step before packaging the project?

Hi Dementium4ever,

really great work! Exactly what I need for my bachelor thesis.
The “Field of View” slider is supposed to scale the projection?! It does not seem to do anything right now, or am I missing something?

Edit: Nevermind. It works when playing, but not in the Preview.

+1 great work. I’ll make sure to follow this, and store a copy somewhere

@Quacker42 For the movies to work after build you have to put them in a folder called “Movies” in the root of your folder structure. Does this solve the problem with your mp4s?