Announcement

Collapse
No announcement yet.

Projection Mapping technique to projects video files on objects?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Projection Mapping technique to projects video files on objects?

    Hello,
    I'm looking for way to create some kind of projection mapping in UE.
    All 3D programs has features such as project animated textures onto objects surface, like Cinema 4D has option to use Camera-View to project texture from there on some objects, so instead of UV it uses camera-angle, here is couple videos what I want to create in UE.



    Can you please help me to create this kind of effect? Let say can you take two cubes and project some video-texture on them at the same time?
    I was thinking to use Decals, but not sure is it going to work or not? I need 10 sec video file to be constantly looped while I walking in the level and it's going to be few of them.
    What is the best technique to do that?
    Maybe there is a way to project video texture from light object?

    Please guide me somewhere, is it possible in Unreal?

    Thank you.

    #2
    depending on the length of the video you could probably use flipbooks and light functions for that.


    https://docs.unrealengine.com/latest...ightFunctions/

    Comment


      #3
      Light Function seems to have only alpha channel?
      Flipbooks - I will look what is it, how to project it.

      Comment


        #4
        Camera projections are possible in Unreal:

        https://forums.unrealengine.com/show...CameraVectorWS

        Not sure of the best way to share it, but I can try to figure something out if you think you could use it.
        Last edited by _Scribbler_; 12-09-2016, 02:13 PM.

        Comment


          #5
          Wow looks very interesting, it may work, looks bit complicated with setup, but if you can share project file I can look for reference, I only have question about video-file or maybe loop image sequence to play? Can you make quick demo file with playing video?

          Comment


            #6
            I'll see what I can do. Been meaning to share this for a while, but haven't had a lot of time.

            Comment


              #7
              Looks like you used "Transform to Z Vector". that will only work as long as the camera is never tilted sideways at all since it derives the XZ vectors assuming an up vector of 0,0,1 to start. That node was meant to create a tangent basis from vertex normals on terrain and things like that.

              If you do a full transform using the camera X, Y and Z vectors it will work anywhere. The bottom half of this image (stopping at the texture sample, the top part is only needed if you are doing a shadow depth comparison and the texture is a shadow depth map):

              Click image for larger version

Name:	PerObjectShadow_07.gif
Views:	2
Size:	147.3 KB
ID:	1119824

              In the above case "Object Position" is simply the center of the object, and will determine the center of the projection coordinates. Use whatever there.

              Object Radius can be changed to equal the size of the texture you want to project.

              There is actually a function in Engine\Content (not exposed to the right click search menu) called "BoundingSphere_LightTransform" that I made a while ago to create shadowmaps for unlit materials, but it will likely be replaced by something closer to the above graph which is much more robust. Details on what its used for here:

              http://shaderbits.com/blog/custom-pe...ing-blueprints
              Last edited by RyanB; 12-09-2016, 06:00 PM.
              Ryan Brucks
              Principal Technical Artist, Epic Games

              Comment


                #8
                Thanks Ryan - that's how I got it working properly in the end. Took a while to figure out, though...

                I'm now trying to figure out a way to turn it into a slick "import camera > choose projection image > choose geometry to project onto" workflow. Will open up some DMP techniques in Unreal.

                Many thanks

                Comment


                  #9
                  I've got it packed up in some kind of transferrable form.

                  Step 1:

                  Download the following .zip file, and unpack it into the 'Content' folder of your project.

                  DOWNLOAD

                  It contains 3 .uassets:

                  >'decal2' material
                  >'projection_decal2' blueprint
                  >'video' media object

                  Step 2:

                  1 - drag the blueprint into the level

                  2 - choose the camera actor you want to project from

                  3 - choose the texture you want to project

                  4 - offset the decal bounding box to enclose the objects to are try to project onto

                  Step 3:

                  I added a blank media texture to the decal material to check that it hooked up okay. It won't do anything without some blueprint shenanigans, but it looks like a viable workflow.


                  Bit 'bare bones' for now, but let me know if it works, and if there is anything I should do to improve it.



                  Comment


                    #10
                    Hello Scribbler..

                    Ive downloaded your zip file...
                    This Blueprint has a great deal of potential!
                    Ive been searching for a way of incorporating a matte painting approach into Unreal.

                    Using an camera to render a shot, bring this image into Photoshop to paint over, then re-project this back.

                    The blueprint works great with the default texture associated with it, but I'm unable to swap textures.
                    Adding a new texture in the "Projection Image" area does nothing.

                    Ive opened the blueprint itself in an attempt to replace it there, but cant find where its located.

                    any ideas?

                    Thanks!

                    Jeff

                    Comment


                      #11
                      Originally posted by _Scribbler_ View Post
                      I've got it packed up in some kind of transferrable form.

                      I added a blank media texture to the decal material to check that it hooked up okay. It won't do anything without some blueprint shenanigans, but it looks like a viable workflow.


                      Bit 'bare bones' for now, but let me know if it works, and if there is anything I should do to improve it.

                      hello there, excellent solution , wonder why a special tool does not exist in the engine, it can have so many uses.
                      Have used your blueprints for a personal test project and here is some feedback:

                      comments:

                      1. used it with a video texture, it works just fine with some tweaks.

                      2. not really sure what i changed in the blueprints to get the new texture working, simply setting as a camera texture did not work.
                      Maybe the video texture induces extra lag in user feedback so it got me confused.


                      issues:

                      1. texture does not update when moving camera or setting camera field of view.

                      Need to manually reselect camera from the Projection camera variable in the projecteddecal options. How can we have this auto-refreshed when camera moves?
                      This is very crucial for precise/interactive texture positioning, or any realtime usage, in example the player has a projector/flashlight of any kind that lightsup stuff while moving.

                      2. scaling from the offset actually distorts (not only crops) projected texture. this does not happen with my video texture but with the original texture you provided.



                      I think that is all for now. Thank you for sharing this, we really appreciate.
                      Last edited by flatform; 05-21-2018, 12:12 PM.

                      Comment


                        #12
                        Is this still NOT part of UE? This is something that should have been there for years already! In regular 3D apps this is used all the time for VFX, and it´s a big omission

                        Comment


                          #13
                          Thank you _Scribbler_ and RyanB for your awesome work! I used this script it in my Bachelors Thesis, without your groundwork it would not have been possible for me!

                          I improved your script quite a bit, I added the capability to cast shadows, and improved the usability.


                          So first the shadows: The projection only shows on the first layer of objects it hits, all objects behind don't recieve any light, just like a real life projection would behave.
                          The shadows are projected from the camera, so that if you view your scene through the eyes of the projection camera, you're going to see no shadows.

                          Projection with Shadows

                          Projection with shadows side


                          Now to the improved usability: I set up the Blueprint in a way, that you only need to drop in the Projection Blueprint and specify your media texture. Thats it!

                          There are a few more options that you'll often use though, First the "X Scale" in the "Offset" Variable. This changes how far your texture will be projected.
                          The second one is the "Edge Bleed". This determines how hard the shadow is blended with the projected image where both areas meet.
                          The last one is the "Field of View" on the Camera Sub-component, which adjusts the size of the projection.

                          Settings 2




                          I hope this helps someone as much as your work helped me! If you have any questions, just leave me a message! Download it here: https://github.com/ChristopherRemde/Projection_shadow



                          Just click on "Clone or Download", download and unzip, then open the project and drop a file into the Media Player, et voila!
                          Last edited by Dementium4ever; 01-21-2019, 11:47 PM.

                          Comment


                            #14
                            Thank you Dementium4ever ! Great work on the script! I have just one question. The projection is bleeding/smearing outside of the screen area. How can I get rid of this? Any ideas?

                            Comment


                              #15
                              That's actually a bug on the shader I still have to fix. Maybe I'll upload a patched version today or tomorrow.

                              ​​​​​​Edit: A thing I missed out on was that my script is currently only made for 16:9 content. But I'll fix that too!
                              Last edited by Dementium4ever; 02-19-2019, 06:56 AM.

                              Comment

                              Working...
                              X