Announcement

Collapse
No announcement yet.

UE4 + Nvidia Ansel + 360 Video Capturing

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    UE4 + Nvidia Ansel + 360 Video Capturing

    Hi guys,

    I am trying to take a 4K 360 30fps stereo video from UE4 projects (cinematic scenes made with Matinee or Sequencer).

    I tried Stereo Panorama Tool plugin (Kite & Lighting), but it failed to render properly some problematic parts of some scenes and it's claimed (by other developers) that it works not good with full scene reflection.

    There's a tool developed by Nvidia called Ansel that can take in game 360 stereo image screenshot. Also there's a plugin for UE4 that integrates this tool using AnselSDK.

    One of the problems is that Ansel can be activated (capturing itself, its session can be activated from code) only by pressing on Ansel UI button (whole Ansel UI injected into the game itself once the session is activated). The only way that I have found to automate the process is to inject Windows keyboard events (mouse events cause rotation of the camera sometimes and instability of the captured images) using Windows API (Native, Winform), AutoIt, etc. (all the tools, techniques are free, didn't try commercial products) in order to press on required buttons by TABing (menu navigation) with proper time stamps.

    The problem with this approach is that the Ansel UI which is injected into the game (UE4 standalone mode) doesn't catch these event sometimes which causes a lot of problems (missed frames and wrong selection in the menu).

    Another problem with Ansel is that if it runs for a long time (depending on a machine), it crashes UE4 itself. So the process should be started again and continue from the point of fail. For this problem I didn't find a proper solution as well, I would like to use it as Kite & Lighting.

    Hope you have some thoughts

    Thanks!

    #2
    I use Ansel with scripted keyboard events and it works great rendering 3000 stereo 8k 360 frames. Never had a problem with crashing. I use the macro program JitBit.

    Comment


      #3
      Originally posted by reedandrader View Post
      I use Ansel with scripted keyboard events and it works great rendering 3000 stereo 8k 360 frames. Never had a problem with crashing. I use the macro program JitBit.
      Could you please explain your process in a bit more detail? I'm interested in how you get frame accurate capture.

      Comment


        #4
        Originally posted by reedandrader View Post
        I use Ansel with scripted keyboard events and it works great rendering 3000 stereo 8k 360 frames. Never had a problem with crashing. I use the macro program JitBit.
        Hi, I'm also interested in knowing more about your process! I've been struggling with 360 video capture using the UE SP Tool and automatizing Ansel seems like the best option...

        Comment


          #5
          Hi, I'm also interested in knowing more about how you did! I've been struggling to capture 360 video with UE SP Tool and automatizing Ansel seems like the best solution so far...

          Comment


            #6
            Hey gang. Just found this thread recently and gave JitBit a try with Ansel. I'm able to spit out an image sequence but I'm not sure if it's exactly capturing each frame at 30 FPS. I suspect it isn't.

            The problem as I see it is that all the cinematic nodes are in values of "float". Because of that, one second of animation in a 30 fps speed is 1/30 which is (.0333333333333333333....). I think we need whole numbers some now as this infinite float value looks like it'll let a lot of error creep in as we render longer image sequences.

            Attached is the level blueprint I am using to advance the game one frame then pause it at a rate of 30 fps. I've tried this with and without forcing the game to an FPS of 30 and I'm not sure if that's helping or not. Any help on playing the game then pausing it, one frame at a time in a 30 fps time frame, would be much appreciated.





            Attached Files

            Comment


              #7
              Thanks everyone who has been contributing to this discussion so far. I put together a short teaser video of my (working) workflow here, expanding upon everything in this thread:

              https://www.youtube.com/watch?v=8NH00PWMjm4

              And will do my best to put together a full tutorial at some point.
              Alex Coulombe
              Creative Director
              Agile Lens: Immersive Design
              New York City
              www.agilelens.com

              Comment


                #8
                Originally posted by iBrews View Post
                Thanks everyone who has been contributing to this discussion so far. I put together a short teaser video of my (working) workflow here, expanding upon everything in this thread:

                https://www.youtube.com/watch?v=8NH00PWMjm4

                And will do my best to put together a full tutorial at some point.
                hi man !

                can you explain how u did?
                it's amazing!

                Comment


                  #9
                  This is a game changer, any more tips or info you can provide about how you are setting this up would be greatly appreciated!

                  Comment


                    #10
                    It's kind of insane how many emails I've gotten about this workflow over the last month. Here you go guys:

                    https://gumroad.com/l/dAgAT

                    If you make something cool, please share it with me on twitter iBrews
                    Alex Coulombe
                    Creative Director
                    Agile Lens: Immersive Design
                    New York City
                    www.agilelens.com

                    Comment


                      #11
                      iBrews Have implemented as described in the video but it doesnt seem like the macro wants to fire off Ansel...I can see the other parts of it running through the motions but it wont start up ansel after I have done the first manual step. Is there a special window configuration or something we need to make sure is always on top for it to work?

                      Comment


                        #12
                        Warrior9VR hmmm don't think so. I'm guessing you were able to use Ansel manually. Make sure your Macro has these checkboxes set up (I think the ControlSend is the key one here) :
                        Attached Files
                        Alex Coulombe
                        Creative Director
                        Agile Lens: Immersive Design
                        New York City
                        www.agilelens.com

                        Comment


                          #13
                          iBrews ah yes that would definitely have been the issue! Got it working now, very cool. One thing I noticed though is that my camera seems to shift during the capture process and as such i am getting some multiple exposure effects going on. Trying to figure it out now

                          Comment


                            #14
                            testing across different levels and indeed the double image im getting is a result of the camera animation being paused seemingly between frames.

                            Comment


                              #15
                              Originally posted by Warrior9VR View Post
                              testing across different levels and indeed the double image im getting is a result of the camera animation being paused seemingly between frames.
                              Is your frame-rate in the sequence the same as your frame-rate in the blueprint? Sounds like this would happen if your sequence framerate is 30 and your blueprint framerate is 60.
                              Alex Coulombe
                              Creative Director
                              Agile Lens: Immersive Design
                              New York City
                              www.agilelens.com

                              Comment

                              Working...
                              X