Announcement

Collapse
No announcement yet.

UE4 + Nvidia Ansel + 360 Video Capturing

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • started a topic UE4 + Nvidia Ansel + 360 Video Capturing

    UE4 + Nvidia Ansel + 360 Video Capturing

    Hi guys,

    I am trying to take a 4K 360 30fps stereo video from UE4 projects (cinematic scenes made with Matinee or Sequencer).

    I tried Stereo Panorama Tool plugin (Kite & Lighting), but it failed to render properly some problematic parts of some scenes and it's claimed (by other developers) that it works not good with full scene reflection.

    There's a tool developed by Nvidia called Ansel that can take in game 360 stereo image screenshot. Also there's a plugin for UE4 that integrates this tool using AnselSDK.

    One of the problems is that Ansel can be activated (capturing itself, its session can be activated from code) only by pressing on Ansel UI button (whole Ansel UI injected into the game itself once the session is activated). The only way that I have found to automate the process is to inject Windows keyboard events (mouse events cause rotation of the camera sometimes and instability of the captured images) using Windows API (Native, Winform), AutoIt, etc. (all the tools, techniques are free, didn't try commercial products) in order to press on required buttons by TABing (menu navigation) with proper time stamps.

    The problem with this approach is that the Ansel UI which is injected into the game (UE4 standalone mode) doesn't catch these event sometimes which causes a lot of problems (missed frames and wrong selection in the menu).

    Another problem with Ansel is that if it runs for a long time (depending on a machine), it crashes UE4 itself. So the process should be started again and continue from the point of fail. For this problem I didn't find a proper solution as well, I would like to use it as Kite & Lighting.

    Hope you have some thoughts

    Thanks!

  • replied
    Originally posted by Jenova723 View Post
    Edit: It says it currently only supports fortnite.......
    Surreal Capture software was improved quite a lot in the past two months. It supports to capture 360 degree video directly from a lot of more UE4 games now. Here is the youtuble channel that demonstrate the sample 360 degree video:
    https://www.youtube.com/channel/UCV-...kqLrxlvLPyn60A

    Here is the link to the Surreal Capture website: https://www.surrealcapture.com

    Leave a comment:


  • replied
    Originally posted by raptoreagle View Post

    uninstall the gforce experience and install an older one that worked or the one you used last.
    Thanks mate.

    Actually i found an even better work around, if we disable in-game overlay in GeForce experience the old Ansel plug in is back ! NVCameraConfiguration.exe is still missing tho.

    Leave a comment:


  • replied
    Originally posted by Davidaurecon View Post
    Hi I have been using this method of capturing 360 video with the Ansel plug in and the macros and it worked really well !
    until recently with the latest Nvidia drivers update it seems that Ansel is not working anymore in the Engine ! I can't bring up the Ansel interface anymore.
    this folder is gone as well : C:\Program Files\NVIDIA Corporation\Ansel . This is extremely disappointing that they broke something that was working fine. Anybody found a workaround for that ?
    uninstall the gforce experience and install an older one that worked or the one you used last.

    Leave a comment:


  • replied
    Hi I have been using this method of capturing 360 video with the Ansel plug in and the macros and it worked really well !
    until recently with the latest Nvidia drivers update it seems that Ansel is not working anymore in the Engine ! I can't bring up the Ansel interface anymore.
    this folder is gone as well : C:\Program Files\NVIDIA Corporation\Ansel . This is extremely disappointing that they broke something that was working fine. Anybody found a workaround for that ?

    Leave a comment:


  • replied
    Hi, I was trying to create a new post but can't, so I am posting here. I am rendering HDR stereo using Ansel. I tried render images with HUD, I have set r.Photography.AutoPostprocess 0. but the rendered image is not showing HUD. How can I get rid of this?

    Leave a comment:


  • replied
    Originally posted by y_mike1 View Post
    You can try Surreal Capture (https://www.surrealcapture.com), it is world first and only UE4 360 degree video capture software, it can plug into UE4 games and capture realtime 360 degree video from it.
    1. It is not the first
    2. It is not the only
    3. I will give it a try and hope all post processing works correctly, will give feedback soon as i test.

    Edit: It says it currently only supports fortnite.......
    Last edited by Jenova723; 07-17-2019, 01:32 AM.

    Leave a comment:


  • replied
    for some reason the ansel plugin is forcing my game resolution to 720p this makes capture slower than it should. i'm trying to force 4k but i'm not sure what i'm doing wrong..

    Attached Files

    Leave a comment:


  • replied
    shame epic wont embed 360 capture in the engine

    Leave a comment:


  • replied
    You can try Surreal Capture (https://www.surrealcapture.com), it is world first and only UE4 360 degree video capture software, it can plug into UE4 games and capture realtime 360 degree video from it.

    Leave a comment:


  • replied
    Originally posted by raptoreagle View Post

    Have you tried this?
    Yes, but that plugin doesnt seem to be enabling Anti-Aliasing, I have also opened a ticket on the github page for that as well. I currently am using the python script to automate but another issue has popped up where the camera doesnt move along with the player because i never end the ansel session, so for scenes where i need to move the camera I use the old jitbit macro method.

    Leave a comment:


  • replied
    Originally posted by Roberto De Ioris View Post
    Hi, just to inform you that i have released a plugin for automating 360 captures via Ansel: https://github.com/rdeioris/AnselCapture
    Have you tried this?

    Leave a comment:


  • replied
    Originally posted by Sigurgeir View Post
    . My problem is no matter what I do i can't seem to have any macro program register the "space" button to actually snap the shot. Other keys it will register. I have even tried to hardcode the location on a mouse and have it click it but the same story. I'm using 4.22 and launching the game in stand alone to get Ansel working.
    Are you using the GeForce Experience Overlay or do you have it disabled? I have it enabled under my Geforce Experience Settings.

    Also, I couldn't get a good method to dish out renders with any macro, so i made this python script to press spacebar after each capture, automatically :
    Code:
    import socket
    import pynput
    import time  
    
    UDP_IP = "127.0.0.1"
    UDP_PORT = 5005  
    
    sock = socket.socket(socket.AF_INET, # Internet
                           socket.SOCK_DGRAM) # UDP
    sock.bind((UDP_IP, UDP_PORT))  
    
    from pynput.keyboard import Key, Controller
    keyboard = Controller()
    FrameCount = 0
    TotalFrames = input("Enter total number of frames to capture")
    print(TotalFrames)  
    
    while FrameCount < int(TotalFrames):
    data, addr = sock.recvfrom(1024) # buffer size is 1024 bytes if data.decode("utf-8") == "Capture":
    print(FrameCount) time.sleep(1.5) keyboard.press(Key.space) keyboard.release(Key.space) FrameCount += 1
    else:
    print("Incorrect string received")
    And the PlayerCameraManager Blueprint :
    Click image for larger version  Name:	Screenshot-21.png Views:	1 Size:	42.8 KB ID:	1628285 As you can see i'm recieving a UDP message from unreal to simulate the spacebar press (which I got working with the udp plugin and Extended BP Library)

    But after using this method I was encountering a lot of jittering in the video being rendered so I had to set r.PhotographySettleFrames to 30 (yes, it takes a lot more time) but it seemed to have solved the problem, but now producing a new problem, my skeleton mesh animations jitter a lot more.

    Does anyone have a better method to capture? I'm almost about to give up and live with the jitters
    Last edited by Jenova723; 06-06-2019, 07:58 AM.

    Leave a comment:


  • replied
    Going insane trying to get this to work with macro. Already downloaded iBrews tutorial to be sure that I wasn't doing anything wrong but it's all the same. My problem is no matter what I do i can't seem to have any macro program register the "space" button to actually snap the shot. Other keys it will register. I have even tried to hardcode the location on a mouse and have it click it but the same story. I'm using 4.22 and launching the game in stand alone to get Ansel working.

    Have you guys made this approach work with 4.22?

    Leave a comment:


  • replied
    Good day!
    I wanted to report that a new method has been created to create 2D, 180, 360 , mono and stereo videos. It allows you to quickly get the desired result.
    If someone is interested, a link to the discussion and a link to the product.
    https://forums.unrealengine.com/unre...tem-blueprints

    Leave a comment:

Working...
X