Announcement

Collapse
No announcement yet.

Screen tearing / Jelly Effect on 4.23 ARKit Camera

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Screen tearing / Jelly Effect on 4.23 ARKit Camera

    *Environment: OSX 10.14, Xcode 11, iPhone XS Max & iPad Air w/ iOS 13.1.3*

    The screen tearing issue seems only exclusive to 4.23 (and .1), whereas the camera feed looks like being splitted into individual transparent boxes which don't sync (a bit like when vsync is not enabled).

    `stat unit` would show the game is running at 60fps, however the camera feed is nowhere near 60f (more like 30f, even with default AR template). Meantime this issue is not affecting 4.22 and 4.21, wonder if this is something introduced with person segmentation or depth recognition in ARKit 3?

    Does anyone also experience this issue?

    #2
    I am also experiencing this issue (iOS 13, 4.23.1). Turning on VSync (r.vsync 1) improves this issue, but it still happens in the top right corner of the screen

    Comment


      #3
      Originally posted by amatthews11 View Post
      I am also experiencing this issue (iOS 13, 4.23.1). Turning on VSync (r.vsync 1) improves this issue, but it still happens in the top right corner of the screen
      Good to know I'm not alone. I suspect something is messed up either with vsync or the tick sync, or depth segmentation. Turning on the tick sync in session config also helps a little bit. But most of the time the tearing seems heavily depends on surrounding brightness and complexity (that's why was wondering if it's related to depth API). Anyway - submitting bug report later.

      Comment


        #4
        Update - issue logged and merged with: https://issues.unrealengine.com/issue/UE-81611

        Comment


          #5
          So 4.24.1 update is here, ARKit still unusable. Apparently Target Fix information in the bug reports mean nothing.
          ARCore even worse, looks like Epic gave up on Mobile AR.

          Comment


            #6
            Any update on this?? I have the same issue and I just got a contract to work with ARKit and Unreal Engine.

            Comment


              #7
              Oh no, same here. My sympathies...

              We will have to check regularly the issue link from Novezeil.
              If it gets marked as fixed, we could get the latest update from GitHub. (https://docs.unrealengine.com/en-US/...ine/index.html)
              But a fix is unlikely during the holidays.

              Using cross-platform ARCore as a possible alternative is out of question as that plugin is even more broken.
              I am investigating Apples Reality Composer App and Unity to create AR Content. But this is nothing for a short term solution i guess.
              Building AR Experiences with Reality Composer: https://developer.apple.com/videos/play/wwdc2019/609

              Comment


                #8
                Had dived into this issue earlier and it might caused by something complicated related to metal shader from Fast Math (although I might be totally wrong here). Long story short - if anyone is looking for ARKit in UE, the current best practice would be stay with 4.21 / 4.22 and Xcode 10.x until further updates from issue tracker.

                4.23 / 4.24 is not totally unusable though and the final result oddly vary from device to device, and iterations of compilation, i.e. compile same project for 10 times and choose one with least artifact (though this might sounds like total non-sense but guess it's not something new for people worked with Java before).

                As for ARCore - never had the chance to make it work since 4.20, as for targeting android, I sincerely think ARFoundation or Vurforia would be the better choice (in a foreseeable future), or directly use the source build from Google.

                Comment


                  #9
                  I had to rollback my Xcode to 10.x and Unreal to 4.22.3. I tested ARCore and ARKit successfully, so I will keep those versions until the bug is fixed.

                  Comment


                    #10
                    We stopped at 4.20 and released an AR app, 4.24 seems like a lot of work to get things running in AR, maybe Epic will take a look.

                    Comment


                      #11
                      Any project settings, that might have an influence? Would you mind to give some insight to your ios-settings in UE4?

                      Comment


                        #12
                        Originally posted by GRIMSK
                        Hmm..... I don't know what's going on here, but in my case, I have no problem with UE 4.24.1 & Xcode 11.3 when I built my iOS game based on ARKit 3.0 now.
                        (UE 4.24.1 from Epic Games Launcher, not from source-built / Xcode 11.3 from AppStore, not beta)
                        I'm not suppose to advertise my poor game, but if someone want to check there's no problem in the result then I would tell him my game.
                        Thanks, I would be very interested to know (presume so do others as this is such a headache).

                        Could you share the appstore link and if it's possible, an empty working project? (Or if you can test if the issue persists in default template)

                        Comment


                          #13
                          Originally posted by GRIMSK

                          OK, I wiped out the whole contents in my project except for settings and plugins.
                          Anyone can download it from here, so have a look into it's settings.
                          ...
                          I wish this has been helpful.

                          Thank you for your effort and sharing this.
                          However i'm afraid there has been a misunderstanding.

                          So your app is VR, it does not use camera footage, that is an unusual case. Naturally everyone means the "real world" camera feed from the ios-device when talking about an Augmented Reality framework, not the virtual camera in the UE4 world that renders actors in a scene.

                          Comment


                            #14
                            Originally posted by CosmicLobster View Post


                            Thank you for your effort and sharing this.
                            However i'm afraid there has been a misunderstanding.

                            So your app is VR, it does not use camera footage, that is an unusual case. Naturally everyone means the "real world" camera feed from the ios-device when talking about an Augmented Reality framework, not the virtual camera in the UE4 world that renders actors in a scene.
                            Oh, I'm sorry if I misunderstood. My app uses camera footage in the background for its tracking like Oculus Quest.
                            Can you check this video whether there's any problem you mentioned above?
                            I built and recorded it just before.

                            Comment


                              #15
                              Thank you for the video.
                              Yes the problem is visible in the video aswell. For example at playtime 17-20 seconds. (See image attached, this is at 19sec)
                              In the video (using iPhone) there is sometimes one tearing in the picture, but on iPads and iPadPros the camera image looks all the time more fractured like a broken chess-board. Endusers would send many complains and give bad reviews, so it is not usable for production.


                              Mobile hardware renders not in full frames, but in small tiles. The small rendered tiles then get composed to one full images. It seems the engine has a problem at that process.

                              By now the issue got 19 votes, target fix updated to 4.24.2, but as we learned last time, this is no reliable information and epic remains silent like always.
                              We are holding back production for an AR-project because of this.

                              Even if this issue gets fixed, AR Environment Capture Probes are broken too.

                              AR - Environment Capture Probe Type: Automatic crash
                              https://issues.unrealengine.com/issue/UE-85533

                              Screen Renders Black in a Packaged AREnvProbe Project
                              https://issues.unrealengine.com/issue/UE-81660

                              AREnvProbe - App crash on iPad if automatic environment capture is enabled
                              https://issues.unrealengine.com/issue/UE-86540


                              I will not trust release notes anymore before testing if they are actually usable for production in the future. This is very sad, so disappointed.

                              https://issues.unrealengine.com/issue/UE-81611
                              Last edited by CosmicLobster; 01-17-2020, 06:51 PM.

                              Comment

                              Working...
                              X