Announcement

Collapse
No announcement yet.

UE 4.19 native spatialization, how to use?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    UE 4.19 native spatialization, how to use?

    Hi, we are trying to use the new UVoipTalker (available since UE 4.19) in order to have users' voices spatialized.
    We've already modified some previous versions of the engine (until 4.18.3) to have the correct association between player id and player voice's AudioComponent. Our modifications are not appliable anymore with UE 4.19, because of some modifications on FRemoteTalkerDataImpl.

    So, we are trying to use the new native way to spatialize users' voices.
    The API seems to be simple, but what we obtain is simply non-spatialized voices.

    Having the player character, we do the following things in its Event BeginPlay:
    Click image for larger version

Name:	232507-image-131071.jpg
Views:	359
Size:	123.9 KB
ID:	1440111

    We do the following for each player character spawned on each machine (client or server): we create the UVoipTalker and assign it the VoiceSettings (with the component to which to attach the voice and the attenuation settings). We already use the same attenuation settings before, by attaching them to the same Avatar Head, and it worked perfectly.
    Last edited by masmil1988; 03-08-2018, 11:23 AM.

    #2
    Hi Masmil1988,

    Sorry I'm not able to see your image here for some reason- looks like it might be a broken link?

    I will say that a common issue with this is that, due to initialization steps, calling Register with Player State on BeginPlay will probably be called before you can actually register anything to the Player State. I've usually assuaged this with a short delay:
    Click image for larger version

Name:	TalkerRegister.PNG
Views:	323
Size:	109.1 KB
ID:	1437156

    Note the 0.2 second delay here. Hope this helps!

    Comment


      #3
      Also, while you're checking out the UVOIPTalker stuff, I should also mention that you can create a blueprint child class of UVOIPTalker, and in doing so get events for when the registered talker starts and stops speaking:
      Click image for larger version

Name:	TalkerFunctions.PNG
Views:	313
Size:	42.7 KB
ID:	1437159
      For example, here I'm just playing a sound when someone starts and stops talking to create a sort of "walkie talkie" effect. If the person speaking is on Windows, this will also take silence detection into account, so you will get Begin Talking events when someone actually starts speaking and End Talking events when they stop, even if you're not using push to talk or if someone is keeping their push to talk button held down.

      Let me know what kind of rad stuff you use the VOIPTalker for!

      Comment


        #4
        How do I add/create this voip talker in the right way? With this setup it worked for me once but now it just crashes everytime I try to speak. The "normal" voip setup just works fine and I tested it on the packaged build and in the editor.

        I will provide a screenshot of my setup which is almost the same as shown by Ethan.

        Comment


          #5
          SnowNOR That crash is probably caused by a recent regression which I just fixed here, if you have source access on GitHub:
          https://github.com/EpicGames/UnrealE...7dee53b6afdfd5

          Comment


            #6
            Ethan.Geller Hey Ethan,
            Is it possible to use the new VOIPTalker with dedicated servers instead of OnlineSubsytem as well?

            Comment


              #7
              Has anyone figured out the "right" way to set this up yet?

              When APlayerState replicates, the possessed pawn may not have replicated over yet, or the PlayerState property on the pawn indicating possession may not have replicated over yet so you can't handle it there.

              When the Pawn replicates over, the pawn's PlayerState property may not have replicated over yet so you can't completely handle it there.

              When the Pawn's PlayerState variable replicates over, the UniqueID on the APlayerState may not have replicated over yet so you can't completely handle it there.

              Once the UniqueID replicates over, you can't easily handle it because there seems to be no way to get the Pawn at that point (in the OnRep_UniqueId handler) unless you iterate through every Pawn in the game and check if it has a PlayerState pointing to the current one (slow), and the Pawn itself or the Pawns's PlayerState variable may not have replicated over yet anyway.

              Only if you attempt handle it in almost all of those places can it work, but I haven't figured out how to get the PlayerState's Pawn in the OnRep_UniqueID without iterating through all Pawns, which may be slow and cause a hitch?

              (Edit: the .2 second delay in the official solution above does not look like good practice. I see solutions like that that given in lots of official tutorials on different network things, but it is just wishing away the problem. You don't see that kind of thing in ShooterGame code, etc.

              You need to way until player state, player state's UniqueId property, the player state's possessed Pawn property, and the possessed pawn's PlayerState PlayerState have all been replicated. They can all happen in many different orders, but you can't fully set things up until they are all there. *Maybe* that all happens .2 seconds after begin play, but maybe your machine gets choppy and it doesn't.)
              Last edited by muchcharles; 03-18-2018, 02:06 PM.

              Comment


                #8
                Hi Ethan.Geller , the solution you suggested works well, only I really don't love Delays because I've seen sometimes PlayerState is replicated after 0.2 seconds, sometimes at 0.4 it has not been replicated yet. At the moment I opted for a continuous polling on the validity of PlayerState with a delay of 0.1 seconds between each check; it is not an optimal solution but is a little more deterministic (sooner or later PlayerState will be valid).

                Click image for larger version

Name:	Untitled.jpg
Views:	302
Size:	113.1 KB
ID:	1448823

                Now, solved that one, we have another related problem...
                Do you know why the native spatialization does not work at all with the Oculus subsystem? When we set to use the Oculus subsystem in the DefaultEngine.ini, neither spatialization/attenuation nor the events EventBeginTalking and EventEndTalking work anymore.

                Comment


                  #9
                  Originally posted by masmil1988 View Post
                  Hi Ethan.Geller , the solution you suggested works well, only I really don't love Delays because I've seen sometimes PlayerState is replicated after 0.2 seconds, sometimes at 0.4 it has not been replicated yet. At the moment I opted for a continuous polling on the validity of PlayerState with a delay of 0.1 seconds between each check; it is not an optimal solution but is a little more deterministic (sooner or later PlayerState will be valid).

                  Click image for larger version

Name:	Untitled.jpg
Views:	302
Size:	113.1 KB
ID:	1448823

                  Now, solved that one, we have another related problem...
                  Do you know why the native spatialization does not work at all with the Oculus subsystem? When we set to use the Oculus subsystem in the DefaultEngine.ini, neither spatialization/attenuation nor the events EventBeginTalking and EventEndTalking work anymore.
                  I'm using OnlineSubsystemSteam and Steam Audio, and hitting the same problem, for what that's worth.

                  Comment


                    #10
                    I've seen from code that, apart from the Null subsystem, no one uses this feature.

                    Comment


                      #11
                      masmil1988 muchcharles
                      I agree that the delay is bad practice. I agree with Masmil1988's idea of using an IsValid check on tick or a delay loop as a more robust way of handling this.

                      masmil1988 Zenteran A7E7
                      re: different online subsystems: UVoipTalker is only supported with Unreal's native VOIP support, which is used in the Null and the MCP subsystems.
                      Last edited by RevolverOSCelot; 04-04-2018, 02:00 PM. Reason: whoops, realized that Masmil1988's blueprint is exactly the thing I described...

                      Comment


                        #12
                        Originally posted by Ethan.Geller View Post
                        masmil1988 muchcharles
                        I agree that the delay is bad practice. I agree with Masmil1988's idea of using an IsValid check on tick or a delay loop as a more robust way of handling this.

                        masmil1988 Zenteran A7E7
                        re: different online subsystems: UVoipTalker is only supported with Unreal's native VOIP support, which is used in the Null and the MCP subsystems.
                        Is there plans soon, to support sub systems with this out of the box? Most users like myself will be using Steam or something else because this is kind of only useful in multiplayer games.
                        In League

                        Comment


                          #13
                          Why is this NOT supported by OSS Steam which, I would like to think at least 95% of your userbase uses for their games?
                          Website/Portfolio: http://www.VictorBurgosGames.com

                          Join me on stream: https://www.twitch.tv/BurgosGames for UE4 Game Dev. If you need help, just stop by and ask!

                          Wishlist Neko Ghost, Jump! a 2D/3D Puzzle-Platformer : https://store.steampowered.com/app/1...ko_Ghost_Jump/

                          Comment


                            #14
                            Hey Guys,

                            I don't understand how you make it work?
                            On my side, using OnlineSubsystemNull as required, I keep failing with this errors in log:
                            Code:
                            LogVoiceEngine: Error: StartLocalVoiceProcessing(): Device is currently owned by another user
                            LogVoiceEngine: Warning: ReadLocalVoiceData: GetVoice failure: VoiceResult: Ok
                            LogVoiceEngine: Warning: Voice data error in ReadLocalVoiceData
                            My DefaultEngine.ini (the section related to voice):
                            Code:
                            [OnlineSubsystem]
                            DefaultPlatformService=NULL
                            bHasVoiceEnabled=true
                            
                            [Voice]
                            bEnabled=true
                            And DefaultGame.ini:
                            Code:
                            [/Script/Engine.GameSession]
                            bRequiresPushToTalk=false
                            Do you have an idea why?

                            Thanks

                            Edit: Ethan.Geller I managed to don't have the "Error: StartLocalVoiceProcessing(): Device is currently owned by another user" error anymore by modifying FOnlineVoiceImpl::RegisterLocalTalker(..) (VoiceInterfaceImpl.cpp:185) and placing the call to Return = VoiceEngine->RegisterLocalTalker(LocalUserEnum) before the call to StartNetworkedVoice(LocalUserEnum). But it still does not work in the end...
                            Last edited by LNaej; 05-19-2018, 11:37 AM.

                            Comment


                              #15
                              Sorry for double post, but do not hesitate to vote for this bug if you work with UE4 VOIP: https://issues.unrealengine.com/issue/UE-59508

                              Comment

                              Working...
                              X