Announcement

Collapse
No announcement yet.

Audio roadmap and APIs

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Audio roadmap and APIs

    It would be great to know what plans, if any, are in store for the audio system in UE4. The lack of low level audio APIs (unless I've missed something so far) is extremely limiting as there is no easy way to implement custom DSP. With so much possible on the visual front, the audio system hugely limits creative possibilities of linking audio + visuals. It would be great to know if there are plans to open up and modularise the the audio engine. Until then, there is no choice but to use other game engines that offer such flexibility — which is disappointing. I'm sure I speak for other audio programmers and designers!

    Thanks

    #2
    Right now we don't have any other choice than to use middleware, like fmod and wwise - it would have been great if the audio system was more of a first class feature rather than current implementation. If we had a simple low level audio API with sample buffer access, then Epic wouldn't have to worry about audio effects since we could implement our own DSP functions and make them available on the future marketplace.

    Comment


      #3
      The short answer is: We don't currently have any near term plans to significantly change the audio API.

      The long answer/explanation is that for the most part we rely on the hardware to do the heavy lifting in our audio system and at the moment we have something like 7 different audio implementations for the different platforms. We definitely have some inconsistencies as it stands (which we are of course addressing) and adding/exposing new features in a way that is consistent across platforms is a challenge.

      Our long term wish is to move to our own software mixer to get consistent behaviors across all platforms (or at least much closer) and new features require one implementation instead of 7+.

      The trouble is that to do our own software mixer requires us to first thread audio handling and both of those tasks are big projects that it is a bit challenging to justify when wWise or fmod are both good options available to people who need the functionality.

      I really do wish I had a better answer than this (as do our in house audio engineers) but unfortunately this is where things stand for the moment.

      Comment


        #4
        Originally posted by Marc Audy View Post
        The short answer is: We don't currently have any near term plans to significantly change the audio API.

        The long answer/explanation is that for the most part we rely on the hardware to do the heavy lifting in our audio system and at the moment we have something like 7 different audio implementations for the different platforms. We definitely have some inconsistencies as it stands (which we are of course addressing) and adding/exposing new features in a way that is consistent across platforms is a challenge.

        Our long term wish is to move to our own software mixer to get consistent behaviors across all platforms (or at least much closer) and new features require one implementation instead of 7+.

        The trouble is that to do our own software mixer requires us to first thread audio handling and both of those tasks are big projects that it is a bit challenging to justify when wWise or fmod are both good options available to people who need the functionality.

        I really do wish I had a better answer than this (as do our in house audio engineers) but unfortunately this is where things stand for the moment.
        I think this is reasonable, but some time might be spent on docs and such for integrating those third party solutions, so that people don't stumble on audio due to it. Maybe there are some? I didn't find them but I wasn't scouring the docs for them, either.
        Storyteller - An immersive VR audiobook player

        Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

        Comment


          #5
          Originally posted by Marc Audy View Post
          The short answer is: We don't currently have any near term plans to significantly change the audio API.

          The long answer/explanation is that for the most part we rely on the hardware to do the heavy lifting in our audio system and at the moment we have something like 7 different audio implementations for the different platforms. We definitely have some inconsistencies as it stands (which we are of course addressing) and adding/exposing new features in a way that is consistent across platforms is a challenge.

          Our long term wish is to move to our own software mixer to get consistent behaviors across all platforms (or at least much closer) and new features require one implementation instead of 7+.

          The trouble is that to do our own software mixer requires us to first thread audio handling and both of those tasks are big projects that it is a bit challenging to justify when wWise or fmod are both good options available to people who need the functionality.

          I really do wish I had a better answer than this (as do our in house audio engineers) but unfortunately this is where things stand for the moment.
          Not exactly what I was hoping for Thank you for the information - looking forward to the long term solution. The current implementation will always have some inconsistencies because of the different implementation of ex. reverberation in the XAudio2, Coreaudio, OpenAl etc. libraries.

          We just need to implement our DSP code (convolution, environment simulation and binaural processing) in wwise/fmod.

          Comment


            #6
            You should bring back hardware audio support, now that Windows 8 supports this.
            Not that Windows 8 is an extremely popular system, but it's more popular than it was, say, a year and a half ago.
            YES, IT'S HAPPENING!
            Next - Unreal III.

            Comment


              #7
              This also makes me sad. As a game consumer (and not developer) I feel audio has taken a huge back seat to graphics.
              I'd love to see sounds occluded/reflected based on the actual level geometry/materials and player position vs. the generic approximations that have been in use for the last 10-15 years. You don't need to be an audiophile to appreciate 'real' sounding sound or have some crazy sound system, either. Adding a flag or two to the materials for audio properties when defining materials in the material editor and letting the engine handle the rest just like it does for graphics would be a beautiful solution!
              Unfortunately, that means some awesome programmer somewhere has to make this audio engine a reality.
              I hope this becomes a long term goal at least...it'd be a shame if it didn't!

              Comment


                #8
                Good to know that FMOD is free for Indies under 100k USD: http://www.fmod.org/sales/
                Sr. Engine Programmer @ www.playspace.com - moritzwundke.com
                Remember: be polite and respect other peoples opinions - Join the Unofficial Unreal Discord Channel - Found a bug? Then use the Bug Report Form to get it fixed ^^

                Comment


                  #9
                  Originally posted by VoxyGon View Post
                  This also makes me sad. As a game consumer (and not developer) I feel audio has taken a huge back seat to graphics.
                  I'd love to see sounds occluded/reflected based on the actual level geometry/materials and player position vs. the generic approximations that have been in use for the last 10-15 years. You don't need to be an audiophile to appreciate 'real' sounding sound or have some crazy sound system, either. Adding a flag or two to the materials for audio properties when defining materials in the material editor and letting the engine handle the rest just like it does for graphics would be a beautiful solution!
                  Unfortunately, that means some awesome programmer somewhere has to make this audio engine a reality.
                  I hope this becomes a long term goal at least...it'd be a shame if it didn't!
                  If I'm right, this stuff was supported back in Unreal 1 and UT99 with A3D hardware.
                  YES, IT'S HAPPENING!
                  Next - Unreal III.

                  Comment


                    #10
                    Originally posted by iLikeTheUDK View Post
                    If I'm right, this stuff was supported back in Unreal 1 and UT99 with A3D hardware.
                    To an extent some of it was, and unfortunately it regressed a lot when Vista came out. Solutions like A3D simulated room environments but were still just DSPs, they didn't truly take geometry into effect. Check this video and this video out for some awesome examples of how audio could be handled in the engine. This is the 'next gen' kind of stuff I'd like to see implemented at some point (not sure if that was what the original thread creator was envisioning or not, though).

                    EDIT:
                    I stand corrected. EAX used DSPs but A3D actually did take geometry into effect.
                    Last edited by VoxyGon; 08-12-2014, 05:35 PM.

                    Comment


                      #11
                      Originally posted by VoxyGon View Post
                      A3D actually did take geometry into effect.
                      Which is why I think this kind of feature is so desperately needed to be implemented. Since A3D support went deprecated with UE2, Unreal Engine, and nearly (if not all) game engines out there, ended up being stuck *behind* in audio technology.
                      YES, IT'S HAPPENING!
                      Next - Unreal III.

                      Comment


                        #12
                        BTW guys, the FMOD developers mentioned contacting support to get access to their (apparently ready soon) early UE4 integration, here: http://www.fmod.org/questions/question/ue4-integration
                        Storyteller - An immersive VR audiobook player

                        Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes

                        Comment


                          #13
                          Just a reminder that Wwise has been available in UE4 since the very beginning, and we have releases for all versions (we are currently working on a UE4.4 integration, which will be out soon).

                          The integration is available here: https://github.com/audiokinetic/WwiseUE4Integration
                          Benoit Santerre - Software developer, Audiokinetic Inc.

                          Comment


                            #14
                            Also to add another example here, FMOD is releasing a plugin in early September for UE4, and will have a pre-release version available soon if you sign up for it.

                            Here is the quote from my forum question about availability last week:

                            Hi Cam, The UE4 integration is being worked on right now! We're only a few working days away from sending out our first pre-release to interested developers.
                            The official release will coincide with the release of FMOD Studio 1.05, which is slated for early September.
                            If you want to start getting pre-release builds then contact support and we can start sending them out to you.
                            Keep an eye on this page in early September for the plugin:
                            http://www.fmod.org/download/

                            Edit: Sorry n00854180t, noticed right after posting you had replied... whoops!
                            Last edited by DotCam; 08-19-2014, 12:09 PM.
                            Free Community Ocean & Sky Project || Join us on Discord! || Trello Roadmap

                            Comment


                              #15
                              Both of those tools are great, no doubt. But I think what we were referring to are less of the developer facing tools and more of how the actual audio processing stack works in the engine. Between those 2, FMod comes closest because it supports Astound Sound. Unfortunately, Astound Sounds is still just a DSP processor, and it's main purpose appears to be to locate an audio source in 3D space using 2 speakers. This doesn't address occlusion/reflection of objects in 3D space (watch the 2 videos in my post above to see what I'm talking about). I don't think FMOD or WWISE consider geometry when handling the audio (correct me if I'm wrong, though).

                              Comment

                              Working...
                              X