Voice capture always gives no data

[Doctor-G]

My team would like to add voice input to our UE4 mobile game using 4.18.3. I have followed the steps outlined in https://answers.unrealengine.com/questions/347976/basic-microphone-input-with-ue4.html. I have used much of his code, adding some logic to kill the session when the game is closed. On both PIE and Android, I consistently get voice capture state NoData with zero bytes available. I am at the limit of my understanding of the engine, and I was hoping someone might be able to spot a place to fix this. I will outline my solution and code below.

DefaultEngine.ini has these entries added:

[OnlineSubsystem]
DefaultPlatformService=Null
bHasVoiceEnabled=true

[Voice]
bEnabled=true

I have a custom gameinstance that initializes a session with OnlineSubsystemNull, using a public InitSession method that is called by the start-up level’s blueprint.

UFairiesGameInstance::UFairiesGameInstance(const FObjectInitializer & ObjectInitializer) : Super(ObjectInitializer)
{
    OnCreateSessionCompleteDelegate = FOnCreateSessionCompleteDelegate::CreateUObject(this, &UFairiesGameInstance::OnCreateSessionComplete);
    OnStartSessionCompleteDelegate = FOnStartSessionCompleteDelegate::CreateUObject(this, &UFairiesGameInstance::OnStartOnlineGameComplete);
    OnDestroySessionCompleteDelegate = FOnDestroySessionCompleteDelegate::CreateUObject(this, &UFairiesGameInstance::OnDestroySessionComplete);
}

bool UFairiesGameInstance::HostSession(TSharedPtr<const FUniqueNetId> UserId, FName SessionName, bool bIsLAN, bool bIsPresence, int32 MaxNumPlayers)
{
    UE_LOG(LogTemp, Warning, TEXT("In HostSession"));
    IOnlineSubsystem* const OnlineSub = IOnlineSubsystem::Get();
    if (OnlineSub)
    {
        IOnlineSessionPtr Sessions = OnlineSub->GetSessionInterface();
        if (Sessions.IsValid() && UserId.IsValid())
        {
            SessionSettings = MakeShareable(new FOnlineSessionSettings());
            SessionSettings->bIsLANMatch = bIsLAN;
            SessionSettings->bUsesPresence = bIsPresence;
            SessionSettings->NumPublicConnections = MaxNumPlayers;
            SessionSettings->NumPrivateConnections = 0;
            SessionSettings->bAllowInvites = true;
            SessionSettings->bAllowJoinInProgress = true;
            SessionSettings->bShouldAdvertise = true;
            SessionSettings->bAllowJoinViaPresence = true;
            SessionSettings->bAllowJoinViaPresenceFriendsOnly = false;

            SessionSettings->Set(SETTING_MAPNAME, FString("MAP_Main_Menu"), EOnlineDataAdvertisementType::ViaOnlineService);
            OnCreateSessionCompleteDelegateHandle = Sessions->AddOnCreateSessionCompleteDelegate_Handle(OnCreateSessionCompleteDelegate);

            return Sessions->CreateSession(*UserId, SessionName, *SessionSettings);
        }
    }
    else
    {
        GEngine->AddOnScreenDebugMessage(-1, 10.f, FColor::Red, TEXT("No OnlineSubsytem found!"));
    }

    return false;
}

void UFairiesGameInstance::OnStartOnlineGameComplete(FName SessionName, bool bWasSuccessful)
{
    GEngine->AddOnScreenDebugMessage(-1, 10.f, FColor::Red, FString::Printf(TEXT("OnStartSessionComplete %s, %d"), *SessionName.ToString(), bWasSuccessful));
    IOnlineSubsystem* OnlineSub = IOnlineSubsystem::Get();
    if (OnlineSub)
    {
        IOnlineSessionPtr Sessions = OnlineSub->GetSessionInterface();
        if (Sessions.IsValid())
        {
            Sessions->ClearOnStartSessionCompleteDelegate_Handle(OnStartSessionCompleteDelegateHandle);
        }
    }

    if (bWasSuccessful)
    {
        UGameplayStatics::OpenLevel(GetWorld(), "/Game/AudioTestLevel", true, "listen");
    }
}

void UFairiesGameInstance::OnCreateSessionComplete(FName SessionName, bool bWasSuccessful)
{
    GEngine->AddOnScreenDebugMessage(-1, 10.f, FColor::Red, FString::Printf(TEXT("OnCreateSessionComplete %s, %d"), *SessionName.ToString(), bWasSuccessful));
    IOnlineSubsystem* OnlineSub = IOnlineSubsystem::Get();

    if (OnlineSub)
    {
        IOnlineSessionPtr Sessions = OnlineSub->GetSessionInterface();
        if (Sessions.IsValid())
        {
            Sessions->ClearOnCreateSessionCompleteDelegate_Handle(OnCreateSessionCompleteDelegateHandle);
            if (bWasSuccessful)
            {
                OnStartSessionCompleteDelegateHandle = Sessions->AddOnStartSessionCompleteDelegate_Handle(OnStartSessionCompleteDelegate);
                Sessions->StartSession(SessionName);
            }
        }

    }
}

void UFairiesGameInstance::OnDestroySessionComplete(FName SessionName, bool bWasSuccessful)
{
    UE_LOG(LogTemp, Warning, TEXT("In OnDestroySessionComplete"));

    IOnlineSubsystem* OnlineSub = IOnlineSubsystem::Get();
    if (OnlineSub)
    {
        IOnlineSessionPtr Sessions = OnlineSub->GetSessionInterface();

        if (Sessions.IsValid())
        {
            Sessions->ClearOnDestroySessionCompleteDelegate_Handle(OnDestroySessionCompleteDelegateHandle);

            if (bWasSuccessful)
            {
                UGameplayStatics::OpenLevel(GetWorld(), "/Game/MainLevel", true);
            }
        }
    }
}

void UFairiesGameInstance::InitSession()
{
    UE_LOG(LogTemp, Warning, TEXT("In InitSession"));
    ULocalPlayer* const Player = GetFirstGamePlayer();

    if (Player != nullptr) {
        HostSession(Player->GetPreferredUniqueNetId(), FName("Game"), true, true, 1);
    }
    else {
        UE_LOG(LogTemp, Warning, TEXT("Init: Player is null"));
    }
}

void UFairiesGameInstance::Shutdown()
{
    UE_LOG(LogTemp, Warning, TEXT("I am shutting down now"));

    IOnlineSubsystem* OnlineSub = IOnlineSubsystem::Get();
    if (OnlineSub)
    {
        IOnlineSessionPtr Sessions = OnlineSub->GetSessionInterface();
        if (Sessions.IsValid())
        {
            Sessions->DestroySession(FName("Game"), OnDestroySessionCompleteDelegate);
        }
    }
    Super::Shutdown();
}

The log messages up to here indicate to me that the session is correctly created, and the game transitions to AudioTestLevel. However, two relevant warnings do come up, and I do not know if they are meaningful to my audio problem or not.

LogOnline: Warning: OSS: AutoLogin missing AUTH_LOGIN=<login id>.
LogOnline: Warning: OSS: No game present to join for session (GameSession)

The logic for capturing audio is handled in my player controller:

void AVoicedPlayerController::BeginPlay()
{
    Super::BeginPlay();
    UE_LOG(LogTemp, Warning, TEXT("Custom player controller initialized"));

    VoiceCapture = FVoiceModule::Get().CreateVoiceCapture();
    bool Success = VoiceCapture->Start();
    if (Success)
    {
        UE_LOG(LogTemp, Warning, TEXT("Voice capture started successfully"));
    }
    else
    {
        UE_LOG(LogTemp, Warning, TEXT("Voice capture not started successfully"));
    }
}

void AVoicedPlayerController::Tick(float DeltaSeconds)
{
    VoiceCaptureTick();
}

void AVoicedPlayerController::VoiceCaptureTick()
{
    if (!VoiceCapture.IsValid())
    {
        UE_LOG(LogTemp, Warning, TEXT("Voice capture is not valid; skipping the rest of voice capture tick"));
        return;
    }

    uint32 VoiceCaptureBytesAvailable;
    EVoiceCaptureState::Type CaptureState = VoiceCapture->GetCaptureState(VoiceCaptureBytesAvailable);

    UE_LOG(LogTemp, Warning, TEXT("Bytes available: %d
Capture state: %s"), VoiceCaptureBytesAvailable, EVoiceCaptureState::ToString(CaptureState));

    VoiceCaptureBuffer.Reset();
    if (CaptureState == EVoiceCaptureState::Ok && VoiceCaptureBytesAvailable > 0)
    {
        int16_t VoiceCaptureSample;
        uint32 VoiceCaptureReadBytes;
        float VoiceCaptureTotalSquared = 0;

        VoiceCaptureBuffer.SetNumUninitialized(VoiceCaptureBytesAvailable);

        VoiceCapture->GetVoiceData(VoiceCaptureBuffer.GetData(), VoiceCaptureBytesAvailable, VoiceCaptureReadBytes);

        for (uint32 i = 0; i < (VoiceCaptureReadBytes / 2); i++)
        {
            VoiceCaptureSample = (VoiceCaptureBuffer[i * 2 + 1] << 8) | VoiceCaptureBuffer[i * 2];
            VoiceCaptureTotalSquared += ((float)VoiceCaptureSample * (float)VoiceCaptureSample);
        }

        float VoiceCaptureMeanSquare = (2 * (VoiceCaptureTotalSquared / VoiceCaptureBuffer.Num()));
        float VoiceCaptureRms = FMath::Sqrt(VoiceCaptureMeanSquare);
        float VoiceCaptureFinalVolume = ((VoiceCaptureRms / 32768.0) * 200.f);

        VoiceCaptureVolume = VoiceCaptureFinalVolume;
    }
}

The logs indicate that there are always zero bytes available and that the state of audio capture is NoData, and hence the main audio processing block is never entered.

There are some errors that come up from the voice system, but I cannot discern what to do about them. Prior to the session’s initialization by the game instance, and again before my player controller is initialized in AudioTestLevel, I get this error:

LogVoiceEngine: Error: StopLocalVoiceProcessing: Ignoring stop request for non-owning user

Also, in the middle of session initialization (between the “In InitSession” and “Session created successfully” logs), I also receive this message:

LogVoiceEngine: Error: StartLocalVoiceProcessing(): Device is currently owned by another user

As I mentioned, I feel that I am at the edge of my understanding, so I am not really sure what to try next. I hoped someone might see something here that could nudge my team in the right direction.

For the sake of completeness, here is the log output.

LogPlayLevel: Creating play world package: /Game/UEDPIE_0_AUdioMainLevel
LogPlayLevel: PIE: StaticDuplicateObject took: (0.001694s)
LogAIModule: Creating AISystem for world AudioMainLevel
LogPlayLevel: PIE: World Init took: (0.000684s)
LogPlayLevel: PIE: Created PIE world by copying editor world from /Game/AUdioMainLevel.AudioMainLevel to /Game/UEDPIE_0_AUdioMainLevel.AudioMainLevel (0.002633s)
LogInit: XAudio2 using 'Speakers (High Definition Audio Device)' : 2 channels at 44.1 kHz using 16 bits per sample (channel mask 0x3)
LogInit: FAudioDevice initialized.
LogLoad: Game class is 'BP_AudioMainLevelGameMode_C'
LogWorld: Bringing World /Game/UEDPIE_0_AUdioMainLevel.AudioMainLevel up for play (max tick rate 0) at 2018.02.20-07.28.58
LogWorld: Bringing up level for play took: 0.000452
LogVoiceEncode: Display: EncoderVersion: libopus 1.1-beta
LogVoiceEngine: Error: StopLocalVoiceProcessing: Ignoring stop request for non-owning user
LogContentBrowser: Native class hierarchy updated for 'MovieSceneCapture' in 0.0005 seconds. Added 11 classes and 0 folders.
LogTemp: Warning: In InitSession
LogTemp: Warning: In HostSession
LogNetVersion: Fairies 1.0.0.0, NetCL: 3709383, EngineNetVer: 2, GameNetVer: 0 (Checksum: 1087104342)
LogInit: WinSock: I am Sanctum (10.220.5.72:0)
LogVoiceEncode: Display: EncoderVersion: libopus 1.1-beta
LogVoiceEngine: Error: StartLocalVoiceProcessing(): Device is currently owned by another user
LogTemp: Warning: Session created successfully
PIE: Play in editor start time for /Game/UEDPIE_0_AUdioMainLevel -0.22
LogBlueprintUserMessages: Late PlayInEditor Detection: Level '/Game/AUdioMainLevel.AudioMainLevel:PersistentLevel' has LevelScriptBlueprint '/Game/AUdioMainLevel.AudioMainLevel:PersistentLevel.AUdioMainLevel' with GeneratedClass '/Game/AUdioMainLevel.AUdioMainLevel_C' with ClassGeneratedBy '/Game/AUdioMainLevel.AudioMainLevel:PersistentLevel.AUdioMainLevel'
LogNet: Browse: /Game/AudioTestLevel?listen
LogLoad: LoadMap: /Game/AudioTestLevel?listen
LogAIModule: Creating AISystem for world AudioTestLevel
LogLoad: Game class is 'BP_AudioTestGameMode_C'
LogTemp: Display: ParseSettings for GameNetDriver
LogTemp: Display: ParseSettings for IpNetDriver_0
LogTemp: Display: ParseSettings for GameNetDriver
LogInit: WinSock: Socket queue 131072 / 131072
PacketHandlerLog: Loaded PacketHandler component: Engine.EngineHandlerComponentFactory (StatelessConnectHandlerComponent)
LogNet: GameNetDriver IpNetDriver_0 IpNetDriver listening on port 7777
LogWorld: Bringing World /Game/UEDPIE_0_AudioTestLevel.AudioTestLevel up for play (max tick rate 0) at 2018.02.20-07.28.58
LogOnline: Warning: OSS: AutoLogin missing AUTH_LOGIN=<login id>.
LogWorld: Bringing up level for play took: 0.000694
LogOnline: Warning: OSS: No game present to join for session (GameSession)
LogGameMode: FindPlayerStart: PATHS NOT DEFINED or NO PLAYERSTART with positive rating
LogVoiceEngine: Error: StopLocalVoiceProcessing: Ignoring stop request for non-owning user
LogTemp: Warning: Custom player controller initialized
LogTemp: Warning: Voice capture started successfully
LogLoad: Took 0.291920 seconds to LoadMap(/Game/AudioTestLevel)
LogTemp: Warning: Bytes available: 0
Capture state: No Data
...
LogOnline: Warning: OSS: No game present to leave for session (GameSession)
LogTemp: Warning: I am shutting down now
LogTemp: Warning: In OnDestroySessionComplete
LogOnline: Display: OSS: FOnlineSubsystemNull::Shutdown()
LogOnline: Display: OSS: FOnlineAsyncTaskManager::Stop() ActiveTask:0000000000000000 Tasks[0/0]
LogOnline: Display: OSS: FOnlineAsyncTaskManager::Exit() started
LogOnline: Display: OSS: FOnlineAsyncTaskManager::Exit() finished
LogPlayLevel: Display: Shutting down PIE online subsystems
LogBlueprintUserMessages: Late EndPlayMap Detection: Level '/Game/AUdioMainLevel.AudioMainLevel:PersistentLevel' has LevelScriptBlueprint '/Game/AUdioMainLevel.AudioMainLevel:PersistentLevel.AUdioMainLevel' with GeneratedClass '/Game/AUdioMainLevel.AUdioMainLevel_C' with ClassGeneratedBy '/Game/AUdioMainLevel.AudioMainLevel:PersistentLevel.AUdioMainLevel'
LogExit: GameNetDriver None shut down
LogPlayLevel: Display: Destroying online subsystem :Context_1
LogOnline: Display: OSS: FOnlineSubsystemNull::Shutdown()

[Konflict]

The voice capture will, internally trim the input audio by applying a certain amount of threshold (in order to cut the analog noise) that will prevent the audio data to be processed. Also, don’t forget to check your system audio settings, the recording will be applied to the device choosen to be the communication input. Windows 7+ will allow you to setup two input devices for different purposes.

[Doctor-G]

Aha! I did not know that there was a built-in low-pass filter. When I shouted into the microphone, I found some data. Thanks for the tip, Konflict. A happy side effect is that if someone else wants to try this, they can copy my code above. Cheers!

[Konflict]

Cheers! Also, if you plan to transmit this voice data over the line (voip) you will have to dig into the opus codec part as well. There are good examples in the voice module of the engine what is the right approach to en-/decode audio packets properly. Distributing the encoded packs can be done by many ways, always check your network consuptions to make sure you’re doing it efficiently as possible. That’s a tip.

[Omar_Vector]

@Konflict

Cheers! Also, if you plan to transmit this voice data over the line (voip) you will have to dig into the opus codec part as well. There are good examples in the voice module of the engine what is the right approach to en-/decode audio packets properly. Distributing the encoded packs can be done by many ways, always check your network consuptions to make sure you’re doing it efficiently as possible. That’s a tip.

When Trying to test it out on android, it works only fine for first time, but once I close the app and restart it, no voice input anymore, and when I debug it it gives me AudioRecord() -38 Error

Have you faced the same problem?

[Konflict]

Nope. And i got no setup for testing things on Android either. Anyways, you apparently have a confirmed (but still unresolved) issue tracked here Unreal Engine Issues and Bug Tracker (UE-62389) that should be fixed and you will be good to go.