FInteractiveProcess not correctly redirecting STDOUT

I am using FInteractiveProcess to use Git LFS to smudge the pointer files into the actual files.

I have got the input redirection working correctly, but when I bind to the output, I appear to be receiving STDERR rather than STDOUT. Specifically, I get from git lfs:

Downloading <unknown file> (30.76 KB)

Which looking at the source code for Git LFS is being printed to STDERR:

fmt.Fprintf(os.Stderr, "Downloading %s (%s)\n", workingfile, pb.FormatBytes(ptr.Size))

The file is being printed to STDOUT as running the command through a terminal yields the correct response.

Full code for launching the command:

FInteractiveProcess SmudgeProcess = FInteractiveProcess(*InPathToGitBinary, *SmudgeCommand, bLaunchHidden);
// SmudgeCommand = lfs smudge

SmudgeProcess.SendWhenReady(ReadInfo);
// ReadInfo = a valid LFS pointer
SmudgeProcess.OnOutput().BindLambda([&](const FString& Output) {
   UE_LOG(LogTemp, Log, TEXT("%s"), *Output)
});

SmudgeProcess.Launch();

I don’t understand FInteractiveProcess enough to say this is definitely a bug, but I’m pretty confident it is.

I think the problem is (but I don’t know why this is resulting in STDERR only) this should be reading ReadPipeChild rather than ReadPipeParent but I haven’t had time to test this as I haven’t built the engine from source. This is only based on how the FPlatformProcess::ReadPipe is used by the git plugin to read the file from git show.

Update based on a simpler test case - looks like there are some problems with FInteractiveProcess

I set up a simple program that prints STDOUT to STDOUT and STDERR to STDERR. When I then run this program through a FInteractiveProcess I get either both (understandable, but not entirely helpful for what git LFS requires), but more frequently I get no output at all (which I guess is how I lost the STDOUT when running Git lfs. Full code:

const FString ProgramPath = TEXT("E:\\dev\\C#\\BasicInputOutputTestingProgram\\BasicInputOutputTestingProgram\\bin\\x64\\Debug\\a.exe");
FInteractiveProcess SmudgeProcess = FInteractiveProcess(*ProgramPath, TEXT(""), true);

SmudgeProcess.OnOutput().BindLambda([&](const FString& Output) {
	UE_LOG(LogTemp, Log, TEXT("%s"), *Output)
});

SmudgeProcess.Launch();

FPlatformProcess::Sleep(1.0);

if (SmudgeProcess.IsRunning())
{
	UE_LOG(LogTemp, Log, TEXT("Done?"));
}

Running this gives no output unless I put a breakpoint inside FInteractiveProcess::Run(). a.exe is a simple program I wrote to produce the stdout/stderr required:

#include <stdio.h>

int main()
{
	fprintf(stdout, "STDOUT\n");
	fprintf(stderr, "STDERR\n");
}

The other odd thing I spotted is that even when the process is definitely done (all output sent) SmudgeProcess.IsRunning() still seems to be returning true.

Hey thnk123,

Can you let me know what the following variables are referencing so I can try to reproduce this issue?

  • InPathToGitBinary
  • SmudgeCommand
  • bLaunchHidden
  • ReadInfo

Thanks.

InPathToGitBinary should be the path to Git on your computer
SmudgeCommand should be “lfs smudge”
bLaunchHidden - I don’t know but I suppose it doesn’t matter
ReadInfo - so this is slightly awkward - you’ll need to get a valid git lfs pointer or otherwise git lfs smudge won’t work. But once you’ve got a file checked into Git LFS, git show HEAD:yourfile.uasset will give you a valid value for ReadInfo.

For context this code is in Plugins/Developer/GitSourceControl/Source/GitSourceControl/Private/GitSourceControlUtils.cpp inside function RunDumpToFile. I’ll update the question with more precise repro steps this afternoon if this isn’t enough.

Hi @Kyle - have you made any progress with this? Thanks.

Hey thk123,

This will take more time. Thanks for your patience.

Did you see my steps at the bottom to reproduce the problem with FInteractiveProcess and a simple program outputting text to STDOUT and STDERR? This doesn’t require Git LFS. The LFS stuff was just how I entered onto this in the first place. I have a separate bug here that details exactly how to set up Git LFS.

Hey thk123,

I have the basic FInteractiveProcess setup but I do not know what you are doing with Git LFS, in regards to the project setup.

At the moment I am getting:

Unknown Command "lfs smudge" for "git-lfs".

I have never used LFS and don’t know much more about it than what is on their website. If you could give me some steps on how you are using it, including what you have committed to GitHub with it, it would help my testing a lot.

Thanks.

I figured your bug was with Git LFS, which is what I was focused on and your steps at the bottom were to reiterate what Git LFS was doing. I am not here to give you a hard time. I am simply trying to get as much clear information as possible. Forgive me if my questions seem redundant or already answered. I can promise you I ask them for a reason.

Thanks again for your patience.

As for what I am seeing. This is what I have setup:

[.h]

#pragma once

#include "GameFramework/Actor.h"
#include "LFSActor.generated.h"

UCLASS()
class AH470412_API ALFSActor : public AActor
{
	GENERATED_BODY()
	
public:	
	ALFSActor( );
    class FInteractiveProcess *InteractiveProcess;
    
    UFUNCTION( BlueprintCallable, Category = "InteractiveProcess" )
    void NewInteractiveProcess( const FString InURL, const FString InParams );
	
    UFUNCTION( )
    void ProcessOutput( const FString &Output );
    
    UFUNCTION( )
    void ProcessCompleted( int32 ReturnCode, bool bCanceling );
};

[.cpp]

#include "AH470412.h"
#include "LFSActor.h"
#include "Runtime/Core/Public/Misc/InteractiveProcess.h"

ALFSActor::ALFSActor()
{ 
	PrimaryActorTick.bCanEverTick = false;   
}

void ALFSActor::NewInteractiveProcess( const FString InURL, const FString InParams )
{
    InteractiveProcess = new FInteractiveProcess( InURL, InParams, false, false );
    
    InteractiveProcess->OnOutput( ).BindUObject( this, &ALFSActor::ProcessOutput );
    InteractiveProcess->OnCompleted( ).BindUObject( this, &ALFSActor::ProcessCompleted );
    InteractiveProcess->Launch( );
 }

void ALFSActor::ProcessOutput( const FString &Output )
{
    UE_LOG( LogTemp, Warning, TEXT("Interactive Process Output: %s"), *Output );
}

void ALFSActor::ProcessCompleted( int32 ReturnCode, bool bCanceling )
{
    UE_LOG( LogTemp, Warning, TEXT("Interactive Process Completed: %d %s"), ReturnCode, bCanceling ? TEXT("True") : TEXT("False") );
    delete InteractiveProcess;
}

What I am doing in Blueprint to call create the FInteractiveProcess with parameters:

And here is my log of using the Interactive Process:

LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 52
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 53
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 54
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 55
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 56
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 57
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 58
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 59
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 60
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 61
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 62
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 63
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 64
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 65
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 66
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 67
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 68
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 69
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 70
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 71
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 72
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 73
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 74
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 75
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 76
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 77
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False
LogInteractiveProcess: Process creation succesfull FInteractiveProcess 78
LogTemp:Warning: Interactive Process Output: STDOUT
LogInteractiveProcess: Child Process  -> STDOUT
LogTemp:Warning: Interactive Process Output: STDERR
LogInteractiveProcess: Child Process  -> STDERR
LogTemp:Warning: Interactive Process Completed: 0 False

Is this in line with what you are seeing?

That wasn’t what I was getting - but I wasn’t newing the interactive process which I thought I had handled correctly but I guess it must have been causing a race condition - I now get both STDOUT and STDERR in my output with that fixed.

My original problem however remains, that I get both STDOUT and STDERR redirected to the same handler. Which is what I need to differentiate between for git lfs. Do I need to open a new question for that?

Thanks for your help so far - let me know if you need any more information about Git LFS.

Hey again,

Sorry for the delay. I had to dive into MSDN to figure out some things for the test program to have a separation of the HANDLE. With that said, I am still not 100% sure this is what you are looking for. But, this is my setup:

[Test Program]

#include <windows.h>

#define BUFFERSIZE 4096 

int main( int argc, char* args[] )
{
	HANDLE hStdin, hStdout;
	hStdout = GetStdHandle(STD_OUTPUT_HANDLE);
	hStdin = GetStdHandle(STD_INPUT_HANDLE);

	if( hStdout && hStdin )
	{
		char ReadBuffer[ BUFFERSIZE ] = {0};
		DWORD BytesRead;
		DWORD BytesWritten;
		if( ReadFile(hStdin, ReadBuffer, BUFFERSIZE - 1, &BytesRead, NULL ) != FALSE )
		{
			// Echo back to Unreal which was passed in through Interactive Process SendWhenReady( FString( ) )
			WriteFile(hStdout, ReadBuffer, BUFFERSIZE, &BytesWritten, NULL);
		}
	}
	return 0;
}

Updated LFSActor:

[.h]

#pragma once

#include "GameFramework/Actor.h"
#include "LFSActor.generated.h"

UCLASS()
class AH470412_API ALFSActor : public AActor
{
	GENERATED_BODY()
	
public:	
	ALFSActor( );
    class FInteractiveProcess *InteractiveProcess;
    
    UFUNCTION( BlueprintCallable, Category = "InteractiveProcess" )
    void NewInteractiveProcess( const FString InURL, const FString InParams );
	
    UFUNCTION( )
    void ProcessOutput( const FString &Output );
    
    UFUNCTION( )
    void ProcessCompleted( int32 ReturnCode, bool bCanceling );
};

[.cpp]

#include "AH470412.h"
#include "LFSActor.h"
#include "Runtime/Core/Public/Misc/InteractiveProcess.h"

ALFSActor::ALFSActor()
{ 
	PrimaryActorTick.bCanEverTick = false;   

	InteractiveProcess = nullptr;
}

void ALFSActor::NewInteractiveProcess( const FString InURL, const FString InParams )
{
	// Probably should have some verifcation that InteractiveProcess isn't already doing something before new
	InteractiveProcess = new FInteractiveProcess(InURL, FString(""), false, false);
	InteractiveProcess->OnOutput().BindUObject(this, &ALFSActor::ProcessOutput);
	InteractiveProcess->OnCompleted().BindUObject(this, &ALFSActor::ProcessCompleted);
	InteractiveProcess->Launch();

	InteractiveProcess->SendWhenReady(InParams);
 }

void ALFSActor::ProcessOutput( const FString &Output )
{
    UE_LOG( LogTemp, Warning, TEXT("Interactive Process Output: %s"), *Output );
}

void ALFSActor::ProcessCompleted( int32 ReturnCode, bool bCanceling )
{
    UE_LOG( LogTemp, Warning, TEXT("Interactive Process Completed: %d %s"), ReturnCode, bCanceling ? TEXT("True") : TEXT("False") );

	delete InteractiveProcess;
}

Blueprint calling the Interactive Process Actor:

And finally, Log:

LogInteractiveProcess: Process creation succesfull FInteractiveProcess 1
LogInteractiveProcess: Parent Process -> Original Message: Hello from Unreal , Written Message: Hello from Unreal
LogTemp:Warning: Interactive Process Output: Hello from Unreal
LogInteractiveProcess: Child Process  -> Hello from Unreal
LogTemp:Warning: Interactive Process Completed: 0 False

And, as an extra note, in the test program, if you do not call WriteFile( ), there will be no call back toUE4, it will however show the original message as UE4 is sending it. This is the log when WriteFile( ) is commented out:

LogInteractiveProcess: Process creation succesfull FInteractiveProcess 1
LogInteractiveProcess: Parent Process -> Original Message: Hello from Unreal , Written Message: Hello from Unreal
LogTemp:Warning: Interactive Process Completed: 0 False

Let me know if this is in the ballpark of what you were talking about in regards to separating In/Out handles.

Thanks.

We have not heard back from you in a few days, so we are marking this post as Resolved for tracking purposes. If you are still experiencing the issue you reported, please respond to this message with additional information and we will follow up.