HTTP Request Streaming is slower with more FPS

Development Programming & Scripting Multiplayer & Networking
, ,
1

Hi,

I have a problem that i find quite strange. Im using the HTTP manager to stream in data from my endpoint using IHTTPRequest.

Im using OnRequestProgress64 for my update and display the results in a widget and also print them to a log.

For some reason this Update is super slow when im tabbed in the game while if i tab out or limit the framereate using t.maxfps the request is way faster.

Its a huge difference, to the point where the stream will just go word by word.

What am i missing here? Is the module not designed to handle something like this and i need to use libcurl myself?

The endpoint is working totally fine and fast for any other client other than unreal (eg libcurl in python)

Things i have tried to resolve this without any success:

  • Changing the thread policy to Gamethread or HTTP Thread
  • Using HTTP thread and collecting data for a few updates before sending my output delegates
  • modifiying the following engine settings:
HttpThreadEnabled
bEnableHttpThread
bUseHttpThread
bUsePlatformHttp
HttpThreadActiveFrameTimeInSeconds
HttpThreadIdleFrameTimeInSeconds
HttpThreadActiveMinimumSleepTimeInSeconds
HttpThreadIdleMinimumSleepTimeInSecond
HttpMaxRequestsPerTick
2

Steps to Reproduce

  • Use HTTP manager with OnRequestProgress64 on a streaming endpoint (application/x-ndjson)
  • Log the data and populate a widget text
  • Limit the framerate (to 30) or tab out. Not sure if its influenced by how heavy on the CPU the game is
  • With a lower framerate or tabbed out the stream will update way faster than with 60fps
3

Are you using one of SetResponseBodyReceiveStream or SetResponseBodyReceiveStreamDelegateV2 for the streaming data?

4

I am using OnRequestProgress like this:

TSharedRef<IHttpRequest, ESPMode::ThreadSafe> Req = FHttpModule::Get().CreateRequest();
Req->SetURL(FinalURL);
Req->SetVerb(TEXT("POST"));
Req->SetHeader(TEXT("Content-Type"), TEXT("application/json"));
Req->SetHeader(TEXT("Accept"), TEXT("application/x-ndjson"));
 
 
Req->OnRequestProgress64()

But I have also tried using both SetResponseBodyReceiveStream and SetResponseBodyReceiveStreamDelegateV2 with similar results. [mention removed]​

it might be related to the HTTP Thread Response stalling when the Gamethread is waiting for other tasks. Since i have noticed differences in speed on different maps.

5

The OnRequestProgress64 delegate, the SetResponseBodyReceiveStream’s FArchive::Serialize call, and SetResponseBodyReceiveStreamDelegateV2 delegate run on the HTTP thread. Ensure that you are not doing anything there that could briefly block or take a long time to process. Other than that, there should be no dependency on the game thread, other than when the request completes, to trigger the OnProcessRequestComplete delegate if the thread policy is set to EHttpRequestDelegateThreadPolicy::CompleteOnGameThread.