Get progress and filesize of http request

Development Programming & Scripting
1

I am currently using IHttpRequest/Response to talk to a web server. When I am downloading a file (such as an image), I need to get the size of the file, and the progress. I thought the delegate OnRequestProgress would do just that, but it only gets called once the file is done downloading (and only passes the request that doesn’t have the file size), so how do I get this information before then?

2

Do you see process of download in browser?

3

You mean in my web browser for the image when accessing the image in my web browser? For a 10MB image, yes. Not when downloading in UE though.

4

Unity sends progress as well, so I know it is done one way or the other.

5

I kind of thouth you HTTP might not send file size, but i guess not

6

You should be able to obtain the “Content-Length” from the http response to determine the size of the payload. Your http request might be too small for the OnRequestProgress to trigger multiple times.

7

Can I get a response from anything other than OnRequestProgress? If not, then my question is still not answered. As I said in a comment below my question, I am downloading a 10MB image, and still getting no response until it finishes (5-10 seconds). The only time I have ever seen OnRequestProgress get called during the download was for a full webpage where multiple parts are downloading.

8

Which platform implementation of the Http code are you using? Ie. is it WinInet, Libcurl, iOS, Mac, etc?

9

I am using windows 7 and the editor, so WinInet?

10

I’m also trying to find the total file size while it’s being downloaded. I can get the amount of bytes downloaded, but to make a progress bar, I need the total size. Did you find a solution for this?

11

Hey I was just looking into this and see what you need to do. You need to bind a function to HttpRequestProgress. That function is passed a HttpRequest pointer, but you can call GetResponse to get the HttpResponse. From the response you can get the size.

Something like this:

DECLARE_LOG_CATEGORY_CLASS(LogHttpTest, Log, All);
void StartRequest()
{
	FHttpModule& HttpModule = FModuleManager::LoadModuleChecked<FHttpModule>("HTTP");
	TSharedRef<class IHttpRequest> HttpRequest = HttpModule.Get().CreateRequest();

	FString Url = "https://d26ilriwvtzlb.cloudfront.net/a/a7/ThirdPerson_FBX.zip";

	HttpRequest->OnProcessRequestComplete().BindUObject(this, &HttpRequestComplete);
	HttpRequest->OnRequestProgress().BindUObject(this, &HttpRequestProgress);
	HttpRequest->SetURL(Url);

	HttpRequest->ProcessRequest();
}


void HttpRequestProgress(FHttpRequestPtr HttpRequest, int32 BytesSend, int32 InBytesReceived)
{
	int32 ReceivedSize = InBytesReceived;
	//int32 TotalSize = HttpRequest->GetContentLength(); // Incorrect, this is the size of the payload SENT in the REQUEST

	FHttpResponsePtr HttpResponse = HttpRequest->GetResponse();

	if (HttpResponse.IsValid())
	{
		int32 TotalSize = HttpResponse->GetContentLength(); // Correct, this is the size of the payload RECEIVED in the RESPONSE
		float PercentDone = ((float)ReceivedSize / (float)TotalSize) * 100;
		UE_LOG(LogHttpTest, Warning, TEXT("Received Size = %d, TotalSize = %d, PercentDone = %.2f%"), ReceivedSize, TotalSize, PercentDone);
	}
}

void HttpRequestComplete(FHttpRequestPtr HttpRequest, FHttpResponsePtr HttpResponse, bool bSucceeded)
{
//...
}

That results in this kind of output:

LogHttpTest:Warning: Received Size = 16384, TotalSize = 1291706, PercentDone = 1.27
LogHttpTest:Warning: Received Size = 32768, TotalSize = 1291706, PercentDone = 2.54
LogHttpTest:Warning: Received Size = 49152, TotalSize = 1291706, PercentDone = 3.81

Thanks

12

Any solutions to getting the total file size?

With the code below TotalSize is the TotalSize on PC and the bytes read on iOS.

request->OnRequestProgress().BindLambda([onProgress](FHttpRequestPtr HttpRequest, int32 BytesSend, int32 InBytesReceived){
        if (onProgress)
        {
            FHttpResponsePtr HttpResponse = HttpRequest->GetResponse();
            if (HttpResponse.IsValid())
            {
                int32 TotalSize = HttpResponse->GetContentLength(); // Correct, this is the size of the payload RECEIVED in the RESPONSE
				Outputf(_T("%d/%d/%d"), BytesSend,InBytesReceived, TotalSize);
				onProgress((float)InBytesReceived / (float)TotalSize);
            }
        }
    });
13

Adapting the iOS HTTP response content length to use the payload size the iOS request is working with seems to work.

I have this and a cache change that fixes issues when working with AWS services that I should look into getting back into the main branch sometime.

return [ResponseWrapper getPayload].Max();
14

Hey ,

The previous question was about the implementation on Windows, which worked with the steps I provided. The iOS case had a bug related to this that was fixed for 4.15.

The details are in UE-38392, but from the details of the checked in fix, it sounds like a similar approach was taken:

“Changed FAppleHttpResponse::GetContentLength to return expected content size instead of payload size so it’s consistent with other implementations”

Upgrading to 4.15 or merging that change should fix the iOS case.

Hope that helps.

15

That would do it.

While we’re talking about it I’m sitting on another change - in order to get iOS talking to AWS services I had to set a cache policy.

[Request setCachePolicy : NSURLRequestReloadIgnoringCacheData];

16

Hello Joe,
Thanks for this code, i want to know is it possible to get file size before starting download progress ?
Regards

17

Haven’t tried it but you could look into the http “HEAD” request.