Is there any multipart/form-data support when sending HTTP POST requests?

Development Programming & Scripting
, ,
1

I’m trying to set up a plugin that can automatically upload files as part of a JIRA bug report and I have everything working except that I cannot figure out how to upload the attachments. According to the JIRA Rest API documentation you have to send all of the files as part of a single POST using the multipart/form-data format, which requires a lot of custom markup, injecting data blobs between markers, etc. As far as I can tell this is impossible using any of the HTTP interfaces that come with UE (FHttpRequest is what I’m trying to use currently), but I would very much like to be proven wrong. Is there any possible way to get this to work with existing functionality or do I just have to write a custom HTTP request class that can handle it? I REALLY hope that’s not the case since doing that for every platform is going to take a ton of work…

From what I can tell, the IHTTPRequest interface only has support for sending a single file/data blob. I’ve tried to combine all of the files and markup into a single data blob, which I feel like is maybe getting close to a working solution, but that always generates error code 400 “Required part ‘file’ is not present.” I somehow need to set the form-data name to “file”, but I have no idea how to accomplish this. I don’t see any way to specify the name of the content being sent, so if someone knows how to do that, that might be the last part I need.

Anyways, here’s a code example of how I’m trying to build the data blob. I may be doing this completely wrong, but it’s the only thing I could figure out with my limited knowledge of how this HTTP stuff works.

TArray<uint8> rawBytes;
FMemoryWriter memoryWriter(rawBytes);
 
const FString boundaryString = FString::Printf(TEXT("---------%lld"), FMath::RandRange(10000000LL, MAX_int64));
 
const auto WriteFileToArchive = [&memoryWriter](const FString& filePath)
{
	IPlatformFile& platformFile = FPlatformFileManager::Get().GetPlatformFile();
	TUniquePtr<IFileHandle> fileHandle(platformFile.OpenRead(*filePath, true));
	if (!ensureAlways(fileHandle.IsValid()))
		return false;
 
	const auto fileSize = fileHandle->Size();
	if (!ensureAlways(fileSize > 0))
		return false;
 
	TArray<uint8> fileData;
	fileData.AddZeroed(fileSize);
 
	if (!ensureAlways(fileHandle->Read(fileData.GetData(), fileSize)))
		return false;
 
	fileData.BulkSerialize(memoryWriter);
	return true;
};
 
for (const FString& filePath : filePaths)
{
	const FString fileExtension = FPaths::GetExtension(filePath);
	const FString fileNameAndExtension = FPaths::GetCleanFilename(filePath);
 
	const FString dispositionHeader = FString::Printf(TEXT("Content-Disposition: form-data; name=\"file\"; filename=\"%s\""), *fileNameAndExtension);
 
	// Determine content type based on file extension.
	FString contentType;
	if (fileExtension == TEXT("png"))
	{
		contentType = TEXT("image/png");
	}
	else
	{
		contentType = TEXT("application/octet-stream");
	}
	const FString typeHeader = FString::Printf(TEXT("Content-Type: %s"), *contentType);
 
	(TEXT("--") + boundaryString).SerializeAsANSICharArray(memoryWriter);
	dispositionHeader.SerializeAsANSICharArray(memoryWriter);
	typeHeader.SerializeAsANSICharArray(memoryWriter);
 
	if (!ensureAlways(WriteFileToArchive(filePath)))
	{
		return
	}
}
 
(TEXT("--") + boundaryString + TEXT("--")).SerializeAsANSICharArray(memoryWriter);
 
// Very minimal example of how the data is being set. Lots of other stuff is missing, this is just showing
// the important parts related to this data.
const auto request = FHttpModule::Get().CreateRequest();
request->SetHeader(TEXT("X-Atlassian-Token"), TEXT("no-check"));
request->SetHeader(TEXT("Content-Type"), FString::Printf(TEXT("multipart/form-data; boundary=%s"), *boundaryString) );
request->SetContent(MoveTemp(rawBytes));

[Attachment Removed]

2

Aha I solved it! Apparently you can throw it all into the single blob as long as you are very careful about how your format it. There were multiple things I was missing/doing wrong:

  1. Had to wrap the boundary part of the Content-Type header in double-quotes.
  2. Needed to add “\r\n” on every line to make the format match what HTTP POST expects. A double newline is required after the type header as well.
  3. I was handling the serialization of the values incorrectly since SerializeAsANSICharArray() and BulkSerialize() add a lot of UE specific markup that HTTP obviously doesn’t care about. Changing those to properly just drop in the entire blob without additional length values or markers was the fix for that.

So this is the final code example for reference:

TArray<uint8> rawBytes;
FMemoryWriter memoryWriter(rawBytes);
 
const FString boundaryString = FString::Printf(TEXT("---------%lld"), FMath::RandRange(10000000LL, MAX_int64));
static const FString newline = TEXT("\r\n");
 
const auto WriteFileToArchive = [&memoryWriter](const FString& filePath)
{
	IPlatformFile& platformFile = FPlatformFileManager::Get().GetPlatformFile();
	TUniquePtr<IFileHandle> fileHandle(platformFile.OpenRead(*filePath, true));
	if (!ensureAlways(fileHandle.IsValid()))
		return false;
 
	const auto fileSize = fileHandle->Size();
	if (!ensureAlways(fileSize > 0))
		return false;
 
	TArray<uint8> fileData;
	fileData.AddZeroed(fileSize);
 
	if (!ensureAlways(fileHandle->Read(fileData.GetData(), fileSize)))
		return false;
 
	memoryWriter.Serialize(fileData.GetData(), fileData.Num());
	return true;
};
 
// This is just a copy of SerializeAsANSICharArray() without the extra length and null characters added.
const auto SerializeStr = [&memoryWriter](const FString& str)
{
	for (int32 i = 0; i < str.Len(); ++i)
	{
		ANSICHAR ansiChar = CharCast<ANSICHAR>((*str)[i]);
		memoryWriter << ansiChar;
	}
};
 
for (const FString& filePath : filePaths)
{
	const FString fileExtension = FPaths::GetExtension(filePath);
	const FString fileNameAndExtension = FPaths::GetCleanFilename(filePath);
	const FString dispositionHeader = FString::Printf(TEXT("Content-Disposition: form-data; name=\"file\"; filename=\"%s\""), *fileNameAndExtension);
 
	// Determine content type based on file extension.
	FString contentType;
	if (fileExtension == TEXT("png"))
	{
		contentType = TEXT("image/png");
	}
	else
	{
		contentType = TEXT("application/octet-stream");
	}
	const FString typeHeader = FString::Printf(TEXT("Content-Type: %s"), *contentType);
 
	SerializeStr(TEXT("--") + boundaryString + newline);
	SerializeStr(dispositionHeader + newline);
	SerializeStr(typeHeader + newline + newline);
 
	if (!WriteFileToArchive(filePath))
	{
		// Error handling here
		return;
	}
 
	SerializeStr(newline);
}
 
SerializeStr(TEXT("--") + boundaryString + TEXT("--"));
 
// Very minimal example of how the data is being set. Lots of other stuff is missing, this is just showing
// the important parts related to this data.
const auto request = FHttpModule::Get().CreateRequest();
request->SetHeader(TEXT("X-Atlassian-Token"), TEXT("no-check"));
request->SetHeader(TEXT("Content-Type"), FString::Printf(TEXT("multipart/form-data; boundary=\"%s\""), *boundaryString) );
request->SetContent(MoveTemp(rawBytes));

[Attachment Removed]

3

Hi,

Looks like you sorted it out. Thanks for sharing the solution. Looks like a cool feature.

Regards,

Patrick

[Attachment Removed]