My + character gets replaced with a space character when I contact my PHP server

[JansenSensei]

This is the code I use to contact my server:

	FHttpRequestRef Request = FHttpModule::Get().CreateRequest();

	FString URL = FString::Printf(TEXT("%s%s"), *Url, *CContentURL);
	Request->SetURL(URL);
	Request->SetVerb("POST");
	Request->SetHeader(TEXT("User-Agent"), "X-UnrealEngine-Agent");
	Request->SetHeader(TEXT("Content-Type"), "application/json");

	if (META.Contains(" "))
	{
		UE_LOG(LogTemp, Error, TEXT("Meta data contains spaces!!!!!!!!!!!!!!!!!!!!!!!!!!!!"));
		return;
	}
	Request->SetContentAsString(META);
	Request->ProcessRequest();

everythign works perfectly except for the fact that any ‘+’ characters in my META field gets replaced by a space on the server side. META can contain a simple FString that was formatted to contain a + character or it can (more importantly) be a base64 encoded string that contains a + character.

In the case of the latter I cannot decode the string on the server since the value is not valid… On the server side I can simply say “replace all spaces with a +” but that could lead to other problems again. For now it gives okay-ish results.

Is this a PHP problem or a C++ problem?

[3dRaven]

Just to narrow down the problem.
You start by contacting the server
I do not see you passing in any payload to the request so I’m guessing it’s just a post with no parameters and a response, correct?

And you want the return response to contain + symbols that are sent out by the php server.
Your final objective is to send a base64 string from the php server to be later decoded by unreal after processing the response string.

example php code server side

<?php

header('Content-Type: application/json; charset=utf-8');
$data = new stdClass();
$data->success = true;
$string ="Coded information this is a test of a longer string + other symbols";
$data->Base64 = base64_encode($string);
echo json_encode($data);
?>

Response from server:

cpp for server interaction

void AServerClass::ContactServer()
{
	FHttpRequestRef Request = FHttpModule::Get().CreateRequest();
	FString URL = FString::Printf(TEXT("%s%s"), *Url, *CContentURL);	
	Request->SetURL(URL);
	Request->SetVerb("POST");
	Request->SetHeader(TEXT("User-Agent"), "X-UnrealEngine-Agent");
	Request->SetHeader(TEXT("Content-Type"), "application/json");				
	Request->OnProcessRequestComplete().BindUObject(this, &AServerClass::ProcessReq);
	Request->ProcessRequest();	
}

void AServerClass::ProcessReq(FHttpRequestPtr Request, FHttpResponsePtr Response, bool bWasSuccessful)
{		
	META = Response.Get()->GetContentAsString();
	GEngine->AddOnScreenDebugMessage(-1,2,FColor::Cyan,*META);
	if (META.Contains(" "))
	{
		UE_LOG(LogTemp, Error, TEXT("Meta data contains spaces!!!!!!!!!!!!!!!!!!!!!!!!!!!!"));
		return;
	}
}

In game:

[JansenSensei]

Hi @3dRaven

I’m afraid you have it backwards. I am building the string in Unreal and seding it TO the server. My goal is to manipulate the data on the server by calling database functions to gather information, format it in a maningful way and send it back to Unreal

That second to last line of code in my OP is where I add the payload (or am I doing that wrong?). I already use this code to do a lot of things and it works perfectly fine EXCEPT in this instance where the payload contains a + character. See the sample string below.

YXJyYXkoLGFycmF5KCdjb2x1bW4nPT4ncG9zdF9kYXRlJywnaG91cic9PjksJ2NvbXBhcmUnPT4nPj0nLCksYXJyYXkoJ2NvbHVtbic9Pidwb3N0X2RhdGUnLCdob3VyJz0+MTcsJ2NvbXBhcmUnPT4nPD0nLCksYXJyYXkoJ2NvbHVtbic9Pidwb3N0X2RhdGUnLCdjb21wYXJlJz0+J0JFVFdFRU4nLCdkYXlvZndlZWsnPT5hcnJheSgyLDYpLCkpLA==

FString License = ServerGlobals->License;
FString META = FString::Printf(TEXT("wfgl=%s"), *License);
FString Token = FString::Printf(TEXT("%s%s"), *ServerGlobals->SecurityString, *License);
for (const TPair<FString, FString>& pair : meta->Defined)
{
	META = FString::Printf(TEXT("%s&%s=%s"),*META, *pair.Key, *pair.Value);
	Token = FString::Printf(TEXT("%s%s"), *Token, *pair.Value);
}
FString EncodedToken = FMD5::HashAnsiString(*Token);
META = FString::Printf(TEXT("%s&token=%s"),*META, *EncodedToken);

That is how I build up the string that I want to send. meta is a TMap from which I then generate a long string (GET style) and cal it META. The base64 string above is one of the values in this TMap.

When this string reaches the website the + at position 132 in that particular field is replaced by a space. This results in base64_decode converting the first 131 characters just fine and then spewing out nonsense for the rest

Is
Request->SetContentAsString(META)
the right wayt o add POST data to a request or is there another way I should be doing it?

[3dRaven]

Ok got it to send. Turned out I needed a slash at the end of my URL for it to register the post vars… the joy of the web

void AServerClass::ContactServer()
{
	FHttpRequestRef Request = FHttpModule::Get().CreateRequest();
	FString URL = FString::Printf(TEXT("%s%s"), *Url, *CContentURL);	
	Request->SetURL(URL);
	Request->SetVerb(TEXT("POST"));
	Request->SetHeader(TEXT("Content-Type"), TEXT("application/json"));
	Request->SetHeader(TEXT("Accepts"), TEXT("application/json"));
		
	FString META = FString::Printf(TEXT("wfgl=%s"), *License);
	FString Token = FString::Printf(TEXT("%s%s"), *SecurityString, *License);
	
	for (const TPair<FString, FString>& pair : Defined)
	{
		META = FString::Printf(TEXT("%s&%s=%s"), *META, *pair.Key, *pair.Value);
		Token = FString::Printf(TEXT("%s%s"), *Token, *pair.Value);
	}
	FString EncodedToken = FMD5::HashAnsiString(*Token);
	META = FString::Printf(TEXT("%s&token=%s"), *META, *EncodedToken);
	
	FPayLoad payload;
	payload.wfgl = License;
	payload.Defined = Defined;
	payload.SecurityString = SecurityString;
	payload.EncodedToken = EncodedToken;
	payload.Token = Token;
	payload.META = META;

	FString outString;
	FJsonObjectConverter::UStructToJsonObjectString(payload, outString);

	Request->SetContentAsString(outString);
	Request->OnProcessRequestComplete().BindUObject(this, &AServerClass::ProcessReq);
	Request->ProcessRequest();	
}

void AServerClass::ProcessReq(FHttpRequestPtr Request, FHttpResponsePtr Response, bool bWasSuccessful)
{		
	if (bWasSuccessful)
	{
		GEngine->AddOnScreenDebugMessage(-1, 5.0f, FColor::Blue, "Success");
	}

	RESULT = Response.Get()->GetContentAsString();
	GEngine->AddOnScreenDebugMessage(-1,2,FColor::Cyan,*RESULT);
	if (RESULT.Contains(" "))
	{
		UE_LOG(LogTemp, Error, TEXT("Meta data contains spaces!!!!!!!!!!!!!!!!!!!!!!!!!!!!"));
		return;
	}
}

struct added in header

USTRUCT()
struct FPayLoad
{
	GENERATED_BODY()

public:
	UPROPERTY()
	FString wfgl;

	UPROPERTY()
	FString SecurityString;
	
	UPROPERTY()
	TMap<FString, FString> Defined;
	
	UPROPERTY()
	FString Token;

	UPROPERTY()
	FString META;

	UPROPERTY()
	FString EncodedToken;
};

I echoed the post that is returned from the server

php echo

<?php
header('Content-Type: application/json; charset=utf-8');
$data = new stdClass();
$data->success = true;
$_POST = json_decode(file_get_contents("php://input"), true);
$data->data =  $_POST;
echo json_encode($data);
?>

Just decode it on the server => do changes => return encoded info as long as success is returned as true.

You can filter out the data in the struct as you need. You can skip the $_GET type ? & symbols.
Serialization is also safer this way.

[JansenSensei]

looking at what you do and what i do it seems the major difference is that you encode to json before you send it off to the server… I wonder if that will solve my issue. I’ll definitely give this a try.

My main problem was the fact that the + becomes a space when it reaches the website. I thought maybe the problem is that it gets encoded in Unreal to something other than UTF8 and thus the space is not ASCII 32 but some other character that just appears as a space because it doesn’t have a glyph defined. Looking at the code, though, I see the SetContentAsString() method does in fact encode text to UTF8 so I had no idea why the data I send doesn’t match the data I receive. Maybe encoding it to JSON will somehow force the data to arrive correctly…

…or maybe it is because you use file_get_contents() that it works for you. I just use $__REQUEST[‘tax_query’] directly. I tried other things but right now I am hacking my code with forced str_replace() comands so now at least I have something new to try…

Thanks a bunch. Touch wood…

EDIT:
I forgot I have this at the start of my PHP file…

<?php
if (empty($_POST))
{
	$rawPost = file_get_contents('php://input');
	$_POST = array();

	mb_parse_str($rawPost, $_POST);
	foreach($_POST as $k => $v)
		$_REQUEST[$k] = $v;
}

if (empty($_REQUEST))
	die('No params received');

EDIT:
I just edited that code like so:

<?php
if (empty($_POST))
{
	$rawPost = file_get_contents('php://input');
	$_POST = array();

	mb_parse_str($rawPost, $_POST);
	foreach($_POST as $k => $v) {
        $_REQUEST[$k] = $v;

        if($k == 'tax_query')
        {
            if($v != $_REQUEST[$k])
                die('CONVERTED VALUE MISMATCH '. $v);
            if (strpos($v,' ') > 0)
                die( 'tax_query has spaces ' . $v);
        }
    }
}

turns out the data already has the spaces when read via file_get_contents. Hmmm… wonder why $__POST is empty when we DID use the verb POST, though. Anyway, let me keep checking what I can do…

[3dRaven]

If I modify the code sent as base64 introducing a + sign like this:

my php returns this:

As you can see the + sign is as it should be.

I’ve also tested it with spaces and they too pass to the server and return intact.

[JansenSensei]

I did some more testing and I find that Unreal does send the data correctly, yes, and it is received correctly also… the problem occurs in the mb_parse_str($rawPost, $_POST); function. It splits the string into variables but it removes the + signs.

I just had to write my own parser function to replace that line and now it works as expected. This was in deed a PHP problem.

Sorry for taking up your time with this. As always, thank you for taking the time to assist!

EDIT: My fix

	$all_fields = explode('&',$rawPost);
	foreach ($all_fields as $single_field)
    {
        $split_values = explode('=',$single_field);
        if(count($split_values) == 1)
        {
            $_POST[$split_values[0]] = '';
            continue;
        }
        if(is_numeric($split_values[1]))
            $_POST[$split_values[0]] = floatval($split_values[1]);
        else
            $_POST[$split_values[0]] = strval($split_values[1]);
    }
	//mb_parse_str($rawPost, $_POST);

[3dRaven]

I would use json_decode like this

<?php
header('Content-Type: application/json; charset=utf-8');
$data = new stdClass();
$data->success = true;
$_POST = json_decode(file_get_contents("php://input"), true);
$data->data =  $_POST;

$base = $_POST['defined']['Base64Test'];
$data->decodedB64 = base64_decode($base);


echo json_encode($data);
?>

and in the decoded base 64 I get
"decodedB64":"array(,array('column'=>'post_date','hour'=>9,'compare'=>'>=',),array('column'=>'post_date','hour'=>17,'compare'=>'<=',),array('column'=>'post_date','compare'=>'BETWEEN','dayofweek'=>array(2,6),)),"}
So it looks like some data for building an sql query. Logical and intact.

If the data is dynamic then after the decode you can iterate over the properties like this

foreach ($obj as $key => $value) {
    echo "$key => $value\n";
}

and do needed transforms based on the key if needed