Crash when Deserializing TArray into Json

Okay, I spoke too soon. Opened it up today and its still crashing. Is Buffer Overflow something that I may have to be worried about? Does the string that I’m receiving cause memory corruption perhaps?

If the string is not null-terminated, the Json reader may read beyond the buffer length for sure.

84440-capture.png

I’ve also tried to ensure that it is Null-Terminated by altering adding a line to the streaming loop:

		while (Socket->HasPendingData(Size))
		{
			ReceivedData.SetNumUninitialized(Size);

			Read = 0;
			Socket->Recv(ReceivedData.GetData(), ReceivedData.Num(), Read);
//I'm hoping that the below will null terminate the string, if it isn't already
			ReceivedData.Add((uint8)0);
			Print(StringFromBinaryArray(ReceivedData));

		}

Its a TCP Socket, forgot to say that in the previous comment

Are you certain that the problem is caused by this particular code? Perhaps comment out everything and then add lines back in one by one. You should definitely log out the received Json string, and you should also step through the Json serializer to see where it fails.

I really don’t understand what I’m doing anymore. I think I may have been trying to access the Json Object before it was filled in the deserialize function. Thats the explanation I have now, given that the code seems to run fine (even though the deserialise function returns -1).

However, while I was stepping through the the code, I got a weird number of strings being printed. It seems that I am getting the Json String coming in, but its in pieces, rather than full size that I need. Given the length of the Json Object that I am attempting to write in, what size should my buffer be? I’m not really familiar with low level things like that, its not really my expertise

TCP will reassemble the packets for you. The buffer size should be at least as large as the maximum expected Json string size (plus one, if the sender null-terminates it).

Is your server perhaps sending the Json string as multiple TCP packets? In that case you would have to reassemble the string yourself prior to parsing it.

I’m not sure how to determine what is a part of the string, and what is just noise. And how do I know when I’ve gotten the last packet? Again, networking really isn’t my strong point, I’m quite lost in this

ReceivedData.Add((uint8)0);

This is not going to work. You are adding a zero to the end of the buffer, not the end of the received string (which may be less than the size of the buffer).

You probably want this:

ReceivedData[Size] = 0;

Or you can null-terminate on the sender’s side if you have control over the server’s code.

Actually, do this instead:

ReceivedData->RemoveAt(Read, ReceivedData->Num() - Read, false);

The Json reader may not check for null-termination.

Even with that line, the code is still spitting out that character set from above as if to say I don’t know whether this is actually a string or not :frowning:

GLog->Logf(TEXT(“Json string: %s”), (TCHAR*)ReceivedData->GetData);

Its still coming up with that undefined sign.

84472-capture.png

What does that refer to exactly? I don’t understand what it means?

Looks like a non-ASCII character.

Do this:

TCHAR* MyString = (TCHAR*)ReceivedData->GetData();

Then set a breakpoint on it and check the value in the Visual debugger.

Yeah, there isn’t anything ASCII about what I’m getting. Is there a chance that there’s some sort of encoding to ensure there isn’t any null characters inside the stream? Might that be changing the values?

Is the server reachable on the internet, or is it some program you’re running locally? You could take a look at the TCP packets with Wireshark.

To fix the errors that I had, I changed my code to the following:

void ANetworkPlayerController::ReceiveData()
{
	TArray<uint8> ReceivedData;
	uint32 Size;
	int32 Read = 0;

		while (Socket->HasPendingData(Size))
		{
			ReceivedData.SetNumUninitialized(FMath::Min(Size, 65507u));

			Read = 0;
			Socket->Recv(ReceivedData.GetData(), ReceivedData.Num(), Read);
			ReceivedData.RemoveAt(Read, ReceivedData.Num() - Read, false);
			TCHAR* MyString = (TCHAR*)ReceivedData.GetData();
			Print(MyString);
			GLog->Logf(TEXT("Json string: %s"), (TCHAR*)(ReceivedData.GetData()));
			Print(StringFromBinaryArray(ReceivedData));

		}

		if (ReceivedData.Num() <= 0)
		{
			VShow("Error, the Server isn't streaming data");
		}
		FMemoryReader MemoryReader(ReceivedData);
		TSharedRef< TJsonReader<> > Reader = TJsonReader<>::Create(&MemoryReader);
		bool result = FJsonSerializer::Deserialize(Reader, JsonObject);
		if (!result)
			Print("Error, Json Deserialization Failed");
}

The main thing that was going wrong I believe was my setup of the JsonReader. Instead of converting to a string, and putting that through the reader, I set up an FMemoryReader object instead. That did stop the code from crashing.

However, my problem is still there, but its not my fault, aside from the fact that I was trying to access the Deserialised values, even when they didn’t exist since the function had failed. That’s to do with the software I was trying to interface with, nothing I can do now.