How to parse string into array correctly

Hello, Im trying to get data from arduino using UE4Duino plugin:https://github.com/RVillani/UE4Duino Im transmitting data in format like: “gyro|GyroSensor00(01, 02, …)|quaternionX,|quaternionY| quaternionZ|quaternionW”. Example of data is screenshot 1. When I transform a string into an array, and then select element by index, I get sometimes trash data. Now connected only 2 sensors(GyroSensor00, GyroSensor01). How can I parse it correctly, or I should use other protocol to connect arduino with ue4?
image
image
There is the blueprint, how it works.


There are many possible sources of error here.

First, the Arduino serial port is, by default. only set at 9600 baud (or is it 19200? very low anyway.) It’s easy to send more than available on the wire, which will make the Arduino just randomly drop characters when there’s not enough space. Dropped characters leads to lines that don’t parse correctly.

Second, the Windows serial port handling isn’t all that great. There’s a limited amount of buffering, and if you don’t keep reading out the data that’s coming in, the serial port will block. It doesn’t matter if you have handshaking or not out the serial port, because the Arduino has no buffering – it, too, will block, and you’ll lose an arbitrary amount of data.

Third, there may be errors in the “UE Arduino” plugin, whatever that is. (Seems kinda weird to me – why wouldn’t you just call the serial port yourself? Why does it need to be Arduino specific?) I have no idea what the quality of that code is, or how it will behave under stress, when not polled frequently enough, etc.

Fourth, there may be some misunderstanding in how you’re using the library – does the library guarantee that you will only get complete lines? If not, you may get some data that contains a linebreak in the middle, and then the beginning (but not end) of the next line. Does your code deal correctly with a case like this?

It’s impossible for us to tell which of these cases you’re running into – you’re going to have to add logging/printing, separate smaller test case, and other debugging methods, until you figure out what’s going wrong along this path.

That being said, using a very long name for each “gyro” sensor is not generally a good idea in embedded land, where space is dear. How about just “G%d” where the %d is the gyro index?

And you don’t need the “gyro” field at all – if the name of the sensor starts with “G” you know it’s a gyro; when you add a thermometer, call it “T” and so on. I doubt you’ll have more than 26 kinds of sensors :slight_smile:

And how about starting each line with a known character (like “A”) and stopping it with another known character (like “Z”) so you can easily tell whether you have a complete line or not?

Also, what about adding a checksum or CRC to the protocol, where you can calculate the checksum on the receiving side, and if it doesn’t matches what’s in the packet, you know you lost some data?

Thanks for your answer!
Now Im using 115200 baud and flushing port after each received data, it helped a little bit. Then I changed protocol to "G%d|q1|q2|q3|q4". Its really good advice! I also noticed that if I use the COM port of a PC, the transfer speed is lower compared to a laptop. Maybe it depends on the port version.
About “UE Arduino” plugin: functions “Read line” or “Read string” are the same, but uses \n or \0 end characters but when I use “Read line” - it reads the data correctly but slowly, perhaps this is due to line breaks. “Read line” is very convenient to use so I didn’t try to write my own SerialReader.

And how about starting each line with a known character (like “A”) and stopping it with another known character (like “Z”) so you can easily tell whether you have a complete line or not?

This feature may help btw.

Also, what about adding a checksum or CRC to the protocol, where you can calculate the checksum on the receiving side, and if it doesn’t matches what’s in the packet, you know you lost some data?

Yes, but what will it give me?

What a CRC or checksum gives you is robustness.

Let’s say I will know that I have an error in data transmission. What should I do in this case? Maybe bluetooth connection will change it?