Saving stats to file

Development Programming & Scripting
1

Hi!

Quick question - is it possible to save stats to file or FString?

I’m currently working on dedicated server and there are some statistics shown with “stat net” command that I’d like to collect. I need them to check if the server behaves well. I need them to be saved to file or to be displayed on console.

Unfortunatelly I can’t find any code to do such thing, the stat system is kinda complicated here.

I found only something like

GET_STATID(STAT_Ping).GetRawPointer()->AnsiString;

to get raw data, but they are not telling me anything.

Do You have any clues how to collect stat data into the game so I could process them or save to file?

1 Like
2

Hi zompi2,

I would have a look at How to improve game thread CPU performance in Unreal Engine - Unreal Engine which explains the process of how to generate a stat file with all the necessary information.

Hope that helps. Cheers,

3

Hi, thanks for a reply!

Unfortunatelly this is not what I’m looking for. Stat file gathers all of the information and I can’t see a clue how to strip it so it can get only stats I need. It also requires an application to read where I need a text format file.

The more I look on it the less I understand how it works… Are these data stored somewhere anyway?

1 Like
4

I would have a look at StatsCommand.cpp (to add your custom command line) and StatsFile.cpp (to write out the correct stats to a file through something similar as what happens in FCommandStatsFile::Start and filter out the necessary stats in FStatsWriteFile::WriteFrame) in that case. I’ve never done this as well but I think that should work.

5

Ok, there was really a lot of digging around but I managed to write a function for gathering stats.

First of all - we need to override FStatsThreadState just to get access to the one protected field:

class MYGAME_API FStatsThreadStateOverlay : public FStatsThreadState
{
public:
	int64 GetLastFullFrameProcessed()
	{
		return LastFullFrameProcessed;
	}
};

Oh, the original FStatsThreadState and all needed methods are in “StatsData.h”

Then I had to prepare a Filter structure for future use:

struct FGroupFilter : public IItemFiler
{
	TSet<FName> const& EnabledItems;

	FGroupFilter(TSet<FName> const& InEnabledItems)
		: EnabledItems(InEnabledItems)
	{
	}

	virtual bool Keep(FStatMessage const& Item)
	{
		const FName MessageName = Item.NameAndInfo.GetRawName();
		return EnabledItems.Contains(MessageName);
	}
};

And now time for some magic. Here’s complete code to get all net stats (with comments):

// Get the reference to the stats thread
FStatsThreadStateOverlay& Stats = FStatsThreadStateOverlay&)FStatsThreadState::GetLocalState();

// Get the number of the last processed frame and check if it is valid (just for sure)
int64 LastGoodGameFrame = Stats.GetLastFullFrameProcessed();
if (Stats.IsFrameValid(LastGoodGameFrame) == false)
{
	return;
}

// This is the name of the group where stats are.
FName GroupName = FName(TEXT("STATGROUP_net"));

// Gather the names of the stats that are in this group.
TArray<FName> GroupItems;
Stats.Groups.MultiFind(GroupName, GroupItems);

// Prepare the set of names and raw names of the stats we want to get
TSet<FName> EnabledItems;
for (const FName& ShortName : GroupItems)
{
	EnabledItems.Add(ShortName);
	if (FStatMessage const* LongName = Stats.ShortNameToLongName.Find(ShortName))
	{
		EnabledItems.Add(LongName->NameAndInfo.GetRawName());
	}
}

// Create a filter (needed by stats gathering function)
FGroupFilter Filter(EnabledItems);
	
// Create empty stat stack node (needed by stats gathering function)
FRawStatStackNode HierarchyInclusive;

// Prepare the array for stat messages
TArray<FStatMessage> NonStackStats;

// COLLECT ALL STATS TO THE ARRAY HERE
Stats.UncondenseStackStats(LastGoodGameFrame, HierarchyInclusive, &Filter, &NonStackStats);

// Go through all stats
// There are many ways to display them, dig around the code to display it as you want :)
for (auto Stat : NonStackStats)
{
	// Here we are getting the raw name
	FName StatName = Stat.NameAndInfo.GetRawName();
	UE_LOG(MyLog, Log, TEXT("Received Stat: %s"), *StatName.ToString());

	// Here we are getting values
	int64 iVal = 0;
	double dVal = 0;
	switch (Stat.NameAndInfo.GetField<EStatDataType>())
	{
		case EStatDataType::ST_int64:
			iVal = Stat.GetValue_int64();
			UE_LOG(DTLog, Log, TEXT("Value int64: %lld"), iVal);
			break;
		case EStatDataType::ST_double:
			dVal = Stat.GetValue_double();
			UE_LOG(MyLog, Log, TEXT("Value double: %f"), dVal);
			break;
		default:
			check(0);
	}		
}

One important thing! The function will work only when the “stat net” has been enabled from console. Without it stats won’t be gathered.

1 Like
6

I was hoping someone was trying to do the same thing and could show how to do this, but I’ve managed to write stat gathering function based on StatsCommand.cpp file you’ve mention :slight_smile: Thanks for helping.

7

One question, in order to get these stats with this code, do you need to run it in the stats thread? If so, how did you accomplish that?

Thanks,

Franco

8

No, I was running it on GameThread. You only have to remember that You must run stat command to make it work. Without it the code will have nothing to parse.

9

I know you’ve found an answer, but figured I would give this in case someone else stumbles on the same problem

This console command will save the information to a csv file along with some meta data.

1 Like
10

This is quite cool function, but still, it captures only fps data. If you need data about memory, package drop etc. it seems there is no easy way :wink:

11

Yeah I guess ;/

12

So if I understood you correctly, this retrieves a lot of information?
Could you help me set this up? Do you have discord or something?

13

The method I’ve shown simply copy the data that are displayed on the screen when you type ‘stat net’ into the game console. You can also get any kind of stat you want - memory, memoryplatform, engine, there are a lot of STATGROUPs. I needed this to save data about ping and packet lost during the online game to the file for QA reasons.
I’m not sure what more can I tell :slight_smile:

14

Thing is, the “stat net” command doesn’t even work for me, any idea why?
I also need info about ping and packet loss … basically I need about everything I can do evaluate my system

15

The only reason ‘stat net’ would not work is even you run your game in shipping build or there is no net driver running. To see ‘stat net’ you must be during an online game.

16

Thanks man, packaging the game and joining a session made it work. No idea why I didn’t think of that before.
Can you guide me through adding the code to save the data to a csv file?
I’m not very familiar with C++ in UE 4, although, I do know C++.

17

To put some data into a file in UE4 you want to use FOutputDeviceFile. Simply create one:

FOutputDeviceFile* OutputDevice = new FOutputDeviceFile(TEXT("UrlToFile"), true);

Then you can write to it by:

OutputDevice->Serialize(TEXT("LogToSave"), ELogVerbosity::Log, FName(TEXT("StatCategory")));

After You finish - don’t forget to close it:

OutputDevice->TearDown();
delete OutputDevice;
OutputDevice = nullptr;

I don’t know the format of CSV file, so this is something You have to investigate: http://stackoverflow.com/questions/25201131/writing-csv-files-from-c

18

Hello, I am working on a plugin which need to collect stats of particles. Could you please tell me how I can run these code in my plugin.

19

Hello, can you collect stats by plugin now? I‘m working on this too. Do you solve this problem?

20

I thnik You just have to put the code inside the plugin and run it… that’s all. Plugin can also enable stats by launching the command:
GetWorld()->Exec(GetWorld(), TEXT(“stat net”));