Hi all,
I’m hitting an issue when I’m staging a NonUFS *.ini file which has a BOM of [0xff, 0xfe] (UTF16 LE). This is caused by “FilterIniFile” in Utils.cs.
This is causing an issue with a third party library. This library has a few ini files which are encoded in UTF16 LE with a BOM. I’m staging these config files with +DirectoriesToAlwaysStageAsNonUFS. Because the file type is an *.ini file, it goes via FilterIniFile() function to remove any sections in the iniKeyDenyList and SectionDenyList. However, it reads and writes the file as a UTF8 encoded file because of the following lines:
`// Assumes UTF8 encoding, ignoring the BOM
string Lines = File.ReadAllLines(SourceName, encoding);
// Writes out UTF8 encoding, breaking the encoding
File.WriteAllText(TargetName, NewLines.ToString());`This breaks my third party library because when the library tries to parse the config file it fails to parse correctly as it’s expecting it to be encoded as UTF16 but it’s been staged as a UTF8 encoded file.
I’ve fixed this locally by simply trying to read the BOM in the ini file and preserve the encoding:
`private static void FilterIniFile(string SourceName, string TargetName, List IniKeyDenyList, List InSectionDenyList, List InSectionAllowList)
{
// Try to see if the file has a byte order mark
Encoding encoding = Encoding.UTF8; // Default to utf-8 (default Unreal behaviour)
try
{
FileStream fs = new FileStream(SourceName, FileMode.Open, FileAccess.Read);
byte bits = new byte[3];
fs.Read(bits, 0, 3);
if (bits[0] == 0x0 && bits[1] == 0x0 && bits[2] == 0xfe && bits[3] == 0xff)
{
encoding = new UTF32Encoding(true, true);
}
else if (bits[0] == 0xff && bits[1] == 0xfe && bits[2] == 0x0 && bits[3] == 0x0)
{
encoding = new UTF32Encoding(false, true);
}
else if (bits[0] == 0xef && bits[1] == 0xbb && bits[2] == 0xbf)
{
encoding = Encoding.UTF8;
}
else if (bits[0] == 0xff && bits[1] == 0xfe)
{
encoding = Encoding.Unicode;
}
else if (bits[0] == 0xfe && bits[1] == 0xff)
{
encoding = Encoding.BigEndianUnicode;
}
}
catch (IOException) { } // Ignore any read errors, default to UTF8 encoding
string Lines = File.ReadAllLines(SourceName, encoding);
// … Snip, rest of code
File.WriteAllText(TargetName, NewLines.ToString(), encoding);
}`
Cheers!
-Steven