Announcement

Collapse
No announcement yet.

Runtime DataTable - Import text CSV or Google Sheet at runtime and fill an array of structs!

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • replied
    Originally posted by Jared Therriault View Post

    Taking this 1-to-1 so I can ask about project-specific information.
    For those reading, we worked this out. For some reason the marketplace gave this an outdated version of the plugin - if you're on 4.24 and experience issues with non-latin writing systems when using this plugin let me know.

    Leave a comment:


  • replied
    Originally posted by Crusader_91 View Post
    Hello Jared,

    This is a duplicate of the letter I sent you personally.

    We were talking about an issue about encoding cyrillic symbols in the downloaded information. You asked me to send the payload string to you. Here it is (partially):


    ========================================================================
    Code:
    LogBlueprintUserMessages: [BP_ItemManager_2] ---,Item Name: Identified,Item Class,Item Type,Item Mesh In World,Item Materials,Item UI Image,Description,Usable,Equipable,Stackable,Max Stack Count,Item Price,OneItemMass,PivotPointShiftInItemInspector,EventsToActivateOnPickup
    00000000,"(RU=""Масло навыка: Перехват"",EN=""None"")",/Game/Blueprints/World/Items/Oil_Items/Skills/BP_Equip_Oil_Skill_General_Intercept.BP_Equip_Oil_Skill_General_Intercept_C,Oil_Skill,/Game/Prototyping/Meshes/Items/Potionbottle_001.Potionbottle_001,(None),/Game/VisualContent/UI/GraphicsAndSprites/EquipmentIconsAndItems/Items/Oil/MI_ItemsUI_Oil_Skill_Gener
    al_Intercept.MI_ItemsUI_Oil_Skill_General_Intercept,"(RU=""Капсула с Маслом, дающая персонажу навык: Перехват"",EN=""English Text"")",FALSE,TRUE,FALSE,0,0,0.500000,"(X=0.000000,Y=0.000000,Z=0.000000)",None
    00000001,"(RU=""Масло навыка: Дезориентирующий приказ"",EN=""None"")",/Game/Blueprints/World/Items/Oil_Items/Skills/BP_Equip_Oil_Skill_General_DisorientOrder.BP_Equip_Oil_Skill_General_DisorientOrder_C,Oil_Skill,/Game/Prototyping/Meshes/Items/Potionbottle_001.Potionbottle_001,(None),/Game/VisualContent/UI/GraphicsAndSprites/EquipmentIconsAndItems/Items/Oil/M
    I_ItemsUI_Oil_Skill_General_Disorient.MI_ItemsUI_Oil_Skill_General_Disorient,"(RU=""Капсула с Маслом, дающая персонажу навык: Дезориентирующий приказ"",EN=""English Text"")",FALSE,TRUE,FALSE,0,0,0.500000,"(X=0.000000,Y=0.000000,Z=0.000000)",None
    ========================================================================

    As I can see, the payload string is correct, so the garbage data is definitely not coming from Google. Each segment of the Russian-language text (after the labels "RU") is reproduced correctly.

    I also attach a link to a copy of this table, so you can make sure the information is correct:

    https://docs.google.com/spreadsheets...it?usp=sharing

    Thanks for your work again!
    Taking this 1-to-1 so I can ask about project-specific information.

    Leave a comment:


  • replied
    Originally posted by joesys View Post
    Hi Jared,

    Thank you for the wonderful plugin.

    I notice the following code in ReadCSVRow()

    Code:
    //Ensure we won't end up with an index under 0 (in case of an empty string)
    int32 FieldIndex = (fields[CSV_FieldIndex].Len() - 1 < 0 ? 0 : fields[CSV_FieldIndex].Len() - 1); //todo: keep an eye on this
    
    if (fields[CSV_FieldIndex].Len() > 1 && fields[CSV_FieldIndex][FieldIndex] == '\\') //if we find a wrapper character and the last character was supposed to escape it then it was obviously included by the user
    {
    fields[CSV_FieldIndex][FieldIndex] = WrapperCharacter;
    StateIndex = 3; //Quote within user string/text
    }
    
    StateIndex = 2; //Wrapped wrapper
    I believe it should be:

    Code:
    ...
    
    StateIndex = 3; //Quote within user string/text
    }
    else
    {
    StateIndex = 2; //Wrapped wrapper
    }
    Otherwise, StateIndex will always be 2.
    You're right joesys! I actually saw this oversight a few weeks ago and fixed it localy but I haven't pushed the fix. I will be pushing a fix soon (in the next month). This should only affect users who have double-nested quotes in their text and heavy use of "\". For most users this won't be an issue, but a bug is a bug and I'll have the fix out soon! Thank you!

    Leave a comment:


  • replied
    Hello Jared,

    This is a duplicate of the letter I sent you personally.

    We were talking about an issue about encoding cyrillic symbols in the downloaded information. You asked me to send the payload string to you. Here it is (partially):


    ========================================================================
    Code:
    LogBlueprintUserMessages: [BP_ItemManager_2] ---,Item Name: Identified,Item Class,Item Type,Item Mesh In World,Item Materials,Item UI Image,Description,Usable,Equipable,Stackable,Max Stack Count,Item Price,OneItemMass,PivotPointShiftInItemInspector,EventsToActivateOnPickup
    00000000,"(RU=""Масло навыка: Перехват"",EN=""None"")",/Game/Blueprints/World/Items/Oil_Items/Skills/BP_Equip_Oil_Skill_General_Intercept.BP_Equip_Oil_Skill_General_Intercept_C,Oil_Skill,/Game/Prototyping/Meshes/Items/Potionbottle_001.Potionbottle_001,(None),/Game/VisualContent/UI/GraphicsAndSprites/EquipmentIconsAndItems/Items/Oil/MI_ItemsUI_Oil_Skill_Gener
    al_Intercept.MI_ItemsUI_Oil_Skill_General_Intercept,"(RU=""Капсула с Маслом, дающая персонажу навык: Перехват"",EN=""English Text"")",FALSE,TRUE,FALSE,0,0,0.500000,"(X=0.000000,Y=0.000000,Z=0.000000)",None
    00000001,"(RU=""Масло навыка: Дезориентирующий приказ"",EN=""None"")",/Game/Blueprints/World/Items/Oil_Items/Skills/BP_Equip_Oil_Skill_General_DisorientOrder.BP_Equip_Oil_Skill_General_DisorientOrder_C,Oil_Skill,/Game/Prototyping/Meshes/Items/Potionbottle_001.Potionbottle_001,(None),/Game/VisualContent/UI/GraphicsAndSprites/EquipmentIconsAndItems/Items/Oil/M
    I_ItemsUI_Oil_Skill_General_Disorient.MI_ItemsUI_Oil_Skill_General_Disorient,"(RU=""Капсула с Маслом, дающая персонажу навык: Дезориентирующий приказ"",EN=""English Text"")",FALSE,TRUE,FALSE,0,0,0.500000,"(X=0.000000,Y=0.000000,Z=0.000000)",None
    ========================================================================

    As I can see, the payload string is correct, so the garbage data is definitely not coming from Google. Each segment of the Russian-language text (after the labels "RU") is reproduced correctly.

    I also attach a link to a copy of this table, so you can make sure the information is correct:

    https://docs.google.com/spreadsheets...it?usp=sharing

    Thanks for your work again!
    Last edited by Crusader_91; 07-17-2020, 02:16 PM.

    Leave a comment:


  • replied
    Originally posted by Crusader_91 View Post
    Hello, Jared!

    I have encountered an issue working with your plugin.

    It doesn't encode cyrillic symbols correctly after import from Google spreadsheets. For example, russian word "Приказ" was encoded to "?@8:07". I think, that's because of default UTF-16 encoding. Is it possible to use UTF-8 encoding during import?
    Hi Crusader_91, maybe you can help me figure out when this is happening. After 'Call On Complete' is called, save the payload and send that to me please!

    I'm currently using wchar_t to parse the payload into structs and it's my understanding that should support Cyrillic, so I would like to know if the garbage data is coming from Google or if the script converts it to garbage. Send some samples when you can! Thank you.

    Leave a comment:


  • replied
    Hello, Jared!

    I have encountered an issue working with your plugin.

    It doesn't encode cyrillic symbols correctly after import from Google spreadsheets. For example, russian word "Приказ" was encoded to "?@8:07". I think, that's because of default UTF-16 encoding. Is it possible to use UTF-8 encoding during import?

    Leave a comment:


  • replied
    Hi Jared,

    Thank you for the wonderful plugin.

    I notice the following code in ReadCSVRow()

    Code:
    //Ensure we won't end up with an index under 0 (in case of an empty string)
    int32 FieldIndex = (fields[CSV_FieldIndex].Len() - 1 < 0 ? 0 : fields[CSV_FieldIndex].Len() - 1); //todo: keep an eye on this
    
    if (fields[CSV_FieldIndex].Len() > 1 && fields[CSV_FieldIndex][FieldIndex] == '\\') //if we find a wrapper character and the last character was supposed to escape it then it was obviously included by the user
    {
        fields[CSV_FieldIndex][FieldIndex] = WrapperCharacter;
        StateIndex = 3; //Quote within user string/text
    }
    
    StateIndex = 2; //Wrapped wrapper
    I believe it should be:

    Code:
    ...
    
        StateIndex = 3; //Quote within user string/text
    }
    else 
    {
        StateIndex = 2; //Wrapped wrapper
    }
    Otherwise, StateIndex will always be 2.

    Leave a comment:


  • replied
    Originally posted by AaronMTodd View Post

    I second this request! I have some complex structs that would be great to export and be able to manipulate. I'm sure I read somewhere that this supports structs within structs, correct? It would be nice to be able to export just to make sure that a complex CSV like that is setup correctly for import.
    That's a fair use case, thank you for the feedback! It is true, the plugin supports structs within structs. For now, I'd recommend creating a normal DataTable in the engine with a nested struct then exporting it to CSV to get a feel for the layout, then create new rows in Google sheets based on that.

    The heat is rising on this feature request, but I'll be honest: This is something I've looked into and the amount of work for implementation wouldn't be trivial. So if I do make this feature happen it will be part of a separate plugin. Reasons being:
    • I don't want to raise the price of RDT as it is
    • People who don't need the export feature might feel burned having to pay more when it didn't use to be like that
    • The time investment for implementation, testing and support will far surpass the amount of money I make from the plugin to begin with
    So it would be a separate buy-in. I'm not sure how customers would react to that. How do you feel?

    Leave a comment:


  • replied
    Originally posted by danibb View Post
    Would it be possible to export a data structure to excell in runtime?
    I second this request! I have some complex structs that would be great to export and be able to manipulate. I'm sure I read somewhere that this supports structs within structs, correct? It would be nice to be able to export just to make sure that a complex CSV like that is setup correctly for import.

    Leave a comment:


  • replied
    Originally posted by danibb View Post
    Would it be possible to export a data structure to excell in runtime?
    Hi danibb - not with this plugin at this time, but this is a request I see a lot. It is something I will consider for the future.

    Leave a comment:


  • replied
    Would it be possible to export a data structure to excell in runtime?

    Leave a comment:


  • replied
    Originally posted by franklzt View Post
    If I fill blank string for google url,it means I need load data from disk, load it from disk directly, instead of requesting from google and then load from disk. Hope next update can fix this problem. I modify it myself first.
    Hey franklzt! That is by design. Rather than having a series of bool parameters to choose what to do, the expected flow is to simply leave the Sheets URL blank to use a local path.

    Leave a comment:


  • replied
    If I fill blank string for google url,it means I need load data from disk, load it from disk directly, instead of requesting from google and then load from disk. Hope next update can fix this problem. I modify it myself first.

    Leave a comment:


  • replied
    Originally posted by franklzt View Post
    I make one test, found that this can only work for packed game for load csv data from disk ? how can it work on editor mode ?
    Fixed the problem

    Leave a comment:


  • replied
    I make one test, found that this can only work for packed game for load csv data from disk ? how can it work on editor mode ?

    Leave a comment:

Working...
X