Announcement

Collapse
No announcement yet.

Importing text files is unnecessarily hard

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    [FEATURE REQUEST] Importing text files is unnecessarily hard

    One would expect that any game engine would let you import text files easily by default, right ? Well, that's not the case in UE4.

    Right now if you want to import something as common as a JSON or a XML you have to create a plugin just for that(well, you first have to search for a plugin that does something similar so you can use it as reference since there are 0 docs about the topic) and then pray so Epic doesn't decide to use that format for something else.

    A few months ago I had to expend two days just to make a plugin to import text files. And now turns out I can't import JSON files anymore because Epic decided that JSON files should be used for DataTables. Setting ImportPriority in my UTextImporterFactory to MAX_int32 doesn't seem to solve anything, so it looks like Epic is bending its own rules here.

    That's the kind of unnecessary friction that makes Unreal so unfriendly sometimes.


    Ideally, I think importing files should be something like this :

    Let us import any binary or text file by default as a standard text/binary asset. Chances are we just want to read some data, but that way we can enjoy from editor integration, auto-imports and auto-reimports.

    If there are one or more factories that uses the extension, just ask us what we want to do instead of using the current priority system. Some extensions like JSON and XML are too common and will probably be used for more than one thing on any non-trivial project. In fact, that's the case with DataTables.

    For common cases, let us choose a default behaviour, so we're not asked every time we import a PNG or a FBX.


    I think that would be easier for everyone and doesn't have any drawback compared to the current behaviour.

    #2
    Heya,

    Thanks for reporting this, we're going to take a look and try to ensure that the engine level factories that consume JSON either have a low priority and/or only engage when the format of the contents in the file matches what is expected.

    If your json is being generated by some other tool or editor that you control, it might also be worth considering using a different extension, although obviously this isn't always possible.

    The disambiguation dialog is a good idea, but it is probably a little further down the road due to the architecture of how the factories currently work.

    Cheers,
    Michael Noland

    Comment


      #3
      I like the idea of Generic Text/Binary Asset. When you drag some file into the editor, editor will ask you to import it as Generic Text/Binary Asset if it's not already recognized asset format.
      | twitter | github | #ue4tip

      Comment


        #4
        Originally posted by Michael Noland View Post
        Heya,

        Thanks for reporting this, we're going to take a look and try to ensure that the engine level factories that consume JSON either have a low priority and/or only engage when the format of the contents in the file matches what is expected.

        If your json is being generated by some other tool or editor that you control, it might also be worth considering using a different extension, although obviously this isn't always possible.

        The disambiguation dialog is a good idea, but it is probably a little further down the road due to the architecture of how the factories currently work.

        Cheers,
        Michael Noland
        Hi Michael, thanks for your attention.

        Looks like Paper2D imports JSONs just fine, is Paper2D handled on the engine as a special case ? Didn't you have to do anything special to make it work on 4.8 ?

        The first thing I tried to do was to look at the PaperJsonImporter module since that was what I used as reference in the first place, but looks like you refactored quite a few things in the plugin since 4.7 and PaperJsonImporter is not there anymore.


        And yeah, using a different extension is not really an option. I could change it manually every time I save, but yet again, reading a text file shouldn't be that hard.

        The good news is that I found a workaround. If I place a JSON file inside one of the folders that the editor scans for auto-imports and click cancel on the DataTables dialog, then the file gets imported by my own plugin. I'd like to get it fixed properly but at least I can work for now.



        @Nate
        I could share my plugin if you are interested. The only catch is that you have to add any extension you are interested in manually and that, well, there can be some conflicts with some extensions sometimes, as you can see.
        The only reason I didn't share it when I made it is that since there are no docs on ImporterFactories I'm not really confident enough to support it.

        Comment


          #5
          Hey,

          I was the one who added the Data Table JSON importing. I'm sorry to hear that it's caused you this trouble.

          I have been looking into this today, but I found that the Paper2D importing was also broken. How are you trying to import your JSON file? I was just drag-dropping the file onto the Content Browser (which should be the same as using the Import option).

          With regards to the issue of the factory priority not being respected, I have found one path through the code where the matching factories for a given file extension weren't being sorted before being tested. This basically resulted in them being processed in the order they were found, rather than based on their priority.

          I have a pending fix for the sorting issue, and have also lowered the priority of the Data Table import factory so that it's below normal by default. Hopefully I'll get these changes submitted tomorrow, and I'll follow up on this thread once the changes are in.

          Thanks,
          Jamie.

          Comment


            #6
            Hi Jamie. Great news! Thank you.

            Originally posted by Jamie Dale View Post
            I have been looking into this today, but I found that the Paper2D importing was also broken. How are you trying to import your JSON file? I was just drag-dropping the file onto the Content Browser (which should be the same as using the Import option).
            Looks like Paper2D only imports JSONs if they are recognized as Tiled maps, if not, I suppose the file is ignored (which I didn't know was possible) and the DataTable import menu appears. Maybe that's why Paper2D importing doesn't work for you ?

            I'm using JSONs for Tiled maps too (but I generate 3D tilemaps instead of 2D ones), that's why Paper2D was working fine for me.

            Comment


              #7
              Heya,

              As a heads up, if you wanted to use a UPaperTileMap asset and visualize it in a different way that should be doable without creating a separate importer, though it all depends on exactly what you are doing.

              Cheers,
              Michael Noland

              Comment


                #8
                The changes I mentioned before have gone in as b97258a.

                This will make the priority work correctly for your factory, but it is also essential that your factory override FactoryCanImport to perform a meaningful test against the JSON data to make sure it's something it can import (see UPaperSpriteSheetImportFactory::FactoryCanImport and UPaperTiledImporterFactory::FactoryCanImport for examples of this).

                Anything not imported by a higher priority factory will simply be consumed as if it were Data Table JSON.

                Comment


                  #9
                  Originally posted by Jamie Dale View Post
                  The changes I mentioned before have gone in as b97258a.
                  Great! Thank you.

                  I don't have access to a proper computer at the moment, but I'll try the fix as soon as I can.

                  Will this be included in the next hotfix release ?

                  Comment


                    #10
                    Originally posted by TS100101 View Post
                    Will this be included in the next hotfix release ?
                    No, it'll be part of 4.9.

                    Our hotfix releases are only for fixing critical blocker issues (such as unavoidable crashes), and aren't supposed to make API changes - the change I made to get this working meets neither of those criteria

                    Comment


                      #11
                      I want to add my two cents. I need to import a text file with some custom data and process it. I can do it using blueprints as it's a simple document, but I have a LOT of these files. Just being able to add a text file to the content browser would be nice.

                      It's 2017, UE should definitely import text files.

                      Comment


                        #12
                        Originally posted by Antidamage View Post
                        I want to add my two cents. I need to import a text file with some custom data and process it. I can do it using blueprints as it's a simple document, but I have a LOT of these files. Just being able to add a text file to the content browser would be nice.

                        It's 2017, UE should definitely import text files.
                        Why not use data tables instead?

                        Comment


                          #13
                          That is the plan. The plan falls down when you realise UE has no way to import the text file you want to parse into the data table. This sounds like a job for blutilities or something, but the basic text file support should be in there. I'm going to take five minutes to add a factory for it later on today, but it'd be nice if it already existed.

                          Comment


                            #14
                            Then why not create the data table outside of UE4 (just save as a .csv file) and import it that way?

                            Comment


                              #15
                              Because the data isn't in csv format. The TEXT needs pre-processing, which if done in a blueprint would be an excellent time to populate it into a data table. Trust me when I say I've thought about this beyond knocking two braincells together.

                              Comment

                              Working...
                              X