How to import thousends of assets?

Development Pipeline & Plugins
,
1

While creating a plugin to import assets into the editor, I keep running out of memory when I run it on our complete dataset. To investigate the issue I’ve tried the following without our plugin, and the Editor still crashes:

  • Have 2500+ PNG images of each 2048x2048 pixels, RGB format. Each image is approx 5MB, total size for all images is ~14GB

  • Open a pretty much empty project

  • Drag and drop all 2500 images into the Content Browser

The editor starts importing the textures, but runs out of memory somewhere between 1000 and 2000 images. My machine has 32GB of RAM and a paging file of 32GB also, and I’m running on Windows 11.

So I have two questions:

1 How do I import so many assets at all?
2 More importantly for my project, how do I import so many assets from C++ code in a plugin? I’m having a hard time freeing memory between imported files.