Working on a VR Comic Book Reader - That’s a shot of the early loading from .cbz (zip) of .jpg files to a texture and dynamically setting them.
I spent some time figuring out how to load a .jpg, ., or .bmp to a Texture2D ('s example works for DDS). That took some doing as I had to search through the engine code to figure out how it loads those formats to textures when importing. Enter the ImageWrapper. Makes it quite easy.
That , I had to find a liberally licensed zip file reading solution, which ended up being JUnzip. Then I spent some time converting it to use UE4’s engine file manipulation stuff.
Unfortunately, reading stuff on the game thread blocks (and reading big 1988x3056 or larger images is slow), so I had to set up an multi-threaded system for doing it.
So now instead of having a node that just gives you the texture immediately, you have a node for requesting a texture be loaded from the zip.
- If data hasn’t already been set up and a request dispatched, FIOSystem::LoadData is called with the appropriate offsets and buffers passed in.
- Because I can’t figure out how to check an FIOSystem request, I’m just looking at two special bytes in the buffer to see if they equal the values I put in there on allocation. (Super hacky, but it works.) If anyone knows the real way to check that, I’d love to know
Thanks
- If the compressed data has been loaded, a runnable thread is fired off to uncompress the data into the final buffer before it’s turned into a Texture2D.
- If the decompress task is finished - SetCompressed (compressed as in JPG not as in ZIP) on the ImageWrapper, get the raw data and copy to mip data of the texture.
The decompress task just does a bit at a time, keeping track of where it was at. It does 65k bytes at a time right now, though that might change. Lots of optimization to be .
Unfortunately, that’s not quite enough, as copying the uncompressed data into the mip data for the texture is slow as well - so I’m spending some more time threading that.
The plugin keeps around a cache of the loaded textures, and then there’s a system for polling against it. So I fire off the requests, then poll to see if a given texture is loaded.
@Epic devs,
As an aside, it seems like the engine could quite easily be extended so that FIOSystem::LoadDataCompressed would handle zip-compressed streams (not the archive formatting, just a compressed chunk of data). I modeled my system after it, and the only thing that’s really different is the way it decompresses using ZLIB, as it uses inflate functions rather than uncompress().
The FulfillCompressedRead function would need to be updated so that it knows not to look for PAK file headers in the first couple of bytes of the compressed data buffer, but other than that it seems pretty straight forward.
I’d be willing to submit my (converted to engine API) version of JUnzip (the original code is actually domain, so no licensing at all - which is quite of the author) for doing the zip structure navigation and such.
would like to see how it plays when its ready!