Questions about zen snapshot export/import + missing download large attachments and troubleshooting

Development Pipeline & Plugins
, ,
1

Hi,

I was wondering how the content of zenfs.manifest gets piped to our cloudddc during an oplog-export -builds using embedloosefiles? Navigating my local zendashboard after an oplog-import I am not seeing those files present but should I? We are using zen and zenserver 5.7.25. I am also observing some “Failed to download large attachment” with some hash when dowloading the zensnapshot, but I am unclear on how to go from those hash to the fileid/path to facilitate debugging. Would you have recommendation on how to go about that?

Note: Also one thing which I found surprising was that when we are missing large attachments and doing incremental cooking, some assets cooked on top of the snapshot seems to be recooked on every subsequent incremental cook. Those same assets won’t get recooked if doing a local cook and doing an incremental cook right after. Is this expected and how can we improve that situation?

Thank you!

[Attachment Removed]

2

If you try to do oplog-import/oplog-export without the option to ignore missing attachments you will get an error with a JSON payload indicating the “key”, “packagestoreentry”, “packagedata”,“bulkdata”. It appears as the hash corresponds to the “bulkdata”, “data” value.

I will now be looking at a way to ask the zenserver for attachments information for other hashes past the first failing missing attachment.

[Attachment Removed]

3

Hi Martin,

The files in zenfs.manifest get captured in an “op” within the zenserver “oplog” named “EndCook”. That EndCook op has an array of files entries on it that should include the files that were gathered up into zenfs.manifest. In the newer releases of zenserver we improved the web frontend to show this list of files for the EndCook op when you browse to it.

[Attachment Removed]