Support #18894
closed[document] how Keep manifest size can be predicted/calculated
Description
Peter wrote this up a while ago, it should be added to the documentation at https://doc.arvados.org/architecture/manifest-format.html
=================================================================================================================
Here's a formula for estimating manifest size, assuming efficiently packed blocks. This is including the block signatures, which makes it bigger than the size of the manifest in the database, because that is what is actually transferred to/from the API server and stored in RAM by arv-mount.
sum(size of all directory paths)
+ (total data size / 64 MB) * 94
+ sum(size of all file names)
+ sum(number of files * 20)
=================================================================================================================
Updated by Peter Amstutz over 2 years ago
- Target version changed from 2022-04-13 Sprint to 2022-04-27 Sprint
Updated by Peter Amstutz over 2 years ago
18894-manifest-size-doc @ 09d4326a8a853f9a5f05acb434d92c06156ec107
Updated by Peter Amstutz over 2 years ago
- Status changed from New to In Progress
Updated by Ward Vandewege over 2 years ago
Peter Amstutz wrote:
18894-manifest-size-doc @ 09d4326a8a853f9a5f05acb434d92c06156ec107
LGTM, just one comment: it would be good to mention API/MaxRequestSize here, and also that it applies to the signed version of the manifest.
Updated by Peter Amstutz over 2 years ago
- Status changed from In Progress to Resolved