Project

General

Profile

Actions

Support #18894

closed

[document] how Keep manifest size can be predicted/calculated

Added by Ward Vandewege over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Due date:
Story points:
-
Release relationship:
Auto

Description

Peter wrote this up a while ago, it should be added to the documentation at https://doc.arvados.org/architecture/manifest-format.html

=================================================================================================================
Here's a formula for estimating manifest size, assuming efficiently packed blocks. This is including the block signatures, which makes it bigger than the size of the manifest in the database, because that is what is actually transferred to/from the API server and stored in RAM by arv-mount.

sum(size of all directory paths)
+ (total data size / 64 MB) * 94
+ sum(size of all file names)
+ sum(number of files * 20) =================================================================================================================


Subtasks 1 (0 open1 closed)

Task #19016: Review 18894-manifest-size-doc ResolvedPeter Amstutz04/15/2022Actions
Actions #1

Updated by Ward Vandewege over 2 years ago

  • Description updated (diff)
Actions #2

Updated by Peter Amstutz over 2 years ago

  • Assigned To set to Peter Amstutz
Actions #3

Updated by Peter Amstutz over 2 years ago

  • Target version changed from 2022-04-13 Sprint to 2022-04-27 Sprint
Actions #4

Updated by Peter Amstutz over 2 years ago

Actions #5

Updated by Peter Amstutz over 2 years ago

  • Status changed from New to In Progress
Actions #6

Updated by Ward Vandewege over 2 years ago

Peter Amstutz wrote:

18894-manifest-size-doc @ 09d4326a8a853f9a5f05acb434d92c06156ec107

LGTM, just one comment: it would be good to mention API/MaxRequestSize here, and also that it applies to the signed version of the manifest.

Actions #7

Updated by Peter Amstutz over 2 years ago

  • Status changed from In Progress to Resolved
Actions #8

Updated by Peter Amstutz over 2 years ago

  • Release set to 51
Actions

Also available in: Atom PDF