Project

General

Profile

Actions

Idea #16446

open

Manifest format improvements

Added by Peter Amstutz almost 4 years ago. Updated 17 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Target version:
Story points:
-
Release:
Release relationship:
Auto

Description

Need to be able to scale manifests to arbitrarily large amounts of data. Currently the 128 MB default request size limit becomes an issue.

Some possible approaches

  • Ability to "mount" another manifest at a directory to support really huge collections
  • API ability to get/put single stream (files in a directory) at a time

Related issues

Related to Arvados - Feature #19198: Feature to normalize and re-pack manifest so that same content always has same hashNewActions
Actions #1

Updated by Peter Amstutz almost 4 years ago

  • Start date set to 08/01/2021
  • Due date set to 11/01/2021
Actions #2

Updated by Peter Amstutz about 3 years ago

  • Start date changed from 08/01/2021 to 01/01/2022
  • Due date changed from 11/01/2021 to 03/31/2022
Actions #3

Updated by Peter Amstutz over 2 years ago

  • Start date changed from 01/01/2022 to 01/01/2023
  • Due date changed from 03/31/2022 to 06/30/2023
Actions #4

Updated by Peter Amstutz almost 2 years ago

  • Subject changed from Manifest format v2 to Manifest format improvements
Actions #5

Updated by Peter Amstutz almost 2 years ago

  • Description updated (diff)
Actions #6

Updated by Peter Amstutz almost 2 years ago

  • Start date changed from 01/01/2023 to 07/01/2022
  • Due date changed from 06/30/2023 to 12/31/2022
Actions #7

Updated by Peter Amstutz almost 2 years ago

  • Start date changed from 07/01/2022 to 01/01/2023
  • Due date changed from 12/31/2022 to 06/30/2023
  • Description updated (diff)
Actions #8

Updated by Peter Amstutz almost 2 years ago

  • Start date changed from 01/01/2023 to 06/01/2022
  • Due date changed from 06/30/2023 to 12/31/2022
Actions #9

Updated by Peter Amstutz almost 2 years ago

  • Start date changed from 06/01/2022 to 07/01/2022
Actions #10

Updated by Peter Amstutz almost 2 years ago

  • Related to Feature #19198: Feature to normalize and re-pack manifest so that same content always has same hash added
Actions #11

Updated by Peter Amstutz over 1 year ago

  • Start date changed from 07/01/2022 to 10/01/2022
  • Due date changed from 12/31/2022 to 03/31/2023
Actions #12

Updated by Peter Amstutz over 1 year ago

  • Start date changed from 10/01/2022 to 03/01/2023
  • Due date changed from 03/31/2023 to 07/31/2023
Actions #13

Updated by Peter Amstutz about 1 year ago

  • Start date changed from 03/01/2023 to 07/01/2023
  • Due date changed from 07/31/2023 to 12/31/2023
Actions #14

Updated by Peter Amstutz 11 months ago

  • Start date changed from 07/01/2023 to 09/01/2023
Actions #15

Updated by Peter Amstutz 11 months ago

  • Start date changed from 09/01/2023 to 01/01/2024
  • Due date changed from 12/31/2023 to 03/31/2024
Actions #16

Updated by Peter Amstutz 2 months ago

  • Start date changed from 01/01/2024 to 01/01/2025
  • Due date changed from 03/31/2024 to 03/31/2025
Actions #18

Updated by Peter Amstutz 17 days ago

  • Target version set to Future
Actions

Also available in: Atom PDF