Project

General

Profile

Actions

Feature #22076

open

keep-web can create a zipfile on the fly of a collection

Added by Peter Amstutz 3 months ago. Updated 17 days ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
Keep
Target version:
Story points:
-

Description

Accessed by making a POST request to the root of the (WebDAV endpoint) for the collection on keep-web. Works by PDH or UUID.

Should work the same whether using the "inline" or "download only" endpoint. Must be the collection root, not a subdirectory.

Indicate that it should be a zipfile by providing the header Accept: application/zip (confirm that is the right MIME type).

The POST body is either empty (get the whole collection), or a JSON array of strings which are paths within the collection to be included in the zip.

These are files or directories, if a path goes to a directory then it gets the entire contents of that directory. If there is both a reference to a subdirectory and to a specific file within that subdirectory, it gets the whole subdirectory (the file reference is redundant).

The list of files/directories should be sorted so they always download in the same order.

The zip file should be streamed to avoid excessive copying or use of staging storage.

If any of the file paths requested do not exist in the collection, return an error.

Check with customer

We probably do not need to support Range requests, this will be confirmed with customer.

We probably don't need to compress the files, but need to check.

Consider including the ".arvados#collection" file in the zip with the collection metadata.

Actions #2

Updated by Stephen Smith 3 months ago

Here is a potentially useful streaming zip library: https://github.com/scosman/zipstreamer

Actions #3

Updated by Peter Amstutz 3 months ago

In fact, the heavy lifting seems to be all in the "archive/zip" standard library module:

https://pkg.go.dev/archive/zip

The standard module even has feature that walks an entire "FS" object and packs all the files -- perfect for packing up a whole collection.

So zipstreamer might not actually add anything we need.

Actions #4

Updated by Peter Amstutz about 2 months ago

  • Target version changed from Future to Development 2024-12-04
Actions #5

Updated by Peter Amstutz about 1 month ago

  • Target version changed from Development 2024-12-04 to Development 2025-01-08
Actions #6

Updated by Peter Amstutz 18 days ago

  • Description updated (diff)
Actions #7

Updated by Peter Amstutz 17 days ago

  • Target version changed from Development 2025-01-08 to Development 2025-01-22
Actions

Also available in: Atom PDF