Project

General

Profile

Actions

Feature #19181

open

[tracking] Workbench 2 file operations

Added by Peter Amstutz almost 2 years ago. Updated 12 months ago.

Status:
New
Priority:
Normal
Assigned To:
-
Category:
Workbench2
Target version:
Story points:
-

Description

Workbench collection operations should use the replace_files API (See: https://doc.arvados.org/v2.5/api/methods/collections.html#replace_files).

Existing operations:

  • Rename files or directories
  • Remove files or directories
  • Create a new collection from selected files

New operations -- these should be basically the same as the copy operations except that files are removed from their original collection locations.

  • Move files or directories to a new collection
  • Move files or directories to an existing collection
  • Copy selected files to an existing collection (was implemented previously, is presently commented out)
  • Copy/move to existing collection should work when the source and destination collection are the same (i.e. within the same collection)
  • When opening a dialog to move or copy files to a collection, the dialog should start with the current collection selected.

Related issues

Related to Arvados Workbench 2 - Bug #18587: "Copy selected into collection" incorrect behaviorNewActions
Related to Arvados Epics - Idea #16943: WB2 Descriptions, project & collection operationsIn Progress10/01/202203/31/2024Actions
Related to Arvados - Feature #16644: Move files into existing collectionNewActions
Related to Arvados - Feature #17586: Merge collectionsNewActions
Related to Arvados - Bug #18198: "copy selected files into collection" function in WB2 extremely slowNewActions
Related to Arvados - Bug #19048: Copying a folder in collection file browser results in empty folderNewActions
Related to Arvados Workbench 2 - Bug #19258: unexpected behaviour in "make collection from selected" in WB2NewActions
Actions #1

Updated by Peter Amstutz almost 2 years ago

  • Related to Bug #18587: "Copy selected into collection" incorrect behavior added
Actions #2

Updated by Peter Amstutz almost 2 years ago

  • Description updated (diff)
Actions #3

Updated by Peter Amstutz almost 2 years ago

  • Related to Idea #16943: WB2 Descriptions, project & collection operations added
Actions #4

Updated by Peter Amstutz almost 2 years ago

  • Target version changed from 2022-07-06 to 2022-07-20
Actions #5

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-07-20 to 2022-08-03 Sprint
Actions #6

Updated by Peter Amstutz over 1 year ago

  • Category set to Workbench2
Actions #7

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-08-03 Sprint to 2022-08-17 sprint
Actions #8

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-08-17 sprint to 2022-09-14 sprint
Actions #9

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-09-14 sprint to 2022-09-28 sprint
Actions #10

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-09-28 sprint to 2022-10-12 sprint
Actions #11

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-10-12 sprint to 2022-10-26 sprint
Actions #12

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-10-26 sprint to 2022-11-09 sprint
Actions #13

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-11-09 sprint to 2022-11-23 sprint
Actions #14

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-11-23 sprint to 2022-12-07 Sprint
Actions #15

Updated by Peter Amstutz over 1 year ago

  • Related to Feature #16644: Move files into existing collection added
Actions #16

Updated by Peter Amstutz over 1 year ago

Actions #17

Updated by Peter Amstutz over 1 year ago

  • Related to Bug #18198: "copy selected files into collection" function in WB2 extremely slow added
Actions #18

Updated by Peter Amstutz over 1 year ago

  • Related to Bug #19048: Copying a folder in collection file browser results in empty folder added
Actions #19

Updated by Peter Amstutz over 1 year ago

  • Related to Bug #19258: unexpected behaviour in "make collection from selected" in WB2 added
Actions #20

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-12-07 Sprint to 2022-12-21 Sprint
Actions #21

Updated by Peter Amstutz over 1 year ago

  • Target version changed from 2022-12-21 Sprint to 2023-01-18 sprint
Actions #22

Updated by Peter Amstutz about 1 year ago

  • Target version changed from 2023-01-18 sprint to 2023-02-01 sprint
Actions #23

Updated by Peter Amstutz about 1 year ago

  • Tracker changed from Idea to Feature
Actions #24

Updated by Peter Amstutz about 1 year ago

  • Target version changed from 2023-02-01 sprint to Future
Actions #27

Updated by Peter Amstutz about 1 year ago

  • Subject changed from Workbench 2 uses replace_files API to Workbench 2 file operations
Actions #28

Updated by Sarah Zaranek about 1 year ago

List from my list of use cases --

Creating New Collections
Summary: Users want to create a new collection in WB.
Details: Wants to make collection in from any of the data sources:
From data already in an existing collection
From a subset of files in an existing collection
By combining files from different subsets of different collections
By Uploading files from my desktop
From the results of a already run workflow
By downloading data from another source into a collection

Annotating Collections
Summary: User wants to annotate their collections with such details as sample
name, sample type, sequencing method, species, etc.
Details:
User wants to add annotations/metadata to existing collection
User want to verify the added metadata to the collection

Examining Collection Files
Summary: User is viewing a collection and wants to examine the files and extract the data or metadata.
Details:
User wants to find certain files within the collection
(visually,using search (basic and advanced search using regexp, etc)
User want to View or download certain files with the collection using a UI

Sharing Collection Files
Summary: User wants to share the Files in a Collection
Details:
User wants make the collection available for downloading via ftp, s3 or other 3rd party application
User wants to share the collection with others (by name) in their organization, others via a defined group, to everyone in their organization, or publicly. Users want those they share the collection with to have
Read permission.
Write permissions
Manage permissions

Finding Existing Collections
Summary: User want to find an existing collection
Details: User wants to find the collection by one of the following ways:
By Browsing to the project that contains the collection
By searching the UUID of the project - and then looking through the items in the collection
By searching the UUID of the collection
By searching the collection name
By searching the PDH
By searching the collection metadata
By searching for the workflow that produced it
By a URL that can share that points to the project on WB

Identifying Collections
Summary: User needs to ID the collection for use as inputs to a workflow and/or inputs to a command using the API or CLI
Details: User wants to find these specific file IDs
User wants to get the collection UUID
User wants to get the collection PDH
User wants to get the collection metadata

Editing an Existing Collection
Summary: User wants to modify a collection for use in their new project.
Details: User will want to
Copy over an existing collection to a new project
Remove files in their collection to suit their needs for their new bioinformatics task.
Add files from either their desktop or from another collection into this new collection

Editing the File Structure of An Existing Collection
Summary: User wants to adjust the file structure of an existing collection to work better as an input to their analysis tools
Details: User will want to
Remove files from folders to have them in all in a single flat directory
Separate different subsets of files and place them in folders
Move files into folders

Verifying Correct Collection
Summary: User wants to download data from a collection, and want to make sure the collect they found is the correct collection
Details: User may want to check the following:
UUID and/or PDH
metadata
file contents
collection “version”
lineage of the collection (i.e. if the collection was generated as the result of a workflow - which workflow created this collection)

Creating Projects
Summary: User is working in the Arvados Workbench and needs a new Arvados project for a new analysis project they are working on.
Details: The user wants to do the following
Create new project
Create a subproject within the new project (e.g. For Testing vs Final Runs)
Name the project/subproject
Add description of the project
Add metadata to the project
Extract UUID of Project for use in workflow inputs or various command line/SDK/API functionality.
Copy a URL that can share to others that points to the project
Mark the projects as “favorites” to be able to it more easily

Moving or Creating Items within Project
Summary. A User wants to set up their project to do their work. They want to move all relevant existing data and workflows as well as upload new data and create new registered workflows.
Details: Users may want to:
Copy existing collections and register workflows into this project
Create new collections within the project
Create new registered workflows in that project
Run workflows in this projects having the output, logs and other created artifacts contained within this project

Archiving or Sharing a Project
Summary: Now that the work is finished and the user has the results they need. They want to get the project ready for sharing with their organization, with those outside their organization or even possible publicly in a publication.
Details: Users may want to
Clean up the project by removing old collections, logs, processes and subdirectories that are not necessary to keep.
Edit the name or metadata for the project
Freeze the project
Share the project with others in their organization, to everyone in their organization, publicly. These are possible configurations:
Others have read permission
Others have write permissions
Others have manage permissions

Identifying an Existing Project
Summary: User needs to identify the correct project in which to run their workflow. They think they found the correct project they want searching on the project name.
Details: The user then may want to:
Find the UUID of the project
Examine aspects of the project to double check it is the project they want to use. They might want to:
Check the contents of the project
Check the project description
User might want to see the history of the project (*not currently available)
User might want to see the metadata for the project
Check to see if the project is frozen

Finding a Project or Subproject
Summary: A user logs back into Arvados after a break and would like to find the project they were working on previously.
Details: Those users would may want to find this project by
Searching for UUID, metadata or the project name
Navigating through the project/subproject hierarchy to find the project
Skimming through projects marked as their “Favorites”
Look for projects owned or created by a specific user (* Not Currently available)

Actions #29

Updated by Sarah Zaranek about 1 year ago

From customer interview -

Ability to split collections by folder
Merging collection (flat merge)
Extract files in collection by pattern
Rename files in collection
Given UUID – download listing of all files

Actions #30

Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)
Actions #31

Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)
Actions #32

Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)
Actions #33

Updated by Lucas Di Pentima about 1 year ago

  • Description updated (diff)
Actions #34

Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)
Actions #35

Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)
Actions #36

Updated by Peter Amstutz about 1 year ago

  • Subject changed from Workbench 2 file operations to [tracking] Workbench 2 file operations
Actions

Also available in: Atom PDF