Project

General

Profile

Actions

Feature #20225

closed

Let users choose collection subdirectories for workflow Directory inputs

Added by Brett Smith about 1 year ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Workbench2
Story points:
2.0
Release relationship:
Auto

Description

The RNAseq tutorial workflow is written with the expectation that one of its inputs will be a collection subdirectory. See genome in yml/RNA-seq-wf.yml.

Workbench 2 doesn't seem to support this. When you select a directory input, it only lets you choose a collection. There's no ability to open the collection and select a specific subdirectories like you can with file inputs.


Files

selectable-groups.mov (3.47 MB) selectable-groups.mov Lucas Di Pentima, 09/25/2023 06:31 PM
cypress.log (58 KB) cypress.log Stephen Smith, 10/12/2023 08:21 PM

Subtasks 1 (0 open1 closed)

Task #20330: Review 20225-directory-input-subfolder-selectionResolvedStephen Smith09/22/2023Actions

Related issues

Related to Arvados Epics - Idea #17001: Arvados uses WB2 by defaultResolvedActions
Is duplicate of Arvados - Feature #18968: Should be able to pick directories within collectionsDuplicateActions
Blocked by Arvados - Feature #20031: New collection file browser copy/move operationsResolvedStephen Smith05/02/2023Actions
Actions #1

Updated by Peter Amstutz about 1 year ago

  • Story points set to 2.0
Actions #2

Updated by Peter Amstutz about 1 year ago

  • Target version changed from Future to To be scheduled
Actions #3

Updated by Peter Amstutz about 1 year ago

  • Target version changed from To be scheduled to Development 2023-04-26 sprint
Actions #4

Updated by Peter Amstutz about 1 year ago

  • Assigned To set to Stephen Smith
Actions #5

Updated by Peter Amstutz 12 months ago

  • Target version changed from Development 2023-04-26 sprint to Development 2023-05-10 sprint
Actions #6

Updated by Peter Amstutz 12 months ago

  • Status changed from New to In Progress
Actions #7

Updated by Peter Amstutz 12 months ago

  • Target version changed from Development 2023-05-10 sprint to Development 2023-05-24 sprint
Actions #8

Updated by Peter Amstutz 12 months ago

  • Blocked by Feature #20031: New collection file browser copy/move operations added
Actions #9

Updated by Peter Amstutz 11 months ago

  • Target version changed from Development 2023-05-24 sprint to Development 2023-06-07
Actions #10

Updated by Peter Amstutz 11 months ago

  • Target version changed from Development 2023-06-07 to Development 2023-06-21 sprint
Actions #11

Updated by Peter Amstutz 10 months ago

  • Tracker changed from Idea to Feature
Actions #12

Updated by Peter Amstutz 10 months ago

  • Target version changed from Development 2023-06-21 sprint to Development 2023-07-05 sprint
Actions #13

Updated by Peter Amstutz 10 months ago

  • Target version changed from Development 2023-07-05 sprint to Development 2023-08-02 sprint
Actions #14

Updated by Peter Amstutz 9 months ago

  • Target version changed from Development 2023-08-02 sprint to Development 2023-08-16
Actions #15

Updated by Peter Amstutz 9 months ago

  • Target version changed from Development 2023-08-16 to Development 2023-08-30
Actions #16

Updated by Peter Amstutz 8 months ago

  • Target version changed from Development 2023-08-30 to Development 2023-09-13 sprint
Actions #17

Updated by Brett Smith 8 months ago

  • Related to Idea #17001: Arvados uses WB2 by default added
Actions #18

Updated by Peter Amstutz 8 months ago

  • Target version changed from Development 2023-09-13 sprint to Development 2023-08-30
Actions #19

Updated by Peter Amstutz 8 months ago

  • Release set to 66
Actions #20

Updated by Peter Amstutz 8 months ago

  • Target version changed from Development 2023-08-30 to Development 2023-09-13 sprint
Actions #21

Updated by Peter Amstutz 8 months ago

  • Is duplicate of Feature #18968: Should be able to pick directories within collections added
Actions #22

Updated by Peter Amstutz 8 months ago

  • Status changed from In Progress to Duplicate
Actions #23

Updated by Peter Amstutz 8 months ago

  • Status changed from Duplicate to In Progress
Actions #24

Updated by Peter Amstutz 8 months ago

  • Release changed from 66 to 67
Actions #25

Updated by Peter Amstutz 7 months ago

  • Target version changed from Development 2023-09-13 sprint to Development 2023-09-27 sprint
Actions #26

Updated by Stephen Smith 7 months ago

Changes at arvados-workbench2|25c0de8aa20f801212dda2dff23216289487d08f branch 20225-directory-input-subfolder-selection
Tests developer-tests-workbench2: #1346

  • All agreed upon points are implemented / addressed.
    • Adds subdirectory selection support to DirectoryInput and DirectoryArrayInput
    • Reworks TreePicker initialization code to handle multiple selection items to initialize
    • Adds cascadeSelection=false mode to tree picker since it doesn't make sense to force users to also select all subfolders when selecting a parent folder/collection
    • Enables users to select a collection without expanding it in cascadeSelection=false mode since the selection won't cascade
  • Anything not implemented (discovered or discussed during work) has a follow-up story.
    • In the future it might be nice to rework the project tree picker to tie together visibility, cascade selection, and selectable objects into something like an enum that determines visibility and selection behavior
    • The preload procedure is a bit sluggish and could benefit from a loading overlay, could be done as part of #20037
  • Code is tested and passing, both automated and manual, what manual testing was done is described
    • Tested selecting a mixture of collection subdirectories and top level collections, including selection of directories residing inside other selected directories, in both home project and shared with me (selections in favorites get re-selected through home or shared with me when re-opening the picker)
  • Documentation has been updated.
    • n/a
  • Behaves appropriately at the intended scale (describe intended scale).
    • The preloading requires loading the ancestor tree for each item, so it's a bit slow. I would expect anyone selecting a large number of files to be not using workbench (eg. cli)
  • Considered backwards and forwards compatibility issues between client and server.
    • n/a
  • Follows our coding standards and GUI style guidelines.
    • No substantial UI changes
Actions #27

Updated by Lucas Di Pentima 7 months ago

Some comments:

  • I think it would be really nice to make smaller sized commits in the future, for readability reasons. I appreciate the detailed descriptions on these but it's a lot easier to follow (at least, for me) having bite sized commits instead of just a couple commits with all the changes.
  • Changes at src/store/tree-picker/tree-picker-actions.ts might require some unit testing.
  • It would be good to have unit tests for the changes made in src/models/tree.ts too.
  • In tree-picker-actions.ts the error message "Some selections failed to load and were removed" may be a bit confusing to the user, do you think it would be worth explaining which selections were skipped on the console (and inform the user about that) or in some other way?
  • When editing a multi-dir input field and expanding a group, I get a checkbox beside the group but it doesn't do anything when clicking on it (see attached video)
    • OK, I realize this bug report is out of scope, so maybe we should make a different ticket for it.
  • When editing a multi-dir input field, the chips added to the bottom only display the dir name, I think it would be super helpful to have some kind of tooltip on each that shows its full tree path or some other data that provides some context data. I think this would be useful for selections on extremely long list of projects, especially when there're equally named subdirs.
    • This also may be an out of scope bug, feel free to delegate it to a different ticket.
  • Question: Is there already a consensus about what to do when selecting a parent dir and a subdir at the same time? Right now the UI treats them as different selections.
  • Can we have a Cypress test exercising these new behaviors?
Actions #28

Updated by Peter Amstutz 7 months ago

  • Target version changed from Development 2023-09-27 sprint to Development 2023-10-11 sprint
Actions #29

Updated by Peter Amstutz 7 months ago

  • Target version changed from Development 2023-10-11 sprint to Development 2023-10-25 sprint
Actions #30

Updated by Stephen Smith 7 months ago

Changes at arvados-workbench2|162665e037ec2de3203e8ed34991b1f443462382
Tests: (jekins still broken so local run log attached)

  • Add unit testing for tree picker initialization / loadInitialValue actions
  • Add unit tests for new tree method
  • Improved error handling
    • Catch errors in tree picker loadProject and show toast + console error with rejection reason
    • Trigger rejected promise if ancestor service returns no ancestors, add console.error to note uuid of item that failed to preselect
  • Added cypress test to test subdirectory selection and reselection when reopening picker
  • There is currently no consensus on how to handle selecting a parent and child directory node in the tree picker because before now, the only occurrences of multi select tree picker are in situations where the parent and child must be different types (with the exception of a multi project picker with nested projects, which I don't think is done anywhere but could be a problem with the previous scheme), thus the picker which should only accept 1 type ignores the parent selection in the setvalue handler (eg a multi collection picker where selecting a project selects all collections within without including the project as a selection despite being selected). The default behavior before was to always select all selectable child nodes when a parent is selected and to limit what can be selected by only loading the desired item types. It also enforced that selecting all selectable child items causes the parent to be selected too. Enabling collection subdirectory picking changes this since now both a parent and child can be a valid selection type, and will both validate as a selected location - so I disabled the cascade selection up/down behavior so that you can select a sole subdirectory nested in a collection without every parent directory up to the root collection also getting submitted as a value (until it reaches a directory that contains a second item). One way where the previous behavior could be a problem is if you have a collection: Collection/subdir/reference, but lets say subdir also contains files that you don't want to give to the tool. With cascading selections, selecting reference would auto select subdir and Collection. With cascading disabled you can just select reference. Depending on how various tools handle directory parameters, it might also be desirable to select subdir and not reference if the tool doesn't recurse automatically and you only want it to process files in the parent directory. I think there's more room for improvement though:
    • If you have a large structure of folders and need to select them all, it would be more tedious this way - but it should only be a problem if you have a tool that doesn't recurse
    • We could possibly further refine how selections cascade so that we can enable selecting all collections in a project - for example instead of disabling selection cascading entirely, we could change it so that a selection cascades downwards until it reaches an item type that matches what the picker accepts. So selecting a project with collections will cascade to the root collections without selecting collection contents for both collection and directory pickers, while a file picker would cascade into each collection to select all files. In the end I decided just disabling the cascading behavior for the directory array picker would be best for the scope of the ticket
Actions #31

Updated by Lucas Di Pentima 6 months ago

Just one suggestion:

  • On the Cypress test, there's a createCollection() call with a manifest text that includes unnecessary block permission signatures (i.e.: +A8f0be20f1a6e28cf4e2c034dc3d4a02a49bebe7e@653aae59), you can skip it as you're using the admin account to create it.

The rest LGTM, thanks

Actions #32

Updated by Stephen Smith 6 months ago

  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF