Project

General

Profile

UseCases » History » Version 24

Sarah Zaranek, 10/05/2022 02:06 PM

1 2 Tom Clegg
2 1 Sarah Zaranek
_Note: these are not full use cases, simplified to just give more of the what and not a lot of the why or who.  We will be boiling these up into higher-level use cases.  This is more a list of items that should be included in all those higher-level use cases.  We want to not only ensure these use cases can be done in WB2 but the experience is equivalent or hopefully better in WB2 than in WB1._ 
3 23 Sarah Zaranek
4
5
h1. Use Cases for WB and Collections/Projects (Keep)
6 1 Sarah Zaranek
7 16 Sarah Zaranek
*Creating New Collections*
8
Summary: Users want to create a new collection in WB. 
9
Details:  Wants to make collection in from any of the data sources:
10 17 Sarah Zaranek
* From data already in an existing collection
11
* From a subset of files in an existing collection
12
* By combining files from different subsets of different collections
13
* By Uploading files from my desktop
14
* From the results of a already run workflow
15
* By downloading data from another source into a collection
16 1 Sarah Zaranek
17 16 Sarah Zaranek
*Annotating Collections*
18
Summary: User wants to annotate their collections with such details as sample
19
name, sample type, sequencing method, species, etc. 
20
Details:
21 17 Sarah Zaranek
* User wants to add annotations/metadata to existing collection
22
* User want to verify the added metadata to the collection
23 1 Sarah Zaranek
24 17 Sarah Zaranek
*Examining Collection Files* 
25 16 Sarah Zaranek
Summary:  User is viewing a collection and wants to examine the files and extract the data or metadata. 
26
Details:
27 17 Sarah Zaranek
* User wants to find certain files within the collection 
28
* (visually,using search (basic and advanced search using regexp, etc)
29
* User want to View or download certain files with the collection using a UI
30 1 Sarah Zaranek
31 18 Sarah Zaranek
*Sharing Collection Files* 
32 16 Sarah Zaranek
Summary:  User wants to share the Files in a Collection
33 17 Sarah Zaranek
Details:
34
* User wants make the collection available for downloading via ftp, s3 or other 3rd party application
35 18 Sarah Zaranek
* User wants to share the collection with others (by name) in their organization, others via a defined group, to everyone in their organization, or publicly.   Users want those they share the collection with to have: Read permission, Write permissions, Manage permissions
36 1 Sarah Zaranek
37 18 Sarah Zaranek
*Finding Existing Collections*
38 16 Sarah Zaranek
Summary: User want to find an existing collection
39
Details:  User wants to find the collection by one of the following ways:
40 18 Sarah Zaranek
* By Browsing to the project that contains the collection
41
* By searching the UUID of the project - and then looking through the items in the collection
42
* By searching the UUID of the collection
43
* By searching the collection name
44
* By searching the PDH
45
* By searching the collection metadata
46
* By searching for the workflow that produced it
47
* By a URL that can share that points to the project on WB
48 1 Sarah Zaranek
49 22 Sarah Zaranek
*Identifying Collections*
50 1 Sarah Zaranek
Summary: User needs to ID the collection for use as inputs to a workflow and/or inputs to a command using the API or CLI
51 22 Sarah Zaranek
Details:  
52
* User wants to find these specific file IDs
53
* User wants to get the collection UUID
54
* User wants to get the collection PDH
55
* User wants to get the collection metadata
56 16 Sarah Zaranek
57 22 Sarah Zaranek
*Editing an Existing Collection* 
58 16 Sarah Zaranek
Summary: User wants to modify a collection for use in their new project.
59 22 Sarah Zaranek
Details: 
60
* User will want to
61
* Copy over an existing collection to a new project
62
* Remove files in their collection to suit their needs for their new bioinformatics task. 
63
* Add files from either their desktop or from another collection into this new collection
64 1 Sarah Zaranek
65 24 Sarah Zaranek
* Editing the File Structure of An Existing Collection*
66
* Summary:  User wants to adjust the file structure of an existing collection to work better as an input to their analysis tools
67
* Details:  User will want to
68
* Remove files from folders to have them in all in a single flat directory
69
* Separate different subsets of files and place them in folders
70 1 Sarah Zaranek
71 16 Sarah Zaranek
Verifying Correct Collection
72
Summary: User wants to download data from a collection, and want to make sure the collect they found is the correct collection
73
Details:  User may want to check the following:
74
UUID and/or PDH
75
metadata
76
file contents
77
collection “version”
78
lineage of the collection (i.e. if the collection was generated as the result of a workflow - which workflow created this collection)
79 1 Sarah Zaranek
80 16 Sarah Zaranek
Creating Projects
81
Summary: User is working in the Arvados Workbench and needs a new Arvados project for a new analysis project they are working on. 
82
Details: The user wants to do the following
83
Create new project
84
Create a subproject within the new project (e.g. For Testing vs Final Runs)
85
Name the project/subproject
86
Add description of the project
87
Add metadata to the project  
88
Extract UUID of Project for use in workflow inputs or various command line/SDK/API functionality. 
89
Copy a URL that can share to others that points to the project
90
Mark the projects as “favorites” to be able to it more easily
91 1 Sarah Zaranek
92 16 Sarah Zaranek
Moving or Creating Items within Project
93
Summary. A User wants to set up their project to do their work.  They want to move all relevant existing data and workflows as well as upload new data and create new registered workflows.  
94
Details: Users may want to:
95
Copy existing collections and register workflows into this project
96
Create new collections within the project
97
Create new registered workflows in that project
98
Run workflows in this projects having the output, logs and other created artifacts contained within this project
99 1 Sarah Zaranek
100 16 Sarah Zaranek
Archiving or Sharing a Project
101
Summary: Now that the work is finished and the user has the results they need.  They want to get the project ready for sharing with their organization, with those outside their organization or even possible publicly in a publication.  
102
Details: Users may want to
103
Clean up the project by removing old collections, logs, processes and subdirectories that are not necessary to keep. 
104
Edit the name or metadata for the project
105
Freeze the project
106
Share the project with others in their organization, to everyone in their organization, publicly.   These are possible configurations:
107
Others have read permission  
108
Others have write permissions
109
Others have manage permissions
110 1 Sarah Zaranek
111 16 Sarah Zaranek
Identifying an Existing Project
112
Summary: User needs to identify the correct project in which to run their workflow.  They think they found the correct project they want searching on the project name.  
113
Details: The user then may want to:
114
Find the UUID of the project
115
Examine aspects of the project to double check it is the project they want to use.  They might want to:
116
Check the contents of the project 
117
Check the project description 
118
User might want to see the history of the project (*not currently available)
119
User might want to see the metadata for the project
120
Check to see if the project is frozen
121 1 Sarah Zaranek
122 16 Sarah Zaranek
Finding a Project or Subproject
123
Summary: A user logs back into Arvados after a break and would like to find the project they were working on previously. 
124
Details:  Those users would may want to find this project by
125
Searching for UUID, metadata or the project name
126
Navigating through the project/subproject hierarchy to find the project
127
Skimming through projects marked as their “Favorites”
128
Look for projects owned or created by a specific user (* Not Currently available)
129 1 Sarah Zaranek
130 19 Sarah Zaranek
h1. Use Cases for WB and Workflows (Crunch)
131 1 Sarah Zaranek
132 16 Sarah Zaranek
Submitting Workflows to Arvados Without Command Line
133
Summary: User wants to submit a workflow to run on Arvados without having to use the command line. The workflow CWL file either lives in a git repository or on the user’s local machine or in a collection on Arvados.  (* Currently this is not available on WB)
134
Details: The user will want to indicate to Arvados which workflow they want to run and have Arvados 
135
Parse the CWL file and generate an interactive form to fill out with input values and then submit the workflow via Workbench
136
Upload a YML file of input values to upload or direct Arvados to an existing YML file in an Arvados collection and then submit the workflow via Workbench. 
137 1 Sarah Zaranek
138 16 Sarah Zaranek
Monitoring Submitted Workflows 
139
Summary: User submits a workflow to run on Arvados using the command line. They want to monitor the workflow that they submitted using the Arvados Workbench. 
140
Details: The User will want to
141
Find the running workflow
142
Through search
143
Via Project Navigation
144
Check the workflow’s current status
145
If run successfully, find output collection 
146
If run successfully, look at how long it took to run
147
If run successfully, estimate costs (* currently only available for CommandLine)
148 1 Sarah Zaranek
149 16 Sarah Zaranek
Debugging Workflows - Part I
150
Summary: User submits a workflow to run on Arvados using the command line. They find out that the workflow and it did not run not run successfully.   They want to figure out why it failed.  
151
Details: The User may wants to:
152
Examine logs
153
Examine inputs
154
Examine command
155
Examine CWL  (* CWL not yet available)
156
Check to see which docker container was used
157
Run crunch-run stats interactively on WB  (*Currently only available via the command line) or have those plots available on WB. 
158 1 Sarah Zaranek
159 16 Sarah Zaranek
Debugging Workflows - Part II
160
Summary:  A User is updating a workflow workflow and the workflow has now stops running after a series of changes.  The User needs to compare the new workflow to the old workflow to see why it failed.
161
Details:  The User may need to 
162
Examine old workflow runs and backtrack to see when was the workflow last working
163
Determine what is the difference between the last working workflow and the broken workflows main process . This could include comparing: Inputs, Command, Resource Allocation, Node Type, and Docker Image metadata.
164
Determine which steps (if any) were re-used from the working workflow
165
Determine which steps failed in the new workflows
166
Determine if any big differences between the different workflows steps
167
Look at information passed to the steps including Inputs, Command, Resource Allocation
168
Find and compare logs from the relevant workflow steps Compare Docker image metadata (docker image ID, name, version, dockerfile)
169
Need to compare compare crunch-run stats between jobs (*Currently only available via command line)
170
171
Integrating Git Commit Information with Submitted Workflow
172
User has been working on changes to an existing workflow.  They have been submitting workflows managed in a git repository.  The workflow stops working or returning different outputs. The User would like to 
173
Find the relevant git commit information for the commit version of the workflow run on Arvados that worked
174
Find the relevant git commit information for the version of the workflow that is returning different results or not working
175
176
Calculating Workflow Costs
177
Summary: A User ran a big job and is now worried about how much it cost to run.  Details: The user wants to
178
Find the container request UUID of the workflow  
179
Run the cost analyzer to see how much it cost to run the entire workflow 
180
(* Currently not available in WB)
181
Run the cost analyzer to see which step is costing the most money
182
(* Currently not available in WB)
183
See if it would be possible to run workflow on less expensive instances (using information now only possible from the command line)
184
Estimate how much would it cost to run another similar workflow
185
 (* Currently not available in Arvados)
186
187
Creating a Register Workflow Using Workbench
188
Summary: User wants to create a registered workflow interactively using the WB (* Only currently available using command line).
189
Details.  The user may want to:
190
Share registered workflow with others in their organization
191
Set default values for the workflow
192
Define the name, metadata and descriptive text for the register workflow
193
194
Finding a Registered Workflow
195
Summary: User wants to run a registered workflow created by their colleague. User needs to find this workflow.
196
Details: The user may want to find the workflow by
197
Name, metadata or unique identifier
198
A URL shared by the creator of the register workflow
199
Looking Inside “project” that contains all the available shared workflows
200
201
Specifying Inputs to a Registered Workflow
202
Summary:  User wants to use the WB to submit a registered workflow made by 
203
their collaque to run on Arvados .  
204
Details: The user wants to:
205
Specify the inputs for the register workflow to use
206
Use the default inputs (if provided)
207
Identify the project the workflow should “run” in (i.e. where the outputs, logs and other collections should be stored)
208
See details about which registered workflow they are running to help guide them with to provide the proper input (eg. Registered workflow , name, description, etc.)
209
210
Examining A Running Workflow
211
Summary: User notices that their workflow has been running a long time and wants to check up on it.They would like to figure out if workflow is hung or is progressing along.  If progressing, the user may want to try and figure out why the workflow is running slower than expected.
212
Details:  The User may want to examine the following on WB
213
Real time logs
214
Resources used
215
Steps run so far and their outputs/inputs
216
Real-time values of RAM, CPU usage etc currently 
217
(* Not available in WB)
218
219
 Rerunning Old Workflows  
220
Summary: A User has successfully run a workflow on Arvados using the command line and would like to rerun that old workflow. 
221
(*Not Currently Available in WB)
222
Details: User may want to specify:
223
New Inputs
224
New Resource Requirements
225
New Docker Container
226
227
 Canceling a Running or Queued Workflow
228
Summary: User has submitted a workflow to Arvados via the command line and releases that they accidentally used the wrong inputs or an outdated function that the workflow leverages.  They want to cancel the workflow before it wastes time and resources.
229
Details:  User would like to
230
Find the running workflow on WB
231
Have an easy (single-button) way to cancel workflow on WB
232
Have a way to verify that the workflow was canceled
233
234
Checking Workflow Inputs or Requirements
235
Summary: User is examining output results of a workflow and wants to remind themselves which input parameter they used for the model they ran in the workflow. 
236
Details:  User would like to
237
Trace output collection to process that created it (i.e. container)
238
Find and examine the inputs to the workflow or workflow step run in that container