Project

General

Profile

UseCases » History » Version 20

Sarah Zaranek, 10/05/2022 02:00 PM

1 20 Sarah Zaranek
h1. Use Cases for WB and Collections/Projects (Keep)
2 2 Tom Clegg
3 16 Sarah Zaranek
Note: these are not full use cases, simplified to just give more of the what and not a lot of the why or who.  We will be boiling these up into higher-level use cases.  This is more a list of items that should be included in all those higher-level use cases.  We want to not only ensure these use cases can be done in WB2 but the experience is equivalent or hopefully better in WB2 than in WB1. 
4 1 Sarah Zaranek
5 16 Sarah Zaranek
*Creating New Collections*
6
Summary: Users want to create a new collection in WB. 
7
Details:  Wants to make collection in from any of the data sources:
8 17 Sarah Zaranek
* From data already in an existing collection
9
* From a subset of files in an existing collection
10
* By combining files from different subsets of different collections
11
* By Uploading files from my desktop
12
* From the results of a already run workflow
13
* By downloading data from another source into a collection
14 1 Sarah Zaranek
15 16 Sarah Zaranek
*Annotating Collections*
16
Summary: User wants to annotate their collections with such details as sample
17
name, sample type, sequencing method, species, etc. 
18
Details:
19 17 Sarah Zaranek
* User wants to add annotations/metadata to existing collection
20
* User want to verify the added metadata to the collection
21 1 Sarah Zaranek
22 17 Sarah Zaranek
*Examining Collection Files* 
23 16 Sarah Zaranek
Summary:  User is viewing a collection and wants to examine the files and extract the data or metadata. 
24
Details:
25 17 Sarah Zaranek
* User wants to find certain files within the collection 
26
* (visually,using search (basic and advanced search using regexp, etc)
27
* User want to View or download certain files with the collection using a UI
28 1 Sarah Zaranek
29 18 Sarah Zaranek
*Sharing Collection Files* 
30 16 Sarah Zaranek
Summary:  User wants to share the Files in a Collection
31 17 Sarah Zaranek
Details:
32
* User wants make the collection available for downloading via ftp, s3 or other 3rd party application
33 18 Sarah Zaranek
* User wants to share the collection with others (by name) in their organization, others via a defined group, to everyone in their organization, or publicly.   Users want those they share the collection with to have: Read permission, Write permissions, Manage permissions
34 1 Sarah Zaranek
35 18 Sarah Zaranek
*Finding Existing Collections*
36 16 Sarah Zaranek
Summary: User want to find an existing collection
37
Details:  User wants to find the collection by one of the following ways:
38 18 Sarah Zaranek
* By Browsing to the project that contains the collection
39
* By searching the UUID of the project - and then looking through the items in the collection
40
* By searching the UUID of the collection
41
* By searching the collection name
42
* By searching the PDH
43
* By searching the collection metadata
44
* By searching for the workflow that produced it
45
* By a URL that can share that points to the project on WB
46 1 Sarah Zaranek
47 16 Sarah Zaranek
Identifying Collections
48
Summary: User needs to ID the collection for use as inputs to a workflow and/or inputs to a command using the API or CLI
49
Details:  User wants to find these specific file IDs
50
User wants to get the collection UUID
51
User wants to get the collection PDH
52
User wants to get the collection metadata
53 1 Sarah Zaranek
54 16 Sarah Zaranek
Editing an Existing Collection 
55
Summary: User wants to modify a collection for use in their new project.
56
Details: User will want to
57
Copy over an existing collection to a new project
58
Remove files in their collection to suit their needs for their new bioinformatics task. 
59
Add files from either their desktop or from another collection into this new collection
60 1 Sarah Zaranek
61 16 Sarah Zaranek
Editing the File Structure of An Existing Collection 
62
Summary:  User wants to adjust the file structure of an existing collection to work better as an input to their analysis tools
63
Details:  User will want to
64
Remove files from folders to have them in all in a single flat directory
65
Separate different subsets of files and place them in folders
66 1 Sarah Zaranek
67 16 Sarah Zaranek
Verifying Correct Collection
68
Summary: User wants to download data from a collection, and want to make sure the collect they found is the correct collection
69
Details:  User may want to check the following:
70
UUID and/or PDH
71
metadata
72
file contents
73
collection “version”
74
lineage of the collection (i.e. if the collection was generated as the result of a workflow - which workflow created this collection)
75 1 Sarah Zaranek
76 16 Sarah Zaranek
Creating Projects
77
Summary: User is working in the Arvados Workbench and needs a new Arvados project for a new analysis project they are working on. 
78
Details: The user wants to do the following
79
Create new project
80
Create a subproject within the new project (e.g. For Testing vs Final Runs)
81
Name the project/subproject
82
Add description of the project
83
Add metadata to the project  
84
Extract UUID of Project for use in workflow inputs or various command line/SDK/API functionality. 
85
Copy a URL that can share to others that points to the project
86
Mark the projects as “favorites” to be able to it more easily
87 1 Sarah Zaranek
88 16 Sarah Zaranek
Moving or Creating Items within Project
89
Summary. A User wants to set up their project to do their work.  They want to move all relevant existing data and workflows as well as upload new data and create new registered workflows.  
90
Details: Users may want to:
91
Copy existing collections and register workflows into this project
92
Create new collections within the project
93
Create new registered workflows in that project
94
Run workflows in this projects having the output, logs and other created artifacts contained within this project
95 1 Sarah Zaranek
96 16 Sarah Zaranek
Archiving or Sharing a Project
97
Summary: Now that the work is finished and the user has the results they need.  They want to get the project ready for sharing with their organization, with those outside their organization or even possible publicly in a publication.  
98
Details: Users may want to
99
Clean up the project by removing old collections, logs, processes and subdirectories that are not necessary to keep. 
100
Edit the name or metadata for the project
101
Freeze the project
102
Share the project with others in their organization, to everyone in their organization, publicly.   These are possible configurations:
103
Others have read permission  
104
Others have write permissions
105
Others have manage permissions
106 1 Sarah Zaranek
107 16 Sarah Zaranek
Identifying an Existing Project
108
Summary: User needs to identify the correct project in which to run their workflow.  They think they found the correct project they want searching on the project name.  
109
Details: The user then may want to:
110
Find the UUID of the project
111
Examine aspects of the project to double check it is the project they want to use.  They might want to:
112
Check the contents of the project 
113
Check the project description 
114
User might want to see the history of the project (*not currently available)
115
User might want to see the metadata for the project
116
Check to see if the project is frozen
117 1 Sarah Zaranek
118 16 Sarah Zaranek
Finding a Project or Subproject
119
Summary: A user logs back into Arvados after a break and would like to find the project they were working on previously. 
120
Details:  Those users would may want to find this project by
121
Searching for UUID, metadata or the project name
122
Navigating through the project/subproject hierarchy to find the project
123
Skimming through projects marked as their “Favorites”
124
Look for projects owned or created by a specific user (* Not Currently available)
125 1 Sarah Zaranek
126 19 Sarah Zaranek
h1. Use Cases for WB and Workflows (Crunch)
127 1 Sarah Zaranek
128 16 Sarah Zaranek
Submitting Workflows to Arvados Without Command Line
129
Summary: User wants to submit a workflow to run on Arvados without having to use the command line. The workflow CWL file either lives in a git repository or on the user’s local machine or in a collection on Arvados.  (* Currently this is not available on WB)
130
Details: The user will want to indicate to Arvados which workflow they want to run and have Arvados 
131
Parse the CWL file and generate an interactive form to fill out with input values and then submit the workflow via Workbench
132
Upload a YML file of input values to upload or direct Arvados to an existing YML file in an Arvados collection and then submit the workflow via Workbench. 
133 1 Sarah Zaranek
134 16 Sarah Zaranek
Monitoring Submitted Workflows 
135
Summary: User submits a workflow to run on Arvados using the command line. They want to monitor the workflow that they submitted using the Arvados Workbench. 
136
Details: The User will want to
137
Find the running workflow
138
Through search
139
Via Project Navigation
140
Check the workflow’s current status
141
If run successfully, find output collection 
142
If run successfully, look at how long it took to run
143
If run successfully, estimate costs (* currently only available for CommandLine)
144 1 Sarah Zaranek
145 16 Sarah Zaranek
Debugging Workflows - Part I
146
Summary: User submits a workflow to run on Arvados using the command line. They find out that the workflow and it did not run not run successfully.   They want to figure out why it failed.  
147
Details: The User may wants to:
148
Examine logs
149
Examine inputs
150
Examine command
151
Examine CWL  (* CWL not yet available)
152
Check to see which docker container was used
153
Run crunch-run stats interactively on WB  (*Currently only available via the command line) or have those plots available on WB. 
154 1 Sarah Zaranek
155 16 Sarah Zaranek
Debugging Workflows - Part II
156
Summary:  A User is updating a workflow workflow and the workflow has now stops running after a series of changes.  The User needs to compare the new workflow to the old workflow to see why it failed.
157
Details:  The User may need to 
158
Examine old workflow runs and backtrack to see when was the workflow last working
159
Determine what is the difference between the last working workflow and the broken workflows main process . This could include comparing: Inputs, Command, Resource Allocation, Node Type, and Docker Image metadata.
160
Determine which steps (if any) were re-used from the working workflow
161
Determine which steps failed in the new workflows
162
Determine if any big differences between the different workflows steps
163
Look at information passed to the steps including Inputs, Command, Resource Allocation
164
Find and compare logs from the relevant workflow steps Compare Docker image metadata (docker image ID, name, version, dockerfile)
165
Need to compare compare crunch-run stats between jobs (*Currently only available via command line)
166
167
Integrating Git Commit Information with Submitted Workflow
168
User has been working on changes to an existing workflow.  They have been submitting workflows managed in a git repository.  The workflow stops working or returning different outputs. The User would like to 
169
Find the relevant git commit information for the commit version of the workflow run on Arvados that worked
170
Find the relevant git commit information for the version of the workflow that is returning different results or not working
171
172
Calculating Workflow Costs
173
Summary: A User ran a big job and is now worried about how much it cost to run.  Details: The user wants to
174
Find the container request UUID of the workflow  
175
Run the cost analyzer to see how much it cost to run the entire workflow 
176
(* Currently not available in WB)
177
Run the cost analyzer to see which step is costing the most money
178
(* Currently not available in WB)
179
See if it would be possible to run workflow on less expensive instances (using information now only possible from the command line)
180
Estimate how much would it cost to run another similar workflow
181
 (* Currently not available in Arvados)
182
183
Creating a Register Workflow Using Workbench
184
Summary: User wants to create a registered workflow interactively using the WB (* Only currently available using command line).
185
Details.  The user may want to:
186
Share registered workflow with others in their organization
187
Set default values for the workflow
188
Define the name, metadata and descriptive text for the register workflow
189
190
Finding a Registered Workflow
191
Summary: User wants to run a registered workflow created by their colleague. User needs to find this workflow.
192
Details: The user may want to find the workflow by
193
Name, metadata or unique identifier
194
A URL shared by the creator of the register workflow
195
Looking Inside “project” that contains all the available shared workflows
196
197
Specifying Inputs to a Registered Workflow
198
Summary:  User wants to use the WB to submit a registered workflow made by 
199
their collaque to run on Arvados .  
200
Details: The user wants to:
201
Specify the inputs for the register workflow to use
202
Use the default inputs (if provided)
203
Identify the project the workflow should “run” in (i.e. where the outputs, logs and other collections should be stored)
204
See details about which registered workflow they are running to help guide them with to provide the proper input (eg. Registered workflow , name, description, etc.)
205
206
Examining A Running Workflow
207
Summary: User notices that their workflow has been running a long time and wants to check up on it.They would like to figure out if workflow is hung or is progressing along.  If progressing, the user may want to try and figure out why the workflow is running slower than expected.
208
Details:  The User may want to examine the following on WB
209
Real time logs
210
Resources used
211
Steps run so far and their outputs/inputs
212
Real-time values of RAM, CPU usage etc currently 
213
(* Not available in WB)
214
215
 Rerunning Old Workflows  
216
Summary: A User has successfully run a workflow on Arvados using the command line and would like to rerun that old workflow. 
217
(*Not Currently Available in WB)
218
Details: User may want to specify:
219
New Inputs
220
New Resource Requirements
221
New Docker Container
222
223
 Canceling a Running or Queued Workflow
224
Summary: User has submitted a workflow to Arvados via the command line and releases that they accidentally used the wrong inputs or an outdated function that the workflow leverages.  They want to cancel the workflow before it wastes time and resources.
225
Details:  User would like to
226
Find the running workflow on WB
227
Have an easy (single-button) way to cancel workflow on WB
228
Have a way to verify that the workflow was canceled
229
230
Checking Workflow Inputs or Requirements
231
Summary: User is examining output results of a workflow and wants to remind themselves which input parameter they used for the model they ran in the workflow. 
232
Details:  User would like to
233
Trace output collection to process that created it (i.e. container)
234
Find and examine the inputs to the workflow or workflow step run in that container