Arvados

Review feedback:

Welcome to Arvados (user/index.html) page

- In “If you are new to Aravdos … “, the link “Run a pipeline” link goes to a page whose title does not match. It should probably be pointed to “Running a pipeline using workbench” section instead.

- Need a comma after “Robust storage of very large files, such as whole genome sequences”

- Extra space at the end of line “Storing and querying ”

Access Arvados Workbench (user/getting_started/workbench.html) page

- Should we add the email address “arvados@curoverse.com” one more time in the sentence “ontact the administrator of the Arvados instance . . . ” ?

Running a pipeline using Workbench

- The previous section says that the default is assumed to be localhost. However, the tutorials mentioned in this page most likely do not exist on someone’s localhost. Should we refer to qr1hi here?

Putting data into Keep (user/tutorials/tutorial-keep.html) section

- If someone is reading the tutorial from the beginning, this is the first time they are coming across Keep and hence just a one sentence intro does not seem sufficient. I think it is better to add some more detail about Keep and make it a separate first paragraph in this section. I think it would be even better if we have a “Introduction to Keep” section similar to “Introduction to Crunch”

- The sentence “Use arv-put actually add the file to Keep:” does not sound correct. You mean “Use arv-put to add the file to Keep:” ?

- Does the “Choose a project” link actually exist in the collections page? “ In order to place your newly uploaded file into a Project . . . ”

Getting data from Keep section

- “This tutorial covers using arv-ls and arv-get to access” is slightly confusing. Can it be “This tutorial covers the arv-ls and arv-get commands to access”

- This section and some other sections have the expectations “This tutorial assumes that you are logged into . . .” a little later into the page. Some other pages have it at the very top of the page. Would it make sense to have this blurb at the top of the page wherever it is included?

Mounting Keep as a file system

- “ To avoid having to fetch a potentially very large list of ” - I think the word “very” is unnecessary

How Keep works section

- I think this section can be titled “Introduction to Keep”. I think this should be the first section under “Working with data sets”

- “ This means that files are managed using special unique identifiers derived from the contents of the file, rather than human-assigned file names (specifically, the MD5 hash)” . I think “(specifically, the MD5 hash)” placement is incorrect. I think it should be placed before “rather than human-assigned …”. With the current placement, it feels like the MD5 hash thing is about human assigned names

- Missing comma in “In this example we will use”

Command line interface page

- I think this page needs an intro at the top of the page.

Writing a pipeline section

- Too much packed into one sentence in “Use the following command to create a new empty . . .” . I think it would be better if you said something like “Using this tutorial, you will create an empty template. You will then edit it . . . ”.

- I am not sure if the title of this section is descriptive enough. Should it be called “Writing a pipeline template” instead?

Writing a script section

- Should we call it “Writing a pipeline template to run a script”?

List of examples included with Arvados section

- how about section title “Scripts provided by Arvados”

- I clicked on the script links in this page and they all are correct

Radhika Chippada wrote:

Review feedback:

Welcome to Arvados (user/index.html) page

- In “If you are new to Aravdos … “, the link “Run a pipeline” link goes to a page whose title does not match. It should probably be pointed to “Running a pipeline using workbench” section instead.

Fixed

- Need a comma after “Robust storage of very large files, such as whole genome sequences”

- Extra space at the end of line “Storing and querying ”

Fixed

Access Arvados Workbench (user/getting_started/workbench.html) page

- Should we add the email address “arvados@curoverse.com” one more time in the sentence “ontact the administrator of the Arvados instance . . . ” ?

We don't currently have a default administrator email address in the doc configuration, I'm not sure if we want to provide the curoverse arvados support address by default, it's possible that a given Arvados instance isn't run by us.

Running a pipeline using Workbench

- The previous section says that the default is assumed to be localhost. However, the tutorials mentioned in this page most likely do not exist on someone’s localhost. Should we refer to qr1hi here?

That's configured with arvados_workbench_host when the docs are generated.

Putting data into Keep (user/tutorials/tutorial-keep.html) section

- If someone is reading the tutorial from the beginning, this is the first time they are coming across Keep and hence just a one sentence intro does not seem sufficient. I think it is better to add some more detail about Keep and make it a separate first paragraph in this section. I think it would be even better if we have a “Introduction to Keep” section similar to “Introduction to Crunch”

I went the other way and took out the mention of Keep by name to keep it task-focused.

- The sentence “Use arv-put actually add the file to Keep:” does not sound correct. You mean “Use arv-put to add the file to Keep:” ?

Fixed.

- Does the “Choose a project” link actually exist in the collections page? “ In order to place your newly uploaded file into a Project . . . ”

It only shows up in certain circumstances, but it should probably show up all the time: https://arvados.org/issues/3452

Getting data from Keep section

- “This tutorial covers using arv-ls and arv-get to access” is slightly confusing. Can it be “This tutorial covers the arv-ls and arv-get commands to access”

Fixed.

- This section and some other sections have the expectations “This tutorial assumes that you are logged into . . .” a little later into the page. Some other pages have it at the very top of the page. Would it make sense to have this blurb at the top of the page wherever it is included?

It should now consistently be the 2nd paragraph, after the 1st introductory paragraph.

Mounting Keep as a file system

- “ To avoid having to fetch a potentially very large list of ” - I think the word “very” is unnecessary

Fixed.

How Keep works section

- I think this section can be titled “Introduction to Keep”. I think this should be the first section under “Working with data sets”

This is intentional, we don't want to bog the user down with details if they just want to know how to use arv-put and arv-get.

- “ This means that files are managed using special unique identifiers derived from the contents of the file, rather than human-assigned file names (specifically, the MD5 hash)” . I think “(specifically, the MD5 hash)” placement is incorrect. I think it should be placed before “rather than human-assigned …”. With the current placement, it feels like the MD5 hash thing is about human assigned names

Fixed

- Missing comma in “In this example we will use”

Fixed

Command line interface page

- I think this page needs an intro at the top of the page.

I didn't update this at all, it does need work but I decided it was out of scope.

Writing a pipeline section

- Too much packed into one sentence in “Use the following command to create a new empty . . .” . I think it would be better if you said something like “Using this tutorial, you will create an empty template. You will then edit it . . . ”.

Fixed.

- I am not sure if the title of this section is descriptive enough. Should it be called “Writing a pipeline template” instead?

Fixed.

Writing a script section

- Should we call it “Writing a pipeline template to run a script”?

Changed to "Writing a crunch script"

List of examples included with Arvados section

- how about section title “Scripts provided by Arvados”

Fixed

Reviewed one more time.

- Writing a pipeline template page

1. I copied and pasted the "Tutorial align using bwa mem and SortSam" and tried to save my template (in qr1hi). I got the following error: Update failed. Server responded 422: ["#<ActiveRecord::UnknownAttributeError: unknown attribute: SortSam>"]

2. It would be nice if you could break up the "Use the following command to create a new empty template ... " as follows:

Use the following command to create a new empty template.

arv edit $(arv --format=uuid pipeline_template create --pipeline-template '{}') name components

. . .

Add the following content to the interactive text editor . . .

- Writing a crunch script page

1. When I ran the pipeline, I got the following error (shell vm pointing to qr1hi):
Error creating job for component do_hash: Docker image locator not found for arvados/jobs
Job submission was: {"job":{"script":"hash.py","script_parameters":{"input":"8b6e2c4916133e1d859c9e812861ce13+70"},"script_version":"master","repository":"radhika","output_is_persistent":true,"runtime_constraints":{"docker_image":"arvados/jobs"},"owner_uuid":"qr1hi-j7d0g-1jc55y5dxmhdtbq"},"find_or_create":true}

- Debugging a Crunch script page

1. When I ran the command arv-crunch-job --job "$(cat ~/the_job)", I got the following message.

Every node has failed -- giving up on this round

output d41d8cd98f00b204e9800998ecf8427e+0

- Parallel crunch tasks page worked all right.

- Writing a pipeline template page

Template creation passed. I saw failures when I tried to run an instance of this template, but I am not sure if you wanted me to run this or not. This could be because I did not select good input data. In case you want to see the instance is qr1hi-d1hrv-37glq780jxrnztw

- Writing a crunch script page

Ran the pipeline instance successfully after creating the template as listed in the doc.

- Debugging a Crunch script page

Ran the job successfully. Got the following output successfully.
2014-08-04_17:54:18 qr1hi-8i9sb-6slz89paadaio2e 30463 output 576c44d762ba241b0a674aa43152b52a+53+A93b3013e76ec949aa06b026bd6a7fff3a1b05843@53f23dc7