Running bcbio-nextgen using Arvados¶

WIP

Uploading data through the web and using it on Arvados¶

1. In your home project, click on the blue + Add data button in the top right.
2. Click Upload files from my computer
3. Click Choose Files and choose the 2 paired end fastq files you would like to run bcbio-nextgen on.
4. Once you're ready, click > Start
5. Feel free to rename your Collection so you can remember it later. Click on the pencil icon in the top left corner next to New collection
6. Once that is uploaded, navigate back to the dashboard and click on Run a pipeline... and choose bcbio-nextgen pipeline.
7. You can change the input by clicking on the [Choose] button next to the R1 parameter and R2 parameter.
8. Input your left fastq file as R1 and your right fastq file as R2.
9. For each fastq file, click on the dropdown menu, click on your newly-created project, and choose your desired input collection.
10. Click Run to run bcbio on your data!

Uploading data through your shell and using it on Arvados¶

Full documentation can be found here

1. Install the Arvados Python SDK on the system from which you will upload the data (such as your workstation, or a server containing data from your sequencer). Doing so will install the Arvados file upload tool, arv-put.
2. To configure the environment with the Arvados instance host name and authentication token, see here
3. Navigate back to your Workbench dashboard and create a new project by clicking on the Projects dropdown menu and clicking Home.
4. Click on [+ Add a subproject]. Feel free to edit the Project name or description by clicking the pencil to the right of the text.
5. To add data, return to your shell, create a folder, and put the two paired-end fastq files you want to upload inside. Use the command arv-put * --project-uuid qr1hi-xxxxx-yyyyyyyyyyyyyyy. The qr1hi tag can be found in the url of your new project. This ensures that all the files you would like to upload are in one collection.
6. The output value xxxxxxxxxxxxxxxxxxxx+yyyy is the Arvados collection locator that uniquely describes this file.
7. Once that is uploaded, navigate back to the dashboard and click on Run a pipeline... and choose bcbio-nextgen pipeline.
8. You can change the input by clicking on the [Choose] button next to the R1 parameter and R2 parameter.
9. Input your left fastq file as R1 and your right fastq file as R2.
10. For each fastq file, click on the dropdown menu, click on your newly-created project, and choose your desired input collection.
11. Click Run to run bcbio on your data!

FAQ¶

WIP