Idea #17049
closed
Make a single chromosome run of the demo
Description
https://github.com/arvados/arvados-tutorial
Run 5 samples on su92l, on a single chromosome.
Related issues
Updated by Peter Amstutz about 4 years ago
- Related to Idea #17001: Arvados uses WB2 by default added
Updated by Jiayong Li almost 4 years ago
- Status changed from New to Feedback
- Description updated (diff)
branch 17049-make-singlechrom, commit d2f6d47da38e96de4597bb58ba1193e261af8fb1
I finished a run for chr19 input fastqs for 5 samples.
https://workbench.su92l.arvadosapi.com/container_requests/su92l-xvhdp-3dqeqodgi1usk0r
I encountered a gatk bug that the memory usage spikes when writing result vcf in the haplotypecaller step, if the region is outside of chr19. (And in general, in regions when very few reads are placed.) I had to schedule 14G ram machines for that step to complete it. (See failure https://workbench.su92l.arvadosapi.com/container_requests/su92l-xvhdp-v0sq9csiz1p94nx)
Also I had to upload a different docker image for curii/clinvar-report to keep in order to finish the gvcf-to-vcf step (see failure https://workbench.su92l.arvadosapi.com/container_requests/su92l-xvhdp-zulp4y3e9f7tv59). The new docker image is https://workbench.su92l.arvadosapi.com/collections/su92l-4zz18-kl40n4746vmjnw2
Input fastqs here https://workbench.su92l.arvadosapi.com/collections/su92l-4zz18-j1jotbx4uqckkec
The run for generating the chr19 input fastqs is https://workbench.su92l.arvadosapi.com/container_requests/su92l-xvhdp-ilo79fgj728afkl
Updated by Peter Amstutz almost 4 years ago
So the goal of this was to have a relatively lightweight pipeline that still had all the pieces of the whole genome pipeline.
I think this still needs a little work to be lightweight --
I see dozens of haplotype caller instances that are each running for 1-2 minutes. Can we consolidate that?
Same for selectvariants/basecalibrator/applyBSQR. Can we either (a) not scatter or (b) use the RunInSingleContainer feature.
Updated by Jiayong Li almost 4 years ago
branch 17049-make-singlechrom, commit d2f6d47da38e96de4597bb58ba1193e261af8fb1
I changed the fullintervallist to reflect chr19 only calling, hopefully this run is better.
https://workbench.su92l.arvadosapi.com/container_requests/su92l-xvhdp-1l4x28f6q1rn955
Updated by Jiayong Li almost 4 years ago
- Status changed from Feedback to Resolved
commit d147d1d1fafeeea06bd09d9479337b0f5aab43b0
Added comments for the singe chrom chr19 yml.