Pipeline Optimization » History » Revision 2
Revision 1 (Bryan Cosca, 04/14/2016 03:08 PM) → Revision 2/31 (Bryan Cosca, 04/14/2016 03:08 PM)
h1. Pipeline Optimization h2. Crunchstat Summary h3. How to install crunchstat-summary h3. How to use crunchstat-summary --text mode using node recommendations, keep cache size --html mode check if you're cpu/io bound check if tasks are being weird, i.e. gatk queue case when to pipe and when to write to keep in general writing straight to keep will reap benefits. If you run crunchstat-summary --html and you see keep io stopping once in a while, then youre cpu bound. h3. How to optimize the number when you don't have native multithreading tools like gatk, blah blah have native multithreading where you pass a -t. tools like varscan/freebayes blah blah don't have native multithreading so you need to find a workaround. generally, some tools have a -L --intervals to pass in certain loci to work on. If you have a bed file you can split on, then you can create a new task per interval. example here h3. piping between tools or writing to a tmpdir. h3. choosing the right number of jobs each job must output a collection, so if you don't want to output a file, then