https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422016-04-12T17:37:30ZArvadosArvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=377512016-04-12T17:37:30ZSarah Guthriesallyeguthrie@gmail.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/37751/diff?detail_id=36835">diff</a>)</li><li><strong>Story points</strong> set to <i>1.0</i></li></ul> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=378362016-04-13T17:01:33ZSarah Guthriesallyeguthrie@gmail.com
<ul><li><strong>Target version</strong> changed from <i>Pipeline Future Sprints</i> to <i>2016-04-27 sprint</i></li></ul> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=378492016-04-13T19:08:48ZBryan Coscabcosca@curii.com
<ul><li><strong>Assigned To</strong> set to <i>Bryan Cosca</i></li></ul> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=380082016-04-19T14:57:24ZBryan Coscabcosca@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=380812016-04-19T20:40:47ZBryan Coscabcosca@curii.com
<ul></ul><p>Ready for review: <a class="external" href="https://dev.arvados.org/projects/arvados/wiki/Pipeline_template_development">https://dev.arvados.org/projects/arvados/wiki/Pipeline_template_development</a></p> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=381332016-04-20T20:24:38ZSarah Guthriesallyeguthrie@gmail.com
<ul></ul><p>We should not include "arvados_sdk_version" in the main example given the reported bugs in installing it. Mentioning it and stating that it's a good idea to require the arvados sdk to be installed in the docker image.</p>
<p>It would be awesome to connect the script_parameters (required, default, dataclass) with the effects they have on the workbench view (from a user's perspective).</p> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=381532016-04-21T14:59:51ZBryan Coscabcosca@curii.com
<ul></ul><p>I've updated the wiki to remove arvados_sdk_version and added screenshots for script_parameters.</p> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=381612016-04-21T16:16:03ZSarah Guthriesallyeguthrie@gmail.com
<ul></ul><p>Feel free to link to this page if it will help you at all: <a class="external" href="https://dev.arvados.org/projects/arvados/wiki/Writing_a_Script_Calling_a_Third_Party_Tool">https://dev.arvados.org/projects/arvados/wiki/Writing_a_Script_Calling_a_Third_Party_Tool</a></p>
<p>We currently are missing a description of "components" in this page, which are a fairly large part of pipeline templates. Simply adding a paragraph saying that pipeline templates are composed of a dictionary of components and that each component maps to a job would be helpful. We can then state that the rest of the document describes the specific pieces of a component.</p>
<p>The main example still has arvados_sdk_version defined.</p>
<p>The introductory paragraph under "Writing script parameters" should talk about the "required" flag so it doesn't come as a surprise later.</p>
<p>"yields this example" is fairly vague. Saying that a particular pipeline template yields a pipeline instance is more specific and accurate.</p>
<p>How does a dataclass "File" influence the view? What about the dataclass "number"?</p>
<p>The following is inconsistent enough to be confusing:</p>
<blockquote>
<p>The inputs tab in the pipeline instance page shows all the required parameters.<br />...<br />For the 'additional_params' parameter, since its not required, its in the 'Components' tab, where you can set it:</p>
</blockquote>
<p>Maybe try?</p>
<blockquote>
<p>The "Inputs" tab in the pipeline instance page shows all the required parameters.<br />...<br />The "Components" tab in the pipeline instance page shows all the parameters. Thus it is the only place where non-required parameters, such as 'additional_params' may be set.</p>
</blockquote>
<p>Why is "read_group" highlighted in red in the Components tab, but nothing else is? That will likely be distracting to a new viewer.</p>
<blockquote>
<p>One runtime constraint is docker_image.</p>
</blockquote>
<p>What does docker_image control? You describe hints for it, but don't say what it actually does. Is it required? What needs to be in the docker image?</p>
<blockquote>
<p>The max_tasks_per_node parameter will allow you to allocate more computations on your node</p>
</blockquote>
<p>What are "computations"? We need to use more specific language here. What happens if max_tasks_per_node is equal to 2? When will multiple jobs be scheduled on that node? When will multiple tasks be scheduled on that node?</p> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=381832016-04-21T19:14:28ZBryan Coscabcosca@curii.com
<ul></ul><blockquote>
<p>Why is "read_group" highlighted in red in the Components tab, but nothing else is? That will likely be distracting to a new viewer.</p>
</blockquote>
<p>I don't know. I could hack it by making the template set it as false, but this is what it looks like when a Text/number parameter is 'required'.</p>
<p>Everything else has been added.</p> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=381882016-04-21T19:45:46ZSarah Guthriesallyeguthrie@gmail.com
<ul></ul><p>Alright, I'm happy with this. Brett is up next.</p> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=384612016-04-29T14:36:08ZBryan Coscabcosca@curii.com
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li></ul> Arvados - Idea #8884: [Docs] Pipeline author guide gives a basic demonstration of writing a pipeline templatehttps://dev.arvados.org/issues/8884?journal_id=384632016-04-29T14:36:56ZBryan Coscabcosca@curii.com
<ul></ul><p>resolving due to time constraints and brett does not have time to review and sally reviewed already</p>