https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422014-09-30T19:43:15ZArvadosArvados - Feature #4042: [Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)https://dev.arvados.org/issues/4042?journal_id=157032014-09-30T19:43:15ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/15703/diff?detail_id=14477">diff</a>)</li><li><strong>Category</strong> set to <i>Crunch</i></li></ul> Arvados - Feature #4042: [Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)https://dev.arvados.org/issues/4042?journal_id=157052014-09-30T19:46:51ZTom Cleggtom@curii.com
<ul><li><strong>Subject</strong> changed from <i>[Crunch] run-command script supports queueing MxN tasks (e.g., M directories x N chromosomes)</i> to <i>[Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/15705/diff?detail_id=14480">diff</a>)</li></ul> Arvados - Feature #4042: [Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)https://dev.arvados.org/issues/4042?journal_id=157072014-09-30T19:55:52ZTom Cleggtom@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/15707/diff?detail_id=14481">diff</a>)</li></ul> Arvados - Feature #4042: [Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)https://dev.arvados.org/issues/4042?journal_id=157922014-10-02T14:12:34ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Target version</strong> changed from <i>Arvados Future Sprints</i> to <i>2014-10-08 sprint</i></li></ul> Arvados - Feature #4042: [Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)https://dev.arvados.org/issues/4042?journal_id=157932014-10-02T14:12:48ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Assigned To</strong> set to <i>Peter Amstutz</i></li></ul> Arvados - Feature #4042: [Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)https://dev.arvados.org/issues/4042?journal_id=158072014-10-02T16:00:10ZPeter Amstutzpeter.amstutz@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul> Arvados - Feature #4042: [Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)https://dev.arvados.org/issues/4042?journal_id=159892014-10-06T15:26:17ZBrett Smithbrett.smith@curii.com
<ul></ul><a name="General"></a>
<h2 >General<a href="#General" class="wiki-anchor">¶</a></h2>
<p>In both error messages and documentation, "accessable" is misspelled; it should be "accessible."</p>
<a name="Code"></a>
<h2 >Code<a href="#Code" class="wiki-anchor">¶</a></h2>
<ul>
<li>Python dictionaries can use tuples as keys. Given this, I think <code>add_to_group</code> would be better off using <code>match.groups()</code> directly as the key. That way, there's no risk that different group matches will look identical because of conflicts with the join string, even as cute as <code>^_^</code> is.</li>
<li>There are a couple of places in <code>expand_item</code> that say <code>p = pattern.match(i)</code>. This masks the original <code>p</code> argument that was passed in. I don't think that causes any behavior bugs in the current code (we never try to read the original <code>p</code> after), but changing it now might help prevent heartache later.</li>
<li>This doesn't need to change for the branch, but in case it's helpful for the future, you might find <code>os.path.isdir()</code> and <code>os.path.isfile()</code> easier to use than mucking around with stat.</li>
</ul>
<a name="Documentation"></a>
<h2 >Documentation<a href="#Documentation" class="wiki-anchor">¶</a></h2>
<ul>
<li>The two run-command include examples are identical. <code>_run_command_foreach_example.liquid</code> doesn't use <code>foreach</code> anywhere.</li>
<li>Please run a spell checker on the run-command reference. Some typos I found: "evaluted", "substition", "commands’s", "concatinated." </li>
<li>The description of the first example says <code>run “echo” with the first argument “hello”</code>, but the code echoes <code>"hello world"</code>.</li>
<li>The description for <code>$(task.outdir)</code> seems to be cut off.</li>
<li>The second hyphen appears to be a typo in "The directory containing the source code for the run-command- script " </li>
<li>I think "Note that parameter expansion is not performed lists produced this way." should say "performed on lists." </li>
<li>"list context, as described below" - it's described above.</li>
<li>"list items functions" should say "list item" (singular) for consistency with the rest of the documentation.</li>
<li>Similarly, the index documentation should use monospace fonts for literal inputs and outputs, for consistency with the other function documentation. Plus Textile turns <code>--</code> into an em dash otherwise.</li>
<li>"write it output files" should say "its".</li>
</ul> Arvados - Feature #4042: [Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)https://dev.arvados.org/issues/4042?journal_id=161472014-10-08T15:17:14ZPeter Amstutzpeter.amstutz@curii.com
<ul></ul><p>Brett Smith wrote:</p>
<blockquote>
<a name="General"></a>
<h2 >General<a href="#General" class="wiki-anchor">¶</a></h2>
<p>In both error messages and documentation, "accessable" is misspelled; it should be "accessible."</p>
</blockquote>
<p>Fixed.</p>
<blockquote>
<a name="Code"></a>
<h2 >Code<a href="#Code" class="wiki-anchor">¶</a></h2>
<ul>
<li>Python dictionaries can use tuples as keys. Given this, I think <code>add_to_group</code> would be better off using <code>match.groups()</code> directly as the key. That way, there's no risk that different group matches will look identical because of conflicts with the join string, even as cute as <code>^_^</code> is.</li>
</ul>
</blockquote>
<p>I tried that with a list and it said "lists are unhashable"; I didn't realize it worked for tuples. Fixed.</p>
<blockquote>
<ul>
<li>There are a couple of places in <code>expand_item</code> that say <code>p = pattern.match(i)</code>. This masks the original <code>p</code> argument that was passed in. I don't think that causes any behavior bugs in the current code (we never try to read the original <code>p</code> after), but changing it now might help prevent heartache later.</li>
</ul>
</blockquote>
<p>Fixed.</p>
<blockquote>
<ul>
<li>This doesn't need to change for the branch, but in case it's helpful for the future, you might find <code>os.path.isdir()</code> and <code>os.path.isfile()</code> easier to use than mucking around with stat.</li>
</ul>
</blockquote>
<p>Good to know.</p>
<blockquote>
<a name="Documentation"></a>
<h2 >Documentation<a href="#Documentation" class="wiki-anchor">¶</a></h2>
<ul>
<li>The two run-command include examples are identical. <code>_run_command_foreach_example.liquid</code> doesn't use <code>foreach</code> anywhere.</li>
<li>Please run a spell checker on the run-command reference. Some typos I found: "evaluted", "substition", "commands’s", "concatinated." </li>
<li>The description of the first example says <code>run “echo” with the first argument “hello”</code>, but the code echoes <code>"hello world"</code>.</li>
<li>The description for <code>$(task.outdir)</code> seems to be cut off.</li>
<li>The second hyphen appears to be a typo in "The directory containing the source code for the run-command- script " </li>
<li>I think "Note that parameter expansion is not performed lists produced this way." should say "performed on lists." </li>
<li>"list context, as described below" - it's described above.</li>
<li>"list items functions" should say "list item" (singular) for consistency with the rest of the documentation.</li>
<li>Similarly, the index documentation should use monospace fonts for literal inputs and outputs, for consistency with the other function documentation. Plus Textile turns <code>--</code> into an em dash otherwise.</li>
<li>"write it output files" should say "its".</li>
</ul>
</blockquote>
<p>All fixed.</p> Arvados - Feature #4042: [Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)https://dev.arvados.org/issues/4042?journal_id=162862014-10-08T20:23:21ZBrett Smithbrett.smith@curii.com
<ul></ul><p>Peter Amstutz wrote:</p>
<blockquote><blockquote>
<ul>
<li>Python dictionaries can use tuples as keys. Given this, I think <code>add_to_group</code> would be better off using <code>match.groups()</code> directly as the key. That way, there's no risk that different group matches will look identical because of conflicts with the join string, even as cute as <code>^_^</code> is.</li>
</ul>
</blockquote>
<p>I tried that with a list and it said "lists are unhashable"; I didn't realize it worked for tuples. Fixed.</p>
</blockquote>
<p>The rule is "keys have to be immutable," since you'd get bizarre results if you put a key in a dictionary and then mutated it. Fortunately, mutability is the differentiator between lists and tuples.</p>
<p>Reviewing <a class="changeset" title="Updated examples." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/bdd309b073b6e836b78de28e82da89baba66a2a9">bdd309b</a>. Just a few small typos, go ahead and merge once these are fixed.</p>
<ul>
<li>The preface for the foreach example says "over multiple sample;" it should say "samples" plural.</li>
<li>The foreach example is missing a comma after the <code>sample</code> parameter hash.</li>
<li>Small capitalization fixes: I think case-sensitive names like "run-command" and "script_parameters" should always be written with their correct case (i.e., lowercase), even at the beginning of sentences, to avoid bad input. Please capitalize the proper name "Unix."</li>
</ul>
<p>Thanks.</p> Arvados - Feature #4042: [Crunch] run-command script supports running MxN tasks (e.g., M directories x N chromosomes)https://dev.arvados.org/issues/4042?journal_id=163102014-10-09T13:35:08ZAnonymous
<ul><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset arvados|commit:dd59a50f9f3933c359930806516b43899a8b4957.</p>