https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422021-10-21T19:52:21ZArvadosArvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978702021-10-21T19:52:21ZWard Vandewegeward@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/97870/diff?detail_id=94532">diff</a>)</li></ul> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978772021-10-21T20:46:39ZWard Vandewegeward@curii.com
<ul></ul><p>I ran a test job on 9tee4 after removing <code>user_allow_other</code> from /etc/fuse.conf on the compute nodes:</p>
<pre><code><a class="external" href="https://workbench.9tee4.arvadosapi.com/container_requests/9tee4-xvhdp-vv11wi7y4t7276m">https://workbench.9tee4.arvadosapi.com/container_requests/9tee4-xvhdp-vv11wi7y4t7276m</a></code></pre>
<p>It failed with</p>
<pre>
2021-10-21T19:00:33.509718889Z fusermount: option allow_other only allowed if 'user_allow_other' is set in /etc/fuse.conf
2021-10-21T19:00:33.512267359Z 2021-10-21 19:00:33 arvados.arv-mount[2698109] ERROR: arv-mount: exception during mount: fuse_mount failed
2021-10-21T19:00:33.512267359Z Traceback (most recent call last):
2021-10-21T19:00:33.512267359Z File "/usr/share/python3/dist/python3-arvados-fuse/lib/python3.7/site-packages/arvados_fuse/command.py", line 386, in _run_standalone
2021-10-21T19:00:33.512267359Z with self:
2021-10-21T19:00:33.512267359Z File "/usr/share/python3/dist/python3-arvados-fuse/lib/python3.7/site-packages/arvados_fuse/command.py", line 141, in __enter__
2021-10-21T19:00:33.512267359Z llfuse.init(self.operations, native_str(self.args.mountpoint), self._fuse_options())
2021-10-21T19:00:33.512267359Z File "src/fuse_api.pxi", line 246, in llfuse.init
2021-10-21T19:00:33.512267359Z RuntimeError: fuse_mount failed
2021-10-21T19:00:32.592708107Z Not starting a gateway server (GatewayAuthSecret was not provided by dispatcher)
2021-10-21T19:00:32.592916306Z crunch-run 2.3.0~dev20211008165008 (go1.17.1) started
2021-10-21T19:00:32.592937800Z Executing container '<a href="https://arvadosapi.com/9tee4-dz642-381zm4m9v1xxhjl">9tee4-dz642-381zm4m9v1xxhjl</a>' using singularity runtime
2021-10-21T19:00:32.592963683Z Executing on host 'compute1.9tee4.arvadosapi.com'
2021-10-21T19:00:32.707807725Z container token "v2/9tee4-gj3su-15pcdcsz9xj1tcg/6dl6cdean4nbqkxq1zawsjcu6annp4nz5dl5brzs1gylzk5ulo/9tee4-dz642-381zm4m9v1xxhjl"
2021-10-21T19:00:32.708404569Z Running [arv-mount --foreground --allow-other --read-write --storage-classes default --crunchstat-interval=10 --file-cache 268435456 --mount-tmp tmp0 --mount-by-pdh by_id --mount-by-id by_uuid /tmp/crunch-run.<a href="https://arvadosapi.com/9tee4-dz642-381zm4m9v1xxhjl">9tee4-dz642-381zm4m9v1xxhjl</a>.3577839884/keep1214158095]
2021-10-21T19:00:33.582210613Z Arv-mount exit error: exit status 1
</pre> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978782021-10-21T20:53:14ZWard Vandewegeward@curii.com
<ul><li><strong>Assigned To</strong> set to <i>Ward Vandewege</i></li><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li></ul> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978792021-10-21T20:55:06ZWard Vandewegeward@curii.com
<ul><li><strong>Release</strong> set to <i>42</i></li></ul> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978802021-10-21T20:55:41ZWard Vandewegeward@curii.com
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/97880/diff?detail_id=94541">diff</a>)</li></ul> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978812021-10-21T21:03:00ZWard Vandewegeward@curii.com
<ul></ul><p><a class="changeset" title="18289: documentation: singularity refinements. Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wa..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/9f6f07fe6790e7c3a8f1b57990c16447c9d2713f">9f6f07fe6790e7c3a8f1b57990c16447c9d2713f</a> on branch 18289-only-pass-allow-other-when-running-docker</p>
<p>I've tested that this works with a custom crunch-run binary on 9tee4, where I removed allow_other from /etc/fuse.conf:</p>
<p>without patch: <a class="external" href="https://workbench.9tee4.arvadosapi.com/container_requests/9tee4-xvhdp-quwtm8w4oia0cu5">https://workbench.9tee4.arvadosapi.com/container_requests/9tee4-xvhdp-quwtm8w4oia0cu5</a><br />with patch: <a class="external" href="https://workbench.9tee4.arvadosapi.com/container_requests/9tee4-xvhdp-i26ori7gjxhr6hx">https://workbench.9tee4.arvadosapi.com/container_requests/9tee4-xvhdp-i26ori7gjxhr6hx</a></p>
<p>Tests passed at <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/2738/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/2738/">developer-run-tests: #2738 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=2738" alt="" /></a></a></p> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978832021-10-22T13:31:42ZTom Cleggtom@curii.com
<ul></ul><p>LGTM, thanks!</p> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978842021-10-22T14:31:49ZWard Vandewegeward@curii.com
<ul></ul><p>Documentation fixes pushed at <a class="changeset" title="18289: Documentation changes to reflect we support both Slurm and LSF. Refactor the comput..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/01698bea4703ce073425e2080c7cad83e2f873cc">01698bea4703ce073425e2080c7cad83e2f873cc</a> on branch 18289-only-pass-allow-other-when-running-docker</p> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978852021-10-22T14:51:53ZTom Cleggtom@curii.com
<ul></ul><p>Looks like these link targets are reversed in doc/install/crunch2-slurm/install-test.html.textile.liquid</p>
<pre><code>Make sure all of your compute nodes are set up with "Docker":../crunch2/install-compute-node-singularity.html or "Singularity":../crunch2/install-compute-node-docker.html.</code></pre>
<p>Errant <code>'</code> char in doc/install/crunch2/install-compute-node-singularity.html.textile.liquid</p>
<pre>
# "Install'python-arvados-fuse and crunch-run":#install-packages
</pre>
<p>Also in doc/install/crunch2/install-compute-node-singularity.html.textile.liquid there is a link to "Set up a Slurm compute node with Docker":install-compute-node-docker.html -- that title should not include "Slurm" any more. The "introduction" section also shouldn't say Slurm.</p>
<p>It seems a bit odd to divide the fairly small amount of singularity instructions into doc/install/singularity and doc/install/crunch2/install-compute-node-singularity -- could we move the doc/install/singularity information into the new install-compute-node page instead of linking to it?</p> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978862021-10-22T16:50:54ZWard Vandewegeward@curii.com
<ul></ul><p>Tom Clegg wrote:</p>
<blockquote>
<p>Looks like these link targets are reversed in doc/install/crunch2-slurm/install-test.html.textile.liquid<br />Make sure all of your compute nodes are set up with "Docker":../crunch2/install-compute-node-singularity.html or "Singularity":../crunch2/install-compute-node-docker.html.</p>
<p>Errant <code>'</code> char in doc/install/crunch2/install-compute-node-singularity.html.textile.liquid</p>
<p>[...]</p>
<p>Also in doc/install/crunch2/install-compute-node-singularity.html.textile.liquid there is a link to "Set up a Slurm compute node with Docker":install-compute-node-docker.html -- that title should not include "Slurm" any more. The "introduction" section also shouldn't say Slurm.</p>
</blockquote>
<p>Thanks, I fixed all that.</p>
<blockquote>
<p>It seems a bit odd to divide the fairly small amount of singularity instructions into doc/install/singularity and doc/install/crunch2/install-compute-node-singularity -- could we move the doc/install/singularity information into the new install-compute-node page instead of linking to it?</p>
</blockquote>
<p>Yes, good idea, I've made that change.</p>
<p>Latest in <a class="changeset" title="18289: add support to set `mksquashfs mem` for Singularity in our compute node image build..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/c5e4fb5838d2f447ae126159a71340b90cfea33c">c5e4fb5838d2f447ae126159a71340b90cfea33c</a> on branch 18289-only-pass-allow-other-when-running-docker</p> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978902021-10-22T18:17:04ZTom Cleggtom@curii.com
<ul></ul><p>LGTM</p> Arvados - Bug #18289: [crunch] allow_other is not required when using Singularityhttps://dev.arvados.org/issues/18289?journal_id=978922021-10-22T18:22:06ZWard Vandewegeward@curii.com
<ul><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li></ul><p>Applied in changeset arvados-private:commit:arvados|2582dc22a24ee7cdaf1a68c6b4b1c639f88c2efe.</p>