https://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422020-01-30T10:44:43ZArvadosArvados - Bug #16100: [keep-web] Avoid sniffing for content type when file extension matches a MIME typehttps://dev.arvados.org/issues/16100?journal_id=814802020-01-30T10:44:43ZMichael Crusoe
<ul></ul><p>Tom Clegg wrote:</p>
<blockquote>
<p>Observed behavior: A file called "bmx.txt" containing the text "BMX bikes are awesome.\n" is currently served with <code>Content-Type: image/bmp</code> because the first two bytes "BM" satisfy the signature for a BMP image file, and this causes it to render incorrectly in the browser.</p>
</blockquote>
<p>FYI, the unix "file" command correctly identifies said file:</p>
<pre>
$ echo "BMX bikes are awesome.\n" > bmx.txt
$ file --mime bmx.txt
bmx.txt: text/plain; charset=us-ascii
$ file --version
file-5.37
</pre> Arvados - Bug #16100: [keep-web] Avoid sniffing for content type when file extension matches a MIME typehttps://dev.arvados.org/issues/16100?journal_id=824782020-02-14T18:06:14ZTom Cleggtom@curii.com
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>In Progress</i></li><li><strong>Description</strong> updated (<a title="View differences" href="/journals/82478/diff?detail_id=79199">diff</a>)</li></ul> Arvados - Bug #16100: [keep-web] Avoid sniffing for content type when file extension matches a MIME typehttps://dev.arvados.org/issues/16100?journal_id=824932020-02-14T20:09:24ZTom Cleggtom@curii.com
<ul></ul><p>16100-mime-types @ <a class="changeset" title="16100: Add package dependency for /etc/mime.types. Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/46d2ed57248419200d5716cfef8de9a1bb911240">46d2ed57248419200d5716cfef8de9a1bb911240</a> -- <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1733/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1733/">developer-run-tests: #1733 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=1733" alt="" /></a></a></p> Arvados - Bug #16100: [keep-web] Avoid sniffing for content type when file extension matches a MIME typehttps://dev.arvados.org/issues/16100?journal_id=824972020-02-14T20:56:40ZLucas Di Pentimalucas.dipentima@curii.com
<ul></ul><p>Although Jenkins says it's all fine, I've ran the <code>services/keep-web</code> tests on my dev VMs (debian9 & debian10) and I'm getting a failure like this:</p>
<pre>
[...]
{"health":"OK"}
arv-git-httpd pid 11288 ok
{"health":"OK"}
{"health":"OK"}
ws pid 11304 ok
ARVADOS_TEST_PROXY_SERVICES=1
ARVADOS_API_TOKEN=4axaw8zxe0qm22wa6urpp5nskcne8z88cvbupv653y1njyi05h
ARVADOS_CONFIG=/media/psf/arvados/tmp/arvados.yml
ARVADOS_API_HOST=0.0.0.0:45751
ARVADOS_TEST_API_INSTALLED=10501
ARVADOS_TEST_API_HOST=0.0.0.0:54431
ARVADOS_API_HOST_INSECURE=true
======= test services/keep-web
time="2020-02-14T17:52:07-03:00" level=error msg="stat.Size()==3 but only wrote 0 bytes; read(1024) returns 0, GET acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77 failed: [http://localhost:39073/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: Get http://localhost:39073/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: dial tcp [::1]:39073: connect: connection refused http://localhost:33737/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: Get http://localhost:33737/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: dial tcp [::1]:33737: connect: connection refused http://localhost:39073/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: Get http://localhost:39073/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: dial tcp [::1]:39073: connect: connection refused http://localhost:33737/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: Get http://localhost:33737/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: dial tcp [::1]:33737: connect: connection refused http://localhost:39073/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: Get http://localhost:39073/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: dial tcp [::1]:39073: connect: connection refused http://localhost:33737/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: Get http://localhost:33737/acbd18db4cc2f85cedef654fccc4a4d8+3+A2b8e58eafe2fb5db03583e062bd1aa7871103fa8@5e597d77: dial tcp [::1]:33737: connect: connection refused]"
2020/02/14 17:52:09 authSettings: map[ARVADOS_API_HOST:0.0.0.0:54431 ARVADOS_API_HOST_INSECURE:true ARVADOS_API_TOKEN:4axaw8zxe0qm22wa6urpp5nskcne8z88cvbupv653y1njyi05h]
child.pid is 11450
child.pid is 11462
{"RequestID":"req-1rjf1w6yc4zgsu5kds3s","level":"info","msg":"request","remoteAddr":"127.0.0.1:50260","reqBytes":0,"reqForwardedFor":"","reqHost":"<a href="https://arvadosapi.com/zzzzz-4zz18-znoi8dpi452e9qi">zzzzz-4zz18-znoi8dpi452e9qi</a>.collections.example.com:36683","reqMethod":"GET","reqPath":"testdata.bin","reqQuery":"","time":"2020-02-14T17:52:12.078006301-03:00"}
[...]
</pre>
<p>Thought that I might have some local problem, but current <code>master</code> runs OK.</p> Arvados - Bug #16100: [keep-web] Avoid sniffing for content type when file extension matches a MIME typehttps://dev.arvados.org/issues/16100?journal_id=824992020-02-16T00:59:51ZTom Cleggtom@curii.com
<ul></ul><p>Here, I see that error in the logs but it doesn't cause a test failure. The test does a GET request for a file whose content can't be retrieved. The handler only fails after it's returned 200, and the test doesn't check the response body. Changing it to an integration test makes the logged error go away:</p>
<p>16100-mime-types @ <a class="changeset" title="16100: Move test to integration suite to avoid logging errors. Arvados-DCO-1.1-Signed-off-by: To..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/3836d53ef13841dad652e3faeb20660576279afd">3836d53ef13841dad652e3faeb20660576279afd</a> -- <a class="external" href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1735/"<a href="https://ci.arvados.org/view/Developer/job/developer-run-tests/1735/">developer-run-tests: #1735 <img src="https://ci.arvados.org/buildStatus/icon?job=developer-run-tests&build=1735" alt="" /></a></a></p> Arvados - Bug #16100: [keep-web] Avoid sniffing for content type when file extension matches a MIME typehttps://dev.arvados.org/issues/16100?journal_id=825022020-02-17T16:33:22ZLucas Di Pentimalucas.dipentima@curii.com
<ul></ul><p>Thanks. Locally I was getting a test failure with errorlevel=29 when running the tests like this:</p>
<pre>
~/arvados/build/run-tests.sh WORKSPACE=~/arvados CONFIGSRC=~/arvados-test-config --temp ~/.cache/arvados-build --only services/keep-web --skip-install
</pre>
<p>Now this last fix makes the test pass, and correctly fail if I move the file <code>/etc/mime.types</code> to some other place, with the following message:</p>
<pre>
----------------------------------------------------------------------
FAIL: handler_test.go:919: IntegrationSuite.TestFileContentType
time="2020-02-17T13:23:06.120200890-03:00" level=warning msg="SystemRootToken missing from cluster config, falling back to ARVADOS_API_TOKEN environment variable"
time="2020-02-17T13:23:06.120236123-03:00" level=warning msg="Services.Controller.ExternalURL missing from cluster config, falling back to ARVADOS_API_HOST(_INSECURE) environment variables"
handler_test.go:974:
c.Check(resp.Header().Get("Content-Type"), check.Equals, trial.contentType)
... obtained string = "image/bmp"
... expected string = "text/plain; charset=utf-8"
handler_test.go:974:
c.Check(resp.Header().Get("Content-Type"), check.Equals, trial.contentType)
... obtained string = "image/bmp"
... expected string = "image/x-ms-bmp"
</pre>
<p>So this LGTM, thanks!!</p> Arvados - Bug #16100: [keep-web] Avoid sniffing for content type when file extension matches a MIME typehttps://dev.arvados.org/issues/16100?journal_id=825042020-02-17T18:55:36ZAnonymous
<ul><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li><li><strong>Status</strong> changed from <i>In Progress</i> to <i>Resolved</i></li></ul><p>Applied in changeset <a class="changeset" title="Merge branch '16100-mime-types' fixes #16100 Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@tomc..." href="https://dev.arvados.org/projects/arvados/repository/arvados/revisions/0a415b6c80c3bf39bb753274aae857eadde2f590">arvados|0a415b6c80c3bf39bb753274aae857eadde2f590</a>.</p> Arvados - Bug #16100: [keep-web] Avoid sniffing for content type when file extension matches a MIME typehttps://dev.arvados.org/issues/16100?journal_id=852762020-07-06T15:49:46ZWard Vandewegeward@curii.com
<ul><li><strong>Related to</strong> <i><a class="issue tracker-6 status-3 priority-4 priority-default closed parent" href="/issues/15348">Idea #15348</a>: [pam] PAM module in Go</i> added</li></ul>