Support #16668

Load demo data from openslide

Added by Peter Amstutz about 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Estimated time:
Story points:
-

Description

http://openslide.cs.cmu.edu/download/openslide-testdata/

Load this into an Arvados instance somewhere

Need to unzip the zip files.

Separate project per technology.

Separate collection per slide.


Related issues

Related to Arvados - Story #16535: [keep-web] Minimal implementation of S3 APIResolved07/21/2020

History

#1 Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)

#2 Updated by Peter Amstutz about 1 year ago

  • Description updated (diff)

#3 Updated by Peter Amstutz about 1 year ago

  • Target version changed from 2020-08-26 Sprint to 2020-09-09 Sprint

#4 Updated by Peter Amstutz about 1 year ago

  • Assigned To set to Tom Clegg

#5 Updated by Tom Clegg about 1 year ago

  • Status changed from New to In Progress

#6 Updated by Tom Clegg about 1 year ago

Aperio: project ce8i5-j7d0g-fno4j907u6zmrm1
Aperio/CMU-1 ce8i5-4zz18-5fyr0721prr9wgv
Aperio/CMU-1-JP2K-33005 ce8i5-4zz18-wzhsm4qma0i7t5a
Aperio/CMU-1-Small-Region ce8i5-4zz18-nffvfqc46d4qt3q
Aperio/CMU-2 ce8i5-4zz18-b6qqog55xypd1wd
Aperio/CMU-3 ce8i5-4zz18-hej7suwuynw4kd7
Aperio/JP2K-33003-1 ce8i5-4zz18-6586ki9et67fhev
Aperio/JP2K-33003-2 ce8i5-4zz18-24dinfgcnsomfg9
MIRAX: project ce8i5-j7d0g-tl2e38y8xgdwxss
MIRAX/CMU-1 ce8i5-4zz18-j45o88d58u7js60
MIRAX/CMU-1-Exported ce8i5-4zz18-nusgt0s5nd7d1zf
MIRAX/CMU-1-Saved-1_16 ce8i5-4zz18-vlcao3kj3yyxdrk
MIRAX/CMU-1-Saved-1_2 ce8i5-4zz18-a2g6om052rez8i1
MIRAX/Mirax2.2-1 ce8i5-4zz18-id62n1o6o7iikwl
MIRAX/Mirax2.2-4-PNG ce8i5-4zz18-b09nwmiqdla67qk
S3 config:

Example:

$ curl -H "Authorization: AWS4-HMAC-SHA256 Credential=3dnjzwsn4y8mhg03u0mdpucp2jrdcwlskvpmrd1ytdvfippou8" "https://download.ce8i5.arvadosapi.com/ce8i5-4zz18-j45o88d58u7js60/?delimiter=/&prefix=CMU-1/" 
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>ce8i5-4zz18-j45o88d58u7js60</Name><Prefix>CMU-1/</Prefix><Delimiter>/...
$ s3cmd --access_key=3dnjzwsn4y8mhg03u0mdpucp2jrdcwlskvpmrd1ytdvfippou8 --secret_key=none --host=download.ce8i5.arvadosapi.com --host-bucket=download.ce8i5.arvadosapi.com ls s3://ce8i5-4zz18-j45o88d58u7js60/
                       DIR   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/
2020-08-27 20:56     73531   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1.mrxs
$ s3cmd --access_key=3dnjzwsn4y8mhg03u0mdpucp2jrdcwlskvpmrd1ytdvfippou8 --secret_key=none --host=download.ce8i5.arvadosapi.com --host-bucket=download.ce8i5.arvadosapi.com ls s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/
2020-08-27 20:56         0   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/
2020-08-27 20:56       725   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0000.dat
2020-08-27 20:56       725   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0001.dat
2020-08-27 20:56     77021   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0002.dat
2020-08-27 20:56     56786   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0003.dat
2020-08-27 20:56 394278767   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0004.dat
2020-08-27 20:56    830540   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0005.dat
2020-08-27 20:56 116063098   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0006.dat
2020-08-27 20:56  36857868   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0007.dat
2020-08-27 20:56  11216029   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0008.dat
2020-08-27 20:56   3222476   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0009.dat
2020-08-27 20:56    861685   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0010.dat
2020-08-27 20:56    256092   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0011.dat
2020-08-27 20:56     94214   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0012.dat
2020-08-27 20:56     29978   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0013.dat
2020-08-27 20:56      8580   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0014.dat
2020-08-27 20:56      6440   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0015.dat
2020-08-27 20:56      1766   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0016.dat
2020-08-27 20:56       588   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0017.dat
2020-08-27 20:56     62772   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0018.dat
2020-08-27 20:56    193544   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0019.dat
2020-08-27 20:56      4233   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0020.dat
2020-08-27 20:56      1060   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0021.dat
2020-08-27 20:56     58420   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0022.dat
2020-08-27 20:56    834477   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Index.dat
2020-08-27 20:56     11260   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Slidedat.ini

#7 Updated by Tom Clegg about 1 year ago

  • Status changed from In Progress to Resolved

#8 Updated by Peter Amstutz about 1 year ago

What's the rationale for having the token in "Access Key" instead of "Secret Key"? Shouldn't the "Access Key" be the token UUID and the "Secret Key" should be the token secret?

#9 Updated by Tom Clegg about 1 year ago

  • Related to Story #16535: [keep-web] Minimal implementation of S3 API added

#10 Updated by Tom Clegg about 1 year ago

What's the rationale for having the token in "Access Key" instead of "Secret Key"? Shouldn't the "Access Key" be the token UUID and the "Secret Key" should be the token secret?

This was quicker to implement for initial testing (Access Key is sent with the request, but Secret Key isn't). Access Key = token UUID / Secret Key = entire token would be more secure, that's now #16809.

Also available in: Atom PDF