Project

General

Profile

Actions
Copy link

Support #16668

closed

Load demo data from openslide

Added by Peter Amstutz over 4 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Tom Clegg
Category:
-
Target version:
2020-09-09 Sprint
Due date:
Story points:
-

Description

http://openslide.cs.cmu.edu/download/openslide-testdata/

Load this into an Arvados instance somewhere

Need to unzip the zip files.

Separate project per technology.

Separate collection per slide.

Related issues 1 (0 open — 1 closed)

Related to Arvados - Idea #16535: [keep-web] Minimal implementation of S3 APIResolvedTom Clegg07/21/2020Actions
Actions
Copy link
 #1

Updated by Peter Amstutz over 4 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by Peter Amstutz over 4 years ago

  • Description updated (diff)
Actions
Copy link
 #3

Updated by Peter Amstutz over 4 years ago

  • Target version changed from 2020-08-26 Sprint to 2020-09-09 Sprint
Actions
Copy link
 #4

Updated by Peter Amstutz over 4 years ago

  • Assigned To set to Tom Clegg
Actions
Copy link
 #5

Updated by Tom Clegg over 4 years ago

  • Status changed from New to In Progress
Actions
Copy link
 #6

Updated by Tom Clegg over 4 years ago

Aperio: project ce8i5-j7d0g-fno4j907u6zmrm1
Aperio/CMU-1 ce8i5-4zz18-5fyr0721prr9wgv
Aperio/CMU-1-JP2K-33005 ce8i5-4zz18-wzhsm4qma0i7t5a
Aperio/CMU-1-Small-Region ce8i5-4zz18-nffvfqc46d4qt3q
Aperio/CMU-2 ce8i5-4zz18-b6qqog55xypd1wd
Aperio/CMU-3 ce8i5-4zz18-hej7suwuynw4kd7
Aperio/JP2K-33003-1 ce8i5-4zz18-6586ki9et67fhev
Aperio/JP2K-33003-2 ce8i5-4zz18-24dinfgcnsomfg9
MIRAX: project ce8i5-j7d0g-tl2e38y8xgdwxss
MIRAX/CMU-1 ce8i5-4zz18-j45o88d58u7js60
MIRAX/CMU-1-Exported ce8i5-4zz18-nusgt0s5nd7d1zf
MIRAX/CMU-1-Saved-1_16 ce8i5-4zz18-vlcao3kj3yyxdrk
MIRAX/CMU-1-Saved-1_2 ce8i5-4zz18-a2g6om052rez8i1
MIRAX/Mirax2.2-1 ce8i5-4zz18-id62n1o6o7iikwl
MIRAX/Mirax2.2-4-PNG ce8i5-4zz18-b09nwmiqdla67qk
S3 config:

Example:

$ curl -H "Authorization: AWS4-HMAC-SHA256 Credential=3dnjzwsn4y8mhg03u0mdpucp2jrdcwlskvpmrd1ytdvfippou8" "https://download.ce8i5.arvadosapi.com/ce8i5-4zz18-j45o88d58u7js60/?delimiter=/&prefix=CMU-1/" 
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>ce8i5-4zz18-j45o88d58u7js60</Name><Prefix>CMU-1/</Prefix><Delimiter>/...

$ s3cmd --access_key=3dnjzwsn4y8mhg03u0mdpucp2jrdcwlskvpmrd1ytdvfippou8 --secret_key=none --host=download.ce8i5.arvadosapi.com --host-bucket=download.ce8i5.arvadosapi.com ls s3://ce8i5-4zz18-j45o88d58u7js60/
                       DIR   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/
2020-08-27 20:56     73531   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1.mrxs
$ s3cmd --access_key=3dnjzwsn4y8mhg03u0mdpucp2jrdcwlskvpmrd1ytdvfippou8 --secret_key=none --host=download.ce8i5.arvadosapi.com --host-bucket=download.ce8i5.arvadosapi.com ls s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/
2020-08-27 20:56         0   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/
2020-08-27 20:56       725   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0000.dat
2020-08-27 20:56       725   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0001.dat
2020-08-27 20:56     77021   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0002.dat
2020-08-27 20:56     56786   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0003.dat
2020-08-27 20:56 394278767   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0004.dat
2020-08-27 20:56    830540   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0005.dat
2020-08-27 20:56 116063098   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0006.dat
2020-08-27 20:56  36857868   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0007.dat
2020-08-27 20:56  11216029   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0008.dat
2020-08-27 20:56   3222476   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0009.dat
2020-08-27 20:56    861685   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0010.dat
2020-08-27 20:56    256092   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0011.dat
2020-08-27 20:56     94214   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0012.dat
2020-08-27 20:56     29978   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0013.dat
2020-08-27 20:56      8580   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0014.dat
2020-08-27 20:56      6440   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0015.dat
2020-08-27 20:56      1766   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0016.dat
2020-08-27 20:56       588   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0017.dat
2020-08-27 20:56     62772   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0018.dat
2020-08-27 20:56    193544   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0019.dat
2020-08-27 20:56      4233   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0020.dat
2020-08-27 20:56      1060   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0021.dat
2020-08-27 20:56     58420   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Data0022.dat
2020-08-27 20:56    834477   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Index.dat
2020-08-27 20:56     11260   s3://ce8i5-4zz18-j45o88d58u7js60/CMU-1/Slidedat.ini
Actions
Copy link
 #7

Updated by Tom Clegg over 4 years ago

  • Status changed from In Progress to Resolved
Actions
Copy link
 #8

Updated by Peter Amstutz over 4 years ago

What's the rationale for having the token in "Access Key" instead of "Secret Key"? Shouldn't the "Access Key" be the token UUID and the "Secret Key" should be the token secret?

Actions
Copy link
 #9

Updated by Tom Clegg over 4 years ago

  • Related to Idea #16535: [keep-web] Minimal implementation of S3 API added
Actions
Copy link
 #10

Updated by Tom Clegg over 4 years ago

What's the rationale for having the token in "Access Key" instead of "Secret Key"? Shouldn't the "Access Key" be the token UUID and the "Secret Key" should be the token secret?

This was quicker to implement for initial testing (Access Key is sent with the request, but Secret Key isn't). Access Key = token UUID / Secret Key = entire token would be more secure, that's now #16809.

Actions
Copy link

Also available in: Atom PDF