Project

General

Profile

Actions

Feature #8688

closed

Accept authenticated API calls from data providers to add datasets to a public profile

Added by Ward Vandewege about 8 years ago. Updated almost 8 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Category:
Third party integration
Story points:
2.0

Description

The API call will have these arguments:

a) a unique identifier for the external data provider

b) the 3rd party activity identifier for the pgp participant which was sent to the data provider when the participant signed up for the activity

c) a payload which indicates what file(s) to import. Details of the format of that payload are TBD, but at a minimum it should probably be a collection of URLs, ideally associated with a file type for each individual file to download. The URLs should include either a manifest file with MD5 digests (like http://cdimage.debian.org/debian-cd/8.3.0/amd64/iso-cd/MD5SUMS), or an *.md5 file for each data file. The size of each file should also be included.

Acceptance criteria

When a researcher or admin shows/edits a third party study, the unique identifier for the study is shown. (This is passed as api_key.)

Given a POST /third_party/add_dataset request:
  1. Ensure api_key corresponds to an approved third-party study
  2. Ensure participant_code param corresponds to a participant who is participating in the study (otherwise HTTP 4xx)
  3. Ensure data_sources_tsv param is a TSV document, where each line has a URI, size, and hex md5 digest of the corresponding file data
  4. Respond HTTP 4xx if any parameters are invalid or there is an error submitting the download job; otherwise 200

Subtasks 1 (0 open1 closed)

Task #8792: Review 8688-pull-data-apiResolvedWard Vandewege03/24/2016Actions

Related issues

Related to Tapestry - Feature #8689: Trigger an arvados job to retrieve the data and import it into KeepIn ProgressTom Clegg03/14/2016Actions
Related to Tapestry - Feature #8707: Arvados job: download data from remote site into KeepIn ProgressTom Clegg03/15/2016Actions
Actions #1

Updated by Ward Vandewege about 8 years ago

  • Description updated (diff)
Actions #2

Updated by Ward Vandewege about 8 years ago

  • Description updated (diff)
Actions #3

Updated by Tom Clegg about 8 years ago

  • Description updated (diff)
Actions #4

Updated by Tom Clegg about 8 years ago

  • Description updated (diff)
Actions #5

Updated by Tom Clegg about 8 years ago

  • Description updated (diff)
Actions #6

Updated by Abram Connelly about 8 years ago

  • Description updated (diff)
Actions #7

Updated by Tom Clegg about 8 years ago

  • Story points set to 2.0
Actions #8

Updated by Tom Clegg about 8 years ago

  • Assigned To set to Tom Clegg
  • Category set to Third party integration
Actions #9

Updated by Tom Clegg about 8 years ago

  • Description updated (diff)
Actions #10

Updated by Tom Clegg about 8 years ago

  • Status changed from New to In Progress
Actions #11

Updated by Tom Clegg about 8 years ago

  • Description updated (diff)
  • Status changed from In Progress to New
Actions #12

Updated by Tom Clegg about 8 years ago

  • Status changed from New to In Progress
Actions #13

Updated by Tom Clegg about 8 years ago

  • Description updated (diff)
Branch 8688-pull-data-api @ 4969df6
  • Accepts valid API calls: POST /third_party/add_dataset with params api_key, data_sources_tsv, participant_code
  • Rejects invalid API calls with 4xx and a JSON response body
  • Tries to submit an Arvados pipeline using arv-run-pipeline-instance (arvados-cli must be installed, but this dependency can't be in Gemfile because it needs Ruby 1.9.3) (this is part of #8689)
  • Introduces a new config knob ruby19_shell_args (default "rvm-exec 1.9.3")
Actions #14

Updated by Ward Vandewege about 8 years ago

I looked at 8688-pull-data-api, and I think it's OK to merge. When merged I can deploy to my-dev and we can test functionality. Thanks for adding a journal table!

Actions #15

Updated by Tom Clegg almost 8 years ago

  • Status changed from In Progress to Resolved
Actions

Also available in: Atom PDF