Collections as regular Arvados objects

(Archived for informational purposes. Implemented August 2014 in #3036)

  • Collections are assigned regular UUIDs
  • Collections have a regular owner_uuid instead
  • Manifest hash goes into the portable_data_hash field
  • manifest_text and portable_data_hash can be edited
  • Can access collections on API by uuid (returns regular record) or portable_data_hash (returns record containing uuid, portable_data_hash and manifest_text with uuid==portable_data_hash)
    • Setting/changing manifest_text on its own updates portable_data_hash
    • Setting/changing portable_data_hash on its own fetches the manifest from Keep
    • manifest_text and portable_data_hash have to match
  • Add mutable name, description, properties fields
    • Use same logic for names/descriptions as already exist for other Arvados objects
  • Continue to use manifest text hash (portable_data_hash) for crunch
    • Choosers set the value to the manifest hash, record the selected collection uuid on link_uuid
    • Task output continues to be a manifest fragment or manifest hash+token stored in Keep
    • Crunch-job continues to set job output to manifest hash+token
    • Crunch-job creates output collection with owner_uuid as the user (which means it will show up in the "home" project when #3499 is merged)
  • Pipeline runner is responsible for creating a collection in the target project and setting the name
    • Add an output_name field to the component description
  • When user clicks on a link to a collection uuid, it should include the expected portable data hash in the URL
    • If there is a mismatch, it should throw up a warning and possibly tell the user what changed
  • Disallow content hashes in head_uuid and tail_uuid fields of links

Migration

  • Update schema to add name, description, properties columns
  • Convert name links to collection objects owned by tail_uuid and delete the name links
  • For remaining collections not accessible through name links, look for can_read permission links and create collections owned to the relevant user or group
  • Any other collections will remain owned by system user and will have to be assigned manually (?)
  • Update links (other than name or permission links) that point to a manifest hash to point to an appropriate collection uuid instead