Collections as regular Arvados objects¶

(Implemented August 2014 #3036)

• Collections are assigned regular UUIDs
• Collections have a regular owner_uuid instead
• Manifest hash goes into the portable_data_hash field
• manifest_text and portable_data_hash can be edited
• Can access collections on API by uuid (returns regular record) or portable_data_hash (returns record containing uuid, portable_data_hash and manifest_text with uuid==portable_data_hash)
  • Setting/changing manifest_text on its own updates portable_data_hash
  • Setting/changing portable_data_hash on its own fetches the manifest from Keep
  • manifest_text and portable_data_hash have to match
• Add mutable name, description, properties fields
  • Use same logic for names/descriptions as already exist for other Arvados objects
• Continue to use manifest text hash (portable_data_hash) for crunch
  • Choosers set the value to the manifest hash, record the selected collection uuid on link_uuid
  • Task output continues to be a manifest fragment or manifest hash+token stored in Keep
  • Crunch-job continues to set job output to manifest hash+token
  • Crunch-job creates output collection with owner_uuid as the user (which means it will show up in the "home" project when #3499 is merged)
• Pipeline runner is responsible for creating a collection in the target project and setting the name
  • Add an output_name field to the component description
• When user clicks on a link to a collection uuid, it should include the expected portable data hash in the URL
  • If there is a mismatch, it should throw up a warning and possibly tell the user what changed
• Disallow content hashes in head_uuid and tail_uuid fields of links

Migration¶

• Update schema to add name, description, properties columns
• Convert name links to collection objects owned by tail_uuid and delete the name links
• For remaining collections not accessible through name links, look for can_read permission links and create collections owned to the relevant user or group
• Any other collections will remain owned by system user and will have to be assigned manually (?)
• Update links (other than name or permission links) that point to a manifest hash to point to an appropriate collection uuid instead