2014-04-16 Dev tools and data/resource management¶

Arvados Private Wiki¶

This wiki is internal to Curoverse, Inc.

Development Process and Tools¶

• Summary of Development Process
• DevOps To Do List
• Release Roadmap
• Sprint Retrospectives
• Gem and Pypi publishing instructions
• Hacking API server
• Hacking Workbench
• Redmine wishlist

Operations¶

• 4xphq private beta instance
• Deploying a new Arvados instance
• Updating an Arvados instance
• Running tests
• Tech FAQ
• Distributing encrypted disks
• Cluster install instructions
• Deploy instructions -- installing and updating Arvados API server and Workbench on production systems
• New PaaS user activation
• Setting up a private VM for a user
• [[gps:Amazon setup and notes]] (in GPS project)

System/component designs and specifications¶

• Functional components and where they live in the source tree

Keep¶

• public Keep server specification @ Arvados
• public Keep index design in progress @ Arvados
• public Keep manifest format specification @ Arvados
• Keep Specification describes current Perl prototype
• Next generation keep
• Using Keep with S3 backing store
• Keep federation

Other infrastructure¶

• Crunch
• Event bus
• Logging facility
• Metadata database
• EC2 compute node life cycle

User interface¶

• Workbench UI design
  • Presenting a set of jobs

Accounts and permissions¶

• User activation states
• Permissions model
• Terms of Service mechanism
• Pre-activation mechanism

Research notes - related projects¶

• Alternative cluster storage systems
• Alternative MapReduce and cluster scheduling systems
• Remote Procedure Call systems
• Metadata database
• Openstack Notes
• Message bus
• Galaxy

Data curation references¶

• Copying PGP data to AWS
• Importing PGP survey data to Arvados

Admin references¶

Introductory material for people new to Curoverse/Arvados¶

• Background Reading

Customer Development¶

Materials such as a user personas, business strategy information, and notes on customer visits are stored in Google Drive (and other places?).

Archives¶

These are pages that are no longer active or relevant

• Release 1
• Meeting notes
• API
• Permission model
• Source tree overview
• Demo 2013-02-05
• Project status
• Ruby client quick start
• Command line examples
• Admin and system API examples
• Explorvos sample API client, orvos-explorer
• Admin Interface (from Orvos days)
• Orvos-server install and bootstrap
• Compute node setup
• Underlying technical decisions