Arvados

h1. Hacking Ruby SDK

{{toc}}

h2. Overview

|@lib/arvados.rb@          |Google API client.  Built from the discovery document.|

|@lib/arvados/keep.rb@     |Keep client functions.  Parse locators and manifests.|

h2. Running tests

Run @bundle exec rake test@.

Note: We have never (more or less) tested the Google API client.  The test infrastructure currently has no way to test against a development API server, like other SDKs do.  If you want to add this, be aware that you could create a knotty dependency loop: the API server needs the Ruby SDK to run, and the Ruby SDK needs to run the API server for tests.  If you can, it would be better to test the API client with mock responses.  See @test_api.py@ in the Python SDK for an illustration of this approach.

h2. Building and testing gems

As of this writing, the API server and Workbench use the Ruby SDK.  When you modify the SDK, you should build and install the Gem, and then test our servers that use it, to make sure you haven't accidentally introduced a bug.  To do that:

<pre>

sdk/ruby$ # Commit your changes to the Gem code first, then:

sdk/ruby$ gem build arvados.gemspec

sdk/ruby$ gem install arvados-0.1.YYYYMMDDHHMMSS.gem

# Then, for each server you're testing:

server$ bundle update arvados

server$ bundle exec rake test

</pre>

h2. Arranging a build

When new code is pushed to git, Jenkins builds the latest SDK Gem and runs all the tests with it.  However, it will crash if a server's Gemfile refers to a nonexistent Gem version.  To ensure a successful test, you must make sure that Gemfiles in your push refers to Gem versions that have been published.  There are two main ways to do this.

h3. Separate development approach

This approach requires you to think about Arvados components as separate pieces of software, but it's easier to wrap your head around.  Change the SDK separately from other components: make your changes, test them, push them for review, and merge them to master.  After Jenkins builds them, you can put the new version number in Gemfiles for server components that need it, in a later branch or commit.

h3. One-shot approach

If you're in a rush or it just doesn't make sense to have two separate branches reviewed -- or you're in a CI stalemate where passing tests and building new gems depend on each other -- you can make a single push to master that builds new Gems and refers to them in Gemfiles. First commit all changes to the @sdk/ruby@ directory and build a package based on that commit date, then commit changes to Gemfile and Gemfile.lock files. The latter are outside the @sdk/ruby@ directory so they don't affect the gem version.

Outline:

<pre>

$ git add sdk/ruby

$ git commit -m 'Update the SDK'

$ (cd sdk/ruby && gem build arvados.gemspec)

$ ls sdk/ruby/*.gem

$ # edit services/api/Gemfile to reflect the new version number

$ (cd services/api && bundle install)

$ git add services/api/Gemfile{,.lock}

$ git commit -m 'Update SDK version'

$ git push

</pre>

In order for this to work, your @sdk/ruby@ commit must be the most recent change in @sdk/ruby@ on the master branch when the CI server tests your @services/api@ change. If someone else changes @sdk/ruby@ in @master@ while you're working on this, start over.

h2. Use notes

When calling client.execute() on the Google API client, be careful to use @:body_object@ (which serializes the the provided hash to JSON) instead of @:body@ (which is the raw text.)

The @each_*@ methods in Keep::Manifest yield results as they go.  This is important for performance.  Many clients only look at the first few files.  Yielding results as they're available makes it possible for those clients to get information without parsing an entire manifest, which could be very large.