Hacking Ruby SDK¶

• Table of contents
• Hacking Ruby SDK
  • Overview
  • Running tests
  • Building and testing gems
  • Arranging a build
    • Separate development approach
    • One-shot approach
  • Use notes

Overview¶

lib/arvados.rb	Google API client. Built from the discovery document.
lib/arvados/keep.rb	Keep client functions. Parse locators and manifests.

Running tests¶

Run bundle exec rake test.

Note: We have never (more or less) tested the Google API client. The test infrastructure currently has no way to test against a development API server, like other SDKs do. If you want to add this, be aware that you could create a knotty dependency loop: the API server needs the Ruby SDK to run, and the Ruby SDK needs to run the API server for tests. If you can, it would be better to test the API client with mock responses. See test_api.py in the Python SDK for an illustration of this approach.

Building and testing gems¶

As of this writing, the API server and Workbench use the Ruby SDK. When you modify the SDK, you should build and install the Gem, and then test our servers that use it, to make sure you haven't accidentally introduced a bug. To do that:

sdk/ruby$ # Commit your changes to the Gem code first, then:
sdk/ruby$ gem build arvados.gemspec
sdk/ruby$ gem install arvados-0.1.YYYYMMDDHHMMSS.gem
# Then, for each server you're testing:
server$ bundle update arvados
server$ bundle exec rake test

Arranging a build¶

When new code is pushed to git, Jenkins builds the latest SDK Gem and runs all the tests with it. However, it will crash if a server's Gemfile refers to a nonexistent Gem version. To ensure a successful test, you must make sure that Gemfiles in your push refers to Gem versions that have been published. There are two main ways to do this.

Separate development approach¶

This approach requires you to think about Arvados components as separate pieces of software, but it's easier to wrap your head around. Change the SDK separately from other components: make your changes, test them, push them for review, and merge them to master. After Jenkins builds them, you can put the new version number in Gemfiles for server components that need it, in a later branch or commit.

One-shot approach¶

If you're in a rush or it just doesn't make sense to have two separate branches reviewed -- or you're in a CI stalemate where passing tests and building new gems depend on each other -- you can make a single push to master that builds new Gems and refers to them in Gemfiles. First commit all changes to the sdk/ruby directory and build a package based on that commit date, then commit changes to Gemfile and Gemfile.lock files. The latter are outside the sdk/ruby directory so they don't affect the gem version.

Outline:

$ git add sdk/ruby
$ git commit -m 'Update the SDK'
$ (cd sdk/ruby && gem build arvados.gemspec)
$ ls sdk/ruby/*.gem
$ # edit services/api/Gemfile to reflect the new version number
$ (cd services/api && bundle install)
$ git add services/api/Gemfile{,.lock}
$ git commit -m 'Update SDK version'
$ git push

In order for this to work, your sdk/ruby commit must be the most recent change in sdk/ruby on the master branch when the CI server tests your services/api change. If someone else changes sdk/ruby in master while you're working on this, start over.

Use notes¶

When calling client.execute() on the Google API client, be careful to use :body_object (which serializes the the provided hash to JSON) instead of :body (which is the raw text.)

The each_* methods in Keep::Manifest yield results as they go. This is important for performance. Many clients only look at the first few files. Yielding results as they're available makes it possible for those clients to get information without parsing an entire manifest, which could be very large.