Project

General

Profile

Hacking Ruby SDK » History » Version 8

Brett Smith, 09/10/2014 04:11 PM

1 1 Peter Amstutz
h1. Hacking Ruby SDK
2
3 3 Brett Smith
{{toc}}
4
5
h2. Overview
6
7
|@lib/arvados.rb@          |Google API client.  Built from the discovery document.|
8
|@lib/arvados/keep.rb@     |Keep client functions.  Parse locators and manifests.|
9
10
h2. Running tests
11
12
Run @bundle exec rake test@.
13
14
Note: We have never (more or less) tested the Google API client.  The test infrastructure currently has no way to test against a development API server, like other SDKs do.  If you want to add this, be aware that you could create a knotty dependency loop: the API server needs the Ruby SDK to run, and the Ruby SDK needs to run the API server for tests.  If you can, it would be better to test the API client with mock responses.  See @test_api.py@ in the Python SDK for an illustration of this approach.
15
16
h2. Building and testing gems
17
18 6 Brett Smith
As of this writing, the API server and Workbench use the Ruby SDK.  When you modify the SDK, you should build and install the Gem, and then test our servers that use it, to make sure you haven't accidentally introduced a bug.  To do that:
19 3 Brett Smith
20
<pre>
21
sdk/ruby$ # Commit your changes to the Gem code first, then:
22
sdk/ruby$ gem build arvados.gemspec
23 1 Peter Amstutz
sdk/ruby$ gem install arvados-0.1.YYYYMMDDHHMMSS.gem
24 6 Brett Smith
# Then, for each server you're testing:
25 7 Brett Smith
server$ bundle update arvados
26 6 Brett Smith
server$ bundle exec rake test
27 4 Brett Smith
</pre>
28 3 Brett Smith
29
h2. Arranging a build
30
31
When new code is pushed to git, Jenkins builds the latest SDK Gem and runs all the tests with it.  However, it will crash if a server's Gemfile refers to a nonexistent Gem version.  To ensure a successful test, you must make sure that Gemfiles in your push refers to Gem versions that have been published.  There are two main ways to do this.
32
33
h3. Separate development approach
34
35
This approach requires you to think about Arvados components as separate pieces of software, but it's easier to wrap your head around.  Change the SDK separately from other components: make your changes, test them, push them for review, and merge them to master.  After Jenkins builds them, you can put the new version number in Gemfiles for server components that need it, in a later branch or commit.
36
37
h3. One-shot approach
38
39
If you're in a rush or it just doesn't make sense to have two separate branches reviewed, you can make a single push to master that builds new Gems and refers to them in Gemfiles.  You do this by making a commit with a known timestamp, effectively controlling the Gem version Jenkins builds so you can refer to it in that same commit.  The process is:
40
41
<pre>
42
$ gem_ts=$(date '+%Y-%m-%dT%H:%M:%S')
43
$ echo 0.1.$(echo "$gem_ts" | tr -dc 0-9)
44 8 Brett Smith
# That prints your Gem version.  For each server that needs it:
45
    server$ # Edit the server's Gemfile.
46
    server$ bundle update arvados
47
    server$ git add Gemfile Gemfile.lock
48
# After you've updated all necessary servers:
49 3 Brett Smith
$ GIT_COMMITTER_DATE=$gem_ts git commit
50
</pre>
51
52
In order for this to work, your new commit must be the most recent commit when it's pushed to the git server.  If new commits are pushed while you're working on this, discard the commit you made and start over.
53
54
h2. Use notes
55
56 2 Peter Amstutz
When calling client.execute() on the Google API client, be careful to use @:body_object@ (which serializes the the provided hash to JSON) instead of @:body@ (which is the raw text.)
57 3 Brett Smith
58
The @each_*@ methods in Keep::Manifest yield results as they go.  This is important for performance.  Many clients only look at the first few files.  Yielding results as they're available makes it possible for those clients to get information without parsing an entire manifest, which could be very large.