Project

General

Profile

Go » History » Version 9

Misha Zatsman, 01/08/2015 07:05 PM

1 1 Misha Zatsman
h1. Go
2
3
{{toc}}
4
5
h2. Using Go with Arvados
6
7
h3. Install Go
8
9 8 Tom Clegg
You'll need *Go 1.3 or newer*. If you're installing Go on your linux machine you'll want to follow the "tarball instructions":http://golang.org/doc/install#tarball to get the latest stable version, because if you use <code>apt-get</code> you'll get an outdated version.
10 1 Misha Zatsman
11
The first time you install Go, you'll need to set @GOPATH@ to an empty directory.  The Go toolchain will install Go packages and dependencies here. 
12
13
<pre>
14
export GOPATH=~/gocode
15
mkdir -p $GOPATH
16
</pre>
17
18
The rest of these instructions assume that you have a working Go installation on your system, and that your @GOPATH@ environment variable is set appropriately.
19
20
h3. Install Arvados source
21
22
Clone the Arvados git repository, if you have not already:
23
24
<pre>
25
cd
26
git clone git://git.curoverse.com/arvados.git
27
</pre>
28
29
*Note:* If you are an authorized committer, clone @git@git.curoverse.com:arvados.git@ instead so you may push directly to git.curoverse.com.
30
31
h3. Tell Go to use your local Arvados source
32
33
This step ensures that your development environment uses your locally-modified code, instead of fetching the master branch from git.curoverse.com:
34
35
<pre>
36
mkdir -p $GOPATH/src/git.curoverse.com
37
ln -s ~/arvados $GOPATH/src/git.curoverse.com/arvados.git
38
</pre>
39
40
Reason: The Keepstore and Keepproxy packages import other Go packages from the Arvados source tree. These packages have names like:
41
42
<pre>
43
git.curoverse.com/arvados.git/sdk/go/keepclient
44
git.curoverse.com/arvados.git/sdk/go/arvadosclient
45
</pre>
46
47
When the Go compiler needs to import one of these packages, it will look in @$GOPATH/src@ for the package source code. If it does not find the code locally, it will fetch the code from git.curoverse.com automatically.  This symlink ensures that Go will find your local source code under @$GOPATH/src/git.curoverse.com/arvados.git/...@
48
49
h3. Run some tests
50
51
e.g.
52
53
<pre>
54
cd ~/arvados/services/keepstore
55
go test
56
</pre>
57
58
The @go test@ command will print a few dozen lines of logging output.  If the tests succeeded, it will print PASS followed by a summary of the packages which passed testing, e.g.:
59
60
<pre>
61
PASS
62
ok  	_/home/you/arvados/services/keepstore	1.023s
63
</pre>
64
65 4 Misha Zatsman
h2. Arvados Go SDK
66
67
h3. Documentation
68
69
For more information consult the "Arvados Go SDK documentation":http://doc.arvados.org/sdk/go/
70
71
h3. Recipes
72
73 5 Misha Zatsman
This section contains code demonstrating aspects of the Arvados Go SDK. Additional examples can be found in the "Arvados Go SDK documentation":http://doc.arvados.org/sdk/go/
74 4 Misha Zatsman
75
h4. Collections filtered and ordered
76
77 6 Misha Zatsman
This example prints collections last modified after a given date, grouped by owner (owner UUID sorted DESCending) and then ordered by oldest modifications (i.e. ASCending modification date).
78 1 Misha Zatsman
79 7 Misha Zatsman
_Although not an issue in this example, it's worth noting that Arvados only stores the latest modification date for each collection. Therefore if you request collections modified before a given date, the SDK will not return collections who had also been modified after that date._
80 6 Misha Zatsman
81 4 Misha Zatsman
<pre>
82
package main
83
84
// *******************
85
// Import the modules.
86
87
import (
88
	"git.curoverse.com/arvados.git/sdk/go/arvadosclient"
89
	"log"
90
)
91
92
type SdkCollectionInfo struct {
93
	Uuid           string   `json:"uuid"`
94
	OwnerUuid      string   `json:"owner_uuid"`
95
	Redundancy     int      `json:"redundancy"`
96
	ModifiedAt     string   `json:"modified_at"`
97
	ManifestText   string   `json:"manifest_text"`
98
}
99
100
type SdkCollectionList struct {
101
	ItemsAvailable   int                   `json:"items_available"`
102
	Items            []SdkCollectionInfo   `json:"items"`
103
}
104
105
func main() {
106
	// ********************************
107
	// Set up an API client user agent.
108
	//
109
110
	arv, err := arvadosclient.MakeArvadosClient()
111
	if err != nil {
112
		log.Fatalf("Error setting up arvados client %s", err.Error())
113
	}
114
115
	GetCollections(arv)
116
}
117
118
func GetCollections(arv arvadosclient.ArvadosClient) () {
119
	fieldsWanted := []string{"manifest_text",
120
		"owner_uuid",
121
		"uuid",
122
		"redundancy",
123
		"modified_at"}
124 1 Misha Zatsman
125
	sdkParams := arvadosclient.Dict{
126 4 Misha Zatsman
		"select": fieldsWanted,
127 6 Misha Zatsman
		"order": []string{"owner_uuid DESC", "modified_at ASC"},
128
		"filters": [][]string{[]string{"modified_at", ">=", "2014-01-03T06:53:08Z"}}}
129 4 Misha Zatsman
130
	var collections SdkCollectionList
131
132
	err := arv.List("collections", sdkParams, &collections)
133
	if err != nil {
134
		log.Fatalf("error querying collections: %v", err)
135
	}
136
137
	for _, collection := range collections.Items {
138
		log.Printf("Seeing owner uuid, modification date: %s %s",
139
			collection.OwnerUuid,
140
			collection.ModifiedAt)
141
	}
142
	return
143
}
144
</pre>
145 3 Misha Zatsman
146 9 Misha Zatsman
h4. Writing a Log Entry
147
148
This example records a minimal log entry.
149
150
For more fields you can write, see "the Log Schema":http://doc.arvados.org/api/schema/Log.html
151
152
<pre>
153
package main
154
155
import (
156
	"git.curoverse.com/arvados.git/sdk/go/arvadosclient"
157
	"log"
158
)
159
160
func main() {
161
	arv, err := arvadosclient.MakeArvadosClient()
162
	if err != nil {
163
		log.Fatalf("Error setting up arvados client %v", err)
164
	}
165
166
	err = arv.Create("logs",
167
		arvadosclient.Dict{"log": arvadosclient.Dict{
168
			"event_type": "experimental-logger-testing",
169
			"properties": arvadosclient.Dict{
170
				"ninja": "Misha"}}},
171
		nil)
172
	if err != nil {
173
		log.Fatalf("Error writing log: %v", err)
174
	}
175
}
176
</pre>
177
178
179 1 Misha Zatsman
h2. Learning Go
180
181
Getting good at Go, and concurrency/goroutines in particular, is an excellent use of time.
182
183
h3. Introductions
184
 
185
* "Go Tour":http://tour.golang.org/ is a great way to spend a few hours getting your feet wet.
186
* "Effective Go":http://golang.org/doc/effective_go.html The web site says that you should do both the tour and the language specification first before reading this, but honestly, I think it's ridiculous to try to read the language specification before you even start writing code in the language. I wouldn't try to read either the specification or "Effective Go" in depth before trying to write any code, but they're both at least worth skimming at this point.
187
* The slides for Rob Pike's talk on "Go Concurrency Patterns":http://talks.golang.org/2012/concurrency.slide#1. I haven't watched all of the talks yet; these slides are really nice because they show very elegant ways of using channels and goroutines to build rich and complex concurrent abstractions.
188
* "The package documentation.":http://golang.org/pkg/ Note that the package docs include links to web versions of the package source code, which help very much in learning idiomatic Go patterns.
189
190
h3. Resources
191
192
h4. Free online resources:
193
194
* http://golang.org
195
* http://golang.org/doc/code.html
196
* http://tour.golang.org/
197
* https://gobyexample.com/
198
* http://learnxinyminutes.com/docs/go/
199
200
h4. Books
201
202
* http://www.golang-book.com/ (Free!)
203
* "Programming in Go":http://www.amazon.com/Programming-Go-Creating-Applications-Developers/dp/0321774639/ref=pd_bxgy_b_img_y (a bit dated now)