Idea #10484
closed[keepstore] Add S3 bucket statistics and current #clients to status.json
Added by Tom Morris about 8 years ago. Updated almost 8 years ago.
Description
- requests by type - PUT, GET, LIST, DELETE
- bytes on the wire - received, sent
- errors, grouped by status codes (HTTP response code + S3)
Related issues
Updated by Tom Clegg about 8 years ago
- Description updated (diff)
- Subject changed from [keepstore] Log S3 statistics to [keepstore] Add S3 bucket statistics to status.json
Updated by Tom Morris about 8 years ago
- Assigned To set to Tom Clegg
- Target version changed from Arvados Future Sprints to 2016-11-23 sprint
- Story points set to 1.0
Updated by Tom Morris about 8 years ago
Not to dive into implementation details, but introducing a general purpose metrics package may provide benefits going forward.
I've used the Dropwizard metrics in the past and like them. A Go port of that package can be found here: https://github.com/armon/go-metrics
There's also: https://github.com/rcrowley/go-metrics which is described in this blog post: http://blog.dekstroza.io/monitoring-software-written-in-golang/
S3 Error codes are listed here: http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html#ErrorCodeList
Updated by Tom Clegg about 8 years ago
- Subject changed from [keepstore] Add S3 bucket statistics to status.json to [keepstore] Add S3 bucket statistics and current #clients to status.json
Updated by Radhika Chippada about 8 years ago
- I am unable to run tests. Seeing this error when I invoke run-tests with install: (with go version go1.7 linux/amd64)
********** Running services/keepstore install ********** package context: unrecognized import path "context" (import path does not begin with hostname) ********** !!!!!! services/keepstore install FAILED !!!!!! **********
- In s3_volume.go -> IndexTo method, why invoke stats.tick twice in a row at lines 459 and 460?
Updated by Tom Clegg about 8 years ago
Radhika Chippada wrote:
- I am unable to run tests. Seeing this error when I invoke run-tests with install: (with go version go1.7 linux/amd64)
package context: unrecognized import path "context" (import path does not begin with hostname)
Hm, not sure what to make of this. https://blog.golang.org/go1.7 says context was added to stdlib in go1.7. Does it help to reinstall (1.7.3 is current)? Maybe make sure you don't have a GOROOT pointing to an older Go installation?
- In s3_volume.go -> IndexTo method, why invoke stats.tick twice in a row at lines 459 and 460?
One for dataL.First() and one for recentL.First(), both on the following line...
Updated by Radhika Chippada about 8 years ago
Tom said:
Maybe make sure you don't have a GOROOT pointing to an older Go installation
Yes, this was the cause. Tests are passing now.
LGTM
Updated by Tom Clegg almost 8 years ago
10484-count-clients @ f3c31e7c71f076f5feafbb3f14e210f4d0de9012
Updated by Tom Clegg almost 8 years ago
- Status changed from In Progress to Resolved
- % Done changed from 67 to 100
Applied in changeset arvados|commit:d13a369065b4e72047c8c267d6ceaa5c84d50c3d.
Updated by Tom Morris almost 7 years ago
- Related to Feature #13025: [keepstore] runtime metrics endpoint added