Tom Clegg's activity

From 04/20/2017 to 05/19/2017

05/19/2017

11:36 pm Arvados Revision a9b15a2f (arvados): Merge branch '11590-log-reuse'
closes #11590 Tom Clegg
11:20 pm Arvados Revision 25fb9f53 (arvados): 9005: Skip starting unused http server in handler tests.
Tom Clegg
11:20 pm Arvados Revision 57fa190e (arvados): 9005: Add connection leak test.
Tom Clegg
11:17 pm Arvados Revision 138eedbe (arvados): Merge branch '9005-disable-keepalive'
refs #9005
refs #11726
refs #11729
Tom Clegg
11:16 pm Arvados Revision 83b73a47 (arvados): 9005: 11726: 11729: Disable http keepalive.
The previous workaround for #9005 did not account for the prefetch
feature: if a goroutine is using the HTTP client t...
Tom Clegg
08:32 pm Arvados Revision 695dee19 (arvados): 9005: Skip starting unused http server in handler tests.
Tom Clegg
08:23 pm Arvados Revision 303424c3 (arvados): Merge branch '9005-conn-leak'
refs #9005 Tom Clegg
08:06 pm Arvados Revision 1e7d0d5f (arvados): 9005: Fix missing Close().
Tom Clegg
08:06 pm Arvados Revision cad24bba (arvados): 9005: Fix missing error checks.
Tom Clegg
07:58 pm Arvados Story #9005: [SDKs] Go SDK's arvadosclient and keepclient should share http.Client objects by def...
Some related fixes:
9005-conn-leak @ commit:cad24bba2240b47f59bc5719a035e85ff5eb60ef
Tom Clegg
06:10 pm Arvados Story #9005: [SDKs] Go SDK's arvadosclient and keepclient should share http.Client objects by def...
Peter Amstutz wrote:
> I don't see how we can share the same HTTP client for both API requests and keep blocks when ...
Tom Clegg
02:15 pm Arvados Story #9005 (In Progress): [SDKs] Go SDK's arvadosclient and keepclient should share http.Client ...
Tom Clegg
07:55 pm Arvados Revision 1eb8881a (arvados): 9005: Fix missing error checks.
Tom Clegg
07:55 pm Arvados Revision ccf1048d (arvados): 9005: Fix missing Close().
Tom Clegg
07:05 pm Arvados Revision 23721f23 (arvados): 9005: Skip starting unused http server in handler tests.
Tom Clegg
05:47 pm Arvados Revision dfe5c5a6 (arvados): 9005: Remove debug printf.
Tom Clegg

05/18/2017

09:25 pm Arvados Revision d95ccc6b (arvados): 11590: Dry up log_reuse_info() calls.
Tom Clegg
09:01 pm Arvados Task #11683 (Resolved): Add logging to container reuse code
Tom Clegg
08:56 pm Arvados Task #11683 (In Progress): Add logging to container reuse code
Tom Clegg
09:01 pm Arvados Task #11728 (Resolved): Review 11590-log-reuse
Tom Clegg
06:54 pm Arvados Bug #11590: job reuse / re-use logic does not log decisions
Container reuse logging samples... Tom Clegg
02:28 pm Arvados Bug #11590: job reuse / re-use logic does not log decisions
updated to... Tom Clegg
06:53 pm Arvados Revision 02d1074b (arvados): 11590: Add container logging tests.
Tom Clegg
06:50 pm Arvados Feature #11644 (Resolved): [keepstore] mount-oriented APIs
Applied in changeset arvados|commit:d121e087ad1b4e91f869dbd57534c6d6ce51d19d. Tom Clegg
05:34 pm Arvados Feature #11644: [keepstore] mount-oriented APIs
Lucas Di Pentima wrote:
> * File @services/keepstore/volume.go@
> ** Lines 329-333: Can you please explain to me wh...
Tom Clegg
06:45 pm Arvados Revision d121e087 (arvados): Merge branch '11644-mounts-api'
closes #11644 Tom Clegg
06:19 pm Arvados Story #11720: Add dependency management for Go packages
Nico César wrote:
> shouldn't be using GOROOT ? I had the impression GOPATH is a colon/comma separated values of the...
Tom Clegg
02:23 pm Arvados Story #11720: Add dependency management for Go packages
11720-govendor @ commit:865d1b22bb42a6c89ca9aa9a3513f3c61c6547c7 (missed adding vendor/.gitignore because it said to ... Tom Clegg
06:12 pm Arvados Story #9005: [SDKs] Go SDK's arvadosclient and keepclient should share http.Client objects by def...
Peter Amstutz wrote:
> API client and keepstore have different socket settings, which probably why they were not com...
Tom Clegg
05:26 pm Arvados Revision 37003688 (arvados): 11644: Add DeviceID() to Volume interface.
Tom Clegg
05:22 pm Arvados Revision d77b362f (arvados): 11644: Unify block-index handlers. Move prefix arg to query string.
Tom Clegg
05:05 pm Arvados Revision 627fbeb0 (arvados): 11590: Log container reuse decisions.
Tom Clegg
02:41 pm Arvados Bug #11717 (Closed): vendor dir for crunch-run dependencies
Tom Clegg
02:26 pm Arvados Revision df55660f (arvados): Merge branch '11590-log-reuse'
refs #11590 Tom Clegg
02:26 pm Arvados Revision 84906c71 (arvados): 11590: Clarify "job state" condition in log message.
Tom Clegg
02:23 pm Arvados Task #11723 (Resolved): Review
Tom Clegg
02:21 pm Arvados Revision 865d1b22 (arvados): 11720: Add vendor/.gitignore.
Tom Clegg

05/17/2017

09:08 pm Arvados Story #11720: Add dependency management for Go packages
11720-govendor @ commit:b1b7a0be3c789795675e731c8c64bee3cb6d718c
example run https://ci.curoverse.com/job/develope...
Tom Clegg
09:07 pm Arvados Story #11720: Add dependency management for Go packages
Procedure for updating dependencies:
# Ensure @$GOPATH@ points somewhere (maybe @~/go@)
# @go get github.com/kardia...
Tom Clegg
07:05 pm Arvados Story #11720 (In Progress): Add dependency management for Go packages
Tom Clegg
07:04 pm Arvados Story #11720: Add dependency management for Go packages
The first godep problem I ran into was "discussed on github":https://github.com/tools/godep/issues/377:
> FWIW: No...
Tom Clegg
04:20 pm Arvados Story #11720 (Resolved): Add dependency management for Go packages
Our Go packages (programs and libraries) need something analogous to bundler, so changes in our dependencies don't br... Tom Clegg
09:07 pm Arvados Revision b1b7a0be (arvados): 11720: Update Go dependencies.
Tom Clegg
08:50 pm Arvados Bug #11546 (Resolved): Expensive container lock/unlock
Applied in changeset arvados|commit:b3f88934721fc97973b40786cae97e55e84b31d5. Tom Clegg
06:31 pm Arvados Bug #11546: Expensive container lock/unlock
indeed
11546-fast-lock @ commit:78390940029ab08efecab923615c545b9da922b4
Tom Clegg
06:03 pm Arvados Bug #11546: Expensive container lock/unlock
11546-fast-lock @ commit:fbc576b76977938cf7b742f9770ab90559136dc8
* threw away the "circumvent AR update" code
* us...
Tom Clegg
04:10 pm Arvados Bug #11546: Expensive container lock/unlock
Another approach would be to go back to the obvious Rails locking mechanism, but use "select for share nowait".
ht...
Tom Clegg
02:41 pm Arvados Bug #11546: Expensive container lock/unlock
11546-fast-lock @ commit:f3d300ce629f1c17ccc83bf1978ff5c7dd41e33d
Still needs some tests, but does this seem like ...
Tom Clegg
08:48 pm Arvados Revision 37470b03 (arvados): 11720: Merge branch 'master' into 11720-govendor
Tom Clegg
08:47 pm Arvados Revision b3f88934 (arvados): Merge branch '11546-fast-lock'
closes #11546 Tom Clegg
07:26 pm Arvados Revision 513b7af1 (arvados): 11720: Control dependencies with govendor.
Tom Clegg
06:40 pm Arvados Revision f3c054cb (arvados): 11720: Control dependencies with govendor.
Tom Clegg
06:28 pm Arvados Revision 78390940 (arvados): 11546: Wrap lock/unlock in transactions.
Tom Clegg
05:51 pm Arvados Revision fbc576b7 (arvados): 11546: Avoid loading/saving non-essential fields in /arvados/v1/cont...
Tom Clegg
04:14 pm Arvados Bug #11137: apt-get upgrade arvados-api-server on Ubuntu 12.04 did not update crunch-job
crunch-job is in the arvados-src package. Perhaps apiserver needs to list "exact same version of arvados-src package"... Tom Clegg
02:49 pm Arvados Revision f3d300ce (arvados): 11546: Avoid loading/saving non-essential fields in /arvados/v1/cont...
Tom Clegg
02:43 pm Arvados Revision 7a02915c (arvados): 11546: Avoid loading/saving non-essential fields in /arvados/v1/cont...
Tom Clegg
02:24 pm Arvados Revision bbb8b8ee (arvados): 11546: Avoid loading/saving non-essential fields in /arvados/v1/cont...
Tom Clegg
02:12 pm Arvados Revision dfc18fd7 (arvados): 11546: Avoid loading/saving non-essential fields in /arvados/v1/cont...
Tom Clegg

05/16/2017

06:56 pm Arvados Story #7928 (Duplicate): [Keep] keepstore identifies underlying volumes to clients
covered by #11644 Tom Clegg
06:38 pm Arvados Bug #8205 (Duplicate): Node manager is not spinning up nodes.
Tom Clegg
06:34 pm Arvados Feature #10765 (Resolved): [arvados-ws] switch our test infrastructure over to arvados-ws
Tom Clegg
06:23 pm Arvados Bug #11209 (Resolved): stuck keep fuse mounts not cleared by crunch-job
Tom Clegg
06:20 pm Arvados Story #11705 (New): [API] Remove async permissions code
We added the option so we could experiment with it, but it's obsolete now that we check permission using a recursive ... Tom Clegg
06:18 pm Arvados Story #9186 (Closed): [API] Test client impact of async_permissions_update=True
Obsoleted by current implementation of permission checks. Tom Clegg
06:13 pm Arvados Story #8180 (Rejected): [Keepstore] Don't trash/delete blocks until they appear on multiple trash...
Tom Clegg
06:10 pm Arvados Bug #3229 (Closed): Keep logging improvements
Tom Clegg
06:08 pm Arvados Story #2877 (Rejected): Script looks for CGI data in a specified folder, scrapes header comments ...
Tom Clegg
06:06 pm Arvados Story #8233 (In Progress): [Arvbox] Fix remaining tests & use arvbox on jenkins
Tom Clegg
06:03 pm Arvados Bug #1668 (Rejected): workbench jobs page does not show who submitted a job
Tom Clegg
05:31 pm Arvados Revision 42f5f3a2 (arvados): 11644: Test fields in /mounts response.
Tom Clegg
05:25 pm Arvados Feature #11644 (In Progress): [keepstore] mount-oriented APIs
11644-mounts-api @ commit:42f5f3a29761ac9a943530157da376c798a4ac6d
Tom Clegg
05:23 pm Arvados Revision 9eff4af0 (arvados): 11644: Ensure generated UUIDs are always 27 chars.
Tom Clegg
05:22 pm Arvados Revision c4b5194c (arvados): 11644: Add volume replication level to /mounts response.
Tom Clegg
05:11 pm Arvados Revision 8cc51c76 (arvados): 11644: Replace linear search with map for looking up mounts by UUID.
Tom Clegg
05:07 pm Arvados Task #11687 (Resolved): Obey MountUUID restrictions in trash list
Tom Clegg
05:07 pm Arvados Task #11688 (Resolved): Obey MountUUID restrictions in pull list
Tom Clegg
05:03 pm Arvados Revision 74544f43 (arvados): 11644: Test non-empty MountUUID in trash list.
Tom Clegg
04:48 pm Arvados Revision 38de76ce (arvados): 11644: Add pull-to-mount-UUID test. Tidy up pull worker and tests.
Tom Clegg
02:57 pm Arvados Story #11580: [Workbench] Improve format of /container_requests index page
The addition to _show_status.html doesn't seem right to me. It looks like it's trying to preload the children of the ... Tom Clegg

05/15/2017

09:36 pm Arvados Task #11688 (In Progress): Obey MountUUID restrictions in pull list
Tom Clegg
09:36 pm Arvados Task #11687 (In Progress): Obey MountUUID restrictions in trash list
Tom Clegg
09:36 pm Arvados Task #11689 (Resolved): Accept index request for a specific mount
Tom Clegg
02:15 pm Arvados Task #11689 (In Progress): Accept index request for a specific mount
Tom Clegg
09:35 pm Arvados Revision 50c67a3e (arvados): 11644: Accept index/pull/trash requests for a specific mount.
Tom Clegg
03:15 pm Arvados Wiki edit: Proposals (#60)
Tom Clegg
02:14 pm Arvados Task #11686 (Resolved): Get filesystem UUIDs from linux APIs
Tom Clegg
02:13 pm Arvados Task #11685 (Resolved): Report current mounts using UUIDs assigned at runtime
Tom Clegg
02:11 pm Arvados Revision 953a9210 (arvados): 11644: Use storage account and bucket name as DeviceID for Azure vol...
Tom Clegg
02:10 pm Arvados Revision 209dbf15 (arvados): 11644: Use endpoint and bucket name as DeviceID for S3 volumes.
Tom Clegg
05:59 am Arvados Revision 61a88734 (arvados): 11644: Use filesystem UUID and path as DeviceID for local disk volumes.
Tom Clegg

05/12/2017

09:21 pm Arvados Revision 86907851 (arvados): 11644: Add /mounts endpoint using random mount UUIDs assigned at run...
Tom Clegg
06:15 pm Arvados Story #11580: [Workbench] Improve format of /container_requests index page
If we move the preload stuff out of ApplicationController then instance variables won't be scoped to the current requ... Tom Clegg
03:13 pm Arvados Bug #11669: [Crunch2] crunch-dispatch-slurm hits scancel too much
We could reduce the number of scancel invocations by ~1 per container by calling scancel() immediately the very first... Tom Clegg
02:56 pm Arvados Task #11685 (In Progress): Report current mounts using UUIDs assigned at runtime
Tom Clegg
02:52 pm Arvados Task #11685 (Resolved): Report current mounts using UUIDs assigned at runtime
Tom Clegg
02:56 pm Arvados Task #11689 (Resolved): Accept index request for a specific mount
Tom Clegg
02:55 pm Arvados Task #11688 (Resolved): Obey MountUUID restrictions in pull list
Tom Clegg
02:54 pm Arvados Task #11687 (Resolved): Obey MountUUID restrictions in trash list
Tom Clegg
02:53 pm Arvados Task #11686 (Resolved): Get filesystem UUIDs from linux APIs
Tom Clegg
02:47 pm Arvados Task #11683 (Resolved): Add logging to container reuse code
Tom Clegg
02:46 pm Arvados Task #11663 (Resolved): Review
Tom Clegg
02:45 pm Arvados Revision ed56c267 (arvados): Merge branch '11590-log-reuse'
refs #11590 Tom Clegg
02:34 pm Arvados Revision 0d5962b7 (arvados): 11590: Reword log messages.
Tom Clegg
02:33 pm Arvados Bug #11590: job reuse / re-use logic does not log decisions
Updated log messages:... Tom Clegg

05/11/2017

08:35 pm Arvados Revision 93e4548d (arvados): Merge branch '11681-str-newstr'
refs #11681 Tom Clegg
07:38 pm Arvados Bug #11681: [SDK] Cannot compare newstr() and str()
11681-str-newstr @ commit:9cd1bd27bada41cd13fb9813144fe694a67c2d00
Tom Clegg
07:37 pm Arvados Revision 9cd1bd27 (arvados): 11681: Return native str type from KeepLocator.__str__() and Collect...
Tom Clegg
05:57 pm Arvados Bug #11669: [Crunch2] crunch-dispatch-slurm hits scancel too much
Sequence is
# crunch-run updates container state to Complete or Cancelled
# crunch-dispatch-slurm waits some amount...
Tom Clegg
03:41 pm Arvados Bug #11590: job reuse / re-use logic does not log decisions
11590-log-reuse @ commit:fb5d279228a7adcafd858d9e137accd8010bb382 Tom Clegg
03:13 pm Arvados Bug #11590: job reuse / re-use logic does not log decisions
... Tom Clegg
03:08 pm Arvados Bug #11590 (In Progress): job reuse / re-use logic does not log decisions
Example logs... Tom Clegg
03:38 pm Arvados Revision fb5d2792 (arvados): 11590: Add "reuse jobs even if conflicting outputs exist" and "log r...
Tom Clegg
02:34 pm Arvados Bug #11679: [Workbench] Logs containers with undefined exit code
arvados-ws sends properties.new_attributes.state but does not send properties.new_attributes.exit_code; see sendObjec... Tom Clegg

05/10/2017

09:47 pm Arvados Bug #11669: [Crunch2] crunch-dispatch-slurm hits scancel too much
11669-tidy-scancel-race @ commit:f29a53dab584dcae3e4a38a7623efa6d3b30a663
Tom Clegg
09:44 pm Arvados Bug #11669 (In Progress): [Crunch2] crunch-dispatch-slurm hits scancel too much
Moved from #11626
Example:...
Tom Clegg
09:46 pm Arvados Revision f29a53da (arvados): 11669: Avoid 0.5 redundant scancel() when ctr disappears from squeue...
Tom Clegg
09:21 pm Arvados Bug #11626: [Crunch2] Still not propagating slurm errors to user
I missed that the example logs above are from a real site, not the test suite. So the mystery is why scancel wasn't e... Tom Clegg
08:04 pm Arvados Bug #11626: [Crunch2] Still not propagating slurm errors to user
oops, just noticed
log.Printf(text)
should be
log.Print(text)
Other than that, LGTM @ commit:f8675ad473b4...
Tom Clegg
07:56 pm Arvados Bug #11626: [Crunch2] Still not propagating slurm errors to user
Peter Amstutz wrote:
> Hhe problem I was trying to solve was that the logs are full of this:
>
> [...]
>
> Thi...
Tom Clegg
09:11 pm Arvados Feature #11668 (New): [SDKs] Exceptions raised by Python SDK should include the "error token" if ...
This will make debugging easier in cases where "check the API server logs" activity is involved.
Tom Clegg
07:45 pm Arvados Task #11664 (Resolved): Review
Tom Clegg
07:45 pm Arvados Task #11663 (Resolved): Review
Tom Clegg
07:45 pm Arvados Task #11661 (Resolved): Review 11644-mounts-api
Tom Clegg
07:22 pm Arvados Story #11308 (Resolved): Support Python3 for arvados-python-client & command line utilities
Packaging moved to #11657 Tom Clegg
07:21 pm Arvados Story #11657 (New): [SDKs] publish python3-arvados-python-client packages
Tom Clegg
07:19 pm Arvados Task #11379 (Resolved): Review 11308-python3
Tom Clegg
07:15 pm Arvados Bug #11629 (Resolved): [API] NoMemoryError in GroupsController#contents
Tom Clegg
02:02 pm Arvados Bug #11629 (In Progress): [API] NoMemoryError in GroupsController#contents
https://ci.curoverse.com/job/run-tests-apps-workbench-units/888/console... Tom Clegg
02:23 pm Arvados Revision c22bd855 (arvados): Merge branch '11629-groups-contents-memory'
refs #11629
refs #11458
Tom Clegg
02:22 pm Arvados Revision 96b33e5a (arvados): 11629: Fix ambiguous column reference.
Tom Clegg
01:58 pm Arvados Story #11458: Jenkins step to run test_with_arvbox.sh
The dict_keys thing was addressed in #11308#note-18 (notes 18 to 21), merged in commit:4bc2f36b8e475b0dc1145fa27e665c... Tom Clegg

05/09/2017

08:45 pm Arvados Bug #11634 (Resolved): [Tests] Rails install fails when bundle specifies a gem included in system...
Applied in changeset arvados|commit:9456885954b0bff02721bbb14da296be212efdc4. Tom Clegg
08:37 pm Arvados Revision 94568859 (arvados): Merge branch '11634-bundler-cache'
closes #11634 Tom Clegg
08:35 pm Arvados Bug #11629 (Resolved): [API] NoMemoryError in GroupsController#contents
Applied in changeset arvados|commit:ba917d72d48615cdd0c6da87d41b6bd0f9f26666. Tom Clegg
08:33 pm Arvados Revision ba917d72 (arvados): Merge branch '11629-groups-contents-memory'
closes #11629 Tom Clegg
07:49 pm Arvados Revision 4bc2f36b (arvados): Merge branch '11308-collection-keys'
refs #11308 Tom Clegg
07:32 pm Arvados Story #11308: Support Python3 for arvados-python-client & command line utilities
11308-collection-keys @ commit:47f29d75759db7e4d81203e110257f1a471f82e7
Tom Clegg
07:31 pm Arvados Revision 47f29d75 (arvados): 11308: Fix modes not tested in test case.
Tom Clegg
07:30 pm Arvados Revision 8e0303a5 (arvados): 11308: collection.keys() supports indexing for python2 callers.
Tom Clegg
07:12 pm Arvados Story #7475: [Node manager] Better communication when job is unsatisfiable
For crunch2, when node manager is not in use, sbatch rejects unsatisfiable jobs and the user gets an error -- however... Tom Clegg
03:38 pm Arvados Wiki edit: Keep_storage_tiers (#7)
Tom Clegg
03:38 pm Arvados Feature #11646 (New): [keepstore] Support X-Keep-Desired-Tier header
Part of the [[Keep storage tiers]] story.
Accept X-Keep-Desired-Tier header in write requests.
* If X-Keep-Desire...
Tom Clegg
03:38 pm Arvados Feature #11645 (New): [keepstore] Add "tier" field to volume config
Part of the [[Keep storage tiers]] story.
Accept a "Tier" int for each configured volume.
* If config is zero, ne...
Tom Clegg
03:19 pm Arvados Feature #11644 (Resolved): [keepstore] mount-oriented APIs
Keepstore's role in the management part of the [[Keep storage tiers]] story.
* A "mounts" request (@GET /mounts@) sh...
Tom Clegg

05/08/2017

08:10 pm Arvados Bug #11638 (Resolved): [arvados-ws] Label and limit database connections
Applied in changeset arvados|commit:08e9c6aff46e182b9cd14fab9d42d21b2d7c6902. Tom Clegg
07:04 pm Arvados Bug #11638: [arvados-ws] Label and limit database connections
11638-limit-conns @ commit:3652f3ed82e994c9eea74ba1afb4aa3c11ebd697
Tom Clegg
06:08 pm Arvados Bug #11638 (In Progress): [arvados-ws] Label and limit database connections
Tom Clegg
06:08 pm Arvados Bug #11638 (Resolved): [arvados-ws] Label and limit database connections
arvados-ws appears to open too many Postgres connections, eventually reaching the Postgres server limit and preventin... Tom Clegg
08:04 pm Arvados Revision 08e9c6af (arvados): Merge branch '11638-limit-conns'
closes #11638 Tom Clegg
08:04 pm Arvados Revision c4a0f7a7 (arvados): 11638: Log a warning if Postgres pool size is unlimited.
Tom Clegg
07:55 pm Arvados Bug #11626: [Crunch2] Still not propagating slurm errors to user
I'm not sure why the HasUUID() check is added in run() in crunch-dispatch-slurm.go. ctx.Done() fires as soon as HasUU... Tom Clegg
07:12 pm Arvados Task #11633 (In Progress): Review 11626-crunch2-log-slurm-errors
Tom Clegg
07:02 pm Arvados Revision 3652f3ed (arvados): 11638: Add PostgresPool config option, and default application_name
Tom Clegg
06:27 pm Arvados Bug #11629: [API] NoMemoryError in GroupsController#contents
Yes to both, thanks. These updates also revealed that the changes had broken items_available in the groups#contents r... Tom Clegg
06:21 pm Arvados Revision 4cacfe7a (arvados): 11629: Apply memory limits to job/pipeline "components" fields.
Tom Clegg
06:17 pm Arvados Revision 3f285d4c (arvados): 11629: Fix items_available in groups#contents response.
Tom Clegg
04:06 pm Arvados Revision 9bf79049 (arvados): Merge branch '11308-python3'
refs #11308 Tom Clegg
04:05 pm Arvados Revision 8976780c (arvados): 11308: Merge branch 'master' into 11308-python3
Tom Clegg
02:16 pm Arvados Bug #11634 (In Progress): [Tests] Rails install fails when bundle specifies a gem included in sys...
Tom Clegg
02:16 pm Arvados Bug #11634: [Tests] Rails install fails when bundle specifies a gem included in system ruby
11634-bundler-cache @ commit:f21a495d7723e72ea61c62293d56aff1a65c130c
Tom Clegg
01:51 pm Arvados Bug #11634 (Resolved): [Tests] Rails install fails when bundle specifies a gem included in system...
If a system gem is used to satisfy a Gemfile dependency, Bundler tries to update the system gem cache:... Tom Clegg
02:07 pm Arvados Revision f21a495d (arvados): 11634: Prevent bundler from trying to use/update system gem cache.
Tom Clegg
01:59 pm Arvados Revision 3b11d2b5 (arvados): 11634: Fix whitespace.
Tom Clegg

05/05/2017

08:57 pm Arvados Wiki edit: Hacking_prerequisites (#34)
Tom Clegg
08:56 pm Arvados Wiki edit: Hacking_prerequisites (#33)
Tom Clegg
08:52 pm Arvados Wiki edit: Hacking_prerequisites (#32)
Tom Clegg
07:32 pm Arvados Bug #11629: [API] NoMemoryError in GroupsController#contents
11629-groups-contents-memory @ commit:cb109bfddd08bd8136b75e90b681e4af3d60ea30
Tom Clegg
06:13 pm Arvados Bug #11629 (In Progress): [API] NoMemoryError in GroupsController#contents
Tom Clegg
05:17 pm Arvados Bug #11629 (Resolved): [API] NoMemoryError in GroupsController#contents
Presumably, we are seeing this error because (unlike collections#index) the groups#contents controller does not respe... Tom Clegg
07:32 pm Arvados Task #11630 (Resolved): Review 11629-groups-contents-memory
Tom Clegg
07:30 pm Arvados Revision cb109bfd (arvados): 11629: Limit database reads for all list responses, not just index.
Tom Clegg
02:46 pm Arvados Story #11308: Support Python3 for arvados-python-client & command line utilities
11308-python3 @ commit:ffb304afad93f50fd6ee43ecda6584dd0ac000c4 https://ci.curoverse.com/job/developer-run-tests/272/
Tom Clegg

05/04/2017

05:02 pm Arvados Revision f10431fe (arvados): 8019: Use tl.stopping channel instead of tl.flush to signal Close. U...
Tom Clegg
03:16 pm Arvados Revision ffb304af (arvados): 11308: Merge branch 'master' into 11308-python3
Tom Clegg
03:09 pm Arvados Revision 5ca261ee (arvados): 11308: Fix bytes vs. strings from recent merge.
Tom Clegg

05/03/2017

02:25 pm Arvados Bug #11606 (Resolved): [CWL] [API] empty array submitted as null
Applied in changeset arvados|commit:e24a892b920fd8cddf091a3c15acba1ab27f510d. Tom Clegg
02:20 pm Arvados Revision e24a892b (arvados): Merge branch '11606-rails4-no-munge'
closes #11606 Tom Clegg
02:20 pm Arvados Revision fa0ccc4c (arvados): 11308: Stop test suite if api etc. cannot be started.
Tom Clegg

05/02/2017

06:46 pm Arvados Bug #5076: [Workbench] Replace schizophrenic generic "groups" page with "roles" page that only cr...
Estimates:
* 0.5 points for a "roles" page that only lists groups with group_class="role", and has a "create new rol...
Tom Clegg
06:25 pm Arvados Bug #11590: job reuse / re-use logic does not log decisions
For crunch1 and crunch2: add a config flag "log_reuse_decisions". If true, in the "find reusable" code, build the Act... Tom Clegg
05:02 pm Arvados Revision 607fe087 (arvados): 11308: Merge branch 'master' into 11308-python3
Conflicts:
sdk/python/arvados/arvfile.py
sdk/python/arvados/commands/put.py
sdk/python/tests/test_collections.py
Tom Clegg
03:48 pm Arvados Task #11607 (Resolved): Review 11606-rails4-no-munge
Tom Clegg
03:41 pm Arvados Bug #11606: [CWL] [API] empty array submitted as null
Our monkeypatch stopped working, and our test case wasn't running because it wasn't *_test.rb. Rails4 has a builtin f... Tom Clegg
03:09 pm Arvados Bug #11606: [CWL] [API] empty array submitted as null
Despite the test case added in #10935, commit:b07d8a573c74e7be56a2fd8c5064032c37d90c67, [] is getting munged to null ... Tom Clegg
02:39 pm Arvados Bug #11606 (Resolved): [CWL] [API] empty array submitted as null
https://workbench.9tee4.arvadosapi.com/container_requests/9tee4-xvhdp-9xb0s71vn96wfjc#Log... Tom Clegg
03:39 pm Arvados Revision 974799e0 (arvados): 11606: Disable deep_munge behavior. Rename test so it runs in test s...
Tom Clegg

05/01/2017

07:52 pm Arvados Task #11482 (Resolved): Review
Tom Clegg
07:50 pm Arvados Wiki edit: Keep_storage_tiers (#6)
Tom Clegg
07:17 pm Arvados Wiki edit: Keep_storage_tiers (#5)
Tom Clegg
06:13 pm Arvados Wiki edit: Keep_storage_tiers (#4)
Tom Clegg
06:10 pm Arvados Feature #10988 (Resolved): [DOC] update documentation to list ubuntu1604
Applied in changeset arvados|commit:ed375727b95e6f1cd56c599a177bbd7979e1cf78. Tom Clegg
06:03 pm Arvados Revision ed375727 (arvados): Merge branch '10988-document-ubuntu1604-support'
closes #10988 Tom Clegg
06:02 pm Arvados Revision 70166077 (arvados): 10988: Remove apt.arvados.org repo priority: we no longer publish/re...
Tom Clegg

04/28/2017

09:04 pm Arvados Revision 059a62b5 (arvados): 11183: In docker-compose test, wait for checks only on head node.
Tom Clegg
07:52 pm Arvados Revision 87f40ab7 (arvados): 11183: Fix "just setup" test.
Tom Clegg
06:09 pm Arvados Bug #11547: [API] Excessive memory and time consumed by API and dispatch when container fields (l...
Indeed. Given that result, I don't think a column size limit seems like a good strategy for avoiding slowness.
See...
Tom Clegg
05:39 pm Arvados Revision 1522e9dc (arvados): 11183: Rearrange source tree
Tom Clegg
05:12 pm Arvados Revision a328dd7d (arvados): 11183: Start API server
Tom Clegg
05:12 pm Arvados Revision 887ca969 (arvados): 11183: Add "setup -wait" flag
Tom Clegg
05:12 pm Arvados Revision 55466cec (arvados): 11183: Avoid reusing httpclient via vault config to prevent "protoco...
Tom Clegg
05:12 pm Arvados Revision 957ec5ff (arvados): 11183: Less noise while waiting for health checks.
Tom Clegg
05:12 pm Arvados Revision fb08faf4 (arvados): 11183: More consul setup.
Tom Clegg
05:12 pm Arvados Revision ff310f43 (arvados): 11183: Fix setup-already-done check.
Tom Clegg
05:12 pm Arvados Revision 1883fbbd (arvados): 11183: Add "arvados-admin setup"
Tom Clegg
05:05 pm Arvados Bug #11537 (Resolved): [keepproxy] detect proxy loop (broken config) -- return an error if keeppr...
Applied in changeset arvados|commit:e56ae6aad06c37d5512537047871d7363dd97620. Tom Clegg
05:01 pm Arvados Task #11573 (Resolved): Review 11537-keepproxy-loop
Tom Clegg
04:59 pm Arvados Revision e56ae6aa (arvados): Merge branch '11537-keepproxy-loop'
closes #11537 Tom Clegg
03:43 pm Arvados Wiki edit: Keep_storage_tiers (#3)
Tom Clegg
12:46 am Arvados Wiki edit: Keep_storage_tiers (#2)
Tom Clegg
02:07 pm Arvados Revision 88a25d41 (arvados): 11537: Add Via header to get/head/post/put responses.
Tom Clegg
02:07 pm Arvados Revision 72900c01 (arvados): 11537: Add Via header to proxied keepstore requests.
Propagate keepclient PUT error messages to caller. Tom Clegg

04/27/2017

08:55 pm Arvados Revision 6a7edde0 (arvados): Merge branch '7709-rails-observers'
refs #7709 Tom Clegg
08:51 pm Arvados Revision 41a7f391 (arvados): 7709: Add rails-observers gem so plugins can use it.
Tom Clegg
08:02 pm Arvados Wiki edit: Keep_storage_tiers (#1)
Tom Clegg
07:29 pm Arvados Bug #11537: [keepproxy] detect proxy loop (broken config) -- return an error if keepproxy receive...
Implementation:
* Instead of adding a Do field to keepclient.KeepClient as described, I changed the Client field's t...
Tom Clegg
07:24 pm Arvados Revision baa1c3db (arvados): 11537: Add Via header to get/head/post/put responses.
Tom Clegg
06:29 pm Arvados Revision f7c0c213 (arvados): 11537: Add Via header to get/head/post/put responses.
Tom Clegg
05:49 pm Arvados Bug #11583: [crunch-run] Fix excessive memory use
Adding \n to the TestWriteLogsLarge Printf format doesn't seem to make any difference, suggesting this isn't just abo... Tom Clegg
05:32 pm Arvados Bug #11583 (New): [crunch-run] Fix excessive memory use
The crunch-run test suite seems to have started leaking memory in #9132. RSS climbs steadily with each test case. Lea... Tom Clegg
05:12 pm Arvados Bug #8675 (Resolved): [Crunch2] Unreliable test TestWriteMultipleLogs in services/crunch-run
Fixed in #9272 commit:a5260a0bd87f83b17744445476aaf7f107ca24e4 Tom Clegg
05:05 pm Arvados Feature #7709 (Resolved): [API] Upgrade API server to Rails 4.2
Applied in changeset arvados|commit:318c49002aea966128a9d37ab29e601a104d79bb. Tom Clegg
05:03 pm Arvados Revision e6201e51 (arvados): 11537: Add Via header to proxied keepstore requests.
Propagate keepclient PUT error messages to caller. Tom Clegg
04:59 pm Arvados Revision 318c4900 (arvados): Merge branch '7709-api-rails4'
closes #7709 Tom Clegg
03:44 pm Arvados Feature #11582 (New): [API] Delete old job/container logs in main server process, instead of rake...
The API server comes with two rake tasks, db:delete_old_job_logs and db:delete_old_container_logs. They are intended ... Tom Clegg
02:51 pm Arvados Revision fb2fd7c8 (arvados): 11537: Add Via header to proxied keepstore requests.
Propagate keepclient PUT error messages to caller. Tom Clegg

04/26/2017

09:11 pm Arvados Revision 8a1a20bc (arvados): 11537: Add Via header to proxied keepstore requests.
Tom Clegg
08:51 pm Arvados Bug #11537 (In Progress): [keepproxy] detect proxy loop (broken config) -- return an error if kee...
Tom Clegg
07:53 pm Arvados Task #11573 (Resolved): Review 11537-keepproxy-loop
Tom Clegg
07:17 pm Arvados Bug #11561 (New): [API] Limit number of lock/unlock cycles for a given container
Currently, if a container cannot be started due to some infrastructure problem (whether or not it's related to the sp... Tom Clegg
07:17 pm Arvados Bug #11190 (Resolved): Containers seem to run more than once, which isn't supposed to happen
Tom Clegg
07:02 pm Arvados Bug #11190: Containers seem to run more than once, which isn't supposed to happen
Allowing multiple dispatch attempts is a deliberate feature: when the dispatch/startup infrastructure fails early eno... Tom Clegg
07:16 pm Arvados Task #11263 (Resolved): Review
Tom Clegg
06:52 pm Arvados Bug #11209 (Feedback): stuck keep fuse mounts not cleared by crunch-job
Tom Clegg
02:45 pm Arvados Bug #11209: stuck keep fuse mounts not cleared by crunch-job
Joshua Randall wrote:
> It may be a workaround for us to get rid of /etc/mtab and just symlink it to /proc/self/mo...
Tom Clegg
04:31 pm Arvados Bug #11547: [API] Excessive memory and time consumed by API and dispatch when container fields (l...
Peter Amstutz wrote:
> Try creating a collection record with a complex "properties" field (e.g. 100,000 dict entries...
Tom Clegg
02:56 pm Arvados Bug #11547: [API] Excessive memory and time consumed by API and dispatch when container fields (l...
AFAIK apiserver regularly handles >75 MB serialized "properties" fields in log records, so I'm not convinced that the... Tom Clegg
02:33 pm Arvados Revision 137ebf94 (arvados): 7709: Merge branch 'master' into 7709-api-rails4
Tom Clegg

04/25/2017

06:45 pm Arvados Wiki edit: Proposals (#59)
Tom Clegg
05:22 pm Arvados Revision 95e2bdda (arvados): Merge branch '11544-bad-collection-id-500'
closes #11544 Tom Clegg
05:01 pm Arvados Revision 629557aa (arvados): 11544: Return 404 instead of 500 for malformed collection IDs.
Tom Clegg
05:14 am Arvados Revision 3ef580c4 (arvados): 7709: Merge branch 'master' into 7709-api-rails4
Conflicts:
services/api/db/structure.sql
Tom Clegg

04/24/2017

07:21 pm Arvados Task #11526 (Resolved): Review 11470-update-task-fields
Tom Clegg
06:57 pm Arvados Revision bf5d77ba (arvados): Fix panic in test case (overwriting a locked sync.Mutex with an unlo...
No issue # Tom Clegg
06:05 pm Arvados Revision 04bd6b08 (arvados): Merge branch '11517-no-count-logs'
refs #11517 Tom Clegg
06:02 pm Arvados Revision 840b855f (arvados): 11517: Add test for count=none.
Tom Clegg
06:00 pm Arvados Revision 1220e218 (arvados): 11517: Skip database reset in some workbench unit tests.
Reduces unit test suite time by half. Tom Clegg
05:42 pm Arvados Revision 17b80c32 (arvados): 11517: Comment with_count() method
Tom Clegg
02:39 pm Arvados Story #11308: Support Python3 for arvados-python-client & command line utilities
11308-python3 @ commit:3a0d849c08f750dca1d6a40153c0107001769c6d https://ci.curoverse.com/job/developer-run-tests/244/... Tom Clegg
02:25 pm Arvados Bug #11537 (Resolved): [keepproxy] detect proxy loop (broken config) -- return an error if keeppr...
The manual setup procedure makes it easy to misconfigure keepproxy and Nginx such that keepproxy fulfills requests by... Tom Clegg
02:04 pm Arvados Revision 35c25727 (arvados): 11517: Use count=none when retrieving logs.
Tom Clegg

04/22/2017

06:29 am Arvados Revision 3a0d849c (arvados): 11308: Fix arvfile append mode: write() changes the file pointer.
Tom Clegg
06:27 am Arvados Revision 2b3b54be (arvados): 11308: Fix bytes vs. strings from recent merge.
Tom Clegg

04/21/2017

08:02 pm Arvados Revision 31d31c01 (arvados): 11308: Merge branch 'master' into 11308-python3
Conflicts:
sdk/python/arvados/arvfile.py
sdk/python/arvados/commands/get.py
sdk/python/arvados/commands/migrate19....
Tom Clegg

04/20/2017

07:36 pm Arvados Bug #11531 (New): [API] clean up stale/conflicting dns data from deleted node records
Problem scenario:
* node compute100 comes up with ip address = 10.2.3.4
* node record compute100 is deleted
* node...
Tom Clegg
06:10 pm Arvados Revision 2333472a (arvados): Merge branch '11470-update-task-fields'
refs #11470 Tom Clegg
05:53 pm Arvados Bug #11510: [SDK] Support writes to offsets beyond end of file
Peter Amstutz wrote:
> > * test return value of seek() is new file position
>
> Also tested in test_truncate3().
...
Tom Clegg
04:44 pm Arvados Bug #11510: [SDK] Support writes to offsets beyond end of file
11510-sdk-extend-files @ commit:f5638851ddb5a859a54a16fe963ee98591af17a9
* remove debugging print statements in arvf...
Tom Clegg
03:50 pm Arvados Task #11528 (In Progress): Review 11510-sdk-extend-files
Tom Clegg
03:50 pm Arvados Task #11528 (Resolved): Review 11510-sdk-extend-files
Tom Clegg
02:17 am Arvados Task #11526 (Resolved): Review 11470-update-task-fields
Tom Clegg
02:17 am Arvados Bug #11470: [API] container_requests#update alternately responds 422 or 404 for no apparent reason
Bugfix encountered by accident while investigating:
11470-update-task-fields @ commit:9aa83ad7b4de05dd2818885ed341...
Tom Clegg
02:16 am Arvados Revision 9aa83ad7 (arvados): 11470: Fix identifiers to conventional Go.
Tom Clegg
02:09 am Arvados Revision 3c34e713 (arvados): 11470: Fix accidentally cleared fields during job_task update.
Updating with a Task struct had the effect of clearing job_uuid,
created_by_job_task_uuid, and parameters fields, and...
Tom Clegg
 

Also available in: Atom