Activity

From 02/15/2017 to 03/16/2017

03/16/2017

08:57 pm Task #11271 (Resolved): Review 11254-nodemanager-no-actor
Peter Amstutz
08:40 pm Task #11271 (In Progress): Review 11254-nodemanager-no-actor
Tom Clegg
08:55 pm Bug #11254 (Resolved): [Node manager] backtrace on node shutdown
Applied in changeset arvados|commit:996b635700d7270229200a56d2c2b9f7c96a84fb. Peter Amstutz
08:39 pm Bug #11254: [Node manager] backtrace on node shutdown
11254-nodemanager-no-actor @ commit:2c69d49 LGTM
...although I'd say the duplicated conditions here were already a...
Tom Clegg
08:52 pm Revision 996b6357 (arvados): Merge branch '11254-nodemanager-no-actor' closes #11254
Peter Amstutz
08:49 pm Revision c733e426 (arvados): 11209: Add arv-mount --unmount and --replace flags
Tom Clegg
08:49 pm Revision b60a21fe (arvados): 11254: Refactor _node_states
Peter Amstutz
08:43 pm Revision 9a7d9243 (arvados): 11209: Add arv-mount --unmount and --replace flags
Tom Clegg
08:28 pm Bug #4124: [Crunch] Socket timed out on send/recv operation causes pipeline failure
I don't know if we ever updated our default puppet configuration.
We've seen some unexplained SLURM timeouts on so...
Peter Amstutz
08:11 pm Revision 2c69d491 (arvados): 11254: Cloud nodes where "actor is None" are considered to be in shu...
only time it should be "None" is the period between a successful shutdown and
when the node disappears from the cloud...
Peter Amstutz
07:16 pm Revision 9f34ff6f (arvados): 10218: Use a []string for the entire command instead of splitting it.
Lucas Di Pentima
07:13 pm Bug #11014: [Workbench] Hide the "idle/busy nodes" part of the dashboard if crunch1 APIs are disa...
Re-ran the tests here: https://ci.curoverse.com/job/developer-run-tests/192/
Lucas Di Pentima
06:22 pm Bug #11014: [Workbench] Hide the "idle/busy nodes" part of the dashboard if crunch1 APIs are disa...
Updates on branch @11014-hide-node-status@ at commit:73dbf5b
Tests run at: https://ci.curoverse.com/job/developer-ru...
Lucas Di Pentima
02:48 pm Bug #11014 (In Progress): [Workbench] Hide the "idle/busy nodes" part of the dashboard if crunch1...
Lucas Di Pentima
06:18 pm Revision 73dbf5bf (arvados): 11014: Check for PipelineIndex#index API to exist for rendering the ...
Added related test. Lucas Di Pentima
05:24 pm Revision 386faadf (arvados): 10218: Wait for container to be started (not just created) before tr...
Tom Clegg
02:01 pm Revision 8513e042 (arvados): 10218: Split multi-line command output so that each line is written ...
Lucas Di Pentima
04:52 am Bug #11278 (Resolved): Trailing space on string pasted into search box should not cause "Oops req...
String was "170302-e00504-0041-bhh5ytalxx "
Deleting the trailing space causes this search to work
Tom Morris
01:43 am Bug #9588 (Resolved): [CWL] cwl-runner parses nested arrays repetively
This is no longer an issue. Jiayong Li

03/15/2017

11:13 pm Task #3718 (Closed): Hash out desired API with science team
Tom Morris
11:13 pm Task #3957 (Closed): crunch-job uses num_retries support in Perl SDK
Tom Morris
11:12 pm Task #3956 (Closed): Add num_retries arguments throughout Perl SDK
Tom Morris
11:12 pm Task #3060 (Closed): List of shared folders should say who owns each one (or description)
Tom Morris
11:12 pm Task #3058 (Closed): Suggest "what to do next" when viewing a new/empty folder
Tom Morris
11:12 pm Task #2645 (Closed): Generate "ready to ingest" message on the event bus
Tom Morris
11:12 pm Task #2633 (Closed): Implement TLS support internally
Tom Morris
11:12 pm Task #2636 (Closed): Test SSL support
Tom Morris
11:09 pm Task #5031 (Closed): Review/feedback
Tom Morris
11:04 pm Task #7726 (Closed): Review 7724-scoped-token
Tom Morris
10:54 pm Task #8846 (Closed): Review
Tom Morris
10:53 pm Task #9533 (Closed): Review 4753-websocket-test-port
Tom Morris
10:53 pm Task #2806 (Closed): Delete active and success attributes from pipeline_instance
Tom Morris
10:43 pm Task #3458 (Closed): Review 3452-put-collection-in-project
Tom Morris
10:27 pm Task #8955 (Closed): Review 8952-node-manager-safer-slurm-shutdown-check-wip
Tom Morris
10:21 pm Revision 1230d8a1 (arvados): 10218: Logging node information (cpu, mem, disk) by storing command ...
Lucas Di Pentima
08:13 pm Bug #10669 (New): googleapiclient.errors.InvalidJsonError retrieving discovery document
Tom Morris
08:12 pm Bug #10669 (Duplicate): googleapiclient.errors.InvalidJsonError retrieving discovery document
Tom Morris
08:12 pm Bug #9521 (Duplicate): Corrupt discovery document should be handled better
Tom Morris
08:10 pm Task #11272 (Resolved): Review 7824-arvls-arvput-collection-api-usage
Lucas Di Pentima
08:10 pm Task #11271 (Resolved): Review 11254-nodemanager-no-actor
Peter Amstutz
08:10 pm Task #11269 (Resolved): Review 11170-stale-squeue-procs
Lucas Di Pentima
08:07 pm Task #11268 (Resolved): Review 11014-hide-node-status
Radhika Chippada
08:07 pm Task #11267 (Resolved): Review 10669-safe-http-cache
Tom Clegg
08:07 pm Task #11266 (Resolved): Review 11065-rotate-logs
Tom Clegg
08:07 pm Task #11265 (Resolved): Review branch 3821-collection-file-manage
Radhika Chippada
08:07 pm Task #11264 (Resolved): Review 7709-api-rails4
Tom Clegg
08:07 pm Task #11263 (Resolved): Review
Tom Clegg
08:06 pm Bug #11254 (In Progress): [Node manager] backtrace on node shutdown
Tom Morris
08:03 pm Bug #11254 (Resolved): [Node manager] backtrace on node shutdown
Fixed slurmd in the compute nodes and it is working ok now. Javier Bértoli
06:53 pm Bug #11254: [Node manager] backtrace on node shutdown
The backtrace looks like an unintended side effect of #10846.
On further research, we determined the underlying re...
Peter Amstutz
03:45 pm Bug #11254 (Resolved): [Node manager] backtrace on node shutdown
After upgrading c97qk to Xenial, trying to run a test job https://workbench.c97qk.arvadosapi.com/pipeline_instances/c... Javier Bértoli
07:43 pm Feature #11255: [Crunchv2] Option to use host networking for containers
summon Nico before discussing implementing Nico César
03:50 pm Feature #11255 (Resolved): [Crunchv2] Option to use host networking for containers
Docker bridge networking has bugs on certain kernels. Add a flag to crunch-run to instruct containers to use host ne... Peter Amstutz
07:35 pm Story #9132: [Crunch2] crunch-run uses official docker client library
Please take over 9132-dockerclient
* Replace API calls to equivalent ones, see https://godoc.org/github.com/docker...
Peter Amstutz
07:31 pm Bug #11261: [API] full text search on pipeline_instances table is extremely slow (wrong index?)
... Tom Clegg
07:30 pm Bug #11261: [API] full text search on pipeline_instances table is extremely slow (wrong index?)
Copied from #11071:
Version query:...
Tom Clegg
07:29 pm Bug #11261 (New): [API] full text search on pipeline_instances table is extremely slow (wrong ind...
Tom Clegg
07:26 pm Bug #11260 (New): [API] "arv group contents" produces a 404 error (seems to POST instead of GET?)
Tom Clegg
07:23 pm Bug #11071: [Workbench] qr1hi performance tests are broken
Version query:... Tom Clegg
04:55 pm Bug #11071: [Workbench] qr1hi performance tests are broken
I searched using "hash" in our staging environment, and I saw about 10% improvement for the first page.
Before:
...
Radhika Chippada
07:21 pm Story #11259 (Duplicate): [VG][myBRCA] Update status API at beginning and end of sample run
Tom Morris
07:13 pm Story #11259 (Duplicate): [VG][myBRCA] Update status API at beginning and end of sample run
Tom Morris
06:54 pm Bug #11257 (New): [CWL] Fails with $schemas referencing remote resources.
Workflows with external http resources listed in $schemas get into trouble with the code to find and upload local dep... Peter Amstutz
05:37 pm Bug #11237: python-arvados-python-client depends on old versions of packages
This pin comes from commit:ec3ff9b4 "Pin google-api-python-client and oauth2client versions to deal with incompatabil... Tom Clegg
05:20 pm Task #11141 (Resolved): Review 6520-nodemanager-docs
Peter Amstutz
03:31 pm Story #11251: [Tests] Make workbench integration tests finish faster (at least in Jenkins)
IMO we should be fixing slow/redundant test cases before we give up and make the test pipeline/server fatter. It look... Tom Clegg
03:23 am Story #11251: [Tests] Make workbench integration tests finish faster (at least in Jenkins)
Branch 11251-split-wb-integration-tests takes an initial stab at using minutest --exclude option.
This branch inte...
Radhika Chippada
03:19 am Story #11251: [Tests] Make workbench integration tests finish faster (at least in Jenkins)
Similar to workbench_units, workbench_functionals, workbench_integration, it would be helpful if we further split wor... Radhika Chippada
01:29 am Story #11251 (New): [Tests] Make workbench integration tests finish faster (at least in Jenkins)
Radhika Chippada
03:11 pm Task #11205 (Resolved): Review branch 11071-fts-perf-test
Radhika Chippada
03:03 pm Revision 54837bca (arvados): 8567: Add check that admin token is used and ensure that migration l...
created owned by system user. Also fix tests now that arv-keepdocker uses
logging instead of printing directly to sy...
Peter Amstutz
02:54 pm Bug #11209: stuck keep fuse mounts not cleared by crunch-job
The "fuse bug":https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=632258 seems to be related to a double-mounted mount... Tom Clegg
11:15 am Bug #11209: stuck keep fuse mounts not cleared by crunch-job
When I run `umount -f` to clear the problem, it has always been as root. Never tried running it as any other user. Joshua Randall
02:41 pm Story #9589: [Workbench] Update collection interface for collections with non-nil trash_at
* Do we want to support "Move collection" option for an expiring collection?
* Do we want to support "Copy to proj...
Radhika Chippada
04:10 am Revision dd1db553 (arvados): 11251: -v works with -e in TESTOPTS
Radhika Chippada
02:58 am Revision 0ed39efc (arvados): 11251: an intial stab at using minitest --exclude option to split wo...
Radhika Chippada
12:15 am Bug #4124: [Crunch] Socket timed out on send/recv operation causes pipeline failure
Peter - please review whether the recommended message timeout has already been incorporated and close this out. Thx Tom Morris

03/14/2017

11:52 pm Bug #4886 (Closed): [Docker] Web upload stuck at "upload in progress"
Sorry this lingered so long, but the code base has change so much since this was reported that it'd be impossible to ... Tom Morris
11:41 pm Bug #8371: [Deployment] Upgrading Rails packages on CentOS uninstalls them
Javier - can you please review the current state of this and either close or update? Thanks! Tom Morris
08:05 pm Bug #11071 (Resolved): [Workbench] qr1hi performance tests are broken
Applied in changeset arvados|commit:b1aa6c85c3db947f0963b1bddb2784c115f5c97d. Radhika Chippada
07:59 pm Bug #11071: [Workbench] qr1hi performance tests are broken
> Do you think that the line at services/api/app/controllers/arvados/v1/groups_controller.rb:99 could be improved in ... Radhika Chippada
07:26 pm Bug #11071: [Workbench] qr1hi performance tests are broken
This lgtm. Only one little question: Do you think that the line at @services/api/app/controllers/arvados/v1/groups_co... Lucas Di Pentima
03:04 pm Bug #11071: [Workbench] qr1hi performance tests are broken
Tests @ https://ci.curoverse.com/job/developer-run-tests/186/ Radhika Chippada
08:01 pm Revision b1aa6c85 (arvados): closes #11071
Merge branch '11071-fts-perf-test' Radhika Chippada
08:00 pm Revision 98329b75 (arvados): 11071: formatting on the long if statement
Radhika Chippada
07:56 pm Bug #11186 (Duplicate): Priority 0 state shouldn't be labeled "Ready"
See #11250 Peter Amstutz
06:29 pm Bug #11186: Priority 0 state shouldn't be labeled "Ready"
Suggest "Stopped" Peter Amstutz
07:35 pm Revision 91118e3a (arvados): Add missing documentation file. refs #6520
Peter Amstutz
07:17 pm Revision bdaa9de9 (arvados): Merge branch '8567-api-select-docker-fmt' refs #8567
Peter Amstutz
07:17 pm Revision d363323b (arvados): Merge branch '8567-cwl-docker-img' refs #8567
Peter Amstutz
07:15 pm Story #11123 (Resolved): Add node manager to install guide
Applied in changeset arvados|commit:449d780f0a470f67d1a28fe1459a8bab79562069. Peter Amstutz
04:02 pm Story #11123: Add node manager to install guide
Made some updates, if you want to take another quick look: 6520-nodemanager-docs Peter Amstutz
03:10 pm Story #11123: Add node manager to install guide
Yeap. My bad, probably run the wrong @git diff@ when checking which files needed to review.
Now, ignoring my last...
Javier Bértoli
02:57 pm Story #11123: Add node manager to install guide
Javier Bértoli wrote:
> Question, in *doc/install/install-keepstore.html.textile.liquid*: does it make sense to docu...
Peter Amstutz
07:09 pm Revision 449d780f (arvados): Merge branch '6520-nodemanager-docs' closes #11123
Peter Amstutz
06:30 pm Story #11250 (Resolved): [Workbench] Better user-facing terms for container states
* When a container is in the slurm queue, its state is displayed as "Locked". This is confusing to users who don't a... Peter Amstutz
06:18 pm Feature #11249 (New): [SDKs] Writable collection files returned by Go SDK should be seekable
The reading+seeking and writing code need to be reconciled so OpenFile and Create return an object that can read, wri... Tom Clegg
05:29 pm Feature #10664 (Duplicate): arv-keepdocker should upload images with image:tag
Tom Clegg
04:44 pm Revision eabc13f2 (arvados): 8567: Add note about updating API server configuration.
Peter Amstutz
04:39 pm Revision dee634b2 (arvados): 8567: Add docker19 migration instructions to install guide.
Peter Amstutz
04:30 pm Revision 205b0776 (arvados): 8567: Creates migration links.
Peter Amstutz
04:30 pm Revision 2ed5325b (arvados): 8567: Add check for ARVADOS_API_HOST_INSECURE
Peter Amstutz
04:30 pm Revision 1d7577d0 (arvados): 8567: Add status reporting to migrate script.
Peter Amstutz
04:30 pm Revision f074ef81 (arvados): 8567: Fix migrate links to use PDH instead of UUID.
Better error reporting.
Migrate script cleans up /var/lib/docker inside container.
Peter Amstutz
04:30 pm Revision 55c719bf (arvados): 8567: Rename docker19-migrate to migrate-docker19 for consistency with
arv-migrate-docker19. Add docstrict to migrate19() function. Peter Amstutz
04:30 pm Revision 6058f336 (arvados): 8567: Move out of tools/ into sdk/python and docker/docker19-migrate.
Peter Amstutz
04:30 pm Revision 0eb52182 (arvados): 8567: Docker image migration WIP.
Peter Amstutz
04:25 pm Revision 5fffad60 (arvados): 8567: Fix tests now that container_image provides docker repo+tag an...
Peter Amstutz
04:03 pm Revision 355fe24b (arvados): 8567: If a search_term looks like a PDH, always treat it as one. Al...
correct optional parameter syntax. Peter Amstutz
03:38 pm Revision 25296def (arvados): 6520: Add page with sample ping script. Improve instructions on cre...
compute node a little bit. Peter Amstutz
03:37 pm Task #11205 (In Progress): Review branch 11071-fts-perf-test
Lucas Di Pentima
03:21 pm Task #11141 (Feedback): Review 6520-nodemanager-docs
Javier Bértoli
03:06 pm Revision c04608e4 (arvados): 6520: Add node_mem_scaling to documentation.
Peter Amstutz
02:59 pm Revision 7eb2eff1 (arvados): 6520: Node manager docs WIP
Peter Amstutz
02:59 pm Revision a1383389 (arvados): 6520: Add information about setting up SLURM to crunchv2 documentation.
Peter Amstutz
02:46 pm Revision 6ea807b2 (arvados): 8567: Use Docker image repo+tag name instead of PDH so that API serv...
Peter Amstutz

03/13/2017

10:05 pm Revision c7515954 (arvados): 11071: test count=none in groups#contents method.
Radhika Chippada
09:37 pm Revision dbb0d22a (arvados): 8567: Tests default to image format v1 to avoid breaking all the tes...
uses the :docker_image collection. Peter Amstutz
09:31 pm Revision 918e16de (arvados): 8567: Adjust job container resolve test now that images are filtered...
support version. Peter Amstutz
08:40 pm Revision a7220572 (arvados): 8567: Refactor code that queries migration links into get_compatible...
Peter Amstutz
06:48 pm Bug #11209 (In Progress): stuck keep fuse mounts not cleared by crunch-job
Tom Clegg
06:48 pm Bug #11209: stuck keep fuse mounts not cleared by crunch-job
Normally crunch-job frees up mount points using @fusermount -u -z@ but for some reason it isn't working here:
<pre...
Tom Clegg
04:55 pm Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardle...
Tom Morris wrote:
> I take it this got instagroomed while I was gone.
>
> Does this mean that #10669 is fixed (an...
Peter Amstutz
04:19 pm Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardle...
I take it this got instagroomed while I was gone.
Does this mean that #10669 is fixed (and can be closed) as well?
Tom Morris

03/11/2017

07:23 pm Bug #11238: job_task creation fails with ApiError - HttpError 422 - ActiveRecord::StatementInvali...
I've followed the directions at https://blog.dob.sk/2012/05/19/fixing-pg_dump-invalid-memory-alloc-request-size/ to c... Joshua Randall

03/10/2017

10:16 pm Story #8773 (Closed): [GATK Queue] Support a mechanism to redirect stderr for children
Tom Clegg
10:15 pm Bug #7000 (Closed): [Docker] API server starts before PostgreSQL is ready
Tom Clegg
10:15 pm Bug #7007 (Closed): [Docker] Bootstrap font fails to load in Workbench Docker image
Tom Clegg
10:15 pm Bug #7006 (Closed): [Docker] Fix SSO configuration in published images
Tom Clegg
10:15 pm Bug #7005 (Closed): [Docker] API server uses omniauth.rb for SSO configuration, not application.yml
Tom Clegg
10:15 pm Bug #6998 (Closed): [Docker] Workbench image doesn't serve HTTPS
Tom Clegg
10:10 pm Bug #5949 (Closed): [DRAFT] [Workbench] Users can be notified about changes to important pipelines
Tom Clegg
10:07 pm Feature #5828 (Closed): [Workbench] Create new collection with selected files copy streams as wel...
Tom Clegg
10:07 pm Bug #5850 (Closed): All clients authenticate directly against SSO server without going through AP...
Tom Clegg
10:04 pm Bug #5876 (Closed): [Docker] Needs smarter dependency ordering when restarting existing containers
Tom Clegg
10:04 pm Bug #5878 (Closed): [Workbench] PipelineInstancesTest#test_Create_and_run_a_pipeline is not reliable
Tom Clegg
10:04 pm Bug #5892 (Closed): [Workbench] collections#index returns 504 Gateway Timeout
Tom Clegg
10:04 pm Bug #5897 (Closed): [FUSE] Catch and recover from any API client exceptions
Tom Clegg
10:03 pm Feature #5900 (Closed): [SDK] Implement arvados.gettaskparam(*args) to match arvados.getjobparam(...
Tom Clegg
10:02 pm Bug #5905 (Closed): [SDKs] Improve certificate validation in Python SDK
Tom Clegg
10:01 pm Bug #5908 (Closed): [Workbench] collection#show performing too many unnecessary API server reques...
Tom Clegg
10:01 pm Bug #5923 (Closed): Fiddlesticks when trying to view a log file
Tom Clegg
10:01 pm Bug #5919 (Closed): Keep errors
Tom Clegg
10:01 pm Feature #5928 (Closed): Syntax highlighting of viewing source in workbench
Tom Clegg
10:01 pm Bug #5945 (Closed): arv-get is slow for this collection
Tom Clegg
10:00 pm Bug #6019 (Closed): [FUSE] `ls` in a large home directory (4400+ items) takes too long
Tom Clegg
10:00 pm Bug #6025 (Closed): [SSO] Authenticate users with OAuth2
Tom Clegg
10:00 pm Story #5098 (Closed): [Keep] Design: using Arvados to browse, and run jobs on, input data that is...
Tom Clegg
10:00 pm Feature #5414 (Closed): [Keep] Use data stored remotely (not in local keepstore)
Tom Clegg
07:06 pm Revision e1e0bec5 (arvados): 8567: Add & tweak tests for selecting compatible Docker image format.
Peter Amstutz
06:51 pm Revision f4a7c5e6 (arvados): 8567: find_all_for_docker_image() returns only Docker images compati...
Peter Amstutz
06:39 pm Story #4443 (Closed): [DRAFT] [Workbench] Tour Guide Dax follows you around for a while, showing ...
Tom Clegg
06:38 pm Story #4284 (Closed): [Crunch] Add unit and functional tests to crunchstat test suite
Tom Clegg
06:38 pm Bug #4123 (Closed): [API] move Job script/repository/docker validations from controller to model
Tom Clegg
06:36 pm Bug #4120 (Closed): [Workbench] [obsolete?] Pipeline state can be failed even though every job is...
Tom Clegg
06:35 pm Feature #3803 (Closed): [SDKs] Add a rate limiting feature to arv-put and arv-get (like rsync's b...
Tom Clegg
06:34 pm Story #3696 (Closed): [Workbench] Improve log viewer to more effectively expose relevant informat...
Tom Clegg
06:34 pm Feature #3987 (Closed): [Docker] Run a 5-second diagnostic suite after bringing up docker contain...
Tom Clegg
06:34 pm Bug #3910 (Closed): [SDKs] Client libraries should retry failed API requests after errors like Ga...
Tom Clegg
06:34 pm Feature #3955 (Closed): [Workbench] Render runtime-formatted timestamps as "8:00 am" or "Sep 23" ...
Tom Clegg
06:33 pm Feature #3923 (Closed): [Workbench] Use web notifications to alert user when pipelines finish
Tom Clegg
06:32 pm Story #3511 (Closed): [Documentation] Present an efficient pattern for developing a pipeline temp...
Tom Clegg
06:31 pm Story #3701 (Closed): [Workbench] Show current user's compute node usage over time
Tom Clegg
06:31 pm Story #3446 (Closed): [Workbench] Sort by name before uuid in search results.
Tom Clegg
06:30 pm Bug #3352 (Closed): [SDK] After ^C, arv-put sometimes exits but leaves processes/threads behind t...
Tom Clegg
06:30 pm Bug #3134 (Closed): [Workbench] Deleting a job in a project doesn't show delete wipe animation, i...
Tom Clegg
06:30 pm Bug #3166 (Closed): [Workbench] Generic "attributes" tab renders uuid as a link to the object, wi...
Tom Clegg
06:30 pm Bug #3483 (Closed): [Keep] Keep uses appropriate HTTP status codes for failed DELETE requests
Tom Clegg
06:29 pm Bug #3262 (Closed): [Documentation] Describe how to run tramp in emacs.
Tom Clegg
04:04 pm Bug #11238: job_task creation fails with ApiError - HttpError 422 - ActiveRecord::StatementInvali...
... Joshua Randall
03:36 pm Bug #11238: job_task creation fails with ApiError - HttpError 422 - ActiveRecord::StatementInvali...
These appear to be the corresponding log from the api server production.log (there does not appear to be any mention ... Joshua Randall
02:47 pm Bug #11238 (New): job_task creation fails with ApiError - HttpError 422 - ActiveRecord::Statement...
For some reason some of our jobs are failing with an API server error indicating a postgres statement was invalid bec... Joshua Randall
02:38 pm Task #11233 (Resolved): Review
Peter Amstutz

03/09/2017

10:44 pm Revision f63c160f (arvados): Bugfix: python Collection class sets _portable_data_hash in _populat...
refs #10956 Peter Amstutz
10:41 pm Revision 669e7f30 (arvados): 8567: Fix migrate links to use PDH instead of UUID.
Better error reporting.
Migrate script cleans up /var/lib/docker inside container.
Peter Amstutz
10:13 pm Bug #11237: python-arvados-python-client depends on old versions of packages
When updating an Ubuntu 16.04 host, @python-arvados-python-client@ complains that the available versions are not suit... Javier Bértoli
10:08 pm Bug #11237 (Resolved): python-arvados-python-client depends on old versions of packages
Javier Bértoli
09:41 pm Bug #11236 (Resolved): Dependencies in Ubuntu 16.04 (xenial) offer to REMOVE arvados-node-manager
Performing a dist-upgrade on an Ubuntu 16.04 (xenial) host offers to **REMOVE** @arvados-node-manager@
* Sources l...
Javier Bértoli
09:22 pm Bug #4344 (Resolved): [SDK] can't make an anonymous token from the cli
This works, but you have to say @"owner_uuid":"X"@, not @"user":"X"@. Tom Clegg
09:17 pm Bug #3999 (Closed): [Workbench] Pipeline timing number oddities
Tom Clegg
09:17 pm Bug #3855 (Closed): [SDK] New keep mount is very slow on initial usage
Tom Clegg
09:17 pm Bug #4132 (Closed): [Workbench] Add unit test infrastructure (and some examples) for JavaScript a...
Tom Clegg
09:16 pm Feature #3333 (Closed): [DRAFT] [Docker] Make local Docker install more useful for development
Tom Clegg
09:16 pm Bug #3847 (Resolved): [SDK] --project-uuid switch behaves strangely without = (seen in arv-keepdo...
Seems to have been fixed, although I don't see where/when.... Tom Clegg
08:28 pm Revision 605d11f2 (arvados): Merge branch '5036-arv-mount-type'
refs #5036 Tom Clegg
08:14 pm Feature #5036: [arv-mount] Change default mount type from "fuse" to "fuse.arvados"
LGTM. Peter Amstutz
07:56 pm Feature #5036 (In Progress): [arv-mount] Change default mount type from "fuse" to "fuse.arvados"
Tom Clegg
07:56 pm Feature #5036: [arv-mount] Change default mount type from "fuse" to "fuse.arvados"
5036-arv-mount-type @ commit:fa8c5cdb36fa89af92087d6465b207d65ffc9618 adds a "--subtype FOO" arg, which makes the fil... Tom Clegg
07:53 pm Revision fa8c5cdb (arvados): 5036: Add "--subtype foo" flag to set mounted filesystem type to "fu...
Tom Clegg
07:50 pm Bug #11226 (Resolved): [SDK] Python client tries to cache discovery document at the same local pa...
Applied in changeset arvados|commit:82697fea93b1c87cdee27d2b9a76c1b7ac07497e. Peter Amstutz
03:49 pm Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardle...
+1 Peter Amstutz
03:48 pm Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardle...
Suggest we
# pass cache_discovery=False to google-api-client so we get back to what we thought we were doing, i.e., ...
Tom Clegg
03:45 pm Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardle...
Also want to do one of:
a) continue to use httplib2 caching, which is apparently not multithread/multiprocess safe...
Peter Amstutz
03:38 pm Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardle...
https://github.com/google/google-api-python-client/pull/127
I think we can use the @cache_discovery@ option to bui...
Peter Amstutz
07:44 pm Revision 82697fea (arvados): Merge branch '11226-discovery-doc-cache' closes #11226
Peter Amstutz
07:40 pm Revision a15fc58f (arvados): 11226: Disable google api client discovery doc caching, use only htt...
Google API client has its own caching mechanism. The problem is, by default
this goes to /tmp/google-api-python-clie...
Peter Amstutz
06:11 pm Revision fe2484ec (arvados): 10401: Bump cwltool pin.
Peter Amstutz
03:55 pm Task #11233 (Resolved): Review
Peter Amstutz
03:53 pm Revision ebb0138d (arvados): 8567: Fix docker_migration_pdh to seach head/tail_uuid (PDH doesn't ...
Peter Amstutz

03/08/2017

10:04 pm Revision c69d4a2b (arvados): 10401: Fix arvcontainer use of pathmapper.
Peter Amstutz
08:33 pm Revision c7143dc9 (arvados): Add ability to use "arvbox start test" to re-run tests without teari...
restarting whole container. no issue # Peter Amstutz
08:33 pm Revision 005595fb (arvados): Arvbox now uses Go websockets server instead of Puma. no issue #
Peter Amstutz
07:31 pm Bug #11226 (Resolved): [SDK] Python client tries to cache discovery document at the same local pa...
The Python SDK instructs httplib2 to cache the discovery document in @~/.cache/arvados/discovery@. However, the goog... Peter Amstutz
06:43 pm Task #11225 (In Progress): Update cwltool
Peter Amstutz
06:43 pm Task #11225 (Resolved): Update cwltool
Peter Amstutz
06:41 pm Revision 16180e3b (arvados): 10401: Passes unit tests.
Peter Amstutz
06:23 pm Revision 08bf82bb (arvados): 10401: Allow Arvados extensions to be used in "requirements" not jus...
Peter Amstutz
06:19 pm Revision dd7710a6 (arvados): 10401: Use use_custom_schema feature for extensions.
Peter Amstutz
06:13 pm Revision e05dddc4 (arvados): 10401: Bump cwltool & schema salad pins.
Peter Amstutz
04:18 pm Bug #4748 (Closed): [API] Explain why there are 21 jobs with log = ""
Tom Clegg
04:17 pm Bug #4451 (Closed): [Workbench] Fiddlesticks when rerunning pipeline with edited template
Tom Clegg
04:16 pm Bug #4416 (Closed): [Workbench] "Manage" permission should be sufficient to edit project's name/d...
Tom Clegg
04:15 pm Bug #4440 (Closed): [Documentation] "Writing a Crunch script" is mysterious
Tom Clegg
04:15 pm Bug #4529 (Closed): [SDKs] Ensure Python and Go SDKs in a given situation always choose either pr...
Tom Clegg
04:14 pm Bug #4488 (Closed): [SDKs] arvados.keep.KeepClient.KeepService should know whether it is a proxy
Tom Clegg
04:12 pm Bug #4335 (Closed): [Crunch] Improve node startup speed. 15 min is too slow.
Tom Clegg
04:11 pm Bug #4218 (Closed): [Keep] Improve speed of /index requests
Tom Clegg
04:11 pm Story #4238 (Closed): [Documentation] Show how to run a pipeline instance without using a template.
Tom Clegg
04:10 pm Story #4305 (Closed): [Node Manager] Investigate and try to address RAM use
Tom Clegg
04:10 pm Bug #3688 (Closed): [API] User should be able to delete a project even though dependent logs exist
Tom Clegg
04:09 pm Story #3820 (Closed): [Crunch] Jobs need code from multiple git repositories
Tom Clegg
04:09 pm Feature #4131 (Closed): [Workbench] Support incremental loading of long job logs, using Range req...
Tom Clegg
04:09 pm Feature #4090 (Closed): Add code coverage report for workbench javascript
Tom Clegg
04:08 pm Story #4424 (Closed): [Keep] [DRAFT] Use stronger hash algorithm (such as SHA-2) for Keep blocks
Moved to [[Changing Keep hash algorithm]] Tom Clegg
04:06 pm Story #11123: Add node manager to install guide
Question, in *doc/install/install-keepstore.html.textile.liquid*: does it make sense to document the runit process, a... Javier Bértoli
03:52 pm Task #11141 (In Progress): Review 6520-nodemanager-docs
Peter Amstutz
03:01 pm Task #11224 (In Progress): Update API usage
Peter Amstutz
03:01 pm Task #11224 (Resolved): Update API usage
Peter Amstutz
02:20 pm Feature #10218 (In Progress): [Crunch2] Gather and record cloud/physical node information for eac...
Lucas Di Pentima
02:14 pm Bug #3624: [API] Remove redundant "updated_at" column
You know, we still have a redundant update_at column... Peter Amstutz

03/07/2017

10:23 pm Story #3812 (Closed): [SDKs] Clients should warn when they're expecting a newer API server
Tom Clegg
10:22 pm Bug #4756 (Rejected): [API] Add migration to change collection uuids to portable_data_hash in old...
Tom Clegg
10:18 pm Revision 5eae182a (arvados): 3853: Do not try to look up collections by PDH for update/delete act...
Tom Clegg
10:03 pm Bug #3954 (Closed): [API] permission_to_update should prohibit all changes to content-addressable...
Tom Clegg
10:03 pm Bug #4003 (Closed): [Crunch] crunch-dispatch errors that relate to a specific job should be avail...
Tom Clegg
10:03 pm Bug #3831 (Closed): [Crunch] inconsistent find_or_create behavior in pipeline runner vs job
Tom Clegg
10:02 pm Bug #3556 (Closed): [Documentation] Avoid using the term "parallel" when referring to Crunch conc...
Tom Clegg
10:01 pm Feature #3768 (Closed): [SDKs] Speculatively pre-fetch data blocks while reading from a collection.
Tom Clegg
10:01 pm Bug #3789 (Closed): [API] Pipeline template (temporarily?) disappears after move to project
Tom Clegg
10:00 pm Bug #3810 (Closed): [Crunch] run-command should bail when $(file) expands to a nonexistent path
Tom Clegg
09:59 pm Story #3795 (Closed): [Crunch/SDKs] Tasks need more retry support
Tom Clegg
09:59 pm Feature #3633 (Closed): [Workbench] Compare pipeline templates, similar to the existing pipeline ...
Tom Clegg
09:59 pm Story #3347 (Closed): [Crunch] Run dev (and real) jobs using a syntax as close as possible to "py...
Tom Clegg
09:58 pm Bug #3470 (Closed): [Workbench] Ajax updated pages don't interact correctly with browser history.
Tom Clegg
09:58 pm Story #3157 (Closed): [Workbench] Admin button to download user list as CSV
Tom Clegg
09:57 pm Bug #3418 (Rejected): [Crunch] crunch-job ignores task sequence
Tom Clegg
09:57 pm Feature #3329 (Closed): [Docker] Loose ends in local docker install
Tom Clegg
09:56 pm Bug #3386 (Closed): [Documentation] Crunch script can exit 111 to indicate transient failure (i.e...
Tom Clegg
09:56 pm Support #2870 (Closed): Write up PGP data curation plan
Tom Clegg
09:56 pm Bug #3624 (Closed): [API] Remove redundant "updated_at" column
Tom Clegg
09:55 pm Feature #3095 (Closed): [Workbench] Add "run" button to each pipeline template in project view
Tom Clegg
09:43 pm Bug #10669: googleapiclient.errors.InvalidJsonError retrieving discovery document
Discovery doc retrieval is done by google-api-client. We ask it to use httplib2's built-in cache mechanism.
Unfort...
Tom Clegg
09:30 pm Bug #11222 (New): Docker load time on compute node is too long
using a-c-r --local, it took 10 minutes to load a 3GB Docker image... Bryan Cosca
09:16 pm Bug #11221: [systemd] Always restart exited services, even after 3 startup failures
11221-always-restart-services @ commit:273a233818ae39e843fab0276f9e381da6645d28 Tom Clegg
09:06 pm Bug #11221 (Resolved): [systemd] Always restart exited services, even after 3 startup failures
Currently, when postgresql is unavailable for ~300ms when arvados-ws is trying to start (which is normal for a bottin... Tom Clegg
09:15 pm Revision 273a2338 (arvados): 11221: Always restart systemd services, even after a few startup fai...
Tom Clegg
08:45 pm Bug #3115 (Resolved): [API] Retrieving a nonexistent keep disk as an admin should not create a ne...
Applied in changeset arvados|commit:533a50bbcee4ece451a65e87066563a5b33ef150. Tom Clegg
08:39 pm Revision 533a50bb (arvados): Merge branch '3115-keep-disk-create-surprise'
closes #3115 Tom Clegg
08:32 pm Revision c97408a5 (arvados): 3115: Create keep_disks implicitly only in #ping action, not #show.
Tom Clegg
08:03 pm Story #3043 (Resolved): [SDKs] Add Go SDK page to doc site with examples similar to Python etc
Tom Clegg
08:03 pm Story #3045 (Closed): [Workbench] Generalize "re-run this job" button
Tom Clegg
08:02 pm Bug #3097 (Closed): [Crunch] Pass script_parameters on stdin instead of JOB_PARAMETER_* environme...
Tom Clegg
08:02 pm Story #3017 (Closed): [Workbench] When comparing pipeline instances, show/highlight the differing...
Tom Clegg
08:02 pm Story #2932 (Closed): Pipeline has default name from pipeline template ("instance of X") and star...
Tom Clegg
08:02 pm Bug #3159 (Closed): Display of the "Add a new keep disk" button in the keep disks page in incorre...
Tom Clegg
08:00 pm Story #2661 (Closed): Security audit
Tom Clegg
08:00 pm Feature #3895 (Closed): set up dev ci server
Tom Clegg
07:59 pm Story #4599 (Closed): [Crunch] crunch-job should not retry tasks after its SLURM allocation is re...
Tom Clegg
07:53 pm Feature #6312 (Resolved): [Workbench] Use download service instead of passing Keep data through R...
Tom Clegg
07:52 pm Feature #7709: [API] Upgrade API server to Rails 4.2
Release notes: http://guides.rubyonrails.org/ Peter Amstutz
07:38 pm Feature #8801 (Duplicate): [Crunch] log free disk space before task starts
Tom Morris
07:37 pm Feature #8196 (Duplicate): [Crunch] Jobs should log the hardware of their compute node(s)
Tom Morris
07:31 pm Bug #4222 (Closed): [Keep] Rename -enforce-permissions to -bypass-read-permissions with default=f...
Tom Morris
07:30 pm Bug #3555 (Closed): [API] Rename nondeterministic to impure
Tom Morris
07:28 pm Feature #6191 (Resolved): Propagate stderr from all srun uses (e.g., clean-tmp and docker-install...
Tom Clegg
07:13 pm Story #4101 (Rejected): Workbench tests: app/controllers/projects_controller.rb
Tom Clegg
07:13 pm Story #4100 (Rejected): Workbench tests: app/helpers/application_helper.rb
Tom Clegg
07:12 pm Story #4099 (Rejected): Workbench tests: app/helpers/pipeline_instances_helper.rb
Tom Clegg
07:12 pm Story #4098 (Rejected): Workbench tests: app/controllers/pipeline_instances_controller.rb
Tom Clegg
07:12 pm Story #4097 (Rejected): Workbench tests: app/controllers/users_controller.rb
Tom Clegg
07:12 pm Story #4096 (Rejected): Workbench tests: app/helpers/provenance_helper.rb
Tom Clegg
07:12 pm Story #4095 (Rejected): Workbench tests: app/controllers/groups_controller.rb
Tom Clegg
07:12 pm Story #4094 (Rejected): Workbench tests: app/controllers/jobs_controller.rb
Tom Clegg
07:12 pm Story #4093 (Rejected): Workbench tests: app/helpers/vcf_pipeline_helper.rb
Tom Clegg
07:12 pm Story #4092 (Rejected): Workbench tests: app/helpers/jobs_helper.rb
Tom Clegg
07:11 pm Bug #6086 (Closed): [API] Creating new job sometimes takes too long
Tom Clegg
07:10 pm Story #6407 (Closed): [Workbench] Collection file trees are collapsible
Tom Clegg
06:59 pm Bug #10350 (Resolved): [Documentation] Update /api/authentication.html with correct information
Fixed in #10346 Peter Amstutz
06:57 pm Bug #9702 (Closed): [Crunch2] Five CWL conformance tests are failing with Crunch2
Closing this as it no longer reflects the current state of CWL on crunch v2.
The latest conformance test results c...
Peter Amstutz
06:55 pm Story #9132 (In Progress): [Crunch2] crunch-run uses official docker client library
Peter Amstutz
06:53 pm Task #8845 (Resolved): Diagnose
Parent ticket is resolved. Peter Amstutz
06:47 pm Task #7601 (Resolved): Review 6358-put-rendezvous
Parent ticket is resolved. Peter Amstutz
06:47 pm Task #6299 (Resolved): Review
Parent ticket is resolved. Peter Amstutz
06:46 pm Task #5680 (Resolved): Async dispatch hooks from CWL engine
CWL is supported an Arvados. Peter Amstutz
06:45 pm Task #5449: Update examples to use new Python Collection SDK and add deprecation notes to old APIs
Workflow examples using CWL don't involve the SDK.
Pipeline examples that use the Python SDK are already marked as...
Peter Amstutz
06:41 pm Feature #4223: [SDKs] arv keep docker --download to get image from Arvados
I still find myself wanting this feature occasionally, although we may want to abandon this particular branch. Peter Amstutz
06:39 pm Task #5681 (Closed): Task dispatch from CWL job runner
arvados-cwl-runner doesn't use job_tasks (which are specific to crunch v1).
Peter Amstutz
06:37 pm Task #5679 (Resolved): CWL job runner framework
CWL is supported on Arvados. Peter Amstutz
06:32 pm Task #2637 (Closed): Add support for filters to generic create API
Parent ticket closed. Peter Amstutz
06:31 pm Task #2628 (Closed): Write wiki page design for simple joins in REST API
Not currently planned. Peter Amstutz
06:31 pm Task #3884 (Closed): Review 3605-improved-dashboard
Revisit dashboard in future tickets. Peter Amstutz
06:30 pm Task #4703 (Resolved): Work on CWL draft 1 spec for tools
CWL supported on Arvados.
Peter Amstutz
06:30 pm Story #11139 (Resolved): [Node manager] Expected MemTotal for each cloud node size
Applied in changeset arvados|commit:4812b5639cfaf724540786fcd331aaa227635c77. Lucas Di Pentima
06:07 pm Story #11139: [Node manager] Expected MemTotal for each cloud node size
Thanks, LGTM Peter Amstutz
05:59 pm Story #11139: [Node manager] Expected MemTotal for each cloud node size
Updates & master branch merge at commit:101e3227b25c16874fa73660bfd7e338fbfe0da2
Tests: https://ci.curoverse.com/job...
Lucas Di Pentima
02:54 pm Story #11139: [Node manager] Expected MemTotal for each cloud node size
Can you add a test that shows that changing the configuration parameter changes the server calculator.
Can you add...
Peter Amstutz
06:29 pm Task #4686 (Resolved): Example wrapping bcbio sub-tool using CWL tool description
bcbio generates CWL that runs on Arvados. Peter Amstutz
06:29 pm Task #4613 (Resolved): Workbench template for generating UI
Workbench generates input form for CWL workflows. Peter Amstutz
06:28 pm Task #4612 (Resolved): Start writing user guide page
CWL support in Arvados is documented. Peter Amstutz
06:27 pm Task #4575 (Resolved): Review 4035-pipeline-support-cwl-tool
CWL is supported in Arvados. Peter Amstutz
06:27 pm Task #4420 (Resolved): Support user+pass database managed locally by SSO server
Implemented commit:49ab94b9d8d2c6e85c22b0e19171255af66db5e2 Peter Amstutz
06:25 pm Revision 4812b563 (arvados): Merge branch '11139-nodemanager-mem-scale-factor'
Closes #11139 Lucas Di Pentima
06:25 pm Task #4146 (Closed): Fix crunch-job
Parent ticket closed. Peter Amstutz
06:24 pm Bug #4028 (Closed): [Crunch] Task stdout appearing in job logs should be labeled "stdout" instead...
Seems unlikely this will be fixed.
Crunch v2 logs distinguish between stdout and stderr.
Peter Amstutz
06:23 pm Task #4144 (Resolved): Use num_retries on regular collection writer instead of robust_put
Was implemented in commit:1281ecab8f2396739ee9232c36796e46cd551426
Also run-command is nearing end-of-life as it i...
Peter Amstutz
06:20 pm Task #3887 (Closed): Fix tests
Revisit dashboard / workbench design in future tickets. Peter Amstutz
06:20 pm Feature #3605 (Closed): [Workbench] improved dashboard page
Will dashboard design in future tickets. Peter Amstutz
06:18 pm Task #3324 (Closed): Build new docker image, run on qr1hi with test data
Bcbio now generates CWL to run on Arvados. Uses upstream bcbio Docker image. Peter Amstutz
06:17 pm Feature #3319 (Closed): [Sample Pipelines] Run monolithic bcbio pipeline on tiny dataset. Package...
Bcbio now generates CWL to run on Arvados. Peter Amstutz
06:16 pm Task #2648 (Closed): Write tests
No longer applicable due to closing of original ticket. Peter Amstutz
06:15 pm Task #2649 (Closed): Assign collection to project
Obsolete by arv-put --project-uuid Peter Amstutz
06:15 pm Task #3230 (Closed): Support bcbio parallelization framework
bcbio generates CWL files which are runnable on Arvados. Peter Amstutz
06:14 pm Task #2647 (Closed): Put files from staging directory into Keep
Obsoleted by enabling arv-put to upload directly to Keep via keepproxy. Peter Amstutz
06:14 pm Task #2643 (Closed): "Ready to ingest" endpoint (in /actions controller?)
Obsoleted by enabling arv-put to write directly to Keep using keepproxy. Peter Amstutz
06:13 pm Feature #2176 (Closed): API server supports atomic find_or_create operation
Containers API uses a different approach. Does not have compelling use case in other APIs. Peter Amstutz
06:12 pm Story #2627 (Closed): Write design for joins in API
Currently no plans to do this. Peter Amstutz
06:07 pm Task #11200 (Resolved): Review 11139-nodemanager-mem-scale-factor
Peter Amstutz
02:46 pm Task #11200 (In Progress): Review 11139-nodemanager-mem-scale-factor
Peter Amstutz
05:54 pm Revision 101e3227 (arvados): 11139: Merge branch 'master' into 11139-nodemanager-mem-scale-factor
Lucas Di Pentima
05:52 pm Revision 9dae7131 (arvados): 11139: Added new test to check for non-default values. Updated examp...
Lucas Di Pentima
03:57 pm Bug #11220 (Resolved): [SDKs] Fix misleading arv-mount/pysdk error messages by removing obsolete ...
Current error message:... Tom Clegg
03:54 pm Bug #9435 (Resolved): keep-balance failing when a small page of collections has the same modified...
In #9998 keep-balance started using count="none" and accommodating multiple pages of identical timestamps, so this sh... Tom Clegg
03:48 pm Story #6830 (Resolved): [API] [keep-balance] Option to return unsigned manifests from collections...
This was implemented in #9998. Tom Clegg
03:39 pm Bug #11190: Containers seem to run more than once, which isn't supposed to happen
... Tom Clegg
03:15 pm Bug #11166 (Resolved): [Crunch2] crunchrun.go should avoid name collisions when creating log coll...
Applied in changeset arvados|commit:a1b0149879593df786807afbefb26d4687ebe161. Tom Clegg
03:01 pm Bug #11166: [Crunch2] crunchrun.go should avoid name collisions when creating log collections
11166-log-name-collision @ commit:cb0b666f86ce87d35667e8e23e3775b703a37841 Tom Clegg
02:45 pm Bug #11166: [Crunch2] crunchrun.go should avoid name collisions when creating log collections
LGTM. Peter Amstutz
03:11 pm Revision a1b01498 (arvados): Merge branch '11166-log-name-collision'
closes #11166 Tom Clegg
03:02 pm Revision fce072c7 (arvados): 8567: Add check for ARVADOS_API_HOST_INSECURE
Peter Amstutz
02:46 pm Task #11196 (Resolved): Review 11166-log-name-collision
Peter Amstutz
02:27 pm Task #11196 (In Progress): Review 11166-log-name-collision
Peter Amstutz
02:26 pm Revision 123251c6 (arvados): 8567: Rename docker19-migrate to migrate-docker19 for consistency with
arv-migrate-docker19. Add docstrict to migrate19() function. Peter Amstutz
02:18 pm Revision b5886da1 (arvados): 8567: Creates migration links.
Peter Amstutz
02:18 pm Revision 80d9b40b (arvados): 8567: Docker image migration WIP.
Peter Amstutz
02:18 pm Revision 57fec89d (arvados): 8567: Add status reporting to migrate script.
Peter Amstutz
02:18 pm Revision 09fc7249 (arvados): 8567: Move out of tools/ into sdk/python and docker/docker19-migrate.
Peter Amstutz
02:18 pm Revision 5780b539 (arvados): 9132: WIP switching to using official Docker Go client.
Peter Amstutz

03/06/2017

10:38 pm Story #11139: [Node manager] Expected MemTotal for each cloud node size
Updates on @11139-nodemanager-mem-scale-factor@ branch at commit:b6423b5
Test run: https://ci.curoverse.com/job/deve...
Lucas Di Pentima
10:04 pm Revision b6423b5e (arvados): 11139: Added default config parameter 'node_mem_scaling' to be appli...
Added test checking the default config and that the value is actually applied to the server calculations. Lucas Di Pentima
08:58 pm Bug #11166 (In Progress): [Crunch2] crunchrun.go should avoid name collisions when creating log c...
Tom Clegg
08:24 pm Task #11212 (Resolved): use ensure_unique_name flag or timestamp
Tom Clegg
03:37 pm Task #11212 (Resolved): use ensure_unique_name flag or timestamp
Tom Clegg
07:59 pm Revision cb0b666f (arvados): 11166: Use ensure_unique_name to avoid collisions when saving logs a...
Tom Clegg
07:52 pm Revision 58342141 (arvados): 8567: Move out of tools/ into sdk/python and docker/docker19-migrate.
Peter Amstutz
07:31 pm Revision f2c45e2c (arvados): 8567: Add status reporting to migrate script.
Peter Amstutz
04:20 pm Story #11171: [Nodemanager] Packaging for compute node setup scripts
relevant modules in puppet:
arvados-compute-and-crunch-dispatch-dependencies
arvados-compute-dependencies
arvado...
Nico César
03:39 pm Revision 93384273 (arvados): 8567: Creates migration links.
Peter Amstutz
03:35 pm Bug #11138 (Resolved): Protection against running new style docker 1.10+ image on old docker host
Applied in changeset arvados|commit:a4a3227c634d8cfd54a23205be727e910032cbe9. Tom Clegg
02:09 pm Bug #11138: Protection against running new style docker 1.10+ image on old docker host
Thanks! LGTM. Lucas Di Pentima
03:31 pm Revision a4a3227c (arvados): Merge branch '11138-docker-load-fail'
closes #11138 Tom Clegg
03:23 pm Revision cee1c18c (arvados): 8567: Docker image migration WIP.
Peter Amstutz
02:10 pm Task #11197 (Resolved): Review 11138-docker-load-fail
Lucas Di Pentima

03/03/2017

10:05 pm Revision 47c56297 (arvados): Add missing install step: add keep-balance token to keepstore configs.
No issue # Tom Clegg
09:53 pm Revision 2e5c44a6 (arvados): Remove obsolete GOMAXPROCS advice.
No issue # Tom Clegg
09:50 pm Bug #11168 (Resolved): [API] Use JSON instead of YAML for serialized fields in database
Applied in changeset arvados|commit:660a6143ecf1e777f33bd84183ba9e821e1d7a8e. Tom Clegg
09:23 pm Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
commit:07e4083ea451913b988d77e8e4c926da8ad844a4
"Double-decode serialized fields if database was mangled by downgr...
Tom Clegg
06:22 pm Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
... Peter Amstutz
05:02 pm Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
... Peter Amstutz
04:59 pm Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
... Peter Amstutz
04:45 pm Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
Hold on, now I'm having trouble starting the API server in arvbox. I don't know if something is just corrupted/confu... Peter Amstutz
04:30 pm Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
LGTM @ commit:594e00f Peter Amstutz
05:43 am Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
> I notice that a couple of places have been switched to use SafeJSON, but they still have @require 'oj'@. The speci... Tom Clegg
09:46 pm Revision 660a6143 (arvados): Merge branch '11168-serialize-json'
closes #11168 Tom Clegg
09:44 pm Bug #11138: Protection against running new style docker 1.10+ image on old docker host
Lucas Di Pentima wrote:
> On line 428: Could it be that "docker load" fails for some reason, and the printed message...
Tom Clegg
07:59 pm Bug #11138: Protection against running new style docker 1.10+ image on old docker host
On line 428: Could it be that "docker load" fails for some reason, and the printed message still say that it exited w... Lucas Di Pentima
06:06 am Bug #11138: Protection against running new style docker 1.10+ image on old docker host
11138-docker-load-fail uses @docker inspect@ (before and after @docker load@) to assure that @docker run aaa123@ will... Tom Clegg
09:41 pm Revision f7d32ebc (arvados): 11138: Show actual image ID when checking whether docker image is lo...
Tom Clegg
09:18 pm Revision 07e4083e (arvados): 11168: Double-decode serialized fields if database was mangled by do...
Tom Clegg
04:31 pm Task #11195 (Resolved): Review 11168-serialize-json
Peter Amstutz
04:22 pm Task #11197 (In Progress): Review 11138-docker-load-fail
Lucas Di Pentima
06:10 am Task #11207 (Resolved): Make docker_install_script more robust
Tom Clegg
05:52 am Revision dd08615e (arvados): 11138: Test for docker image after loading, in case docker-load erro...
Tom Clegg
05:26 am Revision 594e00f9 (arvados): 11168: Always deep-sort before comparing in where_serialized.
Tom Clegg
05:25 am Revision 6fe6fb4a (arvados): 11168: Remove unused import.
Tom Clegg
05:17 am Revision c7a44e53 (arvados): 11168: Revert serialization change in order to avoid breaking job re...
Tom Clegg

03/02/2017

11:12 pm Revision 2ab1b540 (arvados): Merge branch 'master' into 11071-fts-perf-test
Radhika Chippada
11:06 pm Bug #11071: [Workbench] qr1hi performance tests are broken
Branch 11071-fts-perf-test @ commit:cad4856 takes a stab at using count='none' for full text search in workbench.
...
Radhika Chippada
10:53 pm Revision cad4856b (arvados): 11017: Use count='none' for full text search on workbench.
Update groups.load_searchable_objects which is used by contents method
to work with multiple class types when count='...
Radhika Chippada
09:08 pm Bug #11209 (Resolved): stuck keep fuse mounts not cleared by crunch-job
crunch-job attempts to unmount any fuse filesystems that are mounted under $CRUNCH_TMP but it attempts to do so only ... Joshua Randall
08:48 pm Story #11177: [GA4GH] Participate in interoperability challenge
https://www.synapse.org/#!Synapse:syn8080249/wiki/415165 Peter Amstutz
05:43 pm Story #11177 (Resolved): [GA4GH] Participate in interoperability challenge
Peter Amstutz
05:43 pm Task #11193 (Resolved): Submit writeup
Peter Amstutz
03:48 pm Bug #11138 (In Progress): Protection against running new style docker 1.10+ image on old docker host
Tom Clegg
03:48 pm Task #11207 (In Progress): Make docker_install_script more robust
Tom Clegg
03:48 pm Task #11207 (Resolved): Make docker_install_script more robust
Tom Clegg
03:42 pm Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
11168-serialize-json @ commit:766ddd6
I notice that a couple of places have been switched to use SafeJSON, but the...
Peter Amstutz
02:45 pm Story #11139 (In Progress): [Node manager] Expected MemTotal for each cloud node size
Lucas Di Pentima
02:33 pm Task #11195 (In Progress): Review 11168-serialize-json
Peter Amstutz
06:04 am Revision e9696655 (arvados): 11183: Add "setup -wait" flag
Tom Clegg

03/01/2017

11:07 pm Revision 345dfd5a (arvados): 11168: Add missing require.
Tom Clegg
10:59 pm Revision 766ddd6d (arvados): 11168: Prohibit down-migration to YAML-only codebase.
Tom Clegg
09:24 pm Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
11168-serialize-json @ commit:d7c84d69bb62d61bc671b2d5e0ad4ed42dbeb7c0 Tom Clegg
09:06 pm Bug #11168 (In Progress): [API] Use JSON instead of YAML for serialized fields in database
Tom Clegg
09:09 pm Revision d7c84d69 (arvados): 11168: Change db serialize from YAML to JSON.
Tom Clegg
09:01 pm Task #11205 (Resolved): Review branch 11071-fts-perf-test
Radhika Chippada
08:51 pm Task #11200 (Resolved): Review 11139-nodemanager-mem-scale-factor
Lucas Di Pentima
08:51 pm Task #11199 (Resolved): Review 10218-record-node-info
Lucas Di Pentima
08:51 pm Task #11198 (New): Review
Peter Amstutz
08:51 pm Task #11197 (Resolved): Review 11138-docker-load-fail
Tom Clegg
08:51 pm Task #11196 (Resolved): Review 11166-log-name-collision
Tom Clegg
08:51 pm Task #11195 (Resolved): Review 11168-serialize-json
Tom Clegg
08:50 pm Task #11194 (Resolved): Review 10401-limit-dir-expansion
Peter Amstutz
08:49 pm Task #11193 (Resolved): Submit writeup
Peter Amstutz
08:49 pm Task #11192 (Resolved): Review 9132-dockerclient
Peter Amstutz
08:48 pm Task #11191 (Resolved): Review 11158-fuse-projects
Peter Amstutz
08:45 pm Bug #11071 (In Progress): [Workbench] qr1hi performance tests are broken
Tom Morris
08:43 pm Bug #10729: [Crunch2] Propagate error messages if sbatch command succeeds but crunch-run can't ru...
We can also try running "srun" with the same flags, since it is more likely to return useful errors from the subproce... Peter Amstutz
08:37 pm Bug #11190 (Resolved): Containers seem to run more than once, which isn't supposed to happen
Example: tb05z-dz642-eie1eal1059y9bb Tom Clegg
08:28 pm Bug #11166: [Crunch2] crunchrun.go should avoid name collisions when creating log collections
Should be a matter of using the ensure_unique_name flag. Tom Clegg
05:58 pm Bug #11166: [Crunch2] crunchrun.go should avoid name collisions when creating log collections
We've been seeing CR logs with messages like this:... Lucas Di Pentima
08:24 pm Task #11119 (Resolved): Review 7995-keep-balance-docs
Tom Clegg
08:22 pm Task #11164 (Resolved): Review 9277-container-output
Peter Amstutz
08:22 pm Bug #10585 (Resolved): crunch doesn't end jobs when their arv-mount dies
Tom Clegg
08:12 pm Revision 15b33da8 (arvados): 9216: Require size hints in manifest validation.
Tom Clegg
08:11 pm Bug #10035 (In Progress): [FUSE] Determine why bcl2fastq doesn't work with writable keep mount
Peter Amstutz
08:11 pm Bug #10035 (Resolved): [FUSE] Determine why bcl2fastq doesn't work with writable keep mount
Peter Amstutz
08:11 pm Feature #6520 (Resolved): [Node Manager] [Crunch2] Take queued containers into account when compu...
Peter Amstutz
08:10 pm Feature #10764 (Resolved): [arvados-ws] Write unit tests
Applied in changeset arvados|commit:2db0c3a44d8f12e3a566c796bdecaa1de150b155. Tom Clegg
02:14 pm Feature #10764: [arvados-ws] Write unit tests
10764-ws-tests
commit:f36162457a771824059fefa098a3ffb89c59263f
https://ci.curoverse.com/job/developer-run-tests/180/
Tom Clegg
08:09 pm Bug #9437 (Resolved): keep-balance doesn't seem to be reducing replication level of blocks
Tom Clegg
08:04 pm Revision 2db0c3a4 (arvados): Merge branch '10764-ws-tests'
closes #10764 Tom Clegg
08:03 pm Task #11112 (Resolved): unit tests
Tom Clegg
08:03 pm Bug #11186 (Duplicate): Priority 0 state shouldn't be labeled "Ready"
Peter Amstutz
07:28 pm Bug #8113 (Resolved): "Download the full log" in log tab should show the log in the browser, not ...
Tom Clegg
07:00 pm Story #10401 (In Progress): [CWL] Limit expansion of Directory inputs
Peter Amstutz
06:24 pm Story #11185 (Resolved): [Workbench] Add ability to disable job reuse when running a workflow
* Either "re-run container request" button should present a dialog box asking the user if they want job reuse or not,... Tom Morris
05:41 pm Bug #8685: [FUSE] Add timestamp to arv-mount --logfile option
Nico - can you please expand/update the description for this. Tom Morris
05:37 pm Bug #8624: [FUSE] arv-mount `by_tag` directory only shows 100 tags and tags that exist are not ac...
Assigning to Peter for triage/grooming Tom Morris
03:58 pm Revision a58e9ca2 (arvados): 10764: De-duplicate real/test server startup. Add test for broken co...
Tom Clegg
03:37 pm Task #11108 (Resolved): Review 11015-crunch-run-output-upload
Radhika Chippada
02:14 pm Task #11118 (Resolved): Review 10777-die-if-arv-mount-dies
Tom Clegg
07:52 am Revision f3616245 (arvados): 10764: Simplify test server shutdown.
Tom Clegg
07:36 am Revision b3fa9983 (arvados): 10764: Test v0 session.
Tom Clegg

02/28/2017

11:23 pm Feature #11184 (New): [Keep] Support multiple storage tiers
As an Arvados system administrator, I'd want the ability take advantage of the cool/cold storage classes offered by c... Tom Morris
10:20 pm Bug #10777 (Resolved): [Crunch2] crunch-run: stop the container and fail if arv-mount dies before...
Applied in changeset arvados|commit:a98594f82f1786b09d9d1e0f57a651d4eae3474a. Tom Clegg
12:52 am Bug #10777: [Crunch2] crunch-run: stop the container and fail if arv-mount dies before the contai...
LGTM Radhika Chippada
10:08 pm Revision a98594f8 (arvados): Merge branch '10777-die-if-arv-mount-dies'
closes #10777 Tom Clegg
08:54 pm Feature #11183: [Deployment] Use consul to report health of services/components
work in progress on source:/@11183-setup-agent Tom Clegg
08:41 pm Feature #11183 (In Progress): [Deployment] Use consul to report health of services/components
Tom Clegg
08:52 pm Revision c78289e5 (arvados): 11183: Add "arvados-admin setup"
Tom Clegg
07:58 pm Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
This sounds like a nice improvement. That PG page has some warnings to which we should need to probably pay careful a... Javier Bértoli
07:08 pm Story #11139: [Node manager] Expected MemTotal for each cloud node size
Based on discussion: reduce the predicted RAM available for node by 5% Peter Amstutz
06:35 pm Feature #11015 (Resolved): Improve throughput of crunch-run output-uploading stage using multi-th...
Applied in changeset arvados|commit:0b125c52cd816e6a4120c414d3817f354cad1055. Radhika Chippada
06:18 pm Feature #11015: Improve throughput of crunch-run output-uploading stage using multi-threaded tran...
> Are the upload workers number configurable?
Number of writers >1 will help us as this let's us use multiple thre...
Radhika Chippada
04:44 pm Feature #11015: Improve throughput of crunch-run output-uploading stage using multi-threaded tran...
Sorry for taking so long to send comments, was trying to understand the code the best I could.
One question: Are the...
Lucas Di Pentima
06:29 pm Revision 0b125c52 (arvados): closes #11015
Merge branch '11015-crunch-run-output-upload' Radhika Chippada
06:23 pm Revision b7f4b12d (arvados): Merge branch 'master' into 11015-crunch-run-output-upload
Radhika Chippada
02:25 pm Task #11111 (Resolved): Review 11097-reuse-impure
Radhika Chippada
06:09 am Revision 784371c8 (arvados): 10764: Test v0 session. (WIP)
Tom Clegg
02:49 am Revision aea8596a (arvados): 10764: Permission tests. Support PDH permission check.
Tom Clegg

02/27/2017

10:51 pm Revision ca812f58 (arvados): 10764: Add unit tests
Tom Clegg
07:17 pm Story #11177 (Resolved): [GA4GH] Participate in interoperability challenge
https://www.synapse.org/#!Synapse:syn8080249/wiki/412159
# Get tool & test data
# Import / run it on Arvados
# D...
Peter Amstutz
07:07 pm Task #11118 (In Progress): Review 10777-die-if-arv-mount-dies
Radhika Chippada
06:39 pm Task #11108 (In Progress): Review 11015-crunch-run-output-upload
Lucas Di Pentima
06:25 pm Revision e86baeea (arvados): 11015: use multiple writers to increate throughput of goUpload.
Radhika Chippada
06:23 pm Task #11112 (In Progress): unit tests
Tom Clegg
04:54 pm Revision 58ed4ee3 (arvados): 10979: Add missing SqueueChecker initialization.
refs #10979 Tom Clegg
03:30 pm Story #11171 (New): [Nodemanager] Packaging for compute node setup scripts
All software installed into compute node images, including setup scripts, should be packaged so that documentation fo... Peter Amstutz
01:40 pm Bug #11002 (Resolved): [arv-put] crash in arvfile on upload NoneType object has no attribute 'clo...
Applied in changeset arvados|commit:9889021db676c152d5b62f399bd27024b5a51ae1. Lucas Di Pentima
01:37 pm Revision 9889021d (arvados): Merge branch '11002-arvput-crash-fix'
Closes #11002 Lucas Di Pentima
12:42 pm Revision 1130ddc2 (arvados): 11002: Merge branch 'master' into 11002-arvput-crash-fix
Lucas Di Pentima
12:41 pm Revision 954b101b (arvados): 11002: Added note explaining why we're expecting a SystemExit to cat...
Lucas Di Pentima
08:59 am Revision ceb95004 (arvados): more vault
Tom Clegg

02/26/2017

07:03 pm Bug #11170 (Resolved): Stale squeue processes on c97qk caused by "crunch-dispatch --jobs"
There were a total of 4694 of these processes, representing a significant resource leak.... Ward Vandewege
10:12 am Revision 307e7af0 (arvados): vault
Tom Clegg
12:35 am Revision a4512ef7 (arvados): Merge branch 'master' into deploy-agent
Conflicts:
build/run-tests.sh
Tom Clegg
12:32 am Revision cea3337f (arvados): multiple servers, add docker-compose test
Tom Clegg

02/25/2017

10:48 pm Revision 37972640 (arvados): gitignore
Tom Clegg
07:05 am Bug #10777: [Crunch2] crunch-run: stop the container and fail if arv-mount dies before the contai...
10777-die-if-arv-mount-dies @ commit:c106221bf8c90fd80ca2c19a578fd7faec985229 Tom Clegg
07:04 am Task #11169 (Resolved): Detect arv-mount death & error out
Tom Clegg
06:57 am Revision 2ac112e2 (arvados): 10777: Stop container if arv-mount dies before container exits.
Tom Clegg
06:57 am Revision c106221b (arvados): 10777: Close and flush logs right away instead of waiting for next t...
Tom Clegg
12:34 am Revision db371c57 (arvados-dev): arvados-version-server:
* style improvements: don't stutter
* test improvement: stop printing log output during tests
No issue #
Ward Vandewege

02/24/2017

10:46 pm Revision e47bb9c1 (arvados): run-tests.sh exit non-zero if gofmt fails
No issue # Tom Clegg
10:29 pm Revision 1042f200 (arvados): 10777: Stop container if arv-mount dies before container exits.
Tom Clegg
09:55 pm Task #11169 (In Progress): Detect arv-mount death & error out
Tom Clegg
09:55 pm Task #11169 (Resolved): Detect arv-mount death & error out
Tom Clegg
09:46 pm Revision 17f4a03d (arvados): 10979: Check for orphans only once at startup. Add missing Lock() in
squeue checker. Avoid holding mtx while waiting for API response.
Ensure RunContainer actually gets called in test ca...
Tom Clegg
08:56 pm Revision dccfd637 (arvados): build improvement: really include apps/workbench_functionals when
apps/workbench is specified.
No issue #
Ward Vandewege
07:43 pm Revision 353eb4f4 (arvados): Merge branch '6347-log-timestamps'
closes #6347 Tom Clegg
07:41 pm Bug #11168 (Resolved): [API] Use JSON instead of YAML for serialized fields in database
Currently hashes (like log properties) and arrays (like api_client_authorization scopes) are encoded in YAML, which i... Tom Clegg
07:30 pm Bug #10629 (Resolved): [FUSE] slow enumerating files by collection uuid
Applied in changeset arvados|commit:9b2aa42213a7d333bbe93e040c2d152a70e9b5af. Peter Amstutz
07:25 pm Story #9277 (Resolved): [Crunch2] System-owned container outputs should be garbage-collected
Applied in changeset arvados|commit:cca78a112d6a2aa4f76c1956cfe8ec2d43a68759. Peter Amstutz
06:30 pm Story #9277: [Crunch2] System-owned container outputs should be garbage-collected
It would be nice (if it doesn't already exist) to add a test such as "error setting output on running container if tr... Radhika Chippada
04:12 pm Story #9277: [Crunch2] System-owned container outputs should be garbage-collected
Additional fixes on 9277-container-output
Tests:
https://ci.curoverse.com/job/developer-run-tests/178/
Peter Amstutz
07:24 pm Task #11114 (Resolved): Review 10629-fuse-listing-perf
Peter Amstutz
07:24 pm Revision 9b2aa422 (arvados): Merge branch '10629-fuse-listing-perf' closes #10629
Peter Amstutz
07:22 pm Task #11041 (Resolved): Review 11002-arvput-crash-fix
Peter Amstutz
07:21 pm Revision cca78a11 (arvados): Merge branch '9277-container-output' closes #9277
Peter Amstutz
07:20 pm Revision b86728d2 (arvados): 9277: Add test that setting trashed, unreable collection is disallowed.
Peter Amstutz
06:53 pm Story #11167 (Resolved): [Workbench] Remove arv-get file download fallback
Currently, if keep-web isn't configured, Workbench will fallback to arv-get and Rails streaming to serve files. Howe... Peter Amstutz
06:51 pm Bug #11166 (Resolved): [Crunch2] crunchrun.go should avoid name collisions when creating log coll...
Lucas Di Pentima
06:45 pm Revision 00d95f0f (arvados): 6347: Use RFC3339Nano to render timestamps loaded from serialized fi...
Psych (YAML) serializes timestamps as ISO8601-with-space-separators,
and safe_load deserializes them to Time even wit...
Tom Clegg
06:28 pm Task #11164 (In Progress): Review 9277-container-output
Radhika Chippada
04:12 pm Task #11164 (Resolved): Review 9277-container-output
Peter Amstutz
06:17 pm Revision c702045c (arvados): refs #10979
Merge branch '10979-cancelled-job-nodes' Radhika Chippada
06:16 pm Revision 1a06dafa (arvados): 10979: fix failing test
Radhika Chippada
06:14 pm Bug #11002: [arv-put] crash in arvfile on upload NoneType object has no attribute 'closed'
Rest LGTM. Peter Amstutz
05:02 pm Bug #11002: [arv-put] crash in arvfile on upload NoneType object has no attribute 'closed'
From IRC: make a note around line L454 that because we catch signals, we expect a SystemExit, not KeyboardInterrupt.
...
Peter Amstutz
04:10 pm Story #11163 (In Progress): [GA4GH] Participate in workflow interoperability activities
Peter Amstutz
04:04 pm Story #11162 (New): [GA4GH] Support workflow execution API
Peter Amstutz
04:03 pm Story #11161 (New): [GA4GH] Support task execution API using crunch v2 containers
Peter Amstutz
04:02 pm Story #11160 (New): [GA4GH] Expose workflow records via tool registry API
Peter Amstutz
04:02 pm Story #11159 (New): [GA4GH] Import from tool registry API
Support import of tools from tool registry API.
https://github.com/ga4gh/tool-registry-schemas
Initial capabili...
Peter Amstutz
04:01 pm Revision a04b28a6 (arvados): 9277: arvados-cwl-runner sets "is_trashed" when directly setting out...
Peter Amstutz
03:53 pm Revision a6e4bccc (arvados): 9277: Container output check must be unscoped to include trashed col...
Peter Amstutz
09:17 am Revision c7df9c8c (arvados): 6347: Use RFC3339Nano to render timestamps loaded from serialized fi...
Psych (YAML) serializes timestamps as ISO8601-with-space-separators,
and safe_load deserializes them to Time even wit...
Tom Clegg
06:23 am Bug #11158 (Resolved): [FUSE] Hang on simple FUSE operation and when logging in again later
I changed directory on a shell node to a ~/keep/home/... project which each had a single sub project, which each had ... Tom Morris
05:54 am Revision 6029ffdc (arvados): closes #10979
Merge branch '10979-cancelled-job-nodes' Radhika Chippada
05:54 am Revision a6c7c8db (arvados): Merge branch 'master' into 10979-cancelled-job-nodes
Radhika Chippada
05:52 am Revision e2f2a9dd (arvados): 10979: ruby way of doing it
Radhika Chippada

02/23/2017

10:04 pm Bug #11071: [Workbench] qr1hi performance tests are broken
Looks like we need to use count=none for Workbench search... Tom Clegg
09:52 pm Bug #11002: [arv-put] crash in arvfile on upload NoneType object has no attribute 'closed'
Updates at commit:d2a202e
Added a specific exception class for this kind of errors, so it's not confused with othe...
Lucas Di Pentima
08:56 pm Revision d2a202ef (arvados): 11002: Don't save the state and log the stack trace before quitting ...
Updated tests to reflect this new behaviour. Lucas Di Pentima
07:10 pm Bug #11156 (Resolved): [Crunch2] 9tee4-xvhdp-0m3a4ys52od7dhs was stuck until c-d-s was restarted
Applied in changeset arvados|commit:19dcf1ed20214b8349c76037ba8679db07bf44fc. Tom Clegg
07:09 pm Bug #11156: [Crunch2] 9tee4-xvhdp-0m3a4ys52od7dhs was stuck until c-d-s was restarted
c-d-s was stuck in an infinite loop due to a bad condition in its "get next page" loop. Just kept repeating the same ... Tom Clegg
04:56 pm Bug #11156: [Crunch2] 9tee4-xvhdp-0m3a4ys52od7dhs was stuck until c-d-s was restarted
Looks like @9tee4-dz642-2a5iga7lyeg2rog@ was queued the whole time, but c-d-s didn't pick it up until after restart f... Tom Clegg
04:07 pm Bug #11156 (Resolved): [Crunch2] 9tee4-xvhdp-0m3a4ys52od7dhs was stuck until c-d-s was restarted

9tee4-xvhdp-0m3a4ys52od7dhs was stuck in the running state for almost 16 hours, with one container request (step1_4...
Ward Vandewege
07:04 pm Revision 19dcf1ed (arvados): 11156: Fix infinite loop condition.
closes #11156 Tom Clegg
06:55 pm Bug #11157 (New): [arv-put] When debug mode is active, report the cache file name being to be used
When trying to debug cache file locking problems, it's useful that arv-put report which file is going to use. Can be ... Lucas Di Pentima
06:18 pm Story #9277 (Feedback): [Crunch2] System-owned container outputs should be garbage-collected
... Peter Amstutz
04:37 pm Bug #10629: [FUSE] slow enumerating files by collection uuid
LGTM.
Local @sdk/python@ and @services/fuse@ tests ran without issues.
Tried to do some benchmarking using arvbox b...
Lucas Di Pentima
02:56 pm Task #11114 (In Progress): Review 10629-fuse-listing-perf
Lucas Di Pentima

02/22/2017

11:51 pm Bug #5901: [API] Improve performance of large requests in parallel
This has improved by over an order of magnitude(!) since 2015 which is great, but 20 seconds to fetch 75 MB still see... Tom Morris
11:23 pm Bug #9732: [FUSE] performance issues stat() in python
This seems like it could be related to #10629 Tom Morris
10:40 pm Bug #5713: [FUSE] File access sometimes takes too long on su92l
Has this problem been seen recently? It was reported over 2 years ago and a lot has changed since then, include a bun... Tom Morris
09:53 pm Bug #11071: [Workbench] qr1hi performance tests are broken
Another bit of evidence is this long API call from the logs on qr1hi which looks like it basically accounts for all o... Tom Morris
09:48 pm Bug #11155 (New): [Docs] Document keepstore S3 volume configuration
Add a "Configure S3 storage" page similar to the "Configure Azure Blob storage" page (http://doc.arvados.org/install/... Tom Clegg
09:48 pm Bug #11154 (New): [Docs] Migrate install docs from command line flags to YAML configs (keepstore,...
Currently, install docs recommend setting CLI flags, but these are deprecated.
Update install docs to use @/etc/ar...
Tom Clegg
09:45 pm Story #7995 (Resolved): [Documentation] Document Keep Balance setup in the Install Guide
Applied in changeset arvados|commit:1e6a756a10a1c0a77aeea5041844ba3a572bdd70. Tom Clegg
08:35 pm Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Great, it LGTM, then. Javier Bértoli
08:05 pm Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Javier Bértoli wrote:
> I notice you added these two flags and removed *-never-delete=true*. Is that correct, or jus...
Tom Clegg
05:27 pm Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Tom Clegg wrote:
> Updated the dry run instructions:
>
> To do this, edit your keep-balance startup script to use...
Javier Bértoli
04:45 pm Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Javier Bértoli wrote:
> Keepproxy is optional, as I understand it. If so, can I have more than one keep-balance, ins...
Tom Clegg
04:38 pm Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Updated the dry run instructions:
To do this, edit your keep-balance startup script to use the flags @-commit-pull...
Tom Clegg
04:19 pm Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Tom Morris,
* From this text:
bq. Keep-balance can be installed anywhere with network access to Keep services. ...
Javier Bértoli
04:18 pm Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Tom Morris wrote:
> I made a few copy edits and pushed them to the branch. Please review them to make sure that thin...
Tom Clegg
09:44 pm Feature #11153 (New): [API] create_token script should allow operator to provide token scopes and...
h3. Background
Currently the install guide relies on the @script/create_superuser_token.rb@ program in the apiserv...
Tom Clegg
09:41 pm Bug #10629: [FUSE] slow enumerating files by collection uuid
10629-fuse-listing-perf ready for review. Peter Amstutz
08:35 pm Bug #10629 (In Progress): [FUSE] slow enumerating files by collection uuid
... Peter Amstutz
04:16 pm Bug #10629: [FUSE] slow enumerating files by collection uuid
... Peter Amstutz
09:36 pm Revision 1e6a756a (arvados): Merge branch '7995-keep-balance-docs'
closes #7995 Tom Clegg
09:33 pm Revision 2a30eacc (arvados): Remove pidfiles after shutting down test servers.
No issue # Tom Clegg
09:28 pm Revision aedb62c7 (arvados): 10629: Don't flush dirhandles.
Peter Amstutz
09:08 pm Revision fe85406f (arvados): 10629: Make tracking and dirtying of _committed flag efficient.
Peter Amstutz
08:40 pm Revision 04fb9a14 (arvados): 10629: improve debug logging
--debug includes Keep logging.
--logfile includes timestamps.
Peter Amstutz
08:37 pm Task #11119 (Feedback): Review 7995-keep-balance-docs
Javier Bértoli
07:49 pm Revision 25af6c40 (arvados): Merge branch 'master' into 10979-cancelled-job-nodes
Radhika Chippada
07:48 pm Revision 3a31db02 (arvados): 10979: refactor squeue invocations
Radhika Chippada
06:26 pm Revision 56a99279 (arvados): 11002: Track this specific error with its own exception class, for f...
Lucas Di Pentima
06:25 pm Bug #11097 (Resolved): [API] Reuse containers even when multiple matching containers exist with d...
Applied in changeset arvados|commit:0c529ed05805507b4d2c903b9587e9b61cec5ee6. Tom Clegg
06:19 pm Revision 0c529ed0 (arvados): Merge branch '11097-reuse-impure'
closes #11097 Tom Clegg
04:45 pm Revision d8f1b7e8 (arvados): 7995: Add note about one keep-balance process at a time.
Tom Clegg
04:29 pm Revision 11f5e94c (arvados): 7995: Fix up dry-run instructions.
Tom Clegg
04:24 pm Revision ced8b355 (arvados): 7995: Fix up inconsistent "e.g." vs. "e.g.,".
Tom Clegg
04:02 pm Revision d2c9ca97 (arvados): 7995: Copy edits.
Tom Clegg
03:38 pm Feature #11015 (In Progress): Improve throughput of crunch-run output-uploading stage using multi...
Radhika Chippada
03:29 pm Revision 423d27d0 (arvados): Fix dispatch panic when processing an update after tracker has been ...
refs #11151 Tom Clegg

02/21/2017

11:46 pm Bug #11071: [Workbench] qr1hi performance tests are broken
Actually the commits are likely a red herring since it doesn't look like qr1hi was deployed on or immediately after D... Tom Morris
11:15 pm Bug #11071: [Workbench] qr1hi performance tests are broken
Here are the first few commits after the last successful pass (read from bottom up):
commit b27f1106beb0fb321ee123...
Tom Morris
09:53 pm Bug #10729: [Crunch2] Propagate error messages if sbatch command succeeds but crunch-run can't ru...
Thoughts about how to retrieve the error messages:
* If the cluster is set up with shared home directories (which is...
Tom Clegg
09:40 pm Revision fc742c91 (arvados-dev): Merge branch 'master' of git.curoverse.com:arvados-dev
Git sync user
09:39 pm Revision 1b617d62 (arvados-dev): Fix detection of version of arvados/jobs image to install.
No issue # Ward Vandewege
09:35 pm Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
I made a few copy edits and pushed them to the branch. Please review them to make sure that things are still technica... Tom Morris
09:35 pm Revision fc54d2d3 (arvados): A few copy edits
Tom Morris
09:34 pm Revision 6159bebd (arvados): Document epydoc dependency
Tom Morris
09:08 pm Task #11119 (In Progress): Review 7995-keep-balance-docs
Tom Morris
08:57 pm Bug #11097: [API] Reuse containers even when multiple matching containers exist with differing ou...
> Yes, trashing the output or log collection would accomplish this ... "poking re-use in the eye" procedure seems wo... Radhika Chippada
08:36 pm Bug #11097: [API] Reuse containers even when multiple matching containers exist with differing ou...
Radhika Chippada wrote:
> * I think moving “select_readable_pdh” to the line above the declaration of “candidates” a...
Tom Clegg
07:58 pm Bug #11097: [API] Reuse containers even when multiple matching containers exist with differing ou...
* I think moving “select_readable_pdh” to the line above the declaration of “candidates” at line 85 would help improv... Radhika Chippada
08:27 pm Revision 802af81e (arvados): 11097: Clarify reuse query.
Tom Clegg
08:27 pm Revision 1732df24 (arvados): 11097: Update docs to reflect new container reuse behavior.
Tom Clegg
07:58 pm Task #11111 (In Progress): Review 11097-reuse-impure
Radhika Chippada
07:56 pm Bug #11148 (New): [Crunch2] Propagate dispatch error messages (e.g., sbatch fails) to user via lo...
h3. Background
If crunch-dispatch-slurm encounters an error while trying to submit a container to the slurm queue,...
Tom Clegg
07:44 pm Story #9619 (Rejected): [Crunch2] [API] Default vcpus value to fill in for ContainerRequests
Implemented by client a-c-r Tom Morris
07:44 pm Story #9620 (Rejected): [Crunch2] [API] Default ram value to fill in for ContainerRequests
Implemented by client cwl runner Tom Morris
07:41 pm Story #11147 (New): Write scalability test which can be used periodically as needed
Tom Morris
07:00 pm Feature #11146 (New): [Crunch2] [Workbench] Show slurm queue position of containers submitted to ...
h3. Background
From the user's perspective, it's hard to see what (if anything) is happening between the time a co...
Tom Clegg
06:56 pm Bug #10008 (New): [SDKs] [Python] When reading data through Collection et al., signatures should ...
Tom Clegg
06:49 pm Bug #11002: [arv-put] crash in arvfile on upload NoneType object has no attribute 'closed'
From our conversations on IRC, I think KeyboardInterrupt should skip checkpointing but shouldn't log anything. Howev... Peter Amstutz
06:21 pm Bug #11002: [arv-put] crash in arvfile on upload NoneType object has no attribute 'closed'
Updates: commit:a83e4de
Test run: https://ci.curoverse.com/job/developer-run-tests/175/
Changed so when catching ...
Lucas Di Pentima
04:07 pm Bug #11002: [arv-put] crash in arvfile on upload NoneType object has no attribute 'closed'
So if I'm following along, the problem is that it is in repack_small_blocks() which creates a new, un-owned bufferblo... Peter Amstutz
01:50 pm Bug #11002: [arv-put] crash in arvfile on upload NoneType object has no attribute 'closed'
Branch @11002-arvput-crash-fix@ at commit:e2ea7599812d12dd027930069ee92e8816ee9de8
Test run: https://ci.curoverse.co...
Lucas Di Pentima
06:44 pm Bug #11093: "arv keep_service create" should go to an editor.
You want "arv create keep_service"
or do you mean the default behavior should be more user friendly? :-)
Peter Amstutz
06:29 pm Revision 2b34839c (arvados): 10979: scancel orphaned job nodes in crunch1.
Radhika Chippada
05:41 pm Revision a83e4de2 (arvados): 11002: Do not try to save internal state when receiving a KeyboardIn...
Updated test accordingly. Lucas Di Pentima
04:33 pm Story #11123 (In Progress): Add node manager to install guide
Peter Amstutz
04:32 pm Task #11041 (In Progress): Review 11002-arvput-crash-fix
Peter Amstutz
04:29 pm Feature #11015: Improve throughput of crunch-run output-uploading stage using multi-threaded tran...
Possible implementation
In (*CollectionFileWriter)goUpload(), currently we have...
Tom Clegg
04:22 pm Revision 8ed521f7 (arvados): 11002: Merge branch 'master' into 11002-arvput-crash-fix
Lucas Di Pentima
01:00 pm Revision e2ea7599 (arvados): 11002: When trying to save the cache's state before quitting, if an ...
is caught because of a BlockManager problem induced by an interruption,
print a warning message and quit without savi...
Lucas Di Pentima
12:54 pm Revision f69196e8 (arvados): 11002: Added missing assertion to test.
Lucas Di Pentima
11:41 am Revision 3988d2c6 (arvados): 11002: Improved test mocking a more suitable method and catching the...
Lucas Di Pentima

02/20/2017

10:04 pm Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
7995-keep-balance-docs @ commit:14304c7af0b0dd7bc9345b6c5aeb61a3bdc1d3b0 Tom Clegg
10:03 pm Task #11134 (Resolved): Explain limitations re shared volumes and service_type
Tom Clegg
08:50 pm Task #11134 (In Progress): Explain limitations re shared volumes and service_type
Tom Clegg
10:03 pm Revision 14304c7a (arvados): 7995: Add "dry run" note.
Tom Clegg
09:32 pm Task #11141 (Resolved): Review 6520-nodemanager-docs
Peter Amstutz
09:18 pm Revision 8b75947e (arvados): Merge branch '6520-pending-reason' refs #6520
Peter Amstutz
09:17 pm Revision 4bd54c4c (arvados): 6520: Add ReqNodeNotAvail to list of reasons (along with "Resources"...
Peter Amstutz
09:13 pm Revision 53887579 (arvados): 6520: Add information about setting up SLURM to crunchv2 documentation.
Peter Amstutz
08:49 pm Task #11140 (Resolved): Update tests
Tom Clegg
08:34 pm Task #11140 (In Progress): Update tests
Tom Clegg
08:34 pm Task #11140 (Resolved): Update tests
Tom Clegg
08:46 pm Bug #11097: [API] Reuse containers even when multiple matching containers exist with differing ou...
commit:4f0e07d462b7860bb10686c27fac16970220377f with updated test case. Tom Clegg
08:45 pm Revision 4f0e07d4 (arvados): 11097: Update test to match new behavior.
Tom Clegg
08:34 pm Revision 4cd89bd1 (arvados): 11097: Merge branch 'master' into 11097-reuse-impure
Tom Clegg
08:25 pm Revision 142a3c23 (arvados): 11002: Added test to make the bug happen.
Lucas Di Pentima
07:47 pm Story #11139: [Node manager] Expected MemTotal for each cloud node size
This can be fixed entirely through configuration by overriding the "ram" field for each size. However, we need deter... Peter Amstutz
07:07 pm Story #11139: [Node manager] Expected MemTotal for each cloud node size
GCP... Peter Amstutz
06:20 pm Story #11139 (Resolved): [Node manager] Expected MemTotal for each cloud node size
There's a discrepancy between the RAM of a VM used to choose what size node to boot for a job, and the actual amount ... Peter Amstutz
04:45 pm Task #11133 (Resolved): Set -enable-trash in keepstore docs
Tom Clegg
03:12 pm Task #11133 (In Progress): Set -enable-trash in keepstore docs
Tom Clegg
03:41 pm Revision 84bd6883 (arvados): 7995: Add keep-balance to install guide.
Tom Clegg
12:10 pm Bug #11138 (Resolved): Protection against running new style docker 1.10+ image on old docker host
I upgraded docker to 1.13.1 over the weekend (or so I thought). Since then, I've been struggling to get the system wo... Joshua Randall
09:28 am Revision fc33deae (arvados): cmd, lib
Tom Clegg

02/19/2017

12:06 pm Bug #11137 (New): apt-get upgrade arvados-api-server on Ubuntu 12.04 did not update crunch-job
My jobs were failing since I upgraded docker to 1.13.1 and installed v1.10+ versions of the docker images in keep.
...
Joshua Randall

02/17/2017

10:32 pm Revision d069de03 (arvados): Merge branch '11127-delete-trash-with-links'
refs #11127 Tom Clegg
10:30 pm Bug #11136 (New): Nginx config should speak JSON when returning its own response for unproxyable ...
Also update the install documentation to explain to customers how to do this. Tom Clegg
10:27 pm Story #7995 (In Progress): [Documentation] Document Keep Balance setup in the Install Guide
Tom Clegg
10:26 pm Task #11132 (Resolved): Add page to install guide
Tom Clegg
10:23 pm Task #11132 (In Progress): Add page to install guide
Tom Clegg
10:23 pm Task #11132 (Resolved): Add page to install guide
Tom Clegg
10:26 pm Task #11134 (Resolved): Explain limitations re shared volumes and service_type
Tom Clegg
10:26 pm Task #11133 (Resolved): Set -enable-trash in keepstore docs
Tom Clegg
10:25 pm Bug #11121 (Resolved): [Crunch] Set output collection owner_uuid to match job
Applied in changeset arvados|commit:d0ba6eaac1387ad817de1c2df2d7f4f00800aaa5. Lucas Di Pentima
09:45 pm Bug #11121: [Crunch] Set output collection owner_uuid to match job
LGTM, thanks! Tom Clegg
10:23 pm Revision ead2ecb2 (arvados): 7995: Add keep-balance to install guide.
Tom Clegg
10:20 pm Task #11125 (Resolved): Review 11121-crunch-output-collection-owner
Lucas Di Pentima
10:18 pm Revision d0ba6eaa (arvados): Merge branch '11121-crunch-output-collection-owner'
Closes #11121 Lucas Di Pentima
10:17 pm Revision b1fffbeb (arvados): 11121: Merge branch 'master' into 11121-crunch-output-collection-owner
Lucas Di Pentima
09:17 pm Revision 484370cb (arvados): 11127: Delete dependent links too when emptying trash.
Tom Clegg
08:28 pm Feature #10218: [Crunch2] Gather and record cloud/physical node information for each container
Things to consider logging:
$ uname -a
$ cat /proc/cpuinfo
$ cat /proc/meminfo
Peter Amstutz
08:15 pm Revision f29e958f (arvados): 10979: TrackContainer implementation and test
Radhika Chippada
07:49 pm Revision 85887cd7 (arvados): 11127: Do not crash server if SweepTrashedCollections thread has an ...
Tom Clegg
12:30 pm Bug #9998: Improve performance of API server when keep-balance retrieves collections
Using the arv cli, I timed the performance of collection retrieval on our system using a query very similar to what k... Joshua Randall
08:09 am Revision 016a995d (arvados): more vault
Tom Clegg

02/16/2017

11:22 pm Revision f6071ef7 (arvados): Merge branch 'master' into 10979-cancelled-job-nodes
Radhika Chippada
10:20 pm Revision 6f93ea89 (arvados): 6520: Node manager docs WIP
Peter Amstutz
09:46 pm Task #11031 (Resolved): Review 6520-nodemanager-crunchv2
Peter Amstutz
09:08 pm Task #11031 (In Progress): Review 6520-nodemanager-crunchv2
Tom Clegg
09:46 pm Task #11106 (Resolved): Review 6520-skip-compute0
Peter Amstutz
09:08 pm Task #11106 (In Progress): Review 6520-skip-compute0
Tom Clegg
09:44 pm Revision 3fa4a2b6 (arvados): Merge branch '6520-nodemanager-crunchv2' refs #6520
Peter Amstutz
09:43 pm Revision ed1acd6f (arvados): 6520: Crunch v1 recognizes 'mix' and 'drng' as busy states for
consistency (won't affect node manager behavior going forward, however.) Peter Amstutz
09:35 pm Feature #6520: [Node Manager] [Crunch2] Take queued containers into account when computing how ma...
Tom Clegg wrote:
> in ArvadosNodeListMonitorActor it looks like "alloc*" will be reported as "down" -- instead of "a...
Peter Amstutz
09:18 pm Feature #6520: [Node Manager] [Crunch2] Take queued containers into account when computing how ma...
... Peter Amstutz
09:07 pm Feature #6520: [Node Manager] [Crunch2] Take queued containers into account when computing how ma...
in ArvadosNodeListMonitorActor it looks like "alloc*" will be reported as "down" -- instead of "alloc", which is how ... Tom Clegg
09:34 pm Revision 24b1aecb (arvados): Merge branch '6520-skip-compute0' refs #6520
Peter Amstutz
09:31 pm Revision d5ddfd9d (arvados): 6520: Expand recognized 'busy' states.
Peter Amstutz
09:17 pm Task #11125 (Resolved): Review 11121-crunch-output-collection-owner
Lucas Di Pentima
09:17 pm Bug #11121: [Crunch] Set output collection owner_uuid to match job
Test added at commit:3a6966e
Test run: https://ci.curoverse.com/job/developer-run-tests/170/
Lucas Di Pentima
05:03 pm Bug #11121: [Crunch] Set output collection owner_uuid to match job
Branch @11121-crunch-output-collection-owner@ - Update at: commit:0403a0c
Running tests: https://ci.curoverse.com/jo...
Lucas Di Pentima
04:04 pm Bug #11121 (In Progress): [Crunch] Set output collection owner_uuid to match job
Lucas Di Pentima
03:50 pm Bug #11121 (Resolved): [Crunch] Set output collection owner_uuid to match job
Currently, if a job is started with owner_uuid set to a shared project, the job and its log are shared, but its outpu... Tom Clegg
09:11 pm Revision 3a6966e5 (arvados): 11121: Added test.
Lucas Di Pentima
09:07 pm Revision f6bdb550 (arvados): Merge branch 'master' into 10979-cancelled-job-nodes
Radhika Chippada
08:16 pm Task #11107 (Resolved): Review 9277-trash-container-outputs
Radhika Chippada
07:45 pm Task #11107 (In Progress): Review 9277-trash-container-outputs
Radhika Chippada
08:15 pm Story #9277 (Resolved): [Crunch2] System-owned container outputs should be garbage-collected
Applied in changeset arvados|commit:21598295f38998d8028aaa117f192de6b5758808. Peter Amstutz
08:02 pm Story #9277: [Crunch2] System-owned container outputs should be garbage-collected
LGTM Radhika Chippada
07:16 pm Story #9277 (In Progress): [Crunch2] System-owned container outputs should be garbage-collected
Peter Amstutz
06:15 pm Story #9277: [Crunch2] System-owned container outputs should be garbage-collected
Even better than @trash_at=now@, it looks like we can just set @is_trashed: true@
9277-trash-container-outputs @ c...
Peter Amstutz
08:13 pm Revision 21598295 (arvados): Merge branch '9277-trash-container-outputs' closes #9277
Peter Amstutz
08:10 pm Revision 425f9a75 (arvados): 6520: Explicitly mark node as 'down' before shutting down.
Peter Amstutz
08:04 pm Revision be33e85d (arvados): closes #11104
Merge branch '11104-work-unit-expand-on-caret' Radhika Chippada
07:08 pm Revision 73c2bd55 (arvados): 11104: add fa-caret-down class to hyperlink rather than adding a spa...
Radhika Chippada
06:37 pm Story #11123 (Resolved): Add node manager to install guide
Peter Amstutz
05:51 pm Revision 72873aff (arvados): 9277: Include trashed collections when creating copy collection for ...
request. Peter Amstutz
05:49 pm Revision dc086bed (arvados): closes #11103
Merge branch '11103-fts-crs' Radhika Chippada
05:48 pm Revision 197dd558 (arvados): 11103: ensure the index exists before attempting to drop it.
Radhika Chippada
05:28 pm Revision 2d80bb7d (arvados): 9277: crunch-run Log and output collections set is_trashed to true i...
setting trash_at. Peter Amstutz
05:27 pm Revision 1aa3f436 (arvados): 11103: Omit mounts from container_requests and definition from workf...
Radhika Chippada
05:09 pm Task #11122 (New): log partial job output to keep while job is running
Joshua Randall
05:01 pm Revision aaa53694 (arvados): Merge branch 'master' into deploy-agent
Conflicts:
build/package-build-dockerfiles/centos7/Dockerfile
build/package-build-dockerfiles/debian8/Dockerfile
b...
Tom Clegg
05:00 pm Revision 0403a0c8 (arvados): 11121: Add owner_uuid to output collection on crunch-job
Lucas Di Pentima
03:42 pm Bug #9232 (Resolved): datamanager outputs replication level distribution histogram in unsorted order
(incorporated into keep-balance) Tom Clegg
03:25 pm Story #11017 (Resolved): [API Server] Implement Docker version compatibility fallback support
Applied in changeset arvados|commit:5624fec61db977d386ce03ca333241c74ca251b5. Tom Clegg
03:17 pm Revision 5624fec6 (arvados): Merge branch '11017-docker-migration'
closes #11017 Tom Clegg

02/15/2017

10:41 pm Revision 4edf67ae (arvados): Merge branch '10797-ruby-2.3'
closes #10797 Tom Clegg
10:40 pm Revision 31d76600 (arvados): 10797: Merge branch 'master' into 10797-ruby-2.3
Tom Clegg
10:40 pm Revision 03306e21 (arvados): 10797: Ruby 2.3.3
Tom Clegg
10:35 pm Revision 6873eca0 (arvados): Update golang version to 1.7.5. Also make the equivalent changes to the
(new) ubuntu1604 build scripts.
refs #10797
Ward Vandewege
10:25 pm Revision 6b775ec4 (arvados): Merge branch 'master' into 10797-ruby-2.3
refs #10797 Ward Vandewege
09:38 pm Task #11119 (Resolved): Review 7995-keep-balance-docs
Tom Morris
09:37 pm Task #11118 (Resolved): Review 10777-die-if-arv-mount-dies
Tom Clegg
09:35 pm Task #11115 (New): Review
Peter Amstutz
09:35 pm Task #11114 (Resolved): Review 10629-fuse-listing-perf
Peter Amstutz
09:35 pm Task #11112 (Resolved): unit tests
Tom Clegg
09:35 pm Task #11111 (Resolved): Review 11097-reuse-impure
Tom Clegg
09:30 pm Task #11108 (Resolved): Review 11015-crunch-run-output-upload
Radhika Chippada
09:28 pm Bug #7853 (Closed): [Data Manager] Behave appropriately when multiple keepstore nodes share a sin...
Tom Clegg
08:33 pm Task #11107 (Resolved): Review 9277-trash-container-outputs
Peter Amstutz
08:29 pm Feature #10267 (Rejected): Permission-monitoring service keeps a flattened permissions table up t...
Tom Clegg
08:29 pm Feature #10683 (Rejected): [Crunch2] Separate service to poll sinfo and update nodes table
Node manager checks sinfo directly in #6520 Peter Amstutz
08:17 pm Task #11106 (Resolved): Review 6520-skip-compute0
Peter Amstutz
08:15 pm Revision fb60dbc9 (arvados): 6520: Don't ever assign compute0, start at slot 1 when assigning hos...
Peter Amstutz
08:08 pm Revision 314f3c34 (arvados): 6520: Check that hostname is set before setting weight.
Peter Amstutz
07:56 pm Revision 15c13298 (arvados): 6520: Apply node size fixup in CloudNodeListMonitorActor
Peter Amstutz
07:47 pm Revision 264ffa31 (arvados): 11097: Drop check for conflicting outputs when reusing completed con...
Tom Clegg
07:31 pm Revision 47476fa1 (arvados): 9849: Fix expensive "all readable collections" query.
Tom Clegg
05:57 pm Task #11035 (Resolved): Review 11017-docker-migration
Radhika Chippada
04:50 pm Task #11035 (In Progress): Review 11017-docker-migration
Radhika Chippada
05:56 pm Story #11017: [API Server] Implement Docker version compatibility fallback support
LGTM Radhika Chippada
05:28 pm Revision 0e6c1a28 (arvados): 6520: Update cloud_node.size to match server calculator configuration.
Peter Amstutz
05:14 pm Feature #6520: [Node Manager] [Crunch2] Take queued containers into account when computing how ma...
Skipping compute0 ensures that even if every other node compute1-compute255 in slurm are updated to smaller actual no... Peter Amstutz
05:10 pm Feature #6520: [Node Manager] [Crunch2] Take queued containers into account when computing how ma...
It turns out you can't reconfigure hardware from "scontrol". You can only set the hardware configuration in slurm.co... Peter Amstutz
04:30 pm Revision e86743ca (arvados): 11017: Add arv-migrate-docker19 to package.
Tom Clegg
04:29 pm Revision 908959ea (arvados): 6520: Assign weight to node based on price to preferentially schedul...
cheaper nodes. Peter Amstutz
06:19 am Revision bee30106 (arvados): consul acl
Tom Clegg
 

Also available in: Atom