Tom Clegg's activity

From 02/19/2017 to 03/20/2017

03/20/2017

05:22 pm Arvados Feature #10218: [Crunch2] Gather and record cloud/physical node information for each container
Is it necessary to split JSON into multiple lines rather than just writing it as one chunk? If we write it as one chu... Tom Clegg
03:47 pm Arvados Task #11199 (In Progress): Review 10218-record-node-info
Tom Clegg
02:20 pm Arvados Feature #7709: [API] Upgrade API server to Rails 4.2
So far, the most annoying part has been making the websocket tests (or server?) work. Switching to arvados-ws and aba... Tom Clegg
02:14 pm Arvados Bug #3224: [Workbench] Search box on collections page should use infinite scroll instead of (brok...
The collections page is linked from the dashboard ("all collections" button).
All of the bugs noted still exist to...
Tom Clegg

03/19/2017

01:20 am Arvados Task #11298 (Resolved): make tests pass in rails4
Tom Clegg
01:19 am Arvados Task #11297 (In Progress): update bundle
Tom Clegg
01:19 am Arvados Task #11297 (Resolved): update bundle
Tom Clegg

03/17/2017

08:41 pm Arvados Task #11292 (Resolved): Review 11209-unmount-replace
Tom Clegg
08:41 pm Arvados Bug #11209: stuck keep fuse mounts not cleared by crunch-job
11209-unmount-replace @ commit:5752685c137c5e37e13845f5328e9a3930fa3100
This should let us replace the @"mount|awk...
Tom Clegg
08:32 pm Arvados Revision 5752685c (arvados): 11209: "--unmount /path/..." unmounts /path and all fuse mounts belo...
Tom Clegg
05:36 pm Arvados Wiki edit: Emacs_and_Go (#1)
Tom Clegg
03:23 pm Arvados Wiki edit: Hacking (#11)
Tom Clegg
03:16 pm Arvados Wiki edit: Go (#11)
Tom Clegg

03/16/2017

08:58 pm Arvados Wiki edit: Wiki (#96)
Tom Clegg
08:57 pm Arvados Wiki edit: Keep-balance (#7)
Tom Clegg
08:56 pm Arvados Wiki edit: Keep-balance (#6)
Tom Clegg
08:49 pm Arvados Revision c733e426 (arvados): 11209: Add arv-mount --unmount and --replace flags
Tom Clegg
08:43 pm Arvados Revision 9a7d9243 (arvados): 11209: Add arv-mount --unmount and --replace flags
Tom Clegg
08:40 pm Arvados Task #11271 (In Progress): Review 11254-nodemanager-no-actor
Tom Clegg
08:39 pm Arvados Bug #11254: [Node manager] backtrace on node shutdown
11254-nodemanager-no-actor @ commit:2c69d49 LGTM
...although I'd say the duplicated conditions here were already a...
Tom Clegg
05:24 pm Arvados Revision 386faadf (arvados): 10218: Wait for container to be started (not just created) before tr...
Tom Clegg

03/15/2017

08:07 pm Arvados Task #11267 (Resolved): Review 10669-safe-http-cache
Tom Clegg
08:07 pm Arvados Task #11266 (Resolved): Review 11065-rotate-logs
Tom Clegg
08:07 pm Arvados Task #11264 (Resolved): Review 7709-api-rails4
Tom Clegg
08:07 pm Arvados Task #11263 (New): Review
Tom Clegg
07:31 pm Arvados Bug #11261: [API] full text search on pipeline_instances table is extremely slow (wrong index?)
... Tom Clegg
07:30 pm Arvados Bug #11261: [API] full text search on pipeline_instances table is extremely slow (wrong index?)
Copied from #11071:
Version query:...
Tom Clegg
07:29 pm Arvados Bug #11261 (New): [API] full text search on pipeline_instances table is extremely slow (wrong ind...
Tom Clegg
07:26 pm Arvados Bug #11260 (New): [API] "arv group contents" produces a 404 error (seems to POST instead of GET?)
Tom Clegg
07:23 pm Arvados Bug #11071: [Workbench] qr1hi performance tests are broken
Version query:... Tom Clegg
05:37 pm Arvados Bug #11237: python-arvados-python-client depends on old versions of packages
This pin comes from commit:ec3ff9b4 "Pin google-api-python-client and oauth2client versions to deal with incompatabil... Tom Clegg
03:31 pm Arvados Story #11251: [Tests] Make workbench integration tests finish faster (at least in Jenkins)
IMO we should be fixing slow/redundant test cases before we give up and make the test pipeline/server fatter. It look... Tom Clegg
02:57 pm Arvados Wiki edit: Data_Manager_Design_Doc (#5)
Tom Clegg
02:54 pm Arvados Bug #11209: stuck keep fuse mounts not cleared by crunch-job
The "fuse bug":https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=632258 seems to be related to a double-mounted mount... Tom Clegg

03/14/2017

07:05 pm Arvados Wiki edit: Proposals (#54)
Tom Clegg
06:18 pm Arvados Feature #11249 (New): [SDKs] Writable collection files returned by Go SDK should be seekable
The reading+seeking and writing code need to be reconciled so OpenFile and Create return an object that can read, wri... Tom Clegg
05:29 pm Arvados Feature #10664 (Duplicate): arv-keepdocker should upload images with image:tag
Tom Clegg

03/13/2017

06:48 pm Arvados Bug #11209 (In Progress): stuck keep fuse mounts not cleared by crunch-job
Tom Clegg
06:48 pm Arvados Bug #11209: stuck keep fuse mounts not cleared by crunch-job
Normally crunch-job frees up mount points using @fusermount -u -z@ but for some reason it isn't working here:
<pre...
Tom Clegg

03/10/2017

10:16 pm Arvados Story #8773 (Closed): [GATK Queue] Support a mechanism to redirect stderr for children
Tom Clegg
10:15 pm Arvados Bug #7000 (Closed): [Docker] API server starts before PostgreSQL is ready
Tom Clegg
10:15 pm Arvados Bug #7007 (Closed): [Docker] Bootstrap font fails to load in Workbench Docker image
Tom Clegg
10:15 pm Arvados Bug #7006 (Closed): [Docker] Fix SSO configuration in published images
Tom Clegg
10:15 pm Arvados Bug #7005 (Closed): [Docker] API server uses omniauth.rb for SSO configuration, not application.yml
Tom Clegg
10:15 pm Arvados Bug #6998 (Closed): [Docker] Workbench image doesn't serve HTTPS
Tom Clegg
10:10 pm Arvados Bug #5949 (Closed): [DRAFT] [Workbench] Users can be notified about changes to important pipelines
Tom Clegg
10:07 pm Arvados Feature #5828 (Closed): [Workbench] Create new collection with selected files copy streams as wel...
Tom Clegg
10:07 pm Arvados Bug #5850 (Closed): All clients authenticate directly against SSO server without going through AP...
Tom Clegg
10:04 pm Arvados Bug #5876 (Closed): [Docker] Needs smarter dependency ordering when restarting existing containers
Tom Clegg
10:04 pm Arvados Bug #5878 (Closed): [Workbench] PipelineInstancesTest#test_Create_and_run_a_pipeline is not reliable
Tom Clegg
10:04 pm Arvados Bug #5892 (Closed): [Workbench] collections#index returns 504 Gateway Timeout
Tom Clegg
10:04 pm Arvados Bug #5897 (Closed): [FUSE] Catch and recover from any API client exceptions
Tom Clegg
10:03 pm Arvados Feature #5900 (Closed): [SDK] Implement arvados.gettaskparam(*args) to match arvados.getjobparam(...
Tom Clegg
10:02 pm Arvados Bug #5905 (Closed): [SDKs] Improve certificate validation in Python SDK
Tom Clegg
10:01 pm Arvados Bug #5908 (Closed): [Workbench] collection#show performing too many unnecessary API server reques...
Tom Clegg
10:01 pm Arvados Bug #5923 (Closed): Fiddlesticks when trying to view a log file
Tom Clegg
10:01 pm Arvados Bug #5919 (Closed): Keep errors
Tom Clegg
10:01 pm Arvados Feature #5928 (Closed): Syntax highlighting of viewing source in workbench
Tom Clegg
10:01 pm Arvados Bug #5945 (Closed): arv-get is slow for this collection
Tom Clegg
10:00 pm Arvados Bug #6019 (Closed): [FUSE] `ls` in a large home directory (4400+ items) takes too long
Tom Clegg
10:00 pm Arvados Bug #6025 (Closed): [SSO] Authenticate users with OAuth2
Tom Clegg
10:00 pm Arvados Story #5098 (Closed): [Keep] Design: using Arvados to browse, and run jobs on, input data that is...
Tom Clegg
10:00 pm Arvados Feature #5414 (Closed): [Keep] Use data stored remotely (not in local keepstore)
Tom Clegg
06:39 pm Arvados Story #4443 (Closed): [DRAFT] [Workbench] Tour Guide Dax follows you around for a while, showing ...
Tom Clegg
06:38 pm Arvados Story #4284 (Closed): [Crunch] Add unit and functional tests to crunchstat test suite
Tom Clegg
06:38 pm Arvados Bug #4123 (Closed): [API] move Job script/repository/docker validations from controller to model
Tom Clegg
06:36 pm Arvados Bug #4120 (Closed): [Workbench] [obsolete?] Pipeline state can be failed even though every job is...
Tom Clegg
06:35 pm Arvados Feature #3803 (Closed): [SDKs] Add a rate limiting feature to arv-put and arv-get (like rsync's b...
Tom Clegg
06:34 pm Arvados Story #3696 (Closed): [Workbench] Improve log viewer to more effectively expose relevant informat...
Tom Clegg
06:34 pm Arvados Feature #3987 (Closed): [Docker] Run a 5-second diagnostic suite after bringing up docker contain...
Tom Clegg
06:34 pm Arvados Bug #3910 (Closed): [SDKs] Client libraries should retry failed API requests after errors like Ga...
Tom Clegg
06:34 pm Arvados Feature #3955 (Closed): [Workbench] Render runtime-formatted timestamps as "8:00 am" or "Sep 23" ...
Tom Clegg
06:33 pm Arvados Feature #3923 (Closed): [Workbench] Use web notifications to alert user when pipelines finish
Tom Clegg
06:32 pm Arvados Story #3511 (Closed): [Documentation] Present an efficient pattern for developing a pipeline temp...
Tom Clegg
06:31 pm Arvados Story #3701 (Closed): [Workbench] Show current user's compute node usage over time
Tom Clegg
06:31 pm Arvados Story #3446 (Closed): [Workbench] Sort by name before uuid in search results.
Tom Clegg
06:30 pm Arvados Bug #3352 (Closed): [SDK] After ^C, arv-put sometimes exits but leaves processes/threads behind t...
Tom Clegg
06:30 pm Arvados Bug #3134 (Closed): [Workbench] Deleting a job in a project doesn't show delete wipe animation, i...
Tom Clegg
06:30 pm Arvados Bug #3166 (Closed): [Workbench] Generic "attributes" tab renders uuid as a link to the object, wi...
Tom Clegg
06:30 pm Arvados Bug #3483 (Closed): [Keep] Keep uses appropriate HTTP status codes for failed DELETE requests
Tom Clegg
06:29 pm Arvados Bug #3262 (Closed): [Documentation] Describe how to run tramp in emacs.
Tom Clegg

03/09/2017

09:22 pm Arvados Bug #4344 (Resolved): [SDK] can't make an anonymous token from the cli
This works, but you have to say @"owner_uuid":"X"@, not @"user":"X"@. Tom Clegg
09:17 pm Arvados Bug #3999 (Closed): [Workbench] Pipeline timing number oddities
Tom Clegg
09:17 pm Arvados Bug #3855 (Closed): [SDK] New keep mount is very slow on initial usage
Tom Clegg
09:17 pm Arvados Bug #4132 (Closed): [Workbench] Add unit test infrastructure (and some examples) for JavaScript a...
Tom Clegg
09:16 pm Arvados Feature #3333 (Closed): [DRAFT] [Docker] Make local Docker install more useful for development
Tom Clegg
09:16 pm Arvados Bug #3847 (Resolved): [SDK] --project-uuid switch behaves strangely without = (seen in arv-keepdo...
Seems to have been fixed, although I don't see where/when.... Tom Clegg
08:28 pm Arvados Revision 605d11f2 (arvados): Merge branch '5036-arv-mount-type'
refs #5036 Tom Clegg
07:56 pm Arvados Bug #5036 (In Progress): [FUSE] Provide a unique mount subtype
Tom Clegg
07:56 pm Arvados Bug #5036: [FUSE] Provide a unique mount subtype
5036-arv-mount-type @ commit:fa8c5cdb36fa89af92087d6465b207d65ffc9618 adds a "--subtype FOO" arg, which makes the fil... Tom Clegg
07:53 pm Arvados Revision fa8c5cdb (arvados): 5036: Add "--subtype foo" flag to set mounted filesystem type to "fu...
Tom Clegg
03:48 pm Arvados Bug #11226: [SDK] Python client tries to cache discovery document at the same local path regardle...
Suggest we
# pass cache_discovery=False to google-api-client so we get back to what we thought we were doing, i.e., ...
Tom Clegg

03/08/2017

04:18 pm Arvados Bug #4748 (Closed): [API] Explain why there are 21 jobs with log = ""
Tom Clegg
04:17 pm Arvados Bug #4451 (Closed): [Workbench] Fiddlesticks when rerunning pipeline with edited template
Tom Clegg
04:16 pm Arvados Bug #4416 (Closed): [Workbench] "Manage" permission should be sufficient to edit project's name/d...
Tom Clegg
04:15 pm Arvados Bug #4440 (Closed): [Documentation] "Writing a Crunch script" is mysterious
Tom Clegg
04:15 pm Arvados Bug #4529 (Closed): [SDKs] Ensure Python and Go SDKs in a given situation always choose either pr...
Tom Clegg
04:14 pm Arvados Bug #4488 (Closed): [SDKs] arvados.keep.KeepClient.KeepService should know whether it is a proxy
Tom Clegg
04:12 pm Arvados Bug #4335 (Closed): [Crunch] Improve node startup speed. 15 min is too slow.
Tom Clegg
04:11 pm Arvados Bug #4218 (Closed): [Keep] Improve speed of /index requests
Tom Clegg
04:11 pm Arvados Story #4238 (Closed): [Documentation] Show how to run a pipeline instance without using a template.
Tom Clegg
04:10 pm Arvados Story #4305 (Closed): [Node Manager] Investigate and try to address RAM use
Tom Clegg
04:10 pm Arvados Bug #3688 (Closed): [API] User should be able to delete a project even though dependent logs exist
Tom Clegg
04:09 pm Arvados Story #3820 (Closed): [Crunch] Jobs need code from multiple git repositories
Tom Clegg
04:09 pm Arvados Feature #4131 (Closed): [Workbench] Support incremental loading of long job logs, using Range req...
Tom Clegg
04:09 pm Arvados Feature #4090 (Closed): Add code coverage report for workbench javascript
Tom Clegg
04:08 pm Arvados Story #4424 (Closed): [Keep] [DRAFT] Use stronger hash algorithm (such as SHA-2) for Keep blocks
Moved to [[Changing Keep hash algorithm]] Tom Clegg
04:08 pm Arvados Wiki edit: Changing_Keep_hash_algorithm (#2)
Tom Clegg
03:57 pm Arvados Wiki edit: Changing_Keep_hash_algorithm (#1)
Tom Clegg
03:53 pm Arvados Wiki edit: Proposals (#53)
Tom Clegg

03/07/2017

10:23 pm Arvados Story #3812 (Closed): [SDKs] Clients should warn when they're expecting a newer API server
Tom Clegg
10:22 pm Arvados Bug #4756 (Rejected): [API] Add migration to change collection uuids to portable_data_hash in old...
Tom Clegg
10:18 pm Arvados Revision 5eae182a (arvados): 3853: Do not try to look up collections by PDH for update/delete act...
Tom Clegg
10:03 pm Arvados Bug #3954 (Closed): [API] permission_to_update should prohibit all changes to content-addressable...
Tom Clegg
10:03 pm Arvados Bug #4003 (Closed): [Crunch] crunch-dispatch errors that relate to a specific job should be avail...
Tom Clegg
10:03 pm Arvados Bug #3831 (Closed): [Crunch] inconsistent find_or_create behavior in pipeline runner vs job
Tom Clegg
10:02 pm Arvados Bug #3556 (Closed): [Documentation] Avoid using the term "parallel" when referring to Crunch conc...
Tom Clegg
10:01 pm Arvados Feature #3768 (Closed): [SDKs] Speculatively pre-fetch data blocks while reading from a collection.
Tom Clegg
10:01 pm Arvados Bug #3789 (Closed): [API] Pipeline template (temporarily?) disappears after move to project
Tom Clegg
10:00 pm Arvados Bug #3810 (Closed): [Crunch] run-command should bail when $(file) expands to a nonexistent path
Tom Clegg
09:59 pm Arvados Story #3795 (Closed): [Crunch/SDKs] Tasks need more retry support
Tom Clegg
09:59 pm Arvados Feature #3633 (Closed): [Workbench] Compare pipeline templates, similar to the existing pipeline ...
Tom Clegg
09:59 pm Arvados Story #3347 (Closed): [Crunch] Run dev (and real) jobs using a syntax as close as possible to "py...
Tom Clegg
09:58 pm Arvados Bug #3470 (Closed): [Workbench] Ajax updated pages don't interact correctly with browser history.
Tom Clegg
09:58 pm Arvados Story #3157 (Closed): [Workbench] Admin button to download user list as CSV
Tom Clegg
09:57 pm Arvados Bug #3418 (Rejected): [Crunch] crunch-job ignores task sequence
Tom Clegg
09:57 pm Arvados Feature #3329 (Closed): [Docker] Loose ends in local docker install
Tom Clegg
09:56 pm Arvados Bug #3386 (Closed): [Documentation] Crunch script can exit 111 to indicate transient failure (i.e...
Tom Clegg
09:56 pm Arvados Support #2870 (Closed): Write up PGP data curation plan
Tom Clegg
09:56 pm Arvados Bug #3624 (Closed): [API] Remove redundant "updated_at" column
Tom Clegg
09:55 pm Arvados Feature #3095 (Closed): [Workbench] Add "run" button to each pipeline template in project view
Tom Clegg
09:43 pm Arvados Bug #10669: googleapiclient.errors.InvalidJsonError retrieving discovery document
Discovery doc retrieval is done by google-api-client. We ask it to use httplib2's built-in cache mechanism.
Unfort...
Tom Clegg
09:16 pm Arvados Bug #11221: [systemd] Always restart exited services, even after 3 startup failures
11221-always-restart-services @ commit:273a233818ae39e843fab0276f9e381da6645d28 Tom Clegg
09:06 pm Arvados Bug #11221 (Resolved): [systemd] Always restart exited services, even after 3 startup failures
Currently, when postgresql is unavailable for ~300ms when arvados-ws is trying to start (which is normal for a bottin... Tom Clegg
09:15 pm Arvados Revision 273a2338 (arvados): 11221: Always restart systemd services, even after a few startup fai...
Tom Clegg
08:45 pm Arvados Bug #3115 (Resolved): [API] Retrieving a nonexistent keep disk as an admin should not create a ne...
Applied in changeset arvados|commit:533a50bbcee4ece451a65e87066563a5b33ef150. Tom Clegg
08:39 pm Arvados Revision 533a50bb (arvados): Merge branch '3115-keep-disk-create-surprise'
closes #3115 Tom Clegg
08:32 pm Arvados Revision c97408a5 (arvados): 3115: Create keep_disks implicitly only in #ping action, not #show.
Tom Clegg
08:03 pm Arvados Story #3043 (Resolved): [SDKs] Add Go SDK page to doc site with examples similar to Python etc
Tom Clegg
08:03 pm Arvados Story #3045 (Closed): [Workbench] Generalize "re-run this job" button
Tom Clegg
08:02 pm Arvados Bug #3097 (Closed): [Crunch] Pass script_parameters on stdin instead of JOB_PARAMETER_* environme...
Tom Clegg
08:02 pm Arvados Story #3017 (Closed): [Workbench] When comparing pipeline instances, show/highlight the differing...
Tom Clegg
08:02 pm Arvados Story #2932 (Closed): Pipeline has default name from pipeline template ("instance of X") and star...
Tom Clegg
08:02 pm Arvados Bug #3159 (Closed): Display of the "Add a new keep disk" button in the keep disks page in incorre...
Tom Clegg
08:00 pm Arvados Story #2661 (Closed): Security audit
Tom Clegg
08:00 pm Arvados Feature #3895 (Closed): set up dev ci server
Tom Clegg
07:59 pm Arvados Story #4599 (Closed): [Crunch] crunch-job should not retry tasks after its SLURM allocation is re...
Tom Clegg
07:53 pm Arvados Feature #6312 (Resolved): [Workbench] Use download service instead of passing Keep data through R...
Tom Clegg
07:28 pm Arvados Feature #6191 (Resolved): Propagate stderr from all srun uses (e.g., clean-tmp and docker-install...
Tom Clegg
07:13 pm Arvados Story #4101 (Rejected): Workbench tests: app/controllers/projects_controller.rb
Tom Clegg
07:13 pm Arvados Story #4100 (Rejected): Workbench tests: app/helpers/application_helper.rb
Tom Clegg
07:12 pm Arvados Story #4099 (Rejected): Workbench tests: app/helpers/pipeline_instances_helper.rb
Tom Clegg
07:12 pm Arvados Story #4098 (Rejected): Workbench tests: app/controllers/pipeline_instances_controller.rb
Tom Clegg
07:12 pm Arvados Story #4097 (Rejected): Workbench tests: app/controllers/users_controller.rb
Tom Clegg
07:12 pm Arvados Story #4096 (Rejected): Workbench tests: app/helpers/provenance_helper.rb
Tom Clegg
07:12 pm Arvados Story #4095 (Rejected): Workbench tests: app/controllers/groups_controller.rb
Tom Clegg
07:12 pm Arvados Story #4094 (Rejected): Workbench tests: app/controllers/jobs_controller.rb
Tom Clegg
07:12 pm Arvados Story #4093 (Rejected): Workbench tests: app/helpers/vcf_pipeline_helper.rb
Tom Clegg
07:12 pm Arvados Story #4092 (Rejected): Workbench tests: app/helpers/jobs_helper.rb
Tom Clegg
07:11 pm Arvados Bug #6086 (Closed): [API] Creating new job sometimes takes too long
Tom Clegg
07:10 pm Arvados Story #6407 (Closed): [Workbench] Collection file trees are collapsible
Tom Clegg
03:57 pm Arvados Bug #11220 (New): [SDKs] Fix misleading arv-mount/pysdk error messages by removing obsolete "fetc...
Current error message:... Tom Clegg
03:54 pm Arvados Bug #9435 (Resolved): keep-balance failing when a small page of collections has the same modified...
In #9998 keep-balance started using count="none" and accommodating multiple pages of identical timestamps, so this sh... Tom Clegg
03:48 pm Arvados Story #6830 (Resolved): [API] [keep-balance] Option to return unsigned manifests from collections...
This was implemented in #9998. Tom Clegg
03:39 pm Arvados Bug #11190: Containers seem to run more than once, which isn't supposed to happen
... Tom Clegg
03:15 pm Arvados Bug #11166 (Resolved): [Crunch2] crunchrun.go should avoid name collisions when creating log coll...
Applied in changeset arvados|commit:a1b0149879593df786807afbefb26d4687ebe161. Tom Clegg
03:01 pm Arvados Bug #11166: [Crunch2] crunchrun.go should avoid name collisions when creating log collections
11166-log-name-collision @ commit:cb0b666f86ce87d35667e8e23e3775b703a37841 Tom Clegg
03:11 pm Arvados Revision a1b01498 (arvados): Merge branch '11166-log-name-collision'
closes #11166 Tom Clegg

03/06/2017

08:58 pm Arvados Bug #11166 (In Progress): [Crunch2] crunchrun.go should avoid name collisions when creating log c...
Tom Clegg
08:24 pm Arvados Task #11212 (Resolved): use ensure_unique_name flag or timestamp
Tom Clegg
03:37 pm Arvados Task #11212 (Resolved): use ensure_unique_name flag or timestamp
Tom Clegg
07:59 pm Arvados Revision cb0b666f (arvados): 11166: Use ensure_unique_name to avoid collisions when saving logs a...
Tom Clegg
03:46 pm Arvados Wiki edit: Upgrading_to_master (#17)
Tom Clegg
03:35 pm Arvados Bug #11138 (Resolved): Protection against running new style docker 1.10+ image on old docker host
Applied in changeset arvados|commit:a4a3227c634d8cfd54a23205be727e910032cbe9. Tom Clegg
03:31 pm Arvados Revision a4a3227c (arvados): Merge branch '11138-docker-load-fail'
closes #11138 Tom Clegg

03/03/2017

10:05 pm Arvados Revision 47c56297 (arvados): Add missing install step: add keep-balance token to keepstore configs.
No issue # Tom Clegg
09:53 pm Arvados Revision 2e5c44a6 (arvados): Remove obsolete GOMAXPROCS advice.
No issue # Tom Clegg
09:50 pm Arvados Bug #11168 (Resolved): [API] Use JSON instead of YAML for serialized fields in database
Applied in changeset arvados|commit:660a6143ecf1e777f33bd84183ba9e821e1d7a8e. Tom Clegg
09:23 pm Arvados Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
commit:07e4083ea451913b988d77e8e4c926da8ad844a4
"Double-decode serialized fields if database was mangled by downgr...
Tom Clegg
05:43 am Arvados Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
> I notice that a couple of places have been switched to use SafeJSON, but they still have @require 'oj'@. The speci... Tom Clegg
09:46 pm Arvados Revision 660a6143 (arvados): Merge branch '11168-serialize-json'
closes #11168 Tom Clegg
09:44 pm Arvados Bug #11138: Protection against running new style docker 1.10+ image on old docker host
Lucas Di Pentima wrote:
> On line 428: Could it be that "docker load" fails for some reason, and the printed message...
Tom Clegg
06:06 am Arvados Bug #11138: Protection against running new style docker 1.10+ image on old docker host
11138-docker-load-fail uses @docker inspect@ (before and after @docker load@) to assure that @docker run aaa123@ will... Tom Clegg
09:41 pm Arvados Revision f7d32ebc (arvados): 11138: Show actual image ID when checking whether docker image is lo...
Tom Clegg
09:18 pm Arvados Revision 07e4083e (arvados): 11168: Double-decode serialized fields if database was mangled by do...
Tom Clegg
06:10 am Arvados Task #11207 (Resolved): Make docker_install_script more robust
Tom Clegg
05:52 am Arvados Revision dd08615e (arvados): 11138: Test for docker image after loading, in case docker-load erro...
Tom Clegg
05:26 am Arvados Revision 594e00f9 (arvados): 11168: Always deep-sort before comparing in where_serialized.
Tom Clegg
05:25 am Arvados Revision 6fe6fb4a (arvados): 11168: Remove unused import.
Tom Clegg
05:17 am Arvados Revision c7a44e53 (arvados): 11168: Revert serialization change in order to avoid breaking job re...
Tom Clegg

03/02/2017

03:48 pm Arvados Bug #11138 (In Progress): Protection against running new style docker 1.10+ image on old docker host
Tom Clegg
03:48 pm Arvados Task #11207 (In Progress): Make docker_install_script more robust
Tom Clegg
03:48 pm Arvados Task #11207 (Resolved): Make docker_install_script more robust
Tom Clegg
06:04 am Arvados Revision e9696655 (arvados): 11183: Add "setup -wait" flag
Tom Clegg

03/01/2017

11:07 pm Arvados Revision 345dfd5a (arvados): 11168: Add missing require.
Tom Clegg
10:59 pm Arvados Revision 766ddd6d (arvados): 11168: Prohibit down-migration to YAML-only codebase.
Tom Clegg
09:24 pm Arvados Bug #11168: [API] Use JSON instead of YAML for serialized fields in database
11168-serialize-json @ commit:d7c84d69bb62d61bc671b2d5e0ad4ed42dbeb7c0 Tom Clegg
09:06 pm Arvados Bug #11168 (In Progress): [API] Use JSON instead of YAML for serialized fields in database
Tom Clegg
09:09 pm Arvados Revision d7c84d69 (arvados): 11168: Change db serialize from YAML to JSON.
Tom Clegg
08:51 pm Arvados Task #11197 (Resolved): Review 11138-docker-load-fail
Tom Clegg
08:51 pm Arvados Task #11196 (Resolved): Review 11166-log-name-collision
Tom Clegg
08:51 pm Arvados Task #11195 (Resolved): Review 11168-serialize-json
Tom Clegg
08:37 pm Arvados Bug #11190 (New): Containers seem to run more than once, which isn't supposed to happen
Example: tb05z-dz642-eie1eal1059y9bb Tom Clegg
08:28 pm Arvados Bug #11166: [Crunch2] crunchrun.go should avoid name collisions when creating log collections
Should be a matter of using the ensure_unique_name flag. Tom Clegg
08:24 pm Arvados Task #11119 (Resolved): Review 7995-keep-balance-docs
Tom Clegg
08:22 pm Arvados Bug #10585 (Resolved): crunch doesn't end jobs when their arv-mount dies
Tom Clegg
08:12 pm Arvados Revision 15b33da8 (arvados): 9216: Require size hints in manifest validation.
Tom Clegg
08:10 pm Arvados Feature #10764 (Resolved): [arvados-ws] Write unit tests
Applied in changeset arvados|commit:2db0c3a44d8f12e3a566c796bdecaa1de150b155. Tom Clegg
02:14 pm Arvados Feature #10764: [arvados-ws] Write unit tests
10764-ws-tests
commit:f36162457a771824059fefa098a3ffb89c59263f
https://ci.curoverse.com/job/developer-run-tests/180/
Tom Clegg
08:09 pm Arvados Bug #9437 (Resolved): keep-balance doesn't seem to be reducing replication level of blocks
Tom Clegg
08:04 pm Arvados Revision 2db0c3a4 (arvados): Merge branch '10764-ws-tests'
closes #10764 Tom Clegg
08:03 pm Arvados Task #11112 (Resolved): unit tests
Tom Clegg
07:28 pm Arvados Bug #8113 (Resolved): "Download the full log" in log tab should show the log in the browser, not ...
Tom Clegg
03:58 pm Arvados Revision a58e9ca2 (arvados): 10764: De-duplicate real/test server startup. Add test for broken co...
Tom Clegg
02:14 pm Arvados Task #11118 (Resolved): Review 10777-die-if-arv-mount-dies
Tom Clegg
07:52 am Arvados Revision f3616245 (arvados): 10764: Simplify test server shutdown.
Tom Clegg
07:36 am Arvados Revision b3fa9983 (arvados): 10764: Test v0 session.
Tom Clegg

02/28/2017

10:20 pm Arvados Bug #10777 (Resolved): [Crunch2] crunch-run: stop the container and fail if arv-mount dies before...
Applied in changeset arvados|commit:a98594f82f1786b09d9d1e0f57a651d4eae3474a. Tom Clegg
10:08 pm Arvados Revision a98594f8 (arvados): Merge branch '10777-die-if-arv-mount-dies'
closes #10777 Tom Clegg
08:54 pm Arvados Feature #11183: [Deployment] Use consul to report health of services/components
work in progress on source:/@11183-setup-agent Tom Clegg
08:41 pm Arvados Feature #11183 (In Progress): [Deployment] Use consul to report health of services/components
Tom Clegg
08:52 pm Arvados Revision c78289e5 (arvados): 11183: Add "arvados-admin setup"
Tom Clegg
06:09 am Arvados Revision 784371c8 (arvados): 10764: Test v0 session. (WIP)
Tom Clegg
02:49 am Arvados Revision aea8596a (arvados): 10764: Permission tests. Support PDH permission check.
Tom Clegg

02/27/2017

10:51 pm Arvados Revision ca812f58 (arvados): 10764: Add unit tests
Tom Clegg
06:23 pm Arvados Task #11112 (In Progress): unit tests
Tom Clegg
04:54 pm Arvados Revision 58ed4ee3 (arvados): 10979: Add missing SqueueChecker initialization.
refs #10979 Tom Clegg
08:59 am Arvados Revision ceb95004 (arvados): more vault
Tom Clegg

02/26/2017

10:12 am Arvados Revision 307e7af0 (arvados): vault
Tom Clegg
12:35 am Arvados Revision a4512ef7 (arvados): Merge branch 'master' into deploy-agent
Conflicts:
build/run-tests.sh
Tom Clegg
12:32 am Arvados Revision cea3337f (arvados): multiple servers, add docker-compose test
Tom Clegg

02/25/2017

10:48 pm Arvados Revision 37972640 (arvados): gitignore
Tom Clegg
07:05 am Arvados Bug #10777: [Crunch2] crunch-run: stop the container and fail if arv-mount dies before the contai...
10777-die-if-arv-mount-dies @ commit:c106221bf8c90fd80ca2c19a578fd7faec985229 Tom Clegg
07:04 am Arvados Task #11169 (Resolved): Detect arv-mount death & error out
Tom Clegg
06:57 am Arvados Revision c106221b (arvados): 10777: Close and flush logs right away instead of waiting for next t...
Tom Clegg
06:57 am Arvados Revision 2ac112e2 (arvados): 10777: Stop container if arv-mount dies before container exits.
Tom Clegg

02/24/2017

10:46 pm Arvados Revision e47bb9c1 (arvados): run-tests.sh exit non-zero if gofmt fails
No issue # Tom Clegg
10:29 pm Arvados Revision 1042f200 (arvados): 10777: Stop container if arv-mount dies before container exits.
Tom Clegg
09:55 pm Arvados Task #11169 (In Progress): Detect arv-mount death & error out
Tom Clegg
09:55 pm Arvados Task #11169 (Resolved): Detect arv-mount death & error out
Tom Clegg
09:46 pm Arvados Revision 17f4a03d (arvados): 10979: Check for orphans only once at startup. Add missing Lock() in
squeue checker. Avoid holding mtx while waiting for API response.
Ensure RunContainer actually gets called in test ca...
Tom Clegg
07:43 pm Arvados Revision 353eb4f4 (arvados): Merge branch '6347-log-timestamps'
closes #6347 Tom Clegg
07:41 pm Arvados Bug #11168 (Resolved): [API] Use JSON instead of YAML for serialized fields in database
Currently hashes (like log properties) and arrays (like api_client_authorization scopes) are encoded in YAML, which i... Tom Clegg
06:45 pm Arvados Revision 00d95f0f (arvados): 6347: Use RFC3339Nano to render timestamps loaded from serialized fi...
Psych (YAML) serializes timestamps as ISO8601-with-space-separators,
and safe_load deserializes them to Time even wit...
Tom Clegg
09:17 am Arvados Revision c7df9c8c (arvados): 6347: Use RFC3339Nano to render timestamps loaded from serialized fi...
Psych (YAML) serializes timestamps as ISO8601-with-space-separators,
and safe_load deserializes them to Time even wit...
Tom Clegg

02/23/2017

10:04 pm Arvados Bug #11071: [Workbench] qr1hi performance tests are broken
Looks like we need to use count=none for Workbench search... Tom Clegg
07:10 pm Arvados Bug #11156 (Resolved): [Crunch2] 9tee4-xvhdp-0m3a4ys52od7dhs was stuck until c-d-s was restarted
Applied in changeset arvados|commit:19dcf1ed20214b8349c76037ba8679db07bf44fc. Tom Clegg
07:09 pm Arvados Bug #11156: [Crunch2] 9tee4-xvhdp-0m3a4ys52od7dhs was stuck until c-d-s was restarted
c-d-s was stuck in an infinite loop due to a bad condition in its "get next page" loop. Just kept repeating the same ... Tom Clegg
04:56 pm Arvados Bug #11156: [Crunch2] 9tee4-xvhdp-0m3a4ys52od7dhs was stuck until c-d-s was restarted
Looks like @9tee4-dz642-2a5iga7lyeg2rog@ was queued the whole time, but c-d-s didn't pick it up until after restart f... Tom Clegg
07:04 pm Arvados Revision 19dcf1ed (arvados): 11156: Fix infinite loop condition.
closes #11156 Tom Clegg

02/22/2017

09:48 pm Arvados Bug #11155 (New): [Docs] Document keepstore S3 volume configuration
Add a "Configure S3 storage" page similar to the "Configure Azure Blob storage" page (http://doc.arvados.org/install/... Tom Clegg
09:48 pm Arvados Bug #11154 (New): [Docs] Migrate install docs from command line flags to YAML configs (keepstore,...
Currently, install docs recommend setting CLI flags, but these are deprecated.
Update install docs to use @/etc/ar...
Tom Clegg
09:45 pm Arvados Story #7995 (Resolved): [Documentation] Document Keep Balance setup in the Install Guide
Applied in changeset arvados|commit:1e6a756a10a1c0a77aeea5041844ba3a572bdd70. Tom Clegg
08:05 pm Arvados Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Javier Bértoli wrote:
> I notice you added these two flags and removed *-never-delete=true*. Is that correct, or jus...
Tom Clegg
04:45 pm Arvados Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Javier Bértoli wrote:
> Keepproxy is optional, as I understand it. If so, can I have more than one keep-balance, ins...
Tom Clegg
04:38 pm Arvados Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Updated the dry run instructions:
To do this, edit your keep-balance startup script to use the flags @-commit-pull...
Tom Clegg
04:18 pm Arvados Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
Tom Morris wrote:
> I made a few copy edits and pushed them to the branch. Please review them to make sure that thin...
Tom Clegg
09:44 pm Arvados Feature #11153 (New): [API] create_token script should allow operator to provide token scopes and...
h3. Background
Currently the install guide relies on the @script/create_superuser_token.rb@ program in the apiserv...
Tom Clegg
09:36 pm Arvados Revision 1e6a756a (arvados): Merge branch '7995-keep-balance-docs'
closes #7995 Tom Clegg
09:33 pm Arvados Revision 2a30eacc (arvados): Remove pidfiles after shutting down test servers.
No issue # Tom Clegg
06:25 pm Arvados Bug #11097 (Resolved): [API] Reuse containers even when multiple matching containers exist with d...
Applied in changeset arvados|commit:0c529ed05805507b4d2c903b9587e9b61cec5ee6. Tom Clegg
06:19 pm Arvados Revision 0c529ed0 (arvados): Merge branch '11097-reuse-impure'
closes #11097 Tom Clegg
06:17 pm Arvados Wiki edit: Node_Manager (#4)
Tom Clegg
04:45 pm Arvados Revision d8f1b7e8 (arvados): 7995: Add note about one keep-balance process at a time.
Tom Clegg
04:29 pm Arvados Revision 11f5e94c (arvados): 7995: Fix up dry-run instructions.
Tom Clegg
04:24 pm Arvados Revision ced8b355 (arvados): 7995: Fix up inconsistent "e.g." vs. "e.g.,".
Tom Clegg
04:02 pm Arvados Revision d2c9ca97 (arvados): 7995: Copy edits.
Tom Clegg
03:29 pm Arvados Revision 423d27d0 (arvados): Fix dispatch panic when processing an update after tracker has been ...
refs #11151 Tom Clegg

02/21/2017

09:53 pm Arvados Bug #10729: [Crunch2] Propagate error messages if sbatch command succeeds but crunch-run can't ru...
Thoughts about how to retrieve the error messages:
* If the cluster is set up with shared home directories (which is...
Tom Clegg
08:36 pm Arvados Bug #11097: [API] Reuse containers even when multiple matching containers exist with differing ou...
Radhika Chippada wrote:
> * I think moving “select_readable_pdh” to the line above the declaration of “candidates” a...
Tom Clegg
08:27 pm Arvados Revision 802af81e (arvados): 11097: Clarify reuse query.
Tom Clegg
08:27 pm Arvados Revision 1732df24 (arvados): 11097: Update docs to reflect new container reuse behavior.
Tom Clegg
07:56 pm Arvados Bug #11148 (New): [Crunch2] Propagate dispatch error messages (e.g., sbatch fails) to user via lo...
h3. Background
If crunch-dispatch-slurm encounters an error while trying to submit a container to the slurm queue,...
Tom Clegg
07:00 pm Arvados Feature #11146 (New): [Crunch2] [Workbench] Show slurm queue position of containers submitted to ...
h3. Background
From the user's perspective, it's hard to see what (if anything) is happening between the time a co...
Tom Clegg
06:56 pm Arvados Bug #10008 (New): [SDKs] [Python] When reading data through Collection et al., signatures should ...
Tom Clegg
04:29 pm Arvados Feature #11015: Improve throughput of crunch-run output-uploading stage using multi-threaded tran...
Possible implementation
In (*CollectionFileWriter)goUpload(), currently we have...
Tom Clegg

02/20/2017

10:04 pm Arvados Story #7995: [Documentation] Document Keep Balance setup in the Install Guide
7995-keep-balance-docs @ commit:14304c7af0b0dd7bc9345b6c5aeb61a3bdc1d3b0 Tom Clegg
10:03 pm Arvados Task #11134 (Resolved): Explain limitations re shared volumes and service_type
Tom Clegg
08:50 pm Arvados Task #11134 (In Progress): Explain limitations re shared volumes and service_type
Tom Clegg
10:03 pm Arvados Revision 14304c7a (arvados): 7995: Add "dry run" note.
Tom Clegg
08:49 pm Arvados Task #11140 (Resolved): Update tests
Tom Clegg
08:34 pm Arvados Task #11140 (In Progress): Update tests
Tom Clegg
08:34 pm Arvados Task #11140 (Resolved): Update tests
Tom Clegg
08:46 pm Arvados Bug #11097: [API] Reuse containers even when multiple matching containers exist with differing ou...
commit:4f0e07d462b7860bb10686c27fac16970220377f with updated test case. Tom Clegg
08:45 pm Arvados Revision 4f0e07d4 (arvados): 11097: Update test to match new behavior.
Tom Clegg
08:34 pm Arvados Revision 4cd89bd1 (arvados): 11097: Merge branch 'master' into 11097-reuse-impure
Tom Clegg
04:45 pm Arvados Task #11133 (Resolved): Set -enable-trash in keepstore docs
Tom Clegg
03:12 pm Arvados Task #11133 (In Progress): Set -enable-trash in keepstore docs
Tom Clegg
03:41 pm Arvados Revision 84bd6883 (arvados): 7995: Add keep-balance to install guide.
Tom Clegg
03:00 pm Arvados Wiki edit: Upgrading_to_master (#16)
Tom Clegg
09:28 am Arvados Revision fc33deae (arvados): cmd, lib
Tom Clegg
 

Also available in: Atom