Activity

From 06/27/2017 to 07/26/2017

07/25/2017

07:49 pm Story #4442 (Resolved): [API] [Workbench] Site-wide regex searching
Full text search meets the bulk of this need. Tom Morris
07:14 pm Revision a7a07723 (arvados): Merge branch '12025-perm-cache-version'
refs #12025
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
07:06 pm Revision c55c3953 (arvados): 12027: Exit "retry unlocked" if slurm fails during setup phases.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
06:51 pm Story #12032: Allow projects to be deleted (ie placed in the trash can)
Implementation thoughts:
* Add is_trashed flag to groups (projects)
* Permission graph (recursive view) needs an is...
Tom Clegg
06:39 pm Story #12032 (New): Allow projects to be deleted (ie placed in the trash can)
As a user, I would like deleted projects to be placed in the Trash rather than cluttering my Home project and when th... Tom Morris
05:55 pm Revision b40d8d04 (arvados): 12027: Exit "retry unlocked" if slurm fails during setup phases.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:39 pm Revision 86b16fe1 (arvados): 12025: Change permission cache key to avoid upgrade bugs.
Also, set TTL on permission cache entries.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
02:22 pm Feature #12028 (New): Be able to create a new collection based on a subdirectory in workbench
Right now you can only click on files to create a new collection, users should also be able to click on directories. Bryan Cosca

07/24/2017

09:24 pm Bug #12026 (New): crunchstat-summary gives invalid min_cores recommendation
This recommendation doesn't make sense from a couple of points of view:
### Summary for cwl-runner (su92l-8i9sb-ll...
Tom Morris
09:02 pm Bug #11850: [CWL] arv:RunInSingleContainer should take max() of ResourceRequirements of substeps
Taking into account https://dev.arvados.org/issues/11850#note-10, I think the rule should be rephrased as follows.
...
Jiayong Li
07:02 pm Revision f49e418e (arvados): refs #11906
Merge branch '11906-keepproxy-ping'
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Radhika Chippada
06:58 pm Task #12021 (Resolved): Review branch 11906-keepproxy-ping
Lucas Di Pentima
06:16 pm Task #12021 (Resolved): Review branch 11906-keepproxy-ping
Radhika Chippada
06:57 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
Updates at commit:91f976b99d4e7f00d2c1fbfee75812c3b1b780c6 - branch @11906-keepproxy-ping@ LGTM, thanks! Lucas Di Pentima
06:33 pm Revision 741450ac (arvados): Fix unclosed P tag.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
06:32 pm Revision 5ad5450b (arvados): Update workbench dependencies.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
06:14 pm Revision 91f976b9 (arvados): 11906: add /_health/ping to keepproxy
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
02:53 pm Bug #12020 (New): [Tests] Flaky test in Python SDK tests.test_events.WebsocketTest.test_subscribe...
... Tom Clegg
02:33 pm Revision e2807c62 (arvados): Update api server dependencies.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:11 pm Story #11167: [Workbench] Remove arv-get file download fallback
I didn't spend much time looking at the updates, but here are a couple initial comments.
* collections_controller ...
Radhika Chippada
02:10 pm Task #11937 (In Progress): Review 11167-wb-remove-arvget
Radhika Chippada

07/21/2017

10:13 pm Bug #4409 (New): [Workbench] Want to remove projects from Home
Re-opening. The current behavior is just plain silly. Tom Morris
09:09 pm Feature #12018 (New): Synchronize group membership with external data source
As a user in a corporate environment, I want to be able to synchronize the users in my Arvados groups with my corpora... Tom Morris
08:56 pm Feature #12017 (New): Windows file system driver for Keep (arv-mount equiv)
As a user who wants to be able to run mass spectrometry software on data stored in Keep, I'd like to have a Windows f... Tom Morris
08:45 pm Story #9264 (Rejected): [Keep] Cross-compile Keepstore for Windows
Since we've already got an Azure blob storage solution, I don't see the need for this. Tom Morris
08:12 pm Revision 2354f19a (arvados): Add missing dependencies to vendor.json.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
08:06 pm Revision 2aaf62d7 (arvados): Update Go dependencies.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
07:14 pm Task #12000 (Resolved): Review 11906-health-check-lib
Tom Clegg
01:31 pm Task #12000 (In Progress): Review 11906-health-check-lib
Radhika Chippada
07:14 pm Task #11963 (Resolved): Review 11948-cwl-conformance-fix
Tom Clegg
06:17 pm Story #11167: [Workbench] Remove arv-get file download fallback
Fixed a failing integration test at commit:8f3070e16
Test run: https://ci.curoverse.com/job/developer-run-tests/393/
Lucas Di Pentima
04:57 pm Story #11167: [Workbench] Remove arv-get file download fallback
Some new updates at commit:c7187ec72
Test run: https://ci.curoverse.com/job/developer-run-tests/392/
* Removed ou...
Lucas Di Pentima
03:20 pm Story #11167: [Workbench] Remove arv-get file download fallback
Updates at commit:cd698336f - branch @11167-wb-remove-arvget@
Test run: https://ci.curoverse.com/job/developer-run-t...
Lucas Di Pentima
06:15 pm Revision 8f3070e1 (arvados): 11167: Fixed integration test about collection's sharing link.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima
06:01 pm Revision 3574793a (arvados): Merge branch '11906-health-check-lib'
refs #11906
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
06:01 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
11906-health-check-lib LGTM. Thanks. Radhika Chippada
05:54 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
Radhika Chippada wrote:
> * In sdk handler.go, can we can call “type Func” as “type HealthFunc” or something to help...
Tom Clegg
05:17 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
Branch 11906-health-check-lib:
* In sdk handler.go, can we can call “type Func” as “type HealthFunc” or something ...
Radhika Chippada
01:00 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
11906-health-check-lib @ commit:0e0dc3c8ebf442a35c41816fed42fdddb53aed53
Tom Clegg
05:53 pm Revision da037c13 (arvados): 11906: Clarify "zero value" test.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
05:51 pm Revision fc2705f7 (arvados): 11906: Test 404 cases.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
05:49 pm Revision c834c346 (arvados): 1906: Fix up doc comment.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
04:54 pm Revision c7187ec7 (arvados): 11167: Test adjusted.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima
03:25 pm Revision 058e06eb (arvados): 11167: Removed outdated comment.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima
03:15 pm Revision cd698336 (arvados): 11167: Refuse to start when keep-web isn't configured.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima
01:01 pm Task #11996 (Resolved): Refactor health check handlers into an SDK library
Tom Clegg

07/20/2017

09:25 pm Revision 2835665e (arvados): 11906: Use sdk/go/health for keepstore health checks.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
09:25 pm Revision 34c4b300 (arvados): 11906: Refactor health-check handler into SDK.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
09:25 pm Revision 0e0dc3c8 (arvados): 11906: Remove keepstore's health-check unit tests.
Now that the health check handler has its own unit tests, keepstore
only needs to check that it's wired up correctly....
Tom Clegg
09:25 pm Revision a2fc189f (arvados): 11906: Add tests for sdk/go/health.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
08:23 pm Revision 4feae207 (arvados): Style fixes.
No issue #
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
08:20 pm Revision 49b9c5f4 (arvados): 10805: Bump debian version in fuse install doc.
refs #10805
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
06:04 pm Bug #12010 (New): [keep-web] tests hang
From https://ci.curoverse.com/job/run-tests-remainder/1152/console... Tom Clegg
05:16 pm Feature #12009 (New): Find arvados-cwl-runner version easier in a container request
I found the arvados/jobs image commit in the api_response of a job and then i ran some shell script to find all the a... Bryan Cosca
05:10 pm Story #11995 (Resolved): [Workbench] Support more filetypes for in-browser display
Applied in changeset arvados|commit:0b51ce6bfc3c50155de18eb4440021688462fb6a. Radhika Chippada
03:34 pm Story #11995: [Workbench] Support more filetypes for in-browser display
Including ... Tom Clegg
02:20 pm Story #11995: [Workbench] Support more filetypes for in-browser display
Just one comment:
* For the @.cwl@ extension, it only works when it’s in lower case. I tried renaming a @*.bed@ to...
Lucas Di Pentima
01:32 pm Story #11995: [Workbench] Support more filetypes for in-browser display
Branch 11995-collecion-filetypes @ commit:be0830dcdc10ff352d89a20fa2a24000a5ca8984
* The file types .yaml, .csv, ....
Radhika Chippada
12:36 pm Story #11995 (In Progress): [Workbench] Support more filetypes for in-browser display
Radhika Chippada
05:04 pm Revision 0b51ce6b (arvados): closes #11995
Merge branch '11995-collecion-filetypes'
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Radhika Chippada
05:04 pm Revision 236a6679 (arvados): 1195: add .bed and .cwl to file types that can previewed
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
03:58 pm Revision 894c4a57 (arvados): 11995: check for cwl file type
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
03:42 pm Revision e7e1e4ff (arvados): 11898: Remove superfluous uniq(&:id) call
Avoids adding DISTINCT to SQL query, which causes Postgres to do
expensive things.
Arvados-DCO-1.1-Signed-off-by: To...
Tom Clegg
02:52 pm Revision 6cc4c2ed (arvados): 10805: Build llfuse-1.2 for ubuntu1604.
refs #10805
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
02:30 pm Revision 40e2b5df (arvados): 1195: case insensitive cwl file extension check
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
01:50 pm Bug #10805 (Resolved): [FUSE] Upgrade llfuse to 1.2, fix deadlock in test suite
Applied in changeset arvados|commit:5e5bc2b2153a0af624d75a3c348b996ddf458ef2. Tom Clegg
01:45 pm Revision 5e5bc2b2 (arvados): Merge branch '10805-avoid-llfuse-deadlock'
closes #10805
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
01:36 pm Task #11998 (In Progress): Review branch 11995-collecion-filetypes
Lucas Di Pentima
01:20 pm Revision be0830dc (arvados): 1195: add .bed and .cwl to file types that can previewed
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
12:30 pm Revision 07cd679b (arvados): 10805: Skip llfuse.close() to avoid llfuse deadlocks in test suite.
See https://bitbucket.org/nikratio/python-llfuse/issues/108
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse....
Tom Clegg

07/19/2017

09:21 pm Revision f291a43f (arvados): 11167: Removed arv-get calling code from show_file.
Adjusted/deleted related tests.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com>
Lucas Di Pentima
08:53 pm Revision ec20efc4 (arvados): Merge branch '10805-llfuse-1.2'
refs #10805
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
08:50 pm Revision 7e7ada63 (arvados): 10805: Fix crash during shutdown when running with llfuse 1.x.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
08:16 pm Bug #10805: [FUSE] Upgrade llfuse to 1.2, fix deadlock in test suite
10805-llfuse-1.2 @ commit:7e7ada63bca240416584871398076c1bafc90f76
* Without this change, I see stack traces all ove...
Tom Clegg
08:15 pm Revision 861786a8 (arvados): 10805: Fix crash during shutdown when running with llfuse 1.x.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
07:52 pm Task #12006 (New): Review
Lucas Di Pentima
07:52 pm Task #12005 (New): Review
Tom Morris
07:51 pm Task #12003 (New): Review
Tom Morris
07:50 pm Task #12001 (New): Review
Tom Morris
07:50 pm Bug #11850: [CWL] arv:RunInSingleContainer should take max() of ResourceRequirements of substeps
After much deliberation, I think https://dev.arvados.org/issues/11850#note-6 needs to be reconsidered. I think it's r... Jiayong Li
07:42 pm Bug #11850: [CWL] arv:RunInSingleContainer should take max() of ResourceRequirements of substeps
(4:06:18 PM) tom: jiayong: the cwl code base has a few instances of this test: "$(" in sf or "${" in sf
(4:06:38 PM)...
Jiayong Li
07:50 pm Task #12000 (Resolved): Review 11906-health-check-lib
Tom Clegg
07:44 pm Bug #11948 (Resolved): Fix failing CWL conformance tests
Tom Clegg
07:42 pm Task #11998 (In Progress): Review branch 11995-collecion-filetypes
Radhika Chippada
07:40 pm Story #11997 (New): [Spike] Investigate CI packaging improvements
Tom Morris
07:32 pm Task #11996 (Resolved): Refactor health check handlers into an SDK library
Tom Clegg
07:30 pm Story #9502 (Resolved): [API] Update permissions cache as needed after select writes
Obsolete/unnecessary now that we use a recursive query for permission checks. Tom Clegg
07:03 pm Story #11857 (Rejected): [CWL] arvados-cwl-runner defaults to crunch2 if available
Radhika Chippada
07:00 pm Story #11807 (Resolved): [API] Migrate old serialized database content from YAML to JSON
Tom Clegg
06:58 pm Story #11908: Migrate Collections.properties to JSONB
Merge is blocked on PostgreSQL 9.4 dependency. Tom Clegg
06:58 pm Story #11995 (Resolved): [Workbench] Support more filetypes for in-browser display
We should offer the option for in-browser display for more filetypes including: .cwl, .yaml, .csv, .tsv, .vcf, and .b... Tom Morris
06:57 pm Bug #11994 (New): [arv-mount] Do not crash if /sys/fs/fuse/connections is empty
In debian 9 (stretch), due to a systemd-blames-kernel bug, /sys/fs/fuse/connections/ might be empty.
* https://git...
Tom Clegg
06:45 pm Bug #11519 (New): arv-get should abort on ctrl/C
Lucas Di Pentima
04:52 pm Revision ff56222d (arvados): Merge branch '11842-crunchstat-summary-dygraphs'
refs #11842
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
02:10 pm Revision 3f7bde60 (arvados): refs #11906
Merge branch '11906-wb-ping'
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Radhika Chippada
02:10 pm Revision ed27ef88 (arvados): 11906: healthcheck ping for workbench
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
02:09 pm Revision a1767317 (arvados): 11906: indentation
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
01:43 pm Revision b210f531 (arvados): 11842: Remove chartjs.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
01:07 pm Task #11987 (Resolved): Review branch 11906-wb-ping
Lucas Di Pentima
12:41 pm Task #11987 (In Progress): Review branch 11906-wb-ping
Lucas Di Pentima
01:07 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
Updates at commit:ffc117651144b197c9d650039a5403e972d54b7c LGTM, just one tiny detail:
Line 20 at file @apps/workb...
Lucas Di Pentima
02:39 am Revision a143375d (arvados): 11842: Use CDNJS for dygraphs js and css.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:39 am Revision 10138c15 (arvados): 11842: Split generic webchart code from ChartJS to WebChart.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:39 am Revision 85d5f916 (arvados): 11842: Generate charts with dygraphs.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg

07/18/2017

10:02 pm Task #11987 (Resolved): Review branch 11906-wb-ping
Radhika Chippada
10:02 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
Branch 11906-wb-ping @ commit:ffc117651144b197c9d650039a5403e972d54b7c
Adds _health/ping to workbench.
I also m...
Radhika Chippada
06:00 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
All @services/api@ tests pass now, this LGTM. Thanks! Lucas Di Pentima
05:51 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
> File services/api/app/controllers/arvados/v1/healthcheck_controller.rb Lines 25-29
Updated
> There’s a test ...
Radhika Chippada
05:16 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
* File @services/api/app/controllers/arvados/v1/healthcheck_controller.rb@
** Lines 25-29: We could save one level o...
Lucas Di Pentima
03:26 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
11906-api-ping @ commit:943b3e807126ba5dc936ceeeddbee9801dab7308
* Implemented /_health/ping for API server and ad...
Radhika Chippada
09:40 pm Revision ffc11765 (arvados): 11906: healthcheck ping for workbench
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
08:48 pm Bug #11986 (New): [OPs] Jenkins test run ordering issue
The jenkins test run https://ci.curoverse.com/job/run-tests/1142/ failed due to some issue @ commit:e65fffb60a5697ce1... Radhika Chippada
07:27 pm Revision 818daa27 (arvados): refs #11906
Merge branch '11906-api-ping'
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Radhika Chippada
07:18 pm Revision da94fc33 (arvados): 11906: healthcheck ping json updated
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
07:08 pm Revision 2397c4a4 (arvados): Merge branch '11980-debian9'
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nicolas Cesar <nico@curoverse.com>
Nico César
06:34 pm Revision 4442b72c (arvados): 11980: debian9 packaging plus llfuse >= 1.2
Debian Stretch is not stable so we'll be packaging for it,
llfuse >= 1.2 is required for this distro but also we nee...
Nico César
06:29 pm Story #11740: Support display & reseting of expiration dates for collections
The x-editable library we use has a "datetime" picker option: https://vitalets.github.io/x-editable/docs.html Tom Clegg
06:28 pm Story #11983 (New): On the collections page indicate expiring collections
Include a visual indication that collections are expiring and allow the user to click through to find out when the ex... Tom Morris
06:07 pm Story #11884 (Duplicate): Convert Collection properties column type to JSONB
Tom Morris
06:00 pm Task #11981 (Resolved): Review branch 11906-api-ping
Lucas Di Pentima
04:51 pm Task #11981 (In Progress): Review branch 11906-api-ping
Lucas Di Pentima
03:22 pm Task #11981 (Resolved): Review branch 11906-api-ping
Radhika Chippada
05:48 pm Revision e65fffb6 (arvados): refs #11906
Merge branch '11906-api-ping'
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Radhika Chippada
05:36 pm Revision 87ccfac0 (arvados-dev): Merge branch 'master' of git.curoverse.com:arvados-dev
Git sync user
05:34 pm Revision 83026883 (arvados-dev): 11980: debian9 is part of the suite
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curoverse.com>
Nico César
05:30 pm Revision b9411e85 (arvados): 11906: add _health/.* to errors_test routes
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
05:30 pm Revision 4d8e3950 (arvados-dev): Merge branch 'master' of git.curoverse.com:arvados-dev
Git sync user
05:29 pm Revision 1cff8f41 (arvados-dev): Merge branch '11332-arv-cwl-integration-testing'
refs #11332
Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curoverse.com>
Nico César
05:26 pm Revision f9f4786c (arvados-dev): Merge branch 'master' of git.curoverse.com:arvados-dev
Git sync user
05:24 pm Revision c8f73733 (arvados-dev): 11980: debian9 as a new target
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nico Cesar <nico@curoverse.com>
Nico César
05:11 pm Revision 5eb51309 (arvados): 11980: solving python-llfuse >1.2 dependency update
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nicolas Cesar <nico@curoverse.com>
Nico César
04:52 pm Revision add4fc98 (arvados): 11980: solving python-llfuse (< 0.41.2) but 1.2+dfsg-1 is to be inst...
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nicolas Cesar <nico@curoverse.com>
Nico César
04:14 pm Revision 5bcff13d (arvados): 11980:test-package-debian9.sh added
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nicolas Cesar <nico@curoverse.com>
Nico César
03:45 pm Revision 3622d1d6 (arvados): 11980: in debian9 procps is not there by default
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nicolas Cesar <nico@curoverse.com>
Nico César
03:27 pm Revision a70e4626 (arvados): 11980: in debian9 gpg requires a dirmgr running to import keys from ...
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nicolas Cesar <nico@curoverse.com>
Nico César
03:21 pm Revision 943b3e80 (arvados): 11906: added api ping tests
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
02:59 pm Revision 869335ec (arvados): 11980: gpg doesn't come by default anymore
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nicolas Cesar <nico@curoverse.com>
Nico César
02:54 pm Revision 32a733b2 (arvados): 11980: typo
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nicolas Cesar <nico@curoverse.com>
Nico César
02:51 pm Revision faea1310 (arvados): 11980: debian stretch and libperl license change
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nicolas Cesar <nico@curoverse.com>
Nico César
02:05 pm Task #11978 (Resolved): Review branch 11906-keepstore-ping
Radhika Chippada
01:22 pm Revision 5b4d924c (arvados): 11980: initial commit to test out debian9 packaging
refs #11980
Arvados-DCO-1.1-Signed-off-by: Nicolas Cesar <nico@curoverse.com>
Nico César

07/17/2017

08:47 pm Revision 5b7be380 (arvados): refs #11906
Merge branch '11906-keepstore-ping'
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Radhika Chippada
08:47 pm Revision 10e2711b (arvados): 11906: pass a func() to healthCheckDo from HealthCheckPingHandler
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
08:47 pm Revision fd50115d (arvados): 11906: keepstore ping
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
08:45 pm Revision 3bbe6cfe (arvados): 11906: add _health/ping to api server
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
08:25 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
This LGTM, thanks! Lucas Di Pentima
08:20 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
@ commit:e6b3f6f6
> Lines 643-646 of handlers.go: Shouldn’t the response be inside an “else” clause? Go docs say t...
Radhika Chippada
07:24 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
Reviewing branch @11906-keepstore-ping@ - commit:499e303783a9b5430147ac4eef3bd13ff72968b9
Local test run @services...
Lucas Di Pentima
08:16 pm Revision e6b3f6f6 (arvados): 11906: return error if errors during marshalling json
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
06:35 pm Task #11978 (In Progress): Review branch 11906-keepstore-ping
Lucas Di Pentima
06:29 pm Task #11978 (Resolved): Review branch 11906-keepstore-ping
Radhika Chippada
06:32 pm Task #11939 (Resolved): Review 5652-arvputget-silent-mode
Radhika Chippada
03:58 pm Task #11939 (In Progress): Review 5652-arvputget-silent-mode
Radhika Chippada
06:31 pm Task #11936 (Resolved): Review 11682-delete-container-request
Radhika Chippada
06:25 pm Revision 499e3037 (arvados): 11906: pass a func() to healthCheckDo from HealthCheckPingHandler
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
05:55 pm Bug #5652 (Resolved): [SDKs] There is no way to silence arv-put and arv-get
Applied in changeset arvados|commit:247d202b57d7c65e4192e0667446d533e552bed7. Lucas Di Pentima
05:20 pm Bug #5652: [SDKs] There is no way to silence arv-put and arv-get
All suggestions addressed at commit:18f6483ff
Also merged _master_, so the new test run is here: https://ci.curovers...
Lucas Di Pentima
04:00 pm Bug #5652: [SDKs] There is no way to silence arv-put and arv-get
* "Do not produce any output unless an error happens." => "Do not print any debug messages to console. (Any error me... Radhika Chippada
05:50 pm Revision 247d202b (arvados): Merge branch '5652-arvputget-silent-mode'
Closes #5652
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com>
Lucas Di Pentima
05:32 pm Revision 67a56c0f (arvados): refs #11682
Merge branch '11682-fix-test'
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Radhika Chippada
05:30 pm Revision 529d3d90 (arvados): 11682: add infinite scrolling to container_requests index page test.
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
05:18 pm Revision 18f6483f (arvados): 5652: Readability improvements.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima
05:05 pm Revision 85bd5f32 (arvados): 5652: Merge branch 'master' into 5652-arvputget-silent-mode
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima
04:15 pm Bug #11682 (Resolved): [API] Deleting container request doesn't cancel containers
Applied in changeset arvados|commit:3a5b003b5cf784e96799d8ddadc939834620fd34. Radhika Chippada
01:54 pm Bug #11682: [API] Deleting container request doesn't cancel containers
LGTM, thanks! Tom Clegg
03:52 pm Revision 3a5b003b (arvados): closes #11682
Merge branch '11682-delete-container-request'
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Radhika Chippada
03:52 pm Revision 63e12fa3 (arvados): 11682: when a container_request is being destroyed, update it's cont...
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
03:47 pm Task #11940 (Resolved): Review 11917-dont-clear-cache
Tom Clegg
02:05 pm Bug #11917 (Feedback): Error fetching collection (HttpError 422) during call to arvados.util.list...
Tom Clegg
01:55 pm Bug #11973 (Resolved): [CWL] package-builder fails on schema-salad>=2.6
Tom Clegg

07/14/2017

10:17 pm Revision bf87f4ab (arvados): 11973: Tweak fpm dependency spec.
refs #11973
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
08:38 pm Bug #11682: [API] Deleting container request doesn't cancel containers
@ commit:27027c4
> Thanks. I think this version looks cleaner.
It sure does. Thanks.
> This test looks like ...
Radhika Chippada
01:52 pm Bug #11682: [API] Deleting container request doesn't cancel containers
Thanks. I think this version looks cleaner.
This test looks like it can/should be done as a unit test -- calling "...
Tom Clegg
08:32 pm Revision b6756a24 (arvados): 11682: before_destroy callback sets prority to zero only when the co...
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
07:47 pm Revision 436613a0 (arvados): 11906: keepstore ping
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
07:46 pm Feature #11906 (In Progress): Basic authenticated http health check ("ping") for each system service
Radhika Chippada
06:39 pm Revision 41f48ea2 (arvados): 11973: Build avro 1.8.1 for ubuntu1604 and add to schema-salad deps.
refs #11973
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
05:58 pm Revision 6ff6c7a3 (arvados): 11973: Set DEBIAN_FRONTEND=noninteractive in pkg-build/test images.
Clean up some unnecessary inconsistencies in Dockerfiles.
refs #11973
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom...
Tom Clegg
05:16 pm Revision 96ae27be (arvados): 11973: Upgrade setuptools in package-builder images.
Old versions of setuptools cannot build a schema-salad package.
refs #11973
Arvados-DCO-1.1-Signed-off-by: Tom Cleg...
Tom Clegg
05:03 pm Bug #11973: [CWL] package-builder fails on schema-salad>=2.6
Upgrading setuptools helps:... Tom Clegg
05:01 pm Bug #11973 (Resolved): [CWL] package-builder fails on schema-salad>=2.6
Short version:... Tom Clegg
02:55 pm Bug #11960 (Resolved): [Websockets] trash events not delivered reliably
Applied in changeset arvados|commit:eb66b34d22cef6c98ad9ebdc228cf784b3136b72. Tom Clegg
02:50 pm Bug #11960: [Websockets] trash events not delivered reliably
This LGTM, please merge. Thanks! Lucas Di Pentima
02:51 pm Revision eb66b34d (arvados): Merge branch '11960-trash-events'
closes #11960
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
02:50 pm Task #11967 (Resolved): Review 11960-trash-events
Lucas Di Pentima

07/13/2017

10:22 pm Bug #11960: [Websockets] trash events not delivered reliably
I've added tests for the "trash collection" and "delete non-collection object" cases.
In doing so, I tripped on an...
Tom Clegg
09:59 pm Revision d3700e50 (arvados): 11960: Get the full list of old IDs up front in sendOldEvents.
This avoids keeping the "get list of log IDs" database query open
while each log row gets looked up, queued, permissi...
Tom Clegg
09:54 pm Revision 938a0cc2 (arvados): 11960: Fix races in tests.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
09:50 pm Story #11167 (In Progress): [Workbench] Remove arv-get file download fallback
Lucas Di Pentima
09:47 pm Bug #5652: [SDKs] There is no way to silence arv-put and arv-get
Updates at commit:03eee1573
Test run: https://ci.curoverse.com/job/developer-run-tests/383/
Added test to prove t...
Lucas Di Pentima
09:44 pm Revision 03eee157 (arvados): 5652: Added test that proves that arv-get doesn't write the progress
when stderr is not a tty.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com>
Lucas Di Pentima
07:35 pm Bug #11968 (Resolved): [arvados-ws] Fix crash at startup
Applied in changeset arvados|commit:b8977c89db2707dd713cd4eeb11d595717fcff75. Tom Clegg
07:33 pm Revision b8977c89 (arvados): Merge branch '11968-ws-startup-crash'
closes #11968
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
07:33 pm Revision a95c9948 (arvados): Merge branch '11948-cwl-conformance-fix'
refs #11948
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
07:03 pm Revision b6ad4610 (arvados): Add Chen Chen to AUTHORS.
refs #2411
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curoverse.com>
Ward Vandewege
07:03 pm Bug #11850: [CWL] arv:RunInSingleContainer should take max() of ResourceRequirements of substeps
Jiayong Li wrote:
> Discussion with Peter concludes that in the context of "arv:RunInSingleContainer", when Resource...
Jiayong Li
07:00 pm Bug #11850: [CWL] arv:RunInSingleContainer should take max() of ResourceRequirements of substeps
Discussion with Peter concludes that in the context of "arv:RunInSingleContainer", when ResourceRequirement has javas... Jiayong Li
06:45 pm Bug #11850: [CWL] arv:RunInSingleContainer should take max() of ResourceRequirements of substeps
Approach: modify the arvados-cwl-runner code and install it on a virtualenv, then try it on a test cluster with the -... Jiayong Li
05:51 pm Bug #11682: [API] Deleting container request doesn't cancel containers
Branch 11682-delete-container-request @ commit:27b7b66
> I like the approach of calling update_attributes!(priorit...
Radhika Chippada
05:33 pm Revision 27b7b669 (arvados): 11682: add before_destroy filter to set priority to zero
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
01:54 pm Revision 5d081c42 (arvados): 11960: Test that "delete" permissions are not too permissive.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
07:26 am Revision d1562916 (arvados): 11960: Test trashed collection.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
07:25 am Revision 1be8ac0f (arvados): 11960: Fix events leaking between tests.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
06:39 am Revision 1e7fda3f (arvados): 11960: Test permission on "delete" event.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg

07/12/2017

11:32 pm Bug #5652: [SDKs] There is no way to silence arv-put and arv-get
Updates at commit:446e64dc1
Test run: https://ci.curoverse.com/job/developer-run-tests/382/ (not sure if they're wor...
Lucas Di Pentima
09:12 pm Bug #5652 (In Progress): [SDKs] There is no way to silence arv-put and arv-get
Lucas Di Pentima
11:27 pm Revision 446e64dc (arvados): 5652: Code style fixes.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima
11:10 pm Revision 7777370a (arvados): 5652: Added --silent argument to arv-put to make it produce no
output when running in normal conditions.
Added related tests.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <luca...
Lucas Di Pentima
09:43 pm Bug #11965 (Closed): arvados-cwl-runner --help --api default needs to be changed
Tom Clegg
07:46 pm Bug #11965 (In Progress): arvados-cwl-runner --help --api default needs to be changed
We need to change more than the help, I think.
How does the customer do testing for a crunch1 to crunch2 migration...
Tom Morris
03:40 pm Bug #11965 (Resolved): arvados-cwl-runner --help --api default needs to be changed
Applied in changeset arvados|commit:b4b2bf5948c049d28a65629555e6fed4220608dc. Tom Clegg
03:38 pm Bug #11965 (In Progress): arvados-cwl-runner --help --api default needs to be changed
Tom Clegg
02:22 pm Bug #11965 (Closed): arvados-cwl-runner --help --api default needs to be changed
with the merge of https://dev.arvados.org/issues/11857, the default api is now containers, which should be reflected ... Bryan Cosca
09:43 pm Story #11857: [CWL] arvados-cwl-runner defaults to crunch2 if available
Reverted merge. Turns out we weren't really ready for this. Tom Clegg
09:42 pm Story #11857 (In Progress): [CWL] arvados-cwl-runner defaults to crunch2 if available
Tom Clegg
09:41 pm Revision 281cf578 (arvados): Revert "Merge branch '11857-acr-default-crunch2'" (edf37e92f885ed4d5...
refs #11857
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
09:39 pm Revision 48cfc124 (arvados): Revert "11965: Update help message to match new --api default."
refs #11965
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
06:47 pm Bug #11960: [Websockets] trash events not delivered reliably
* Should this conditional delete event emission have a test?
* Running @services/ws@ tests many times produces 1 tes...
Lucas Di Pentima
03:00 pm Bug #11960 (In Progress): [Websockets] trash events not delivered reliably
Tom Clegg
03:00 pm Bug #11960: [Websockets] trash events not delivered reliably
11960-trash-events @ commit:187382bb88eee65d887c004d73eddf46cbd86bc2
Tom Clegg
02:14 pm Bug #11960: [Websockets] trash events not delivered reliably
I see this frame if I create and trash a collection in a different tab:... Tom Clegg
05:43 pm Revision bd29cfa5 (arvados): 11968: Wait in DB() for a database connection if necessary.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
05:43 pm Task #11967 (In Progress): Review 11960-trash-events
Lucas Di Pentima
03:50 pm Task #11967 (Resolved): Review 11960-trash-events
Tom Clegg
05:43 pm Bug #11968 (Resolved): [arvados-ws] Fix crash at startup
Nil pointer dereference if a client session uses the database from (*v0subscribe)sendOldEvents() before (*pgEventSour... Tom Clegg
05:11 pm Bug #11948: Fix failing CWL conformance tests
Local unit test worked ok on my end.
Updates LGTM.
Conformance tests couldn't be run because I wasn't able to start...
Lucas Di Pentima
03:02 pm Revision b4b2bf59 (arvados): 11965: Update help message to match new --api default.
fixes #11965
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
02:58 pm Revision 187382bb (arvados): 11960: Fix permission checking for "delete" events.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:57 pm Revision 0041af13 (arvados): 11960: Fix permission checking for events on trashed collections.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:17 pm Revision a3bfffe9 (arvados): 11960: Include is_trashed in old/new attributes sent to client.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
01:32 pm Task #11932 (Resolved): Review 11843-arpi-transient-error
Tom Clegg

07/11/2017

10:07 pm Bug #11948: Fix failing CWL conformance tests
Lucas Di Pentima wrote:
> Some questions:
>
> * File sdk/cwl/arvados_cwl/runner.py
> ** Line 53: Could that @if@...
Peter Amstutz
09:01 pm Bug #11948: Fix failing CWL conformance tests
Some questions:
* File sdk/cwl/arvados_cwl/runner.py
** Line 53: Could that @if@ be changed to an @elif@, or ther...
Lucas Di Pentima
05:26 pm Bug #11948: Fix failing CWL conformance tests
11948-cwl-conformance-fix @ commit:d1a7f9691ff2b9d77106f953c714b1455b228c64
The main change in this branch is to a...
Peter Amstutz
10:06 pm Revision 2feec8cb (arvados): 11948: if -> elif style fix
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz
10:04 pm Revision 7e38c190 (arvados): 11948: Fix unit tests for additional file checking and nameroot/name...
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz
09:34 pm Revision c4670f70 (arvados): 11944: Ignore arv-put info on stderr.
refs #11944
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
09:05 pm Task #11963 (In Progress): Review 11948-cwl-conformance-fix
Lucas Di Pentima
07:26 pm Task #11963 (Resolved): Review 11948-cwl-conformance-fix
Lucas Di Pentima
08:42 pm Revision 4492163a (arvados): 11944: Turn off warnings while testing command line tool stderr.
refs #11944
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
08:21 pm Task #11924 (Closed): Load fixtures as JSON
Rails will do this automatically when we change column types to jsonb. Tom Clegg
08:03 pm Revision 5fc3cc69 (arvados): 11944: Remove invalid assertRaises(None, ...)
refs #11944
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
07:08 pm Bug #11682: [API] Deleting container request doesn't cancel containers
Would it work just as well to use the existing container and CR called "running" instead of adding new fixtures?
I...
Tom Clegg
06:58 pm Story #10717: Deprecate arv-run
Remove user docs and mark as deprecated Tom Morris
06:56 pm Bug #11960: [Websockets] trash events not delivered reliably
Should add "is_trashed" to sendObjectAttributes in source:services/ws/session_v0.go Tom Clegg
03:38 pm Bug #11960 (Resolved): [Websockets] trash events not delivered reliably
What I did:
* Subscribe to "create", "update" and "delete" events.
* On workbench, go to a subproject under home
...
Peter Amstutz
06:54 pm Bug #11794: Deprecate / remove arv-web
Remove arv-web from user documentation Tom Morris
06:21 pm Story #11942: [CWL] arvados-cwl-runner should support tagging output collection using properties
Example... Tom Clegg
06:14 pm Story #11942: [CWL] arvados-cwl-runner should support tagging output collection using properties
Use a new command line option for "tag via properties" so old code continues to work as before until we figure out th... Tom Clegg
05:10 pm Story #7475: [Node manager] Better communication when job is unsatisfiable
Updates at commit:7d4a10bcc
Added integration test following the above instructions.
Lucas Di Pentima
02:21 pm Story #7475: [Node manager] Better communication when job is unsatisfiable
Writing an integration test:
Start by copying "test_single_node_azure".
The format of the test case is (steps, ...
Peter Amstutz
05:08 pm Revision 7d4a10bc (arvados): 7475: Added integration test that checks for scancel to be called an...
added to an unsatisfiable container.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com>
Lucas Di Pentima
04:40 pm Revision d1a7f969 (arvados): 11948: Strip out default values which reference unavailable files.
Passes CWL conformance test "Test warning instead of error when default path is not found"
Arvados-DCO-1.1-Signed-of...
Peter Amstutz
03:53 pm Story #11908: Migrate Collections.properties to JSONB
> On my system the down-migration worked without dropping/recreating the index. I added a comment to the up-migration... Radhika Chippada
03:18 pm Story #11908: Migrate Collections.properties to JSONB
Radhika Chippada wrote:
> * Do we not need to drop the full text index on collection before changing column type bac...
Tom Clegg
03:36 pm Task #11936 (In Progress): Review 11682-delete-container-request
Tom Clegg
03:14 pm Revision ebc65675 (arvados): 11908: Add comment about dropping/recreating FT index.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:58 pm Revision 40e42b7c (arvados): 11158: Support incremental update of project contents based on webso...
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz
02:33 pm Revision b4dd6bcd (arvados): 11908: Fix crash if index being deleted is already deleted.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg

07/10/2017

09:56 pm Bug #11917: Error fetching collection (HttpError 422) during call to arvados.util.listdir_recursive
"expiry times" == "clear out old entries"
Perhaps we could put a rake tmp:cache:clear in the API server boot scrip...
Tom Morris
08:17 pm Bug #11917: Error fetching collection (HttpError 422) during call to arvados.util.listdir_recursive
We use expiry times and a finite number of keys, so the cache shouldn't grow indefinitely.
We should add a cache-c...
Tom Clegg
09:33 pm Story #11857: [CWL] arvados-cwl-runner defaults to crunch2 if available
This LGTM, thanks. Lucas Di Pentima
09:10 pm Story #11857: [CWL] arvados-cwl-runner defaults to crunch2 if available
Added the following to [[Upgrading to master]]:
2017-07-10: #11857 commit:edf37e9 defaults to containers api if av...
Radhika Chippada
09:06 pm Revision b1f306a0 (arvados): Merge branch '11950-stretch'
refs #11950
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
08:59 pm Task #11955 (Resolved): Review 11857-acr-default-crunch2
Lucas Di Pentima
08:18 pm Task #11955 (In Progress): Review 11857-acr-default-crunch2
Lucas Di Pentima
06:17 pm Task #11955 (Resolved): Review 11857-acr-default-crunch2
Radhika Chippada
08:58 pm Revision edf37e92 (arvados): refs #11857
Merge branch '11857-acr-default-crunch2'
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Radhika Chippada
08:53 pm Revision 23f0d2b4 (arvados): 11950: Recognize new (2.11.0, debian9) git error messages.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
08:52 pm Revision 670f4b62 (arvados): Revert python sdk changes.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz
08:38 pm Bug #11158: [FUSE] Hang on simple FUSE operation and when logging in again later
On further examination, it seems what is actually happening is that it is getting a steady stream of events from webs... Peter Amstutz
04:53 pm Bug #11158: [FUSE] Hang on simple FUSE operation and when logging in again later
items_available as reported:... Peter Amstutz
01:39 pm Bug #11158: [FUSE] Hang on simple FUSE operation and when logging in again later
I'll split the Python SDK and FUSE changes into separate branches.
Peter Amstutz
08:25 pm Bug #11945 (Resolved): keep-web returns empty/truncated content when asked for same collection us...
Applied in changeset arvados|commit:9b2d0d31de409556fcd759e27efdb260ced6f3d3. Tom Clegg
08:17 pm Bug #11945: keep-web returns empty/truncated content when asked for same collection using differe...
Updates @ commit:a8aec576a LGTM, thanks. Lucas Di Pentima
08:21 pm Revision 9b2d0d31 (arvados): Merge branch '11945-collection-cache-key'
fixes #11945
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
08:19 pm Revision d084ca24 (arvados): Merge branch '11917-dont-clear-cache'
refs #11917
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
08:17 pm Task #11946 (Resolved): Review 11945-collection-cache-key
Lucas Di Pentima
06:22 pm Task #11946 (In Progress): Review 11945-collection-cache-key
Lucas Di Pentima
06:54 pm Revision 71419a78 (arvados): 11950: Fix nodemanager watchdog test.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
06:28 pm Revision 5e6f21c3 (arvados): 11950: Recognize new (2.11.0, debian9) git error messages.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
06:13 pm Revision 7f3cb29c (arvados): 11857: arvados-cwl-runner defaults to crunch2 if available
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
01:22 pm Revision 17cd77ac (arvados): 7475: Merge branch 'master' into 7475-nodemgr-unsatisfiable-job-comms
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima

07/08/2017

08:56 pm Bug #11948 (In Progress): Fix failing CWL conformance tests
Peter Amstutz
08:55 pm Bug #11948 (Resolved): Fix failing CWL conformance tests
New CWL conformance tests were introduced which fail on Arvados.
Fix arvados-cwl-runner to pass these tests.
Peter Amstutz
08:01 pm Revision e953b9ec (arvados): Update cwltool & schema-salad and fix for changes.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz

07/07/2017

08:24 pm Task #11946 (Resolved): Review 11945-collection-cache-key
Tom Clegg
08:24 pm Bug #11945: keep-web returns empty/truncated content when asked for same collection using differe...
11945-collection-cache-key @ commit:a8aec576a4047a27f6ec4227258393d2f7fef59d
Tom Clegg
07:58 pm Bug #11945 (Resolved): keep-web returns empty/truncated content when asked for same collection us...
The collection cache is keyed by PDH, but has different content depending on which token requested it. It should be k... Tom Clegg
08:23 pm Revision a8aec576 (arvados): 11945: Key collection cache on pdh+token.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
08:11 pm Revision 13d40db2 (arvados): Merge branch '11945-disable-cache'
refs #11945
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
08:08 pm Revision 0f650530 (arvados): 11945: Disable collection cache.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
06:52 pm Revision ad47ed8a (arvados): arvados-cwl-runner always submits runner job when invoked with --no-...
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz
06:50 pm Task #11933 (In Progress): Review 11908-properties-column-json
Radhika Chippada
06:50 pm Story #11908: Migrate Collections.properties to JSONB
* Do we not need to drop the full text index on collection before changing column type back in down migration?
* D...
Radhika Chippada
05:53 pm Bug #11158: [FUSE] Hang on simple FUSE operation and when logging in again later
In this version of @list_all()@, the "filters" logic assumes the response is ordered by uuid, but it requests whateve... Tom Clegg
03:47 pm Task #11890 (Resolved): Review 10584-fuse-stop-threads
Peter Amstutz
03:27 pm Revision 1e31815d (arvados): Merge branch '10584-fuse-stop-threads' refs #10584
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz
02:47 pm Bug #10584: [FUSE] high memory consumption (possible leak) in long-running arv-mount
Ok, so this looks good to me. Thanks! Lucas Di Pentima
12:55 pm Bug #10584: [FUSE] high memory consumption (possible leak) in long-running arv-mount
Lucas Di Pentima wrote:
> The thread stopping code was added on a @CollectionDirectoryBase@ subclass, is it possible...
Peter Amstutz
02:04 pm Bug #11843 (Feedback): Timeout attempting to create job for component gatk-haplotypecaller-cram-g...
Tom Clegg
01:13 pm Revision e2b3986e (arvados): Merge branch '11843-arpi-transient-error'
refs #11843
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
01:32 am Bug #11917: Error fetching collection (HttpError 422) during call to arvados.util.listdir_recursive
Is the cache LRU? http://guides.rubyonrails.org/caching_with_rails.html says: "As the cache will grow until the disk ... Radhika Chippada
01:16 am Task #11940 (In Progress): Review 11917-dont-clear-cache
Radhika Chippada
12:46 am Revision 84b73bac (arvados): 11682: when a container_request is being destroyed, update it's cont...
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada

07/06/2017

09:33 pm Story #11908 (In Progress): Migrate Collections.properties to JSONB
Changing the column type in-place is trivial ("alter table foo alter column bar type jsonb using bar::jsonb"). Unfort... Tom Clegg
09:19 pm Revision bb821d03 (arvados): 11908: Migrate collections.properties to jsonb.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
09:14 pm Story #7475: [Node manager] Better communication when job is unsatisfiable
Updates at commit:f507162f3
Test run: https://ci.curoverse.com/job/developer-run-tests/378/
Added support for uns...
Lucas Di Pentima
09:10 pm Revision f507162f (arvados): 7475: Check for job unsatisfiable type (job/container) and cancel it...
the proper method.
Updated test to check for both cases.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curo...
Lucas Di Pentima
07:12 pm Bug #11843: Timeout attempting to create job for component gatk-haplotypecaller-cram-gvcf: Net::R...
Thanks for the explanation, so this LGTM. Thanks. Lucas Di Pentima
06:05 pm Bug #11843: Timeout attempting to create job for component gatk-haplotypecaller-cram-gvcf: Net::R...
Lucas Di Pentima wrote:
> * Is it worth writing a test for this issue?
I wish, but given that crunch1's days are ...
Tom Clegg
05:25 pm Bug #11843: Timeout attempting to create job for component gatk-haplotypecaller-cram-gvcf: Net::R...
* Is it worth writing a test for this issue?
* Taking into account #3005, would it be convenient to only retry a def...
Lucas Di Pentima
07:08 pm Bug #10584: [FUSE] high memory consumption (possible leak) in long-running arv-mount
The thread stopping code was added on a @CollectionDirectoryBase@ subclass, is it possible for this problem to happen... Lucas Di Pentima
02:17 am Bug #10584: [FUSE] high memory consumption (possible leak) in long-running arv-mount
10584-fuse-stop-threads
Ensure get/put threads are stopped before releasing reference to Collection object. Uncl...
Peter Amstutz
05:44 pm Story #11807: [API] Migrate old serialized database content from YAML to JSON
and qr1hi...... Nico César
03:15 pm Story #11807: [API] Migrate old serialized database content from YAML to JSON
this is a deploy in su92l ... Nico César
05:39 pm Task #11890 (In Progress): Review 10584-fuse-stop-threads
Lucas Di Pentima
02:59 pm Bug #11682 (In Progress): [API] Deleting container request doesn't cancel containers
Radhika Chippada
02:14 pm Revision 8e92b7b1 (arvados): 11807: Migrate data from YAML to JSON.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:01 pm Task #11932 (In Progress): Review 11843-arpi-transient-error
Lucas Di Pentima
01:46 pm Task #11924 (In Progress): Load fixtures as JSON
Tom Clegg
01:34 pm Bug #11927 (Resolved): [Workbench] test_Report_network_error failure
Tom Clegg

07/05/2017

10:47 pm Revision c5b9f2d2 (arvados): 2411: Fix crunch-job license statement.
crunch-job began as a copy of whjobmanager from the Free Factories
project.
refs #2411
Arvados-DCO-1.1-Signed-off-b...
Tom Clegg
09:38 pm Revision 446d2d89 (arvados): Merge branch '11927-tags-test'
refs #11927
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
09:38 pm Bug #11927: [Workbench] test_Report_network_error failure
11927-tags-test @ commit:e914c48b16e6bb8199ebad89096a09e477a7efe8
Tom Clegg
08:55 pm Bug #11927: [Workbench] test_Report_network_error failure
This test passed (yay)... Tom Clegg
08:28 pm Bug #11927: [Workbench] test_Report_network_error failure
updated the stuff and kicked https://ci.curoverse.com/job/run-tests-apps-workbench-integration/1084/console which FA... Nico César
07:13 pm Bug #11927 (In Progress): [Workbench] test_Report_network_error failure
Tom Clegg
07:12 pm Bug #11927: [Workbench] test_Report_network_error failure
Probably this:... Tom Clegg
06:17 pm Bug #11927 (Resolved): [Workbench] test_Report_network_error failure
https://ci.curoverse.com/job/run-tests-apps-workbench-integration/1082/console Peter Amstutz
09:34 pm Revision e914c48b (arvados): 11927: Eliminate unnecessary use of selenium.
This test case works without selenium, if we don't try to click an
"edit" link that's disabled because we already cli...
Tom Clegg
09:30 pm Revision 7d751b9d (arvados): 11927: Remove errant type attr from anchor element.
Anchor elements do have a "type" attribute, but it specifies a MIME
type, not a data/control type like "button" as wi...
Tom Clegg
08:55 pm Story #11942: [CWL] arvados-cwl-runner should support tagging output collection using properties
The current output collection tagging using Links, but when tag support is available in the Collection.properties fie... Tom Morris
08:41 pm Story #11942 (New): [CWL] arvados-cwl-runner should support tagging output collection using prope...
Tom Morris
08:04 pm Task #11940 (Resolved): Review 11917-dont-clear-cache
Radhika Chippada
08:02 pm Task #11939 (Resolved): Review 5652-arvputget-silent-mode
Lucas Di Pentima
08:01 pm Task #11937 (In Progress): Review 11167-wb-remove-arvget
Lucas Di Pentima
08:01 pm Task #11936 (Resolved): Review 11682-delete-container-request
Radhika Chippada
07:53 pm Bug #10705 (New): [Crunch2] [API] return a more specific 422 error message when a client calls co...
Tom Morris
07:51 pm Bug #11682: [API] Deleting container request doesn't cancel containers
Deleting a CR should invoke the same hook that runs when priority changes, so (if a container is assigned and the del... Tom Clegg
07:34 pm Task #11933 (In Progress): Review 11908-properties-column-json
Tom Clegg
07:34 pm Task #11932 (Resolved): Review 11843-arpi-transient-error
Tom Clegg
07:34 pm Bug #11843 (In Progress): Timeout attempting to create job for component gatk-haplotypecaller-cra...
11843-arpi-transient-error Tom Clegg
07:04 pm Bug #11850 (In Progress): [CWL] arv:RunInSingleContainer should take max() of ResourceRequirement...
Jiayong Li
07:03 pm Story #11807 (In Progress): [API] Migrate old serialized database content from YAML to JSON
Tom Clegg
07:03 pm Story #8784 (Resolved): [Workbench] Use keep-web to generate directory listings
Tom Clegg
07:00 pm Bug #8624 (Resolved): [FUSE] arv-mount `by_tag` directory only shows 100 tags and tags that exist...
Applied in changeset arvados|commit:fcf3ca5baf89bdd944e3a7dcdc1b65f8ff4945ca. Peter Amstutz
06:25 pm Bug #8624 (In Progress): [FUSE] arv-mount `by_tag` directory only shows 100 tags and tags that ex...
Peter Amstutz
06:58 pm Revision a5b458ca (arvados): 10584: When evicting a Collection from the cache, ensure that get/pu...
threads are stopped.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com>
Peter Amstutz
06:55 pm Revision fcf3ca5b (arvados): Merge branch '8624-fuse-tags' closes #8624
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz
06:51 pm Revision c8cb8059 (arvados): 8624: Move check that tag exists from __contains__ to __getitem__ me...
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz
06:45 pm Bug #10349 (In Progress): [DOC] add cookbook section with code snippets
Tom Morris
06:29 pm Bug #11158: [FUSE] Hang on simple FUSE operation and when logging in again later
Sorry, that branch was a little half baked. Tests should pass now.
Now @ commit:b25cbdbd1b54a622f4428e29ccffeb165...
Peter Amstutz
06:24 pm Bug #10805 (In Progress): [FUSE] Upgrade llfuse to 1.2, fix deadlock in test suite
Peter Amstutz
05:57 pm Bug #11925 (New): [Nodemanager] Fix watchdog test
Peter Amstutz
03:58 pm Task #11759 (In Progress): Review 7475-nodemgr-unsatisfiable-job-comms
Peter Amstutz
03:56 pm Story #7475: [Node manager] Better communication when job is unsatisfiable
7475-nodemgr-unsatisfiable-job-comms @ commit:f77d08dd57a1021525717c8669296eb3e463c5f7
* In _got_response, the uui...
Peter Amstutz
02:31 pm Story #7475: [Node manager] Better communication when job is unsatisfiable
New updates at commit:f77d08dd5
Test run: https://ci.curoverse.com/job/developer-run-tests/377/
* Enhanced error ...
Lucas Di Pentima
03:39 pm Revision 0eb72b52 (arvados): Merge branch '8784-dir-listings'
refs #8784
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
03:33 pm Revision 9aeb8d55 (arvados): 11843: Do not fail pipelines on transient API failures.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
03:32 pm Revision ef8ff20b (arvados): 11843: Improve formatting.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:28 pm Revision f77d08dd (arvados): 7475: Catch exceptions when trying to cancel an unsatisfiable job,
logging an error message in case of problems.
Added tests.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@cu...
Lucas Di Pentima
01:27 pm Task #11923 (Resolved): Migration for other serialized columns
Tom Clegg
01:24 pm Revision ad77601b (arvados): Add missing copyright headers.
refs #2411
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
01:23 pm Revision 94b75d06 (arvados): Update .licenseignore.
refs #2411
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
01:21 pm Revision 20dfea76 (arvados): Fix regex usage.
refs #2411
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
08:06 am Revision 02b3b0a0 (arvados): Add AUTHORS, update COPYING and license file names.
refs #2411
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curoverse.com>
Ward Vandewege
02:14 am Revision 9849da2f (arvados): 11807: Migrate data from YAML to JSON.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:13 am Revision db948650 (arvados): 2411: Fix regex usage.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg

07/04/2017

08:01 pm Task #11923 (In Progress): Migration for other serialized columns
Tom Clegg
08:00 pm Task #11923 (Resolved): Migration for other serialized columns
Tom Clegg
08:01 pm Task #11924 (Closed): Load fixtures as JSON
Tom Clegg
08:00 pm Task #11921 (Resolved): Update "upgrading" wiki with dev cluster times
Tom Clegg

07/03/2017

09:41 pm Revision 79517991 (arvados): Quote subshell expression in test_with_arvbox and make container res...
no issue #
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com>
Peter Amstutz
08:28 pm Revision b25cbdbd (arvados): 11158: Fixes & test updates for ProjectDirectory.
* list_all() orders by uuid and uses it for paging.
* Move dynamic item lookup from __contains__ to __getitem__ to f...
Peter Amstutz
08:14 pm Story #7475: [Node manager] Better communication when job is unsatisfiable
Updates @ commit:3dad67f27
Test run: https://ci.curoverse.com/job/developer-run-tests/376/
Modified @ServerCalcul...
Lucas Di Pentima
08:03 pm Revision 3dad67f2 (arvados): 7475: Cancel jobs that cannot be satisfied instead of endlessly retr...
Add a log entry for every cancelled job explaining why it isn't possible
to run it.
Updated tests.
Arvados-DCO-1.1-S...
Lucas Di Pentima
02:21 pm Task #11825 (Resolved): Review 11789-arvput-exclude-flag
Lucas Di Pentima
02:07 pm Story #8784: [Workbench] Use keep-web to generate directory listings
LGTM @ commit:e59c1d365d9b6e1eff9b5cb030a8b1a3aaf14353 Peter Amstutz
02:06 pm Task #11887 (Resolved): Review 8784-dir-listings
Peter Amstutz
01:42 pm Bug #10584: [FUSE] high memory consumption (possible leak) in long-running arv-mount
Some theories:
* This might be related/due to https://dev.arvados.org/issues/11158 &rarr; it is that it is trying ...
Peter Amstutz

06/30/2017

09:36 pm Task #11191 (In Progress): Review 11158-fuse-projects
Tom Clegg
09:11 pm Bug #11158: [FUSE] Hang on simple FUSE operation and when logging in again later
looking at 11158-fuse-projects @ commit:62f1bc5...
It looks like @list_all(..., limit=1000)@ makes @list_all()@ as...
Tom Clegg
02:41 pm Bug #11158: [FUSE] Hang on simple FUSE operation and when logging in again later
New behavior for project directories. Allow 'cd' into any collection or subproject that exists on the server, even i... Peter Amstutz
02:26 pm Bug #11158 (In Progress): [FUSE] Hang on simple FUSE operation and when logging in again later
Peter Amstutz
08:27 pm Revision 62f1bc50 (arvados): 11158: Improve arvados.util.list_all() implementation to avoid relyi...
items_available.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com>
Peter Amstutz
07:41 pm Story #8784: [Workbench] Use keep-web to generate directory listings
Peter Amstutz wrote:
> What's the rationale for using query_token instead of path_token?
None -- changed to path_...
Tom Clegg
07:34 pm Revision e59c1d36 (arvados): 8784: Fix test for latest firefox.
Given "http://host:99999/path", Firefox 54 returns an error
"SyntaxError: The URI is malformed" instead of trying to ...
Tom Clegg
06:54 pm Task #11891 (Resolved): Review 8624-fuse-tags
Radhika Chippada
06:09 pm Task #11891 (In Progress): Review 8624-fuse-tags
Radhika Chippada
06:53 pm Bug #8624: [FUSE] arv-mount `by_tag` directory only shows 100 tags and tags that exist are not ac...
I am having difficulty running fuse tests in my development env. So, assuming tests are passing to you, LGTM Radhika Chippada
02:30 pm Bug #8624: [FUSE] arv-mount `by_tag` directory only shows 100 tags and tags that exist are not ac...
New behavior for by_tag/ dir. Allow 'cd' into any tag directory that exists on the server, even if it isn't returned... Peter Amstutz
06:52 pm Revision 024a666e (arvados): 11158: Restore paging to get full list of collections and projects (...
don't load other object types). Defer full listing load until first call to
items() to support efficient access of i...
Peter Amstutz
06:34 pm Revision 1b8d6697 (arvados): 8784: Add token to path (not query string) in download links.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
06:31 pm Task #11892 (Resolved): Review 11807-yaml-to-json
Tom Clegg
06:31 pm Task #11921 (Resolved): Update "upgrading" wiki with dev cluster times
Tom Clegg
06:30 pm Revision 55aafbb0 (arvados): Merge branch '11807-yaml-to-json'
refs #11807
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
06:15 pm Story #11870 (Resolved): [Workbench] We have two pages /repositories, and /users/<uuid>/repositor...
Applied in changeset arvados|commit:019c7c67a37cd2a6e43ceed73771c1cd87c5f615. Radhika Chippada
03:34 pm Story #11870: [Workbench] We have two pages /repositories, and /users/<uuid>/repositories. Move t...
LGTM, thanks! Lucas Di Pentima
03:09 pm Story #11870: [Workbench] We have two pages /repositories, and /users/<uuid>/repositories. Move t...
* Updated repositories_controller.rb and _show_help.html.erb accordingly @ commit:30186c3f928d280e4af97a4989d7a82dfe6... Radhika Chippada
02:53 pm Story #11870: [Workbench] We have two pages /repositories, and /users/<uuid>/repositories. Move t...
Reviewing commit:d898b4cb9
* File @apps/workbench/app/controllers/repositories_controller.rb@
** Lines 37, 38: Th...
Lucas Di Pentima
12:13 pm Story #11870: [Workbench] We have two pages /repositories, and /users/<uuid>/repositories. Move t...
Branch 11870-repositories-index
* /repositories page is improved to show URL column and display order (owned first...
Radhika Chippada
06:00 pm Bug #11910 (Resolved): [arvados-ws] Fix racy test case
Applied in changeset arvados|commit:3fe48e9f9ce4f29ececb07fa8934565992a8d52f. Tom Clegg
01:52 pm Bug #11910: [arvados-ws] Fix racy test case
The test failures are exposing a real race bug. I didn't see a quick way to fix it, so in the meantime this avoids hi... Tom Clegg
05:59 pm Revision 019c7c67 (arvados): closes #11870
Merge branch '11870-repositories-index'
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Radhika Chippada
05:57 pm Revision 619efdfa (arvados): Merge branch 'master' into 11870-repositories-index
Radhika Chippada
05:56 pm Revision 3fe48e9f (arvados): Merge branch '11910-ws-test-race'
closes #11910
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
05:55 pm Revision 23ddb633 (arvados): 11910: Fix racy tests: ignore non-matching logs from previous tests.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
05:55 pm Revision 21a3b4fb (arvados): 11910: Ensure test passes despite race between old and new events.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
05:53 pm Bug #11917 (In Progress): Error fetching collection (HttpError 422) during call to arvados.util.l...
Tom Clegg
05:53 pm Bug #11917: Error fetching collection (HttpError 422) during call to arvados.util.listdir_recursive
11917-dont-clear-cache @ commit:e8d11e963248fada51bdb1ecc59446adccff440f
Tom Clegg
05:29 pm Revision 856d0e91 (arvados): 11870: add copyright note to new files
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
05:25 pm Revision 0561bd0c (arvados): Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com>
Merge branch 'master' into 11870-repositories-index
Conflicts:
apps/workbench/app/views/repositories/_show_help.htm...
Radhika Chippada
05:13 pm Revision cecd50eb (arvados): 11158: Fix order="modified_at".
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz
03:34 pm Task #11880 (Resolved): Review 11870-repositories-index
Lucas Di Pentima
01:25 pm Task #11880 (In Progress): Review 11870-repositories-index
Lucas Di Pentima
03:30 pm Revision 55368647 (arvados): 11870: minor update
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
03:05 pm Revision 30186c3f (arvados): 11870: minor updates
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
02:55 pm Story #11807: [API] Migrate old serialized database content from YAML to JSON
I noticed this during my testing of this branch: I migrated my arvados_test db and jobs text fields are now json form... Radhika Chippada
02:37 pm Revision e8d11e96 (arvados): 11917: Do not clear rails cache at boot time.
This seems like a good idea when there is only one process that has
access to the cache. However, in real life the cr...
Tom Clegg
02:36 pm Revision 32c2f3d8 (arvados): Merge branch '2411-rake-copyright'
refs #2411
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
02:34 pm Revision 3a0c9ae2 (arvados): 2411: Maintain copyright header in structure.sql.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:27 pm Revision 8cf65d89 (arvados): 11158: New behavior for project directories. Allow 'cd' into any co...
subproject that exists on the server, even if it isn't returned in the listing.
Now applies listing limit, ordered by...
Peter Amstutz
02:24 pm Bug #10805: [FUSE] Upgrade llfuse to 1.2, fix deadlock in test suite
Filed a bug report on llfuse, with recommendation to remove pthread_cancel().
https://bitbucket.org/nikratio/pytho...
Peter Amstutz
01:48 pm Revision 738c46ba (arvados): 11910: Ensure test passes despite race between old and new events.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
01:48 pm Revision 42de8da8 (arvados): 11910: Fix racy tests: ignore non-matching logs from previous tests.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
01:40 am Revision 2e06098f (arvados): Fix typo.
No issue #
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward@curoverse.com>
Ward Vandewege

06/29/2017

10:28 pm Revision 048821c9 (arvados): 8624: New behavior for by_tag/ dir. Allow 'cd' into any tag directo...
exists on the server, even if it isn't returned in the listing. Also increase
listing limit.
Arvados-DCO-1.1-Signed...
Peter Amstutz
08:24 pm Revision d898b4cb (arvados): 11870: update tests to be compatible with the user menu repositories...
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
08:20 pm Story #11789 (Resolved): [arv-put] add a --exclude flag to allow specific files/directories to be...
Applied in changeset arvados|commit:8a4a2ee17807f35eaff8be99cca780ef1fd0ba64. Lucas Di Pentima
07:51 pm Story #11789: [arv-put] add a --exclude flag to allow specific files/directories to be skipped
Updates at commit:d53ae1523
Test run: https://ci.curoverse.com/job/developer-run-tests/371/
Cleaned up all @filte...
Lucas Di Pentima
02:40 pm Story #11789: [arv-put] add a --exclude flag to allow specific files/directories to be skipped
Wouldn't that be
@arv-put --exclude './*.txt' .@
Otherwise it would be
@arv-put --exclude 'dir1/*.txt' dir1@...
Peter Amstutz
02:36 pm Story #11789: [arv-put] add a --exclude flag to allow specific files/directories to be skipped
Lucas Di Pentima wrote:
> Updates at commit:441ef97e93a951b349356df96d8a6ef604c6cab7
> Test run: https://ci.curover...
Peter Amstutz
08:16 pm Revision 8a4a2ee1 (arvados): Merge branch '11789-arvput-exclude-flag'
Closes #11789
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com>
Lucas Di Pentima
08:10 pm Revision 583cb3b5 (arvados): Add missing test file for noreuse.cwl. refs #11095
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz@curoverse.com> Peter Amstutz
07:43 pm Revision d53ae152 (arvados): 11789: Added support for the corner case of needing to use a pathname
pattern that refers to a file or directory directoy beneath a given
input dir, the user should use './file-to-exclude...
Lucas Di Pentima
06:51 pm Bug #10805: [FUSE] Upgrade llfuse to 1.2, fix deadlock in test suite
Other options include:
# Work around deadlock by refactoring test cases so that llfuse only runs in a short-lived ...
Peter Amstutz
06:12 pm Bug #10805: [FUSE] Upgrade llfuse to 1.2, fix deadlock in test suite
The crash may be a red herring. Not long after it crashing, I had to reboot my compute due to a kernel panic (it has... Peter Amstutz
04:57 pm Bug #10805: [FUSE] Upgrade llfuse to 1.2, fix deadlock in test suite
Working backwards, here is my theory of the deadlock:
# It gets stuck in @PyEval_RestoreThread()@ attempting to ac...
Peter Amstutz
04:16 pm Story #11807: [API] Migrate old serialized database content from YAML to JSON
I was able to verify that the migration is working correctly in my test database.
LGTM
Radhika Chippada
04:00 pm Story #8784: [Workbench] Use keep-web to generate directory listings
What's the rationale for using query_token instead of path_token? Peter Amstutz
03:51 pm Bug #11917: Error fetching collection (HttpError 422) during call to arvados.util.listdir_recursive
We do this in source:services/api/config/environment.rb:... Tom Clegg
03:42 pm Bug #11917: Error fetching collection (HttpError 422) during call to arvados.util.listdir_recursive
In activesupport-4.2.5.2/lib/active_support/cache/file_store.rb... Tom Clegg
03:27 pm Bug #11917: Error fetching collection (HttpError 422) during call to arvados.util.listdir_recursive
This code looks like a supported use of Rails cache:... Tom Clegg
03:24 pm Bug #11917: Error fetching collection (HttpError 422) during call to arvados.util.listdir_recursive
From production.log:... Joshua Randall
03:20 pm Bug #11917 (Feedback): Error fetching collection (HttpError 422) during call to arvados.util.list...
... Joshua Randall
03:34 pm Revision 5936cf79 (arvados): 11789: Replaced filter() with list comprehensions.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima
03:00 pm Bug #11910 (In Progress): [arvados-ws] Fix racy test case
Tom Clegg
02:48 pm Revision 44c95f99 (arvados): 11789: Merge branch 'master' into 11789-arvput-exclude-flag
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima
02:38 pm Task #11887 (In Progress): Review 8784-dir-listings
Peter Amstutz
02:18 pm Story #7988 (Closed): [Keep] Single keepstore responsible for trash lists on S3
Tom Clegg
02:17 pm Story #7988: [Keep] Single keepstore responsible for trash lists on S3
Superseded by the keepstore trash/untrash API, implemented in #8555. Tom Clegg
02:11 pm Story #8061 (Duplicate): [Keep] keepstore checks if a GET client is still there before loading a ...
Tom Clegg
02:09 pm Story #6311 (Rejected): [Maybe] [SDKs] Support caching Keep blocks in memcached
Tom Clegg
02:07 pm Feature #7595 (Resolved): [keep-web] Cache keep_services list between requests
Resolved by #9005 Tom Clegg
02:06 pm Story #8403 (Closed): [SDK] Test to see if increasing keep cache and prefetching produces better ...
Tom Clegg
02:06 pm Feature #8409 (Resolved): [crunchstat-summary] Look for signs of cache thrashing and suggest incr...
Tom Clegg
02:05 pm Task #11892 (In Progress): Review 11807-yaml-to-json
Radhika Chippada
11:51 am Revision 060d38d6 (arvados): Merge branch '2411-check-copyright'
refs #2411
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
03:02 am Revision 23d6cfd5 (arvados): 2411: Eliminate whitespace to placate test.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
12:47 am Revision 528ce62a (arvados): 11870: link repositories menu item in user menu to repositories inde...
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada

06/28/2017

10:53 pm Revision 004c230c (arvados): 2411: Updated check-copyright-notices script to correctly add the co...
block to coffeescript files.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com>
Lucas Di Pentima
09:46 pm Revision 259d991a (arvados): 11870: add infinite scrolling and "add new repository" button to rep...
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika@curoverse.com> Radhika Chippada
09:04 pm Task #11914 (Resolved): Migrate jobs table
Tom Clegg
09:03 pm Task #11914 (Resolved): Migrate jobs table
Tom Clegg
08:57 pm Story #11807: [API] Migrate old serialized database content from YAML to JSON
11807-yaml-to-json @ commit:7a5eb1b19c698f39b7cfdaafa4b3deefe556b07e
* migrates content in the serialized columns of...
Tom Clegg
08:32 pm Revision cc4e8f68 (arvados): 2411: Add exceptions.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
08:27 pm Revision 0f644e24 (arvados): 2411: Add copyright notices to everything.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
08:27 pm Revision ec88c9ee (arvados): 2411: Remove golang<1.4 shim.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
07:38 pm Revision aa5cf740 (arvados): 2411: Remove golang<1.4 shim.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
07:32 pm Revision 7a5eb1b1 (arvados): 11807: Migrate old records in jobs table from YAML to JSON.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
03:44 pm Story #11789: [arv-put] add a --exclude flag to allow specific files/directories to be skipped
Updates at commit:441ef97e93a951b349356df96d8a6ef604c6cab7
Test run: https://ci.curoverse.com/job/developer-run-test...
Lucas Di Pentima
03:37 pm Revision 40a318a7 (arvados): 2411: Add copyright notices to everything.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
03:37 pm Revision 7041d6b1 (arvados): 2411: Add copyright after <?xml declaration in svg files.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
03:37 pm Revision 62ed8312 (arvados): 2411: Fix tests to accommodate copyright notices.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
03:37 pm Revision 9fb7687e (arvados): 2411: Add exceptions.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
03:37 pm Revision 13abd324 (arvados): 2411: Set template for more nodemanager test files.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
03:36 pm Revision 2a97a353 (arvados): 2411: Fix up whitespace.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:47 pm Revision 622d86f9 (arvados): 2411: Fix liquid comment template and skip top matter in docs.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
02:17 pm Bug #11910 (Resolved): [arvados-ws] Fix racy test case
From https://ci.curoverse.com/job/developer-run-tests-remainder/366/console... Tom Clegg
01:25 pm Feature #11095 (Resolved): [CWL] Hint to control reuse of individual workflow steps
Applied in changeset arvados|commit:a27e0874320e408124fee837b355995350bbc7ee. Peter Amstutz
01:24 pm Revision 441ef97e (arvados): 11789: Added tests.
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima
01:23 pm Revision a27e0874 (arvados): Merge branch '11095-cwl-control-reuse' closes #11095
Peter Amstutz
02:29 am Revision 3551a039 (arvados): 11789: Converting a filter() iterable to list, for python3 compatibi...
Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <lucas@curoverse.com> Lucas Di Pentima

06/27/2017

09:04 pm Story #11870 (In Progress): [Workbench] We have two pages /repositories, and /users/<uuid>/reposi...
Radhika Chippada
09:02 pm Task #11889 (Resolved): Review 11095-cwl-control-reuse
Radhika Chippada
07:16 pm Task #11889 (In Progress): Review 11095-cwl-control-reuse
Radhika Chippada
09:02 pm Feature #11095: [CWL] Hint to control reuse of individual workflow steps
LGTM Radhika Chippada
08:24 pm Task #11904 (Resolved): Review 11901-ws-db-conns
Radhika Chippada
05:02 pm Task #11904 (In Progress): Review 11901-ws-db-conns
Radhika Chippada
08:24 pm Task #11881 (Resolved): Review 11860-list-n-index-discovery-methods
Radhika Chippada
08:24 pm Task #11885 (Resolved): Review 11840-unique-constraint-untrash-coll
Radhika Chippada
08:18 pm Task #11872 (Resolved): Workbench redirect
Tom Clegg
08:17 pm Story #8784: [Workbench] Use keep-web to generate directory listings
8784-dir-listings @ commit:ff723f5f08e286df4c4c58a27a9db574ca604a6f -- passed https://ci.curoverse.com/job/developer-... Tom Clegg
08:05 pm Bug #10805: [FUSE] Upgrade llfuse to 1.2, fix deadlock in test suite
According to gdb, this is where it hangs:... Peter Amstutz
06:55 pm Story #11908 (In Progress): Migrate Collections.properties to JSONB
Handle (deserialize) both YAML and JSON and convert column type to JSONB Tom Morris
06:10 pm Feature #11906: Basic authenticated http health check ("ping") for each system service
including
* apiserver
* workbench
* arv-git-httpd
* nodemanager
* keepstore
* keepproxy
* keep-web
* -arvados...
Tom Clegg
01:51 pm Feature #11906 (In Progress): Basic authenticated http health check ("ping") for each system service
Functional details:
* respond to “GET /_health/ping” on same addr:port as the microservice’s main http server (or “m...
Tom Clegg
06:05 pm Bug #11901 (Resolved): [arvados-ws] Fix leaking postgres connections and subsequent stall
Applied in changeset arvados|commit:8051c3a14d40f0d410e4ddf54d89a084475d807e. Tom Clegg
05:47 pm Bug #11901: [arvados-ws] Fix leaking postgres connections and subsequent stall
Radhika Chippada wrote:
> * Should "/_health/ping" and "/_health/db" also check if the ManagementToken is configured...
Tom Clegg
05:22 pm Bug #11901: [arvados-ws] Fix leaking postgres connections and subsequent stall
* Should "/_health/ping" and "/_health/db" also check if the ManagementToken is configured and bearer token matches? ... Radhika Chippada
02:22 pm Bug #11901: [arvados-ws] Fix leaking postgres connections and subsequent stall
The health-check specs here (authentication, URLs, responses) are the ones Nico and I developed last week based on ex... Tom Clegg
06:02 pm Revision 8051c3a1 (arvados): Merge branch '11901-ws-db-conns'
closes #11901
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com>
Tom Clegg
05:53 pm Revision b02577b1 (arvados): 11901: Try more tokens in 404 test.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
05:46 pm Feature #5413: [DRAFT] [API/SSO] Use externally-defined (e.g., LDAP) groups as permission groups
We've been talking about asking the LDAP server for groups memberships when a user logs into Arvados (from the SSO, f... Lucas Di Pentima
05:43 pm Revision 5c860fdb (arvados): 11901: Test 404 when health checks disabled.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
05:23 pm Story #9426 (Resolved): [Workbench] Display/Add/Edit/Delete tags on collection front page
Radhika Chippada
03:39 pm Revision 4533d132 (arvados): 3853: Do not try to look up collections by PDH for update/delete.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom@curoverse.com> Tom Clegg
03:36 pm Feature #5036 (New): [arv-mount] Change default mount type from "fuse" to "fuse.arvados"
Tom Clegg
03:29 pm Bug #4048 (Closed): [Crunch] arv-run-pipeline-instance --run-jobs-here cannot run jobs with local...
Tom Clegg
 

Also available in: Atom