Activity

From 05/17/2015 to 06/15/2015

06/15/2015

11:05 pm Bug #5951: GET-Evidence login via google login no longer works - OpenID 2.0 is gone
At commit:get-evidence|bbb1ed9
The regexp in openid_verify_oauth2.php still doesn't look right -- now the initial ...
Tom Clegg
09:40 pm Revision 4db76202 (arvados): 4410: Crunch retries jobs when all SLURM nodes fail.
See the ticket for detailed background discussion and implementation
rationale, especially notes 13 and 14.
Brett Smith
09:25 pm Bug #4410: [Crunch] crunch-job should exit tempfail when a SLURM node fails
4410-slurm-fails-are-tempfails-wip is up for review. One way to test it is to hack crunch-job to have this block at ... Brett Smith
02:35 pm Bug #4410 (In Progress): [Crunch] crunch-job should exit tempfail when a SLURM node fails
Brett Smith
09:20 pm Revision 85c53a99 (arvados): 4410: Crunch retries jobs when all SLURM nodes fail.
See the ticket for detailed background discussion and implementation
rationale, especially notes 13 and 14.
Brett Smith
09:20 pm Revision fbe079ac (arvados): 4410: crunch-dispatch logs crunch-job exit later.
This makes it easier to log the exit code, and makes the logs look
nicer because the exit log doesn't interrupt crunc...
Brett Smith
09:10 pm Story #2256: [Workbench] Users can access an Arvados shell through the Web
some improvement has been done.
I've been researching Shell in a box: https://code.google.com/p/shellinabox/wiki...
Nico César
07:14 pm Story #6277: [API] Make manifest format validation more strict, to make "munge" methods simpler a...
Branch 6277-manifest-validation-api:
Added the check_manifest_validity before_filter in collection model. However,...
Radhika Chippada
07:12 pm Task #6324 (Resolved): Review branch: 6277-manifest-validation-api
Radhika Chippada
07:02 pm Revision 88038b52 (arvados): 6277: Add check_manifest_validity before_filter in collection model;...
Radhika Chippada
06:38 pm Story #6096: [OPS] Implement a process to regularly deploy a Docker image for running jobs to Arv...
I'm checking 2b93735fc87e447301afafe6556d8571afef2bcf
I see that there are 3 commits on 6096-deploy-jobs-image:
2...
Nico César
06:00 pm Story #6218: [SDKs] [Performance] Add performance tests for Python SDK collection methods
See https://arvados.org/issues/3198#note-39 and https://arvados.org/issues/3198#note-40 for some benchmarking notes a... Peter Amstutz
06:00 pm Bug #6219: [FUSE] [Performance] Add performance tests for FUSE
See https://arvados.org/issues/3198#note-39 and https://arvados.org/issues/3198#note-40 for some benchmarking notes a... Peter Amstutz
05:54 pm Story #3198: [FUSE] Writable streaming arv-mount
A couple additional data points:
* I wrote a small script to iterate over the same file using the Python SDK direc...
Peter Amstutz
03:38 pm Story #3198: [FUSE] Writable streaming arv-mount
I used iozone (@apt-get install iozone3@) for performance testing, this uncovered a few bugs and performance issues. ... Peter Amstutz
05:25 pm Task #6119 (In Progress): Review 6096-deploy-jobs-image
Nico César
03:34 pm Revision 912d5c4e (arvados): refs #6277
Merge branch '6277-manifest-validation' Radhika Chippada
03:34 pm Revision e0196b6e (arvados): Merge branch 'master' into 6277-manifest-validation
Radhika Chippada
03:33 pm Revision 771dae6f (arvados): 6277: all that work and missed the basic nil and empty string check!!!
Radhika Chippada
02:50 pm Feature #6254 (Resolved): [Workbench] Support specifying group memberships for shell accounts
Applied in changeset arvados|commit:832486e363889d47adecf8ad898ab9d21384dca3. Radhika Chippada
02:43 pm Feature #6254: [Workbench] Support specifying group memberships for shell accounts
Brett said:
> I think you can say vm_login_link = Link.new(vm_link) to instantiate a proper Link object from the att...
Radhika Chippada
02:23 pm Feature #6254: [Workbench] Support specifying group memberships for shell accounts
commit:d3fc80c is good to merge. One last follow-up:
Radhika Chippada wrote:
> * Brett said:
> > user setup met...
Brett Smith
02:49 pm Task #6297 (Resolved): Review branch: 6254-groups-during-setup
Radhika Chippada
02:49 pm Revision a830b5b5 (arvados): 3198: Performance tuning based on benchmarking. Limit number of seg...
look ahead for prefetch, don't prefetch blocks over and over again. Peter Amstutz
02:45 pm Revision 832486e3 (arvados): closes #6254
Merge branch '6254-groups-during-setup' Radhika Chippada
02:33 pm Support #6266 (In Progress): Support users and triage bugs (first half)
Ward Vandewege
01:51 pm Story #6268 (Resolved): Write sprint summary blog post in first three days of sprint
Brett Smith
01:39 pm May/June development review: Performance across the board
The Arvados development team just concluded another sprint, continuing to build on a lot of "our work from last time"... Brett Smith
01:15 pm Support #6055 (Feedback): [Support] Cannot log into VM
Tom Clegg

06/14/2015

01:31 am Revision 4617a33d (arvados): refs #6277 : ruby sdk with manifest validation method
Merge branch '6277-manifest-validation' Radhika Chippada
01:31 am Task #6287 (Resolved): Review branch: 6277-manifest-validation (ruby sdk updates only)
Radhika Chippada
01:30 am Story #6277: [API] Make manifest format validation more strict, to make "munge" methods simpler a...
Tom, thanks for the updates in the -TC branch and also updating the wiki with more clarity.
I ran all tests and sa...
Radhika Chippada
01:21 am Revision c07144a7 (arvados): 6277: extra white space
Radhika Chippada
12:47 am Revision 248834fb (arvados): Merge branch 'master' into 6277-manifest-validation
Radhika Chippada

06/13/2015

05:29 am Story #6277: [API] Make manifest format validation more strict, to make "munge" methods simpler a...
Doh... From "the Ruby docs for split":http://ruby-doc.org/core-2.2.0/String.html#method-i-split, "If pattern is a sin... Tom Clegg
02:24 am Revision d3fc80cd (arvados): 6254: minor update to Groups text field label to avoid conflict with...
Radhika Chippada
01:34 am Feature #6254: [Workbench] Support specifying group memberships for shell accounts
* Brett said:
> This has a head_uuid of the Arvados VirtualMachine's UUID and a tail_uuid of the user UUID. This is ...
Radhika Chippada
01:19 am Revision db9ef7d6 (arvados): 6254: better groups text field label
Radhika Chippada
01:06 am Revision a2ad62ca (arvados): 6254: instead of tooltip, use a self explanatory label for groups te...
Radhika Chippada
12:45 am Revision 0f696afd (arvados): Fix asset generation in the workbench package. Make sure the build s...
No issue # Ward Vandewege
12:45 am Revision 0f696afd (arvados-dev): Fix asset generation in the workbench package. Make sure the bui...
No issue # Ward Vandewege
12:26 am Revision af88dd5a (arvados): 7254: add groups to vm link.
Radhika Chippada

06/12/2015

08:47 pm Revision 2b93735f (arvados-dev): Do not build and upload gems that already exist on rubygems.
No issue # Ward Vandewege
08:47 pm Revision 6f1a2265 (arvados-dev): Add very rudimentary support for pushing the rpm packages into a...
repository.
No issue #
Ward Vandewege
08:46 pm Story #6320 (Resolved): [OPS] Shell account creation script supports `groups` properties in can_l...
Update the appropriate script in Puppet to respect the @groups@ property specified after #6254. Brett Smith
08:29 pm Feature #6254: [Workbench] Support specifying group memberships for shell accounts
Reviewing 6254-groups-during-setup at commit:cdec4d7
I'm sorry this wasn't spelled out better in the story descrip...
Brett Smith
06:42 pm Feature #6254: [Workbench] Support specifying group memberships for shell accounts
* Updated the setup popup to also include "groups"
** Store these, after a successful setup step, in the can_login l...
Radhika Chippada
01:35 pm Feature #6254 (In Progress): [Workbench] Support specifying group memberships for shell accounts
Radhika Chippada
07:54 pm Revision 05ff1098 (arvados): 3198: Fix frontrunning (subsequent updates after a commit causing the
collection to conflict with itself). Also fix filename character encoding. Peter Amstutz
07:47 pm Task #6297 (In Progress): Review branch: 6254-groups-during-setup
Brett Smith
07:29 pm Bug #6156 (In Progress): [API] Node record accommodates setups where hostnames are set statically...
Radhika Chippada
06:49 pm Revision 591ddfb6 (arvados): Do not build and upload gems that already exist on rubygems.
No issue # Ward Vandewege
06:49 pm Revision 591ddfb6 (arvados-dev): Do not build and upload gems that already exist on rubygems.
No issue # Ward Vandewege
06:30 pm Revision cdec4d7f (arvados): 6254: remove redundant statement
Radhika Chippada
06:26 pm Revision 5b8709e2 (arvados): 6254: add groups and verify in test.
Radhika Chippada
06:10 pm Task #6284 (Resolved): Review 6149-quiet-egg-info-stderr-wip
Applied in changeset arvados|commit:f0a92e384da79e0a17efb42de17031f45f006e44. Brett Smith
04:37 pm Task #6284 (In Progress): Review 6149-quiet-egg-info-stderr-wip
Tom Clegg
06:10 pm Bug #6149 (Resolved): [Crunch] Install step logs error message from `setup.py egg_info`
Applied in changeset arvados|commit:f0a92e384da79e0a17efb42de17031f45f006e44. Brett Smith
05:05 pm Bug #6149: [Crunch] Install step logs error message from `setup.py egg_info`
The only suspicious thing I noticed is that @$? != 0@ does not guarantee @$? >> 8 != 0@, which means there's a possib... Tom Clegg
03:29 pm Bug #6149: [Crunch] Install step logs error message from `setup.py egg_info`
6149-quiet-egg-info-stderr-wip is up for review. It captures egg_info's stderr, and since we're doing that anyway, i... Brett Smith
06:05 pm Revision f0a92e38 (arvados): Merge branch '6149-quiet-egg-info-stderr-wip'
Closes #6149, #6284. Brett Smith
06:05 pm Revision 1658729c (arvados): 6149: crunch-job installer handles egg_info errors better.
Capture the command's stderr and confirm that the error refers to
git. If it does, ignore the stderr and set a build...
Brett Smith
05:53 pm Bug #6318 (New): [SDKs] Go SDK locator regexp (and its tests) should agree with the documented fo...
Incorrect test cases in source:sdk/go/manifest/manifest_test.go that disagree with the documented [[Keep locator form... Tom Clegg
03:37 pm Story #6316 (Rejected): [OPS] run-build-packages publishes RPMs over HTTP
Brett Smith
03:37 pm Story #6315 (Duplicate): [OPS] run-build-packages builds RPMs
Three story points because you must test that the packages are actually installable and functional. Target the curre... Brett Smith
03:25 pm Revision 148f31f9 (arvados): 6149: crunch-job installer handles egg_info errors better.
Capture the command's stderr and confirm that the error refers to
git. If it does, ignore the stderr and set a build...
Brett Smith
03:07 pm Revision c159d1d0 (arvados-dev): Add very rudimentary support for pushing the rpm packages into a...
repository.
No issue #
Ward Vandewege
03:07 pm Revision c159d1d0 (arvados): Add very rudimentary support for pushing the rpm packages into a yum
repository.
No issue #
Ward Vandewege
03:02 pm Revision 33dd05d2 (arvados): 6254: slight performance improvement where the can_login link is not...
Radhika Chippada
02:51 pm Revision c0818b26 (arvados): 6254: add "groups" to user setup process; these comma separated grou...
Radhika Chippada
02:33 pm Revision 27dc0051 (arvados): 3198: Fix tests for change in ~conflict~ format. Tweak comments.
Peter Amstutz
02:11 pm Revision c0654241 (arvados-dev): Add rpm generation support to run-build-packages.sh
No issue # Ward Vandewege
02:11 pm Story #3198: [FUSE] Writable streaming arv-mount
Tom Clegg wrote:
> I think conflict filenames should avoid using ":" characters. They have a tendency to confuse pro...
Peter Amstutz
02:10 pm Revision 38f341f9 (arvados): Add rpm generation support to run-build-packages.sh
No issue # Ward Vandewege
02:10 pm Revision 38f341f9 (arvados-dev): Add rpm generation support to run-build-packages.sh
No issue # Ward Vandewege
12:21 am Story #6277: [API] Make manifest format validation more strict, to make "munge" methods simpler a...
Tom:
* Updated to add more conditions to the manifest format, such as stream names and file names should not conta...
Radhika Chippada
12:13 am Revision b92d4bfa (arvados): 6277: Add more restrictions to manifest format such as cannot start ...
as well as stream names and files names should not contain '//'.
Added the tests provided by Tom during review.
Radhika Chippada

06/11/2015

09:45 pm Task #6314 (In Progress): Crunch1-in-Crunch2 wiki page
Tom Clegg
09:45 pm Task #6314 (Resolved): Crunch1-in-Crunch2 wiki page
Tom Clegg
09:44 pm Task #6296 (In Progress): Review 6272-index-eof
Tom Clegg
08:55 pm Bug #6149 (In Progress): [Crunch] Install step logs error message from `setup.py egg_info`
Brett Smith
08:32 pm Bug #6308 (Feedback): Words in description breaks awkwardly in Public Projects page
Tom Clegg
06:28 pm Bug #6308 (In Progress): Words in description breaks awkwardly in Public Projects page
Tom Clegg
02:50 pm Bug #6308 (Resolved): Words in description breaks awkwardly in Public Projects page
In the public projects page, it would be nice if the words were not cut off when messing with the screen dimensions.
...
Bryan Cosca
08:31 pm Story #6313 (New): [Node Manager] Booting nodes shouldn't satisfy min_nodes
h2. The scenario
* Imagine Node Manager is configured with min_nodes = 1, running on a cluster where a compute nod...
Brett Smith
08:19 pm Revision deba5b86 (arvados): 6277: valid manifest must end with new line.
Radhika Chippada
08:11 pm Revision a0ce38b0 (arvados): 6277: improve error message for missing file tokens.
Radhika Chippada
07:31 pm Story #3198: [FUSE] Writable streaming arv-mount
Tom Clegg wrote:
> Suggest changing "Collection must be writable" error message to "Collection is not writable" or "...
Peter Amstutz
07:22 pm Revision 349e1ee2 (arvados): 3198: Additional fixes based on review feedback.
* Adjust how ~conflict~ files are renamed
* Add finalize() on fuse inodes() to terminate put and get threads.
* Impro...
Peter Amstutz
07:16 pm Story #6277: [API] Make manifest format validation more strict, to make "munge" methods simpler a...
At commit:0c082b7
None of the "valid" test cases are actually valid: they are all missing a trailing newline.
I...
Tom Clegg
07:07 pm Story #6277 (In Progress): [API] Make manifest format validation more strict, to make "munge" met...
Radhika Chippada
06:49 pm Task #6287 (In Progress): Review branch: 6277-manifest-validation (ruby sdk updates only)
Tom Clegg
06:33 pm Revision 0c082b74 (arvados): 6277: add Manifest::valid? method in ruby sdk.
Radhika Chippada
06:01 pm Task #6070 (In Progress): Refactor arv-git-httpd authenticator into a package
Tom Clegg
05:52 pm Feature #6312 (Resolved): [Workbench] Use download service instead of passing Keep data through R...
May require additional work on the download server to support whatever authentication mechanism is needed.
* One pos...
Tom Clegg
05:18 pm Story #6311 (New): [Maybe] [SDKs] Support caching Keep blocks in memcached
We could potentially improve job performance by running memcached on each compute node to store Keep blocks. When a ... Brett Smith
05:16 pm Feature #6310 (New): [FUSE] Support scaling the internal block cache based on number of open files
FUSE's current caching strategy can serve you well if you're reading files sequentially, or a bunch of small files. ... Brett Smith
05:12 pm Revision aff4a730 (arvados): 3198: arvfile.py and collection.py updates based on feedback.
* Add retryable ERROR state to buffer blocks.
* Fix race conditions in testing a bufferblock state before trying to c...
Peter Amstutz
05:04 pm Story #6309 (Resolved): [Crunch] Support a runtime_constraint that specifies how much RAM arv-mou...
For jobs that access many files simultaneously, FUSE's default block cache is usually not sufficient, and will thrash... Brett Smith
04:17 pm Bug #5951: GET-Evidence login via google login no longer works - OpenID 2.0 is gone
At commit:40a50a0
In @lib/openid.php@
It looks like a number of possible errors culminating in a "no idea which...
Tom Clegg
03:18 pm Bug #5288: [Documentation] Write user guide for arv-copy
Free suggestions:
* a couple of worked examples, maybe the simplest case (collection) and the hairiest case (pipelin...
Tom Clegg
03:08 pm Bug #6306: [Workbench] Indicate that job progress/state is out-of-date when showing pipeline inst...
Each pipeline instance has a cache of the last known state of each component job, in addition to the jobs' UUIDs.
...
Tom Clegg
02:26 pm Bug #6306 (New): [Workbench] Indicate that job progress/state is out-of-date when showing pipelin...
h2. summary
When a pipeline fails, it stops updating its own cache of job status. However, some jobs may continue ...
Bryan Cosca
03:06 pm Story #2256: [Workbench] Users can access an Arvados shell through the Web
It seems like the method for doing websockets/TLS -> SSH is for @authorized_keys@ on the login host to include a key ... Peter Amstutz
02:57 pm Bug #6305 (Duplicate): [SDK] arv keep docker argument parsing is too sensitive/wrong
Duplicates #3847. Brett Smith
01:20 am Bug #6305 (Duplicate): [SDK] arv keep docker argument parsing is too sensitive/wrong
arv keep docker is stupid about collecting the project-uuid argument:... Ward Vandewege
02:46 pm Story #6307 (New): [Workbench] Public project listing should exclude subprojects
Some public projects use subprojects for their internal organization. For example, the public Pathomap project on qr... Brett Smith
01:44 pm Story #6268 (In Progress): Write sprint summary blog post in first three days of sprint
Brett Smith
01:43 pm Task #6150 (Closed): Test correct index usage with "explain analyze"
Tom Clegg
11:30 am Task #6115 (Resolved): Review branch: 6057-public-projects-page
Radhika Chippada
11:27 am Task #6113 (Resolved): Review branch: 6203-collection-perf-api
Radhika Chippada
05:58 am Revision 9f2325b6 (arvados): 6277: Catch whitespace errors, and "." and ".." in paths. Rename val...
Tom Clegg
01:30 am Revision af519091 (arvados-dev): Make run-deploy.rb install the correct version of the arvados/jobs
docker image on deploy.
refs #6096
Ward Vandewege
01:13 am Revision f288c6cc (arvados-dev): Store the full git hash of the checkout in the arvados-src packa...
refs #6096 Ward Vandewege
01:13 am Revision f288c6cc (arvados): Store the full git hash of the checkout in the arvados-src package, ...
refs #6096 Ward Vandewege

06/10/2015

09:13 pm Story #2256: [Workbench] Users can access an Arvados shell through the Web
Some alternatives to tty.js to look at:
* GateOne https://github.com/liftoff/GateOne (AGPLv3)
** Designed to be ...
Peter Amstutz
09:09 pm Bug #6303 (Resolved): missing block 2
My fault, some volumes not mounted on keep0. Fixed now. Ward Vandewege
09:08 pm Bug #6303: missing block 2
stderr 2015-06-10 20:11:05 arvados.arvados_fuse[18369] WARNING: Block not found: ee0d6affaa2d1918367a0a875e117bbe+363... Bryan Cosca
08:38 pm Bug #6303 (Resolved): missing block 2
https://cloud.curoverse.com/collections/8371a54ae05c830af92466afaa629f52+85/qr1hi-8i9sb-gg3lf2s3lhlyozb.log.txt
fr...
Bryan Cosca
09:07 pm Bug #6276 (New): [SDKs] Python Keep block-r/w errors should indicate which collection/file was un...
Ward Vandewege
09:05 pm Bug #6276 (Resolved): [SDKs] Python Keep block-r/w errors should indicate which collection/file w...
My fault, some volumes were not mounted properly on keep0. Fixed now. Ward Vandewege
03:49 pm Bug #6276: [SDKs] Python Keep block-r/w errors should indicate which collection/file was unreadab...
It looks like the job still can't find that block...
https://cloud.curoverse.com/collections/dbd2d9192f3e19870e216...
Bryan Cosca
03:45 pm Bug #6276: [SDKs] Python Keep block-r/w errors should indicate which collection/file was unreadab...
forced an arv-copy and it works
arv-copy -f --src su92l --dst qr1hi su92l-4zz18-kk9vpg66od6cmua
su92l-4zz18-kk9vp...
Bryan Cosca
03:26 pm Bug #6276: [SDKs] Python Keep block-r/w errors should indicate which collection/file was unreadab...
arv-copy did not help me here:
arv-get 3514b8e5da0e8d109946bc809b20a78a+5698/ .
Traceback (most recent call last)...
Bryan Cosca
03:25 pm Bug #6276: [SDKs] Python Keep block-r/w errors should indicate which collection/file was unreadab...
I successfully used it here: https://cloud.curoverse.com/jobs/qr1hi-8i9sb-a5pa7ekx6wpavpp on 6/5.
Bryan Cosca
03:14 pm Bug #6276: [SDKs] Python Keep block-r/w errors should indicate which collection/file was unreadab...
It was this collection 3514b8e5da0e8d109946bc809b20a78a+5698
arv-get 3514b8e5da0e8d109946bc809b20a78a+5698/ .
Tra...
Bryan Cosca
02:54 pm Bug #6276 (New): [SDKs] Python Keep block-r/w errors should indicate which collection/file was un...
h2. summmary
Errors like "block not found" provide a block locator, but it's generally not obvious why a process w...
Bryan Cosca
08:53 pm Task #6104 (Resolved): Review
Tom Clegg
08:53 pm Task #5711 (Resolved): Requirements / API for Crunch v2
Tom Clegg
08:53 pm Story #5623 (Resolved): [Crunch] Specify Crunch2 features and APIs
Tom Clegg
08:44 pm Bug #6304 (Resolved): [API] Fix config file race condition and improve test cases for DNS update ...
Node.dns_server_update() in source:services/api/app/models/node.rb is currently vulnerable to the following race (fix... Tom Clegg
08:29 pm Task #6301 (Resolved): Review
Tom Clegg
08:27 pm Task #6300 (Resolved): Review branch: 6234-user-edit-self
Radhika Chippada
08:27 pm Task #6299 (Resolved): Review
Radhika Chippada
08:26 pm Task #6298 (Resolved): Review branch: 6013-cleanup-setup-impl
Radhika Chippada
08:25 pm Task #6297 (Resolved): Review branch: 6254-groups-during-setup
Radhika Chippada
08:24 pm Task #6296 (Resolved): Review 6272-index-eof
Tom Clegg
08:24 pm Task #6295 (Resolved): Review
Tom Clegg
08:24 pm Task #6294 (Resolved): Review 6264-cwl-runner
Peter Amstutz
08:24 pm Task #6293 (Resolved): Review 6218-python-sdk-perf @ commit:d4c6c7c
Tom Clegg
08:24 pm Task #6292 (Resolved): Review branch: 6279-web-shell-client
Radhika Chippada
08:24 pm Task #6291 (Resolved): Review 5989-bug_1
Nico César
08:23 pm Task #6290 (Resolved): Review specification
Peter Amstutz
08:23 pm Task #6289 (Resolved): Review branch: 6156-hostnames-in-nodes
Tom Clegg
08:23 pm Task #6288 (Closed): Review
Nico César
08:23 pm Task #6287 (Resolved): Review branch: 6277-manifest-validation (ruby sdk updates only)
Radhika Chippada
08:22 pm Task #6286 (Closed): Review branch: 5288-arv-copy-documentation
Brett Smith
08:22 pm Task #6285 (Closed): Review branch: 6014-repo-creation-doc
Brett Smith
08:22 pm Task #6284 (Resolved): Review 6149-quiet-egg-info-stderr-wip
Brett Smith
08:22 pm Task #6283 (Resolved): Review 4410-slurm-fails-are-tempfails-wip
Brett Smith
07:17 pm Story #6282 (Resolved): [Crunch] Write stories for implementation of Crunch v2
Brett Smith
07:17 pm Story #6281 (Resolved): [Crunch] Describe how it looks when Crunch v1 jobs run under Crunch v2
Brett Smith
06:57 pm Feature #6280 (Resolved): [SDKs] Go SDK can return an io.Reader for a given collection record and...
Given a collection (as retrieved from the API server via "list" or "get"), the new method (CollectionFileReader?) sho... Tom Clegg
06:49 pm Bug #4410 (New): [Crunch] crunch-job should exit tempfail when a SLURM node fails
Brett Smith
06:43 pm Revision c5dfc8a4 (arvados-dev): CWL script won't fail when there are no documentation changes to...
Peter Amstutz
06:43 pm Revision c5dfc8a4 (arvados): CWL script won't fail when there are no documentation changes to com...
Peter Amstutz
06:35 pm Story #6279 (Resolved): [Workbench] Integrate a Web shell client
* Create a page where a user can connect to an Arvados shell they have access to. Integrate the Web client developed... Brett Smith
05:34 pm Task #6116 (In Progress): Review 5883-oauth2-update at get-evidence.git
Tom Clegg
04:45 pm Support #6055: [Support] Cannot log into VM
Hi Mohammad,
Sorry for the delay - this shell image is not fully functional at the moment, it doesn't know how to ...
Ward Vandewege
04:27 pm Revision 0477856a (arvados): added shell-image back again to the build. I checked on my local mac...
no issue # Nico César
04:24 pm Task #5631 (Resolved): create Jenkins pipeline
Ward Vandewege
03:56 pm Bug #4560: [Documentation] We need a link to arv-copy in user guide.
It looks like we'll likely do #5288 instead. Brett Smith
03:30 pm Bug #5790 (Resolved): [SDKs] arv-copy does not select the correct Docker image to copy
Applied in changeset arvados|commit:679ea52a117414fc33768aac887615f9b701fdf8. Brett Smith
02:53 pm Bug #5790 (In Progress): [SDKs] arv-copy does not select the correct Docker image to copy
Looks like this change broke arv keep docker on 4xphq:... Ward Vandewege
03:24 pm Revision 679ea52a (arvados): 5790: Fix PySDK Docker image listing comparing ints and datetimes.
Closes #5790. Brett Smith
03:18 pm Story #6277 (Resolved): [API] Make manifest format validation more strict, to make "munge" method...
A valid manifest format has three kinds of tokens: stream name, block locator, and file.
The current manifest pars...
Tom Clegg
02:49 pm Revision 84cfa56a (arvados): Set default identity for cwl repo pushes. no issue #
Peter Amstutz
02:49 pm Revision 84cfa56a (arvados-dev): Set default identity for cwl repo pushes. no issue #
Peter Amstutz
02:32 pm Revision 54cfeb7c (arvados): Add instructions to the 'Create standard objects' page to create a c...
refs #6096 Ward Vandewege
02:28 pm Revision ef45c867 (arvados): Since jenkins reuses working directories, handle updating existing g...
common-workflow-language site that will be automatically pushed. no issue # Peter Amstutz
02:28 pm Revision ef45c867 (arvados-dev): Since jenkins reuses working directories, handle updating existi...
common-workflow-language site that will be automatically pushed. no issue # Peter Amstutz
02:24 pm Story #6091 (Resolved): Support users and triage their bugs (second half of sprint)
Radhika Chippada
02:24 pm Story #6092 (Resolved): [TBD] Improve the performance of the worst-performing collections component
Implemented api performance improvements in #6203 Radhika Chippada
02:20 pm Story #6203 (Resolved): [API] [Performance] Optimize time spent on the API server side during a l...
Applied in changeset arvados|commit:1a98070dcdd379ab6effdc3fc4301c9435a137b1. Radhika Chippada
01:22 pm Story #6203: [API] [Performance] Optimize time spent on the API server side during a large collec...
6203-collection-perf-api-TC @ commit:02156d7 Tom Clegg
06:32 am Story #6203: [API] [Performance] Optimize time spent on the API server side during a large collec...
Updates look good. I've pushed a few more fixes on 6203-collection-perf-api-TC. Mostly comments and clarity, but also... Tom Clegg
02:15 pm Revision 1a98070d (arvados): closes #6203
Merge branch '6203-collection-perf-api' Radhika Chippada
02:06 pm Revision db6c09cc (arvados): Merge branch '6203-collection-perf-api-TC' of git.curoverse.com:arva...
Conflicts:
services/api/app/models/collection.rb
Radhika Chippada
01:54 pm Bug #6240: [API] "order" params should be rejected or used (not silently ignored) even when they ...
The easy fix (reject instead of ignoring) looks something like this, in source:services/api/lib/load_param.rb
<pre...
Tom Clegg
01:30 pm Revision f9b617b7 (arvados): 3198: Adjust language of errors to say what the error is instead of ...
input should be. Peter Amstutz
01:10 pm Revision 900b5480 (arvados): Merge branch 'master' into 3198-writable-fuse
Peter Amstutz

06/09/2015

10:28 pm Revision bc5680ed (arvados): 6272: Add EOF marker at end of index response. Error out of data man...
Tom Clegg
08:41 pm Revision fbce5ced (arvados): Automatically build and upload documentation site. no issue #
Peter Amstutz
08:41 pm Revision fbce5ced (arvados-dev): Automatically build and upload documentation site. no issue #
Peter Amstutz
08:35 pm Revision b9bcac3e (arvados-dev): Make the run-deploy.sh script much more general. It can now be u...
deploy any of our clusters after an upgrade of the versions in hiera.
refs #6096
Ward Vandewege
08:35 pm Revision b9bcac3e (arvados): Make the run-deploy.sh script much more general. It can now be used to
deploy any of our clusters after an upgrade of the versions in hiera.
refs #6096
Ward Vandewege
07:16 pm Revision 361e83e8 (arvados): Update arvados in bundle. refs #6203 refs #6277
Tom Clegg
07:12 pm Revision 9bd0cf5b (arvados): Use default word wrap (no mid-word line breaks). refs #6057
Tom Clegg
07:05 pm Bug #6020: [Workbench] The collection information box values for content address and content size...
Also see #4770 Radhika Chippada
07:01 pm Bug #6076: [API] walk api server installations and ensure modified_at for collections is unique +...
The goal here is to provide a safe and easy way for a client to get the next page of results. Note-3 is safe but not ... Tom Clegg
06:48 pm Revision 3d067159 (arvados): qr1hi-automated-performance-suite is failing because the test doesnt...
Nico César
06:38 pm Revision 2f3cb5d4 (arvados): qr1hi-automated-performance-suite is failing because the test doesnt...
no issue # Nico César
06:35 pm Bug #4410: [Crunch] crunch-job should exit tempfail when a SLURM node fails
(Draft) approach to the "retry" vs. "state machine" problem:
* crunch-job exits EX_RETRY_UNLOCKED if it gets a slurm...
Tom Clegg
06:31 pm Revision 3812c3e8 (arvados): Added unit test for protocols in From field.
Misha Zatsman
06:07 pm Revision f7cf309e (arvados): Fixed outdated comments in arvadosclient.Discovery().
Misha Zatsman
06:00 pm Revision 6a59d473 (arvados): Merge branch 'master' into 3408-production-datamanager
Misha Zatsman
05:51 pm Revision 982f7db5 (arvados): Renamed BlockToReplication BlockToDesiredReplication.
Added protocol field to servers in pull list. Misha Zatsman
04:04 pm Bug #6274 (New): [API] [Workbench] Home project should be a real object (group with group_class="...
Implementation
* Add default_project_uuid attribute to users table. Must not be null. Must resolve to a group with g...
Tom Clegg
03:33 pm Bug #6156: [API] Node record accommodates setups where hostnames are set statically by the sysadmin
Add note in application.default.yml that slot_numbers assigned by clients are exempt from max_compute_nodes. Tom Clegg
03:32 pm Revision efc42e74 (arvados): openssl self cert creation failed because missing of "tmp" directory.
api server failed to start because missing "tmp/logs" and "tmp/api" ditectories
no issue #
Nico César
03:24 pm Story #6088 (Resolved): Contribute to CWL specification
Peter Amstutz
03:22 pm Revision 1c03f0c1 (arvados): Merge branch '6235-go-sdk-discovery' closes #6235
Peter Amstutz
03:21 pm Task #6100 (Resolved): Improve draft 2 spec docs
Peter Amstutz
03:10 pm Bug #6272 (Resolved): [Keep] keepstored index response should have an EOF marker to enable the cl...
The response size is not known when the headers are written, so (without an EOF marker) it is impossible for the clie... Tom Clegg
02:55 pm Task #6103 (Resolved): Review 5790-copy-most-recent-docker-image-wip
Applied in changeset arvados|commit:e8ae9364dce380d305833f35dfb25578175664d7. Brett Smith
02:08 pm Task #6103 (In Progress): Review 5790-copy-most-recent-docker-image-wip
Nico César
02:55 pm Bug #5790 (Resolved): [SDKs] arv-copy does not select the correct Docker image to copy
Applied in changeset arvados|commit:e8ae9364dce380d305833f35dfb25578175664d7. Brett Smith
02:41 pm Bug #5790: [SDKs] arv-copy does not select the correct Docker image to copy
I finally checked and tested out.
This is a comparison of the output from master and the branch :...
Nico César
02:48 pm Revision e8ae9364 (arvados): Merge branch '5790-copy-most-recent-docker-image-wip'
Closes #5790, #6103. Brett Smith
02:43 pm Revision 1b7af451 (arvados): 5790: Improve Docker image listing in Python SDK.
* Always fetch all relevant Docker links.
* Support finding images by image hash.
* Show image hashes when listing im...
Brett Smith
02:35 pm Bug #6152 (Resolved): [Docs] Eliminate `arv` dependency in compute node installer
Applied in changeset arvados|commit:456967e1991ea8adc30038b60f5c34703b47b694. Brett Smith
02:19 pm Bug #6152: [Docs] Eliminate `arv` dependency in compute node installer
Confirmed that this works, LGTM. Ward Vandewege
02:29 pm Revision 456967e1 (arvados): Merge branch '6152-compute-node-no-arv-wip'
Closes #6152, #6256. Brett Smith
02:26 pm Revision 6a6cd451 (arvados): 6152: Use Python SDK for compute node installation.
This eliminates an otherwise-needless dependency on `arv` and the
entire Ruby stack.
Brett Smith
02:25 pm Revision a9c1ba7b (arvados): closes #6093
Merge branch '6093-refresh-docs' Radhika Chippada
02:21 pm Revision 18e24e8b (arvados): 6093: remove "Alternate way to add SSH keys" and add the "Manage acc...
Radhika Chippada
02:20 pm Task #6241 (Resolved): review 6096-package-rails-apps
Ward Vandewege
02:19 pm Task #6256 (Resolved): Review 6152-compute-node-no-arv-wip
Ward Vandewege
02:18 pm Task #6256 (In Progress): Review 6152-compute-node-no-arv-wip
Ward Vandewege
02:11 pm Revision bed1ec60 (arvados): Merge branch '6203-locator-regexp' refs #6203 refs #6277
Tom Clegg
02:10 pm Revision 43cfd422 (arvados): 6203: Add tests for LOCATOR_REGEXP. Fix regexp to reject "++" and tr...
Tom Clegg
02:10 pm Revision dad32377 (arvados): 6203: Fix loophole allowing locators in bogus manifests to be accepted
without verification and then signed in responses. refs #6277 Tom Clegg
02:09 pm Revision e56061a9 (arvados): Merge branch 'master' into 6093-refresh-docs
Radhika Chippada
02:03 pm Revision 6268a275 (arvados-dev): Fix links to workbench on run-tests.sh. Also shut up the PyYAML ...
No issue # Ward Vandewege
02:03 pm Revision 6268a275 (arvados): Fix links to workbench on run-tests.sh. Also shut up the PyYAML buil...
No issue # Ward Vandewege
02:33 am Story #6203: [API] [Performance] Optimize time spent on the API server side during a large collec...
Regarding "Review branch: 6203-collection-perf-ruby-sdk" and https://arvados.org/issues/6203#note-5:
I removed thi...
Radhika Chippada
02:29 am Story #6203: [API] [Performance] Optimize time spent on the API server side during a large collec...
Responses to https://arvados.org/issues/6203#note-4
* Addressed the comments in “compute_pdh and caching”
* Upd...
Radhika Chippada
02:23 am Story #6270: [SDKs] [Performance] Ruby SDK performance metrics
See note https://arvados.org/issues/6203#LOCATOR_REGEXP Radhika Chippada
01:59 am Story #6270 (New): [SDKs] [Performance] Ruby SDK performance metrics
AS noted in https://arvados.org/projects/arvados/wiki/Performance_profiling and #6203, there are huge performance gai... Radhika Chippada
02:22 am Story #6271: [SDKs] [Performance] Improve ruby SDK performance
Also address the comments in https://arvados.org/issues/6203#LOCATOR_REGEXP
And checkout the comments in https://a...
Radhika Chippada
02:02 am Story #6271 (New): [SDKs] [Performance] Improve ruby SDK performance
Using the metrics information collection in #6270, implement necessary performance improvements in ruby SDK.
These...
Radhika Chippada
01:42 am Revision f560e8ae (arvados): 6203: Use LOCATION_REGEXP from sdk; also add back updated each_manif...
with the same updates as munge method since this is simpler than the munge method. This
has seems further improved pe...
Radhika Chippada

06/08/2015

07:39 pm Revision 0e93a99c (arvados): 6203: Use manifest.each_line and line.rstrip! instead of manifest.sp...
Performance was comparable in both cases; though each_line itself is twice as fast (we need to do strip, which eats a... Radhika Chippada
07:06 pm Story #6268 (Resolved): Write sprint summary blog post in first three days of sprint
Brett Smith
06:49 pm Bug #6263: [Git] Can't push via arv-git-httpd
Tom Clegg wrote:
> gitlab seems to have addressed this by "patching gitolite":https://github.com/gitlabhq/gitolite/p...
Ward Vandewege
06:45 pm Bug #6263: [Git] Can't push via arv-git-httpd
gitlab seems to have addressed this by "patching gitolite":https://github.com/gitlabhq/gitolite/pull/1. The patch did... Tom Clegg
06:10 pm Bug #6263: [Git] Can't push via arv-git-httpd
Confirmed as a bug when trying to push to a repo checked out via arv-git-httpd Ward Vandewege
06:01 pm Bug #6263 (Resolved): [Git] Can't push via arv-git-httpd
$ git push origin b303
Counting objects: 7, done.
Delta compression using up to 2 threads.
Compressing objects: 1...
Bryan Cosca
06:37 pm Support #6267 (Resolved): Support users and triage bugs (second half)
This includes:
* Be the front-line person to answer questions from the science team and other users
* Resolve bug...
Brett Smith
06:36 pm Support #6266 (Resolved): Support users and triage bugs (first half)
This includes:
* Be the front-line person to answer questions from the science team and other users
* Resolve bug...
Brett Smith
06:34 pm Story #6265 (Resolved): [CWL] Contribute to the CWL specification
Brett Smith
06:32 pm Revision d80be38b (arvados): 6203: further optimization of munge method; also, match[0].sub(/\+A[...
resulted in another 2 seconds saving in each of create and update operations! Radhika Chippada
06:18 pm Story #6264 (Resolved): [CWL] Prototype Arvados CWL pipeline runner
Implement a tool that acts like arv-run-pipeline-instance that accepts CWL workflows as inputs. It creates and monit... Brett Smith
05:38 pm Revision 848ccc7e (arvados-dev): Fix directory when dropping API server database connections.
The Python one-liner is written with the expectation that it's in this
directory, and usually fails to find <config/d...
Brett Smith
05:38 pm Revision 848ccc7e (arvados): Fix directory when dropping API server database connections.
The Python one-liner is written with the expectation that it's in this
directory, and usually fails to find <config/d...
Brett Smith
04:24 pm Revision 5f3df47f (arvados): 6203: compute_pdh, computed_pdh etc etc etc confusion. clean up to m...
Radhika Chippada
04:15 pm Bug #4410: [Crunch] crunch-job should exit tempfail when a SLURM node fails
We recently ran a set of jobs on a cloud-backed site which caused so much node churn that the one "compute node faile... Tom Clegg
03:27 pm Story #6260 (Resolved): [Keep] Integration test between data manager, API and keepstore.
Write one or more integration tests to verify that Data Manager's _existing_ delete functionality (i.e., deletes bloc... Peter Amstutz
03:24 pm Story #3408: [Keep] Implement Production Data Manager
Misha Zatsman wrote:
> Peter Amstutz wrote:
> > Misha Zatsman wrote:
> >
> > > Sounds great, I added it as a tod...
Peter Amstutz
02:26 pm Revision 02156d71 (arvados): 6203: Accept (and discard) hints in client-provided portable_data_hash.
Tom Clegg
02:26 pm Revision a46c4dd4 (arvados): 6203: Remove unused vars. Remove unnecessary newline manipulation.
Tom Clegg
02:10 pm Revision 5c80d297 (arvados): 6203: Accept (and discard) hints in client-provided portable_data_hash.
Tom Clegg
01:55 pm Story #6203: [API] [Performance] Optimize time spent on the API server side during a large collec...
6203-collection-perf-ruby-sdk @ commit:77bf746
h3. each_file_spec
The @splits@ variable is poorly named and nev...
Tom Clegg
03:17 am Story #6203: [API] [Performance] Optimize time spent on the API server side during a large collec...
6203-collection-perf-api @ commit:12afe0e
h3. compute_pdh and caching
I think the new @compute_pdh@ method is d...
Tom Clegg
01:44 pm Feature #6187: [Crunch] Flag to disable job reuse with GATK Queue plugin
Use @-jobNative no-reuse@ when running Queue Peter Amstutz
01:43 pm Feature #6187 (Resolved): [Crunch] Flag to disable job reuse with GATK Queue plugin
Peter Amstutz
01:36 pm Revision 833c4e90 (arvados): 6235: Discovery() returns the requested value directly instead of a ...
Peter Amstutz
07:11 am Revision 7d838bbf (arvados): 6203: Remove unused vars. Remove unnecessary newline manipulation.
Tom Clegg
06:23 am Revision 1cd7fd38 (arvados): 6203: Use faster =~ instead of match.
Tom Clegg
06:23 am Revision ecd7fa01 (arvados): 6203: Fix cheating test.
Tom Clegg
06:23 am Revision b61f9837 (arvados): 6203: Use each_line instead of split.each.
Tom Clegg
06:22 am Revision a39f7077 (arvados): 6203: Remove redundant split before each_line.
Tom Clegg
05:59 am Revision cee6438c (arvados): 6203: Apply special case only to a 0-byte manifest: don't ignore whi...
Tom Clegg
05:59 am Revision 48b208a0 (arvados): 6203: Eliminate unneeded variable.
Tom Clegg
05:59 am Revision a42a2a77 (arvados): 6203: Merge pdh validations into one method. Update comments. Add te...
Tom Clegg
03:18 am Task #6113 (In Progress): Review branch: 6203-collection-perf-api
Tom Clegg

06/07/2015

02:48 am Revision 77bf7469 (arvados): 6203: update arvados gem file version for workbench
Radhika Chippada
02:45 am Revision 363cf392 (arvados): 6203: further performance improvement by replacing Hash usage with Set.
Radhika Chippada

06/06/2015

11:36 pm Revision 12afe0eb (arvados): 6203: Benchmarking revealed that regexp.match(string) is 2.5x more e...
Radhika Chippada
10:59 pm Revision c8a274ec (arvados): Merge branch 'master' into 6203-collection-perf-api
Radhika Chippada
10:43 pm Revision f5ba3202 (arvados): 6203: files_count method performance improved by using split + file ...
time spent executing files_count and files_size method for qr1hi-4zz18-tcnxylwkxg0nfhi is reduced to 4s, from earlier... Radhika Chippada
08:29 pm Revision 3564cd93 (arvados): 6222: Precompile all regexps. Remove wasted effort in GetBlockHandler.
Tom Clegg
05:32 pm Story #6092 (In Progress): [TBD] Improve the performance of the worst-performing collections comp...
Radhika Chippada
04:29 pm Revision 66573d95 (arvados): 6098: Fix up record_filters API. Add (but skip) test that full-text ...
Tom Clegg
12:54 am Revision 4be7ac53 (arvados-dev): contrib: redmine user alias
gitolite tester

06/05/2015

09:08 pm Bug #6234: [Workbench] Admins should be able to edit their own user records, and view other users...
This bug will be trivial to fix if we resolve the "owner_uuid can be a user or a project" design oddity: i.e., change... Tom Clegg
08:52 pm Story #6091 (In Progress): Support users and triage their bugs (second half of sprint)
Ward Vandewege
08:44 pm Story #6096: [OPS] Implement a process to regularly deploy a Docker image for running jobs to Arv...
Cool, merged 6096-package-rails-apps into arvados-dev. Ward Vandewege
08:29 pm Revision 855e16ba (arvados): Merge branch '6096-package-rails-apps'
refs #6096 Ward Vandewege
08:29 pm Revision 855e16ba (arvados-dev): Merge branch '6096-package-rails-apps'
refs #6096 Ward Vandewege
08:29 pm Revision 530468d1 (arvados-dev): Merge branch 'master' into 6096-package-rails-apps
Ward Vandewege
08:29 pm Revision 530468d1 (arvados): Merge branch 'master' into 6096-package-rails-apps
Ward Vandewege
08:22 pm Feature #6191 (In Progress): Propagate stderr from all srun uses (e.g., clean-tmp and docker-inst...
Tom Clegg
08:18 pm Story #6088 (In Progress): Contribute to CWL specification
Peter Amstutz
07:48 pm Task #6256 (Resolved): Review 6152-compute-node-no-arv-wip
Brett Smith
07:46 pm Bug #6152 (In Progress): [Docs] Eliminate `arv` dependency in compute node installer
Brett Smith
07:46 pm Revision 9e9ff16e (arvados): 6152: Use Python SDK for compute node installation.
This eliminates an otherwise-needless dependency on `arv` and the
entire Ruby stack.
Brett Smith
06:58 pm Feature #6254 (Resolved): [Workbench] Support specifying group memberships for shell accounts
h2. Original requirements
We need a way to list the groups a user should be part of on the node they can log in to...
Ward Vandewege
06:54 pm Revision 511daa2d (arvados): bundler 1.10 brakes workbench build because
https://github.com/lucasefe/themes_for_rails/blob/master/themes_for_rails.gemspec
has a bug. And it's a NOTICE in 1....
Nico César
06:40 pm Story #6090 (Resolved): Support users and triage their bugs (first half of sprint)
Ward Vandewege
05:00 pm Revision b33597e9 (arvados): Merge branch '6146-log-squeue-lost-tasks' refs #6146
Tom Clegg
02:55 pm Task #6242 (Closed): Test to make sure it works
Based on feedback from lunch yesterday, this is working as intended. Bryan Cosca
01:51 pm Revision 1a4ea520 (arvados-dev): Fix "docker-node-engine.sh" renamed to "build-node-docker.sh", a...
Peter Amstutz
01:51 pm Revision 1a4ea520 (arvados): Fix "docker-node-engine.sh" renamed to "build-node-docker.sh", add a...
Peter Amstutz

06/04/2015

10:02 pm Bug #6149: [Crunch] Install step logs error message from `setup.py egg_info`
The original report is correct: this doesn't interfere with the running job at all. Crunch "expects" this error, and... Brett Smith
09:58 pm Bug #6153 (Closed): SLURM_JOB_ID environment variable expiration persistence on compute node
On June 2 we made changes to our SLURM configuration that should prevent these kinds of communication mishaps between... Brett Smith
09:38 pm Bug #6094 (Resolved): [Crunch] Install script exited 141
Really fixed in commit:178d3f36. Brett Smith
09:29 pm Task #6246 (Closed): Add test case for dns configs
Tom Clegg
09:29 pm Task #6245 (Closed): Improve race and exception handling in dns updates
Tom Clegg
09:18 pm Story #3408: [Keep] Implement Production Data Manager
Peter Amstutz wrote:
> Misha Zatsman wrote:
>
> > Sounds great, I added it as a todo for when the SDK method is a...
Misha Zatsman
09:11 pm Revision c63ff556 (arvados): Added size to block locators, touching most of the code.
Moved BlockLocator (and associated parsing code) from sdk/go/manifest to sdk/go/blockdigest.
Added some helper method...
Misha Zatsman
09:09 pm Revision 869f3f0a (arvados): 6146: Document how --steps really works. Simplify squeue output form...
Tom Clegg
08:49 pm Revision 9943f1ed (arvados): 6146: Document how --steps really works. Simplify squeue output form...
Tom Clegg
07:46 pm Task #6241: review 6096-package-rails-apps
updated #6096 LGTM Nico César
01:20 pm Task #6241 (Resolved): review 6096-package-rails-apps
Ward Vandewege
07:46 pm Story #6096: [OPS] Implement a process to regularly deploy a Docker image for running jobs to Arv...
I reviewed 381b79bf5cfefe790bdcc24dd33296b8518e4c19 6096-package-rails-apps
looks good to me. Let's merge
Nico César
07:35 pm Bug #6074 (Resolved): [API] collections are being returned too slowly + NoMemoryError
Applied in changeset arvados|commit:a12cf91db3ececed782d9718af30553f8866ad81. Tom Clegg
07:10 pm Bug #6074: [API] collections are being returned too slowly + NoMemoryError
Thanks for the detailed explanation, and for fixing my bugs. commit:602706ef is good to merge. Brett Smith
05:37 am Bug #6074: [API] collections are being returned too slowly + NoMemoryError
Brett Smith wrote:
> This change should help mitigate Misha's concern about slowness, but it doesn't seem to do anyt...
Tom Clegg
07:30 pm Task #6110 (Resolved): Review 6074-collections-index @ 22c8b63
Tom Clegg
07:29 pm Revision a12cf91d (arvados): Merge branch '6074-collections-index' closes #6074
Tom Clegg
06:47 pm Bug #6244 (New): Looking at a collection through workbench via it's PDH does not have the name of...
(all data in this ticket is public)
When viewing https://workbench.qr1hi.arvadosapi.com/collections/20ad3ae515aa57...
Abram Connelly
06:10 pm Revision 5881feb4 (arvados): Merge branch 'master' into 6093-refresh-docs
Radhika Chippada
06:09 pm Revision a341306e (arvados): 6093: Add button-override css to make any buttons added inside the d...
Radhika Chippada
05:43 pm Story #6203: [API] [Performance] Optimize time spent on the API server side during a large collec...
Below are the numbers after I tweaked the above mentioned methods:
Workbench tests: test/integration_performance/c...
Radhika Chippada
04:51 pm Revision b39d4108 (arvados): 6203: Corrected one dumb switched order of if conditions that caused...
Radhika Chippada
04:40 pm Revision 381b79bf (arvados): Be sure to include the pre-generated assets (tmp/cache/assets) in the
workbench packages.
refs #6096
Ward Vandewege
04:40 pm Revision 381b79bf (arvados-dev): Be sure to include the pre-generated assets (tmp/cache/assets) i...
workbench packages.
refs #6096
Ward Vandewege
04:16 pm Revision 85bdb48b (arvados): Merge branch 'master' into 6203-collection-perf-api
Radhika Chippada
03:21 pm Revision 7cd190ac (arvados): 6087: Fix MissingAttribute check, and change it to a debug warning f...
because it reveals too many bugs. refs #6087 Tom Clegg
03:18 pm Task #6242: Test to make sure it works
So that made it actually run, I tried with my magic parameter of 1 : qr1hi-d1hrv-vilqlsxko3em3js and a magic paramete... Bryan Cosca
02:37 pm Task #6242: Test to make sure it works
Did not work: See qr1hi-d1hrv-nwdcxnawf854k2b and try hitting re-run with latest. I'm guessing this is now because ar... Bryan Cosca
02:17 pm Task #6242: Test to make sure it works
Testing with a small sample and a scatter of 3 here: qr1hi-d1hrv-z6afo1m08lhjfrc. Once this completes then I'll run a... Bryan Cosca
01:34 pm Task #6242 (In Progress): Test to make sure it works
Peter Amstutz
01:34 pm Task #6242 (Closed): Test to make sure it works
Peter Amstutz
02:40 pm Revision 301f4e3a (arvados-dev): Name change for the arvados-api-server and arvados-workbench pac...
refs #6096 Ward Vandewege
02:40 pm Revision 301f4e3a (arvados): Name change for the arvados-api-server and arvados-workbench package...
refs #6096 Ward Vandewege
02:38 pm Revision 0d540b1f (arvados): 6087: Fix MissingAttribute firing for new records during changes_app...
Tom Clegg
02:27 pm Revision fb049c7f (arvados): Merge branch 'master' into 6203-collection-perf-api
Conflicts:
apps/workbench/test/integration_performance/collection_unit_test.rb
apps/workbench/test/integration_perf...
Radhika Chippada
02:16 pm Revision 05ee30be (arvados-dev): Name change for the arvados-api-server and arvados-workbench pac...
arvados-api-server
arvados-workbench
and the bigger version of the package that includes vendor/bundle is called
...
Ward Vandewege
02:16 pm Revision 05ee30be (arvados): Name change for the arvados-api-server and arvados-workbench package...
arvados-api-server
arvados-workbench
and the bigger version of the package that includes vendor/bundle is called
...
Ward Vandewege
01:37 pm Revision b193c499 (arvados): Merge branch '6087-collection-timing' closes #6087
Tom Clegg
01:33 pm Task #6210 (Resolved): Review 6194-python-arvfile-large-write
Peter Amstutz
01:20 pm Task #6236 (Resolved): create debian package for API server
Ward Vandewege
01:20 pm Task #6237 (Resolved): create debian package for Workbench
Ward Vandewege
12:56 pm Revision ebdaedbc (arvados): 6087: If attributes are accessed but not loaded due to select(), rai...
Tom Clegg
12:56 pm Revision 24e15ff4 (arvados): 6087: Strengthen "manifest_text is not lost in update" test.
Tom Clegg
12:56 pm Revision 4e27d97b (arvados): 6087: Add big-manifest tests, with some finer-grained performance nu...
Tom Clegg
12:56 pm Revision 375e0f68 (arvados): 6087: Remove unneeded CollectionsController#update special case.
ArvadosBase#save now covers the general case of omitting unchanged attributes. Tom Clegg
12:56 pm Revision 55f5486d (arvados): 6087: Reset changed-attrs list after saving. Fix only-send-changed-a...
Tom Clegg
05:12 am Bug #6240: [API] "order" params should be rejected or used (not silently ignored) even when they ...
Noticed during #6074. Tom Clegg
05:07 am Bug #6240 (New): [API] "order" params should be rejected or used (not silently ignored) even when...
Works:... Tom Clegg
04:52 am Revision 602706ef (arvados): 6074: Use each instead of find_each, so our order() and limit() cons...
According to http://apidock.com/rails/ActiveRecord/Batches/find_each,
both order and limit are ignored.
The existing...
Tom Clegg
03:57 am Revision 01cc6184 (arvados): 6203: add trailing newline character in munge_manifest_locators meth...
Radhika Chippada
03:41 am Revision 39a1340d (arvados): 6074: Update config docs to match new max_index_database_read behavior.
Tom Clegg
03:13 am Revision 5263be21 (arvados): 6203: Merge strip_manifest_text and maybe_clear_replication_confirme...
Radhika Chippada

06/03/2015

09:45 pm Bug #6074: [API] collections are being returned too slowly + NoMemoryError
Reviewing commit:22c8b63. To be clear upfront: this is a worthwhile change, and the code is good to merge. My point... Brett Smith
07:54 pm Bug #6074: [API] collections are being returned too slowly + NoMemoryError
At least part of the slowness is fixed by using octet_length() instead of length() in ApplicationController#limit_dat... Tom Clegg
09:18 pm Task #6110 (In Progress): Review 6074-collections-index @ 22c8b63
Brett Smith
09:15 pm Story #3408: [Keep] Implement Production Data Manager
Misha Zatsman wrote:
> Sounds great, I added it as a todo for when the SDK method is added.
Can you review the ...
Peter Amstutz
09:10 pm Revision 0dd51747 (arvados-dev): Simplified run-deploy.sh now that we have moved away from Capist...
refs #6096 Ward Vandewege
09:10 pm Revision 0dd51747 (arvados): Simplified run-deploy.sh now that we have moved away from Capistrano.
refs #6096 Ward Vandewege
09:08 pm Bug #5790 (In Progress): [SDKs] arv-copy does not select the correct Docker image to copy
Brett Smith
09:08 pm Bug #5790: [SDKs] arv-copy does not select the correct Docker image to copy
5790-copy-most-recent-docker-image-wip is up for review. It ends up revamping the Docker image list function in arv-... Brett Smith
09:02 pm Task #6107 (Resolved): Fix Docker image search/list
Brett Smith
09:01 pm Revision 72c13a9f (arvados): 5790: Improve Docker image listing in Python SDK.
* Always fetch all relevant Docker links.
* Support finding images by image hash.
* Show image hashes when listing im...
Brett Smith
08:45 pm Revision 626ea7db (arvados-dev): Default BUILD_BUNDLE_PACKAGES to false.
refs #6096 Ward Vandewege
08:45 pm Revision 626ea7db (arvados): Default BUILD_BUNDLE_PACKAGES to false.
refs #6096 Ward Vandewege
08:40 pm Revision 7761d1a6 (arvados): Fix license names in API and workbench packages.
refs #6096 Ward Vandewege
08:40 pm Revision 7761d1a6 (arvados-dev): Fix license names in API and workbench packages.
refs #6096 Ward Vandewege
08:40 pm Task #6123 (Resolved): Determine why still NoMemoryError even with API memory-limiting feature
Tom Clegg
08:08 pm Task #6123 (In Progress): Determine why still NoMemoryError even with API memory-limiting feature
Tom Clegg
08:40 pm Revision c9e40c6e (arvados): 6235: Add method to get parameters from API discovery document.
Peter Amstutz
08:37 pm Revision 22c8b636 (arvados): 6074: Never exceed the configured max_index_database_read (even by one
record) unless necessary to return one row.
Fix copy&paste error in test case.
Tom Clegg
08:28 pm Revision de8324b3 (arvados): 6074: Clear any existing ActiveRecord select() before adding our own,
otherwise we'll read all the big values when we're really just trying
to predict the result size.
Tom Clegg
08:27 pm Revision 6c26aa2c (arvados-dev): Build API server and Workbench debian packages.
Refs #6096 Ward Vandewege
08:27 pm Revision 6c26aa2c (arvados): Build API server and Workbench debian packages.
Refs #6096 Ward Vandewege
08:24 pm Task #6237 (In Progress): create debian package for Workbench
Ward Vandewege
08:24 pm Task #6237 (Resolved): create debian package for Workbench
Ward Vandewege
08:24 pm Task #6236 (In Progress): create debian package for API server
Ward Vandewege
08:23 pm Task #6236 (Resolved): create debian package for API server
Ward Vandewege
07:39 pm Revision 69f592e0 (arvados): 6074: Speed up db query by using octet_length() instead of length()....
Tom Clegg
07:19 pm Bug #6234 (Resolved): [Workbench] Admins should be able to edit their own user records, and view ...
Admins need to be able to see their own user page to manage their group memberships, etc.
Right now, going to the ...
Ward Vandewege
07:11 pm Revision b6283bc4 (arvados): Merge branch 'master' into 6203-collection-perf-api
Radhika Chippada
07:08 pm Revision 49f6fea2 (arvados): 6203: Do not use Keep::Locator.parse to parse locator in some of the...
Radhika Chippada
05:37 pm Revision ab90543d (arvados): 6087: Remove unneeded CollectionsController#update special case.
ArvadosBase#save now covers the general case of omitting unchanged attributes. Tom Clegg
05:37 pm Revision 14c74eb1 (arvados): 6087: Reset changed-attrs list after saving. Fix only-send-changed-a...
Tom Clegg
04:06 pm Revision 2f7a97d6 (arvados): 6087: Remove unneeded CollectionsController#update special case.
ArvadosBase#save now covers the general case of omitting unchanged attributes. Tom Clegg
04:06 pm Revision 0fb683f7 (arvados): 6087: Reset changed-attrs list after saving. Fix only-send-changed-a...
Tom Clegg
04:04 pm Revision f4c41070 (arvados): 6087: Strengthen "manifest_text is not lost in update" test.
Tom Clegg
04:01 pm Revision 9a9e910f (arvados): 6087: Add big-manifest tests, with some finer-grained performance nu...
Tom Clegg
04:01 pm Revision 95a1e7bb (arvados): 6087: Use app-configured key by default for blob signing and verific...
Tom Clegg
04:01 pm Revision 4004979f (arvados): 6087: Use HTTPClient's compression feature (instead of adding the
Content-Encoding header ourselves). Rename config knob to describe
purpose instead of implementation.
Tom Clegg
04:01 pm Revision c9f03478 (arvados): 6087: Compute portable_data_hash only once during check_signatures.
Tom Clegg
01:50 pm Revision c4c8977e (arvados): Merge branch '6146-log-squeue-lost-tasks' refs #6146
Tom Clegg
01:50 pm Revision d9ab8c81 (arvados): 6146: Use new SLURM_JOB_ID env var instead of old SLURM_JOBID
Tom Clegg
01:50 pm Revision 1da9a2a6 (arvados): 6146: Improvements to "kill srun process if slurm task disappears" f...
* Log when we notice a process is orphaned.
* Log when we decide to kill an orphaned process.
* Use `squeue --jobs ...
Tom Clegg
01:50 pm Revision de1e5fd5 (arvados): 6146: Better log message.
Tom Clegg
01:40 pm Feature #6187 (In Progress): [Crunch] Flag to disable job reuse with GATK Queue plugin
Peter Amstutz
02:26 am Revision 622a60e8 (arvados): Merge branch 'master' into 6203-collection-perf-api
Radhika Chippada
02:19 am Revision ac4a24f9 (arvados): refs #6093
Merge branch '6093-refresh-docs' Radhika Chippada
02:17 am Revision 214ad0d5 (arvados): 6093: delete the redundant details in "alternate way to add ssh keys...
Radhika Chippada
02:08 am Revision b5da9565 (arvados): Merge branch 'master' into 6093-refresh-docs
Radhika Chippada
12:24 am Task #6052 (Closed): Review branch: 5930-smalldocfix
Nancy Ouyang
12:15 am Bug #5930 (Resolved): [Documentation] Misc. documentation fixes
Applied in changeset arvados|commit:78ddad37d72c6c3a728530dc6932fb91f7d81b87. Anonymous
12:11 am Bug #5930 (In Progress): [Documentation] Misc. documentation fixes
Nancy Ouyang
12:10 am Bug #5930 (Closed): [Documentation] Misc. documentation fixes
Nancy Ouyang
12:10 am Bug #5930: [Documentation] Misc. documentation fixes
Changed as per Radhika's comments and merged to master. Nancy Ouyang
12:09 am Revision 78ddad37 (arvados): closes #5930. Merge branch '5930-smalldocfix'
Nancy Ouyang
12:08 am Revision 7ca02089 (arvados): 5930: fixed as per code review
Nancy Ouyang

06/02/2015

10:42 pm Revision 47d1bdc0 (arvados): Added string to error message to help with debugging.
Misha Zatsman
08:25 pm Bug #6194 (Resolved): [SDK] Support writing more than 64MiB at a time in Python SDK
Applied in changeset arvados|commit:4b3b3064b87a07b2ba8035dd5c8f3660dd3b2a67. Peter Amstutz
08:16 pm Bug #6194: [SDK] Support writing more than 64MiB at a time in Python SDK
Tom Clegg wrote:
> Peter Amstutz wrote:
> > I changed the test to do two writes:
> >
> > [...]
>
> Hm. The new...
Peter Amstutz
08:12 pm Bug #6194: [SDK] Support writing more than 64MiB at a time in Python SDK
Sorry, cancel that... misread the MockApi as being the initial content.
The boilerplate assertions still look kind...
Tom Clegg
07:50 pm Bug #6194: [SDK] Support writing more than 64MiB at a time in Python SDK
Peter Amstutz wrote:
> I changed the test to do two writes:
>
> [...]
Hm. The new test supplies a non-zero offs...
Tom Clegg
05:42 pm Bug #6194: [SDK] Support writing more than 64MiB at a time in Python SDK
Simplified the write loop. Thanks.
I changed the test to do two writes:...
Peter Amstutz
08:23 pm Revision 4b3b3064 (arvados): Merge branch '6194-python-arvfile-large-write' closes #6194
Peter Amstutz
08:17 pm Revision 7a0a91fa (arvados): 6194: Simplify test_large_write a little bit.
Peter Amstutz
07:48 pm Feature #5645: [Workbench] Social media optimize anonymous collections/projects/pipelines
From the sprint review a week ago: we could also consider adding either user-upload(able) "thumbnail" images to the c... Nancy Ouyang
07:44 pm Bug #5946: [Workbench] "other collections have the same content" page is confusing
(from latest sprint input) We could just add a sentence, such as "this is a disambiguation page" and maybe a brief me... Nancy Ouyang
07:39 pm Revision a0ee198f (arvados): Merge branch 'master' into 6093-refresh-docs
Radhika Chippada
07:39 pm Bug #5947: [Workbench] Provide additional guidance to users running their first pipeline
Ah, the idea is to limit the pipelines available to match the ones we use in our in-depth step-by-step tutorials, by ... Nancy Ouyang
07:23 pm Revision d06ffd22 (arvados): 6087: Reset changed-attrs list after saving. Fix only-send-changed-a...
Tom Clegg
07:14 pm Task #6190 (Resolved): Improve schema
Peter Amstutz
07:14 pm Task #6097 (Resolved): Implement "id" loading
Peter Amstutz
07:14 pm Task #6189 (Resolved): Fix command line input binding ordering
Peter Amstutz
07:10 pm Revision a6471880 (arvados): 6087: Reset changed-attrs list after saving. Fix only-send-changed-a...
Tom Clegg
07:10 pm Revision 9b04f22c (arvados): 6087: Add big-manifest tests, with some finer-grained performance nu...
Tom Clegg
07:10 pm Revision 249f231c (arvados): 6087: Compute portable_data_hash only once during check_signatures.
Tom Clegg
07:10 pm Revision bd76ea1e (arvados): 6087: Use app-configured key by default for blob signing and verific...
Tom Clegg
07:10 pm Revision 00d96755 (arvados): 6087: Use HTTPClient's compression feature (instead of adding the
Content-Encoding header ourselves). Rename config knob to describe
purpose instead of implementation.
Tom Clegg
05:44 pm Task #6102 (Resolved): IRC support
Peter Amstutz
05:44 pm Task #6104 (In Progress): Review
Peter Amstutz
05:32 pm Revision baac22ee (arvados): 6194: Make splitting loop simpler since [n:n+KEEP_BLOCK_SIZE] return...
slice when there isn't KEEP_BLOCK_SIZE data. Update test. Peter Amstutz
05:23 pm Task #5631: create Jenkins pipeline
check if it makes sense to just close this... Nico César
05:22 pm Story #5548 (Resolved): [CWL/OPS] Jenkins runs cwltool tests
... Nico César
05:20 pm Bug #6221 (Resolved): [Data Manager] implement Delete
By design, Data Manager deletes unneeded blocks by sending "trash lists" to the keepstore servers.
Currently, Data...
Ward Vandewege
03:12 pm Feature #6220 (New): [Node Manager + Images] Optimization for local SSD drives in GCE
looking at https://cloud.google.com/compute/docs/disks/local-ssd
hey use a interface=nvme parameter. (and a differ...
Nico César
02:46 pm Bug #6219 (Resolved): [FUSE] [Performance] Add performance tests for FUSE
Short version: implement a separate test suite that reports timing profiles. Eventually we'll have comprehensive per... Brett Smith
02:45 pm Story #6218 (Resolved): [SDKs] [Performance] Add performance tests for Python SDK collection methods
Short version: implement a separate test suite that reports timing profiles. Eventually we'll have performance tests... Brett Smith
01:44 pm Revision 96069fd4 (arvados): Merge branch 'master' into 6087-collection-timing
Radhika Chippada
01:41 pm Story #6203 (In Progress): [API] [Performance] Optimize time spent on the API server side during ...
Radhika Chippada
08:38 am Story #6092: [TBD] Improve the performance of the worst-performing collections component
Radhika Chippada wrote:
> Based on the extensive amount of profiling of collection performance, it appears that belo...
Brett Smith
02:19 am Story #6092: [TBD] Improve the performance of the worst-performing collections component
Based on the extensive amount of profiling of collection performance, it appears that below are some of the big areas... Radhika Chippada
07:01 am Story #3408: [Keep] Implement Production Data Manager
Peter Amstutz wrote:
> Reviewing commit:5824ee2
Thanks!
> * "DefaultReplicationLevel" should come from the "de...
Misha Zatsman
12:26 am Revision d7635fbe (arvados): Changes in response to code review.
Created DataFetcher type so that we can abstract whether our data is read from remote servers or local files.
Moved c...
Misha Zatsman

06/01/2015

09:15 pm Story #6096: [OPS] Implement a process to regularly deploy a Docker image for running jobs to Arv...
docker images now HAVE tags.
see: https://registry.hub.docker.com/u/arvados/jobs/tags/manage/
you can already...
Nico César
09:14 pm Bug #6194: [SDK] Support writing more than 64MiB at a time in Python SDK
commit:c766dd2
It looks like the second condition will always be True:...
Tom Clegg
09:00 pm Task #6210 (In Progress): Review 6194-python-arvfile-large-write
Tom Clegg
08:56 pm Task #6210 (Resolved): Review 6194-python-arvfile-large-write
Peter Amstutz
08:57 pm Revision c766dd23 (arvados): Merge branch 'master' into 6194-python-arvfile-large-write
Peter Amstutz
08:56 pm Revision f645742a (arvados): 6194: Fix test. Lots of small writes break across blocks differentl...
one. Peter Amstutz
08:24 pm Task #5631 (In Progress): create Jenkins pipeline
The first step here was to create a jenkins job called
https://ci.curoverse.com/job/developer-test-job/
If thi...
Nico César
02:05 pm Revision 11af4c66 (arvados): Merge branch 'master' into 6087-collection-timing
Radhika Chippada
02:00 pm Story #6203 (Resolved): [API] [Performance] Optimize time spent on the API server side during a l...
See https://arvados.org/issues/6087#note-10 for complete detail.
It seems we can reduce several seconds in the ove...
Radhika Chippada
12:51 pm Revision 52e32bbe (arvados): 6194: Fix typo in invocation of writeto() and use memoryview to avoi...
Peter Amstutz

05/31/2015

12:39 pm Revision 469356d1 (arvados): Remove non-existent migration from structure.sql. refs #3036
Tom Clegg
12:38 pm Revision 121625ab (arvados): Update example dns_server_update_command. refs #6146
Tom Clegg
12:29 pm Revision 76e42f16 (arvados): Merge branch '6146-dns-update-command' refs #6146
Tom Clegg
12:23 pm Revision c6fe632f (arvados): 6146: Add dns_server_update_command. Update docs & tests for DNS upd...
Tom Clegg
12:18 pm Revision 7ed3ab7a (arvados): 6146: Add dns_server_update_command. Update docs & tests for DNS upd...
Tom Clegg
12:07 pm Revision a8008a0e (arvados): 6146: Add dns_server_update_command. Update docs & tests for DNS upd...
Tom Clegg
09:48 am Revision b43c667b (arvados): 6146: Better log message.
Tom Clegg
09:32 am Revision 6fefc722 (arvados): 6146: Use new SLURM_JOB_ID env var instead of old SLURM_JOBID
Tom Clegg
06:02 am Revision a1b5da5e (arvados): 6146: Improvements to "kill srun process if slurm task disappears" f...
* Log when we notice a process is orphaned.
* Log when we decide to kill an orphaned process.
* Use `squeue --jobs ...
Tom Clegg
05:18 am Revision fc647306 (arvados): 6146: Improvements to "kill srun process if slurm task disappears" f...
* Log when we notice a process is orphaned.
* Log when we decide to kill an orphaned process.
* Use `squeue --jobs ...
Tom Clegg
01:35 am Revision 2b918fc3 (arvados-dev): Alias.pm learns a few more tricks...
(thanks to Nicholas Clark for the first requirement)
- allow replacement by "prefix"
- allow customised warning ...
Sitaram Chamarty

05/30/2015

02:01 am Revision 178d3f36 (arvados): Tell tar to read to EOF (even if it detects trailing NULs).
Avoids SIGPIPE when feeding a tarball made with tar -A.
refs #6146 refs #6094
Tom Clegg
01:23 am Revision 512cc931 (arvados): Use --blocking-factor 1 with tar -A to reduce excess trailing paddin...
Tom Clegg

05/29/2015

08:28 pm Revision 299c01cb (arvados): 6194: Chunk large ArvadosFile writes automatically instead of raisin...
Peter Amstutz
08:26 pm Bug #6194 (In Progress): [SDK] Support writing more than 64MiB at a time in Python SDK
Peter Amstutz
08:13 pm Bug #6194 (Resolved): [SDK] Support writing more than 64MiB at a time in Python SDK
Peter Amstutz
08:20 pm Bug #6098 (Resolved): [API] slow search? SQL query
Ward later confirmed that the note #11 is quoting an incorrect (old) log snippet and search is working much faster now. Radhika Chippada
06:11 pm Feature #6191 (Resolved): Propagate stderr from all srun uses (e.g., clean-tmp and docker-install...
Tom Clegg
06:09 pm Bug #6141 (Resolved): [Docs] Don't hardcode https:// in links to workbench
Peter Amstutz
06:08 pm Task #6137 (Resolved): Fix documentation links in docker build
Peter Amstutz
06:08 pm Task #6155 (Resolved): Review 6141-doc-workbench-links
Peter Amstutz
06:04 pm Revision 9413eb73 (arvados): Merge branch '6146-ignore-tar-sigpipe' refs #6146 refs #6094
Tom Clegg
05:32 pm Revision b25916b9 (arvados): 6146: Ignore SIGPIPE while feeding data to tar. Rely on close() retv...
Tom Clegg
05:25 pm Task #6100 (In Progress): Improve draft 2 spec docs
Peter Amstutz
05:25 pm Task #6190 (In Progress): Improve schema
Peter Amstutz
05:24 pm Task #6190 (Resolved): Improve schema
Peter Amstutz
05:25 pm Task #6189 (In Progress): Fix command line input binding ordering
Peter Amstutz
05:24 pm Task #6189 (Resolved): Fix command line input binding ordering
Peter Amstutz
04:35 pm Revision 90fd1991 (arvados): 6146: Emit a log message if we get SIGPIPE while extracting our git ...
Tom Clegg
04:02 pm Revision 1ec1d552 (arvados): In install script, log archive hash before running tar. refs #6146
Tom Clegg
03:42 pm Feature #6187 (Resolved): [Crunch] Flag to disable job reuse with GATK Queue plugin
Peter Amstutz
03:31 pm Revision 1e0d7703 (arvados): Merge branch '6146-job-runtime-sanity' refs #6146
Tom Clegg
03:02 pm Story #6186 (New): [API] [Performance] Improve "destroy" implementation
The destroy method in API -> application_controller.rb is as follows: ... Radhika Chippada

05/28/2015

09:13 pm Revision 5e0dd2c6 (arvados): 6146: Exit TEMPFAIL early (without failing the job) if worker nodes ...
This is meant to improve the way we handle a couple of edge cases.
1. A worker node doesn't get bootstrapped properl...
Tom Clegg
08:16 pm Revision 36dd29f8 (arvados): 6146: Exit TEMPFAIL early (without failing the job) if worker nodes ...
This is meant to improve the way we handle a couple of edge cases.
1. A worker node doesn't get bootstrapped properl...
Tom Clegg
08:11 pm Story #6089 (Resolved): Write development update blog (first three days of sprint)
Brett Smith
08:05 pm May development review: Project promotion and Keep performance
We want Arvados to be the premier way for bioinformaticians and data scientists to collaborate. And as an open sourc... Brett Smith
06:53 pm Revision 8884a9ff (arvados): Merge branch 'master' into 6087-collection-timing
Radhika Chippada
05:08 pm Bug #6175 (New): [Workbench] Add social sharing buttons to public projects [and other data?]
You know, for Twitter, Facebook, Google+, etc. Brett Smith
02:57 pm Bug #6174 (New): [Workbench] Propagate old search engine meta tags in sync with Open Graph data
We set some very generic meta tags in the header of all Workbench pages. For a more consistent appearance across sea... Brett Smith
02:54 pm Bug #6173 (New): [Workbench] Make public data crawlable
Right now our application layout sets the noindex and nofollow meta tags on every page, telling search engines to not... Brett Smith
02:48 pm Bug #6172 (New): [Workbench] Breadcrumbs can occupy multiple rows when the screen is narrow
This looks bad. If your window actually is small, it can cause the breadcrumbs to take up too much space. Make sure... Brett Smith
02:44 pm Bug #3588 (Duplicate): [Workbench] When looking at a project shared with you (but not owned by yo...
Duplicates #3573. Brett Smith
02:37 pm Story #6171 (Resolved): [FUSE] Document the writable FUSE mount
In an appropriate place in the user documentation, introduce the writable FUSE mount. Discuss how it can be used in ... Brett Smith
02:00 am Bug #6141: [Docs] Don't hardcode https:// in links to workbench
Radhika Chippada wrote:
> Review comments: LGTM
>
> Please note that doc.arvados.org probably needs to update the...
Ward Vandewege

05/27/2015

09:34 pm Bug #6098: [API] slow search? SQL query
After deploying this evening, still slow (though the api server was under very heavy load at the time):... Ward Vandewege
12:58 pm Bug #6098: [API] slow search? SQL query
LGTM
Just one additional comment request: can you please add a comment to the effect "check if search indexes are ...
Radhika Chippada
08:24 pm Story #6089 (In Progress): Write development update blog (first three days of sprint)
Brett Smith
08:10 pm Revision 7a53d874 (arvados): Merge branch '6146-retry-install' refs #6146
Tom Clegg
07:48 pm Revision 1b5c30eb (arvados): 6146: Retry install (max 3 attempts) if install script fails with no...
Also: if install fails, croak() instead of exit(1) so we still get a log file. Tom Clegg
07:38 pm Revision fa9fc5ab (arvados): Merge branch 'master' into 6093-refresh-docs
Conflicts:
doc/user/tutorials/tutorial-submit-job.html.textile.liquid
Radhika Chippada
07:35 pm Story #6057 (Resolved): [Workbench] Provide an attractive index of public projects
Applied in changeset arvados|commit:32038d56fbfe5b635b1c53f247aea9abcca2285c. Radhika Chippada
03:45 pm Story #6057: [Workbench] Provide an attractive index of public projects
At commit:a2eb98f
In the new controller tests, the first assertion is superfluous since it's a strict subset of th...
Tom Clegg
07:28 pm Task #6158 (Resolved): Review 5548-test-cwltool in arvados-dev
Peter Amstutz
05:50 pm Task #6158 (In Progress): Review 5548-test-cwltool in arvados-dev
Nico César
04:31 pm Task #6158 (Resolved): Review 5548-test-cwltool in arvados-dev
Peter Amstutz
07:27 pm Revision 32038d56 (arvados): closes #6057
Merge branch '6057-public-projects-page' Radhika Chippada
07:26 pm Revision c51a3888 (arvados): 6057: few more minor tweaks
Radhika Chippada
07:22 pm Revision a73f5150 (arvados): 6146: Retry install (max 3 attempts) if install script fails with no...
Also: if install fails, croak() instead of exit(1) so we still get a log file. Tom Clegg
07:11 pm Revision 3101fea5 (arvados): Merge branch '6141-doc-workbench-links' refs #6141
Peter Amstutz
07:03 pm Revision 6fed8498 (arvados): Merge branch 'master' into 6057-public-projects-page
Radhika Chippada
07:03 pm Bug #6141: [Docs] Don't hardcode https:// in links to workbench
Review comments: LGTM
Please note that doc.arvados.org probably needs to update the *arvados_workbench_host* attri...
Radhika Chippada
07:00 pm Task #6155 (In Progress): Review 6141-doc-workbench-links
Radhika Chippada
02:28 pm Task #6155 (Resolved): Review 6141-doc-workbench-links
Peter Amstutz
06:40 pm Revision 44eab398 (arvados-dev): Merge branch '5548-test-cwltool' refs #5548
Peter Amstutz
06:40 pm Revision 44eab398 (arvados): Merge branch '5548-test-cwltool' refs #5548
Peter Amstutz
06:31 pm Task #5629 (Resolved): Write test script
Peter Amstutz
04:30 pm Task #5629 (In Progress): Write test script
Peter Amstutz
06:25 pm Story #6138 (Resolved): [Docker] Docker install uses standalone SSO server
Applied in changeset arvados|commit:e73222a8d0c18b159ae3d8b53b54474650bdda16. Peter Amstutz
06:19 pm Story #6138: [Docker] Docker install uses standalone SSO server
checked a7d558d6c2db242a6555c7f397cb4d91618aa13c
looks good to me
Nico César
06:23 pm Task #6128 (Resolved): Test Docker install
Peter Amstutz
06:23 pm Task #6133 (Resolved): Review 6090-docker-use-local-sso
Peter Amstutz
06:23 pm Revision e73222a8 (arvados): Merge branch '6090-docker-use-local-sso' closes #6138
Peter Amstutz
06:18 pm Revision a572ef64 (arvados): 5548: Remove git commands from run-cwl-tests. Remove cwltool from b...
Peter Amstutz
06:18 pm Revision a572ef64 (arvados-dev): 5548: Remove git commands from run-cwl-tests. Remove cwltool fr...
Peter Amstutz
06:07 pm Story #5548: [CWL/OPS] Jenkins runs cwltool tests
reviewing commit:ac6798733ce11d6a3410bf1da639cb3e5a84047f
L160 I see that this logic wont be ever excecuted in jen...
Nico César
04:42 pm Revision d9214af1 (arvados): Add GPG key for RVM installation in the doc.
No issue # Ward Vandewege
04:30 pm Revision ac679873 (arvados): 5548: Script to test, build and upload common workflow language pack...
docker images. Peter Amstutz
04:30 pm Revision ac679873 (arvados-dev): 5548: Script to test, build and upload common workflow language ...
docker images. Peter Amstutz
04:13 pm Bug #6157 (Resolved): [Documentation] Explain extra steps needed when compute hostnames are not fooN
h3. background
Changing slurm config files, and keeping them synchronized across controller+workers, is a bit pain...
Tom Clegg
04:07 pm Bug #6156 (Resolved): [API] Node record accommodates setups where hostnames are set statically by...
h2. Goals
Have the process documented in our "compute node install guide":http://doc.arvados.org/install/install-c...
Tom Clegg
02:51 pm Task #6150 (In Progress): Test correct index usage with "explain analyze"
Tom Clegg
01:09 pm Task #6150 (Closed): Test correct index usage with "explain analyze"
Tom Clegg
02:27 pm Task #6137 (In Progress): Fix documentation links in docker build
Peter Amstutz
02:26 pm Task #6136 (Resolved): review 6135-docker-git-tag
Nico César
02:20 pm Task #6135 (Resolved): 'docker' jenkins jobs has to tag with the git revision on push
Applied in changeset arvados-dev|commit:845e3998bb2b4175e947a20eea4ed530629a778b. Anonymous
02:16 pm Revision 845e3998 (arvados-dev): Merge branch '6135-docker-git-tag'
closes #6135
Conflicts:
jenkins/run-docker-tests.sh
Nico Cesar
02:16 pm Revision 845e3998 (arvados): Merge branch '6135-docker-git-tag'
closes #6135
Conflicts:
jenkins/run-docker-tests.sh
Nico Cesar
01:43 pm Task #6099 (Resolved): Finish fixing tests for draft2-updates
Peter Amstutz
01:43 pm Task #6097 (In Progress): Implement "id" loading
Peter Amstutz
01:23 pm Bug #6153 (Closed): SLURM_JOB_ID environment variable expiration persistence on compute node
A new user ran qr1hi-d1hrv-nfqsgixupqykgok and immediately ran into:
2015-05-27_12:02:56 srun: error: slurm_receiv...
Bryan Cosca
01:20 pm Bug #6152: [Docs] Eliminate `arv` dependency in compute node installer
gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 Peter Amstutz
01:16 pm Bug #6152 (Resolved): [Docs] Eliminate `arv` dependency in compute node installer
The ping script on the compute node install page uses @arv node create@. This adds a dependency on the entire Ruby s... Peter Amstutz
01:09 pm Task #6112 (Resolved): Review 6098-full-text-index 64cfaed
Tom Clegg
12:58 pm Task #6112 (In Progress): Review 6098-full-text-index 64cfaed
Radhika Chippada
01:06 pm Revision 59d68bda (arvados): Merge branch '6098-full-text-index' refs #6098
Tom Clegg
01:05 pm Revision 15009c10 (arvados): 6098: Recreate full text indexes with leading spaces, to persuade Po...
Tom Clegg
12:18 am Revision fc8cdf0a (arvados-dev): make single quotes around reponame optional...
apparently the git client, when using ext:: transports, does not send those
quotes
https://groups.google.com/forum/#...
Sitaram Chamarty
12:18 am Revision 4ed15ad8 (arvados-dev): create command: remove race condition...
Because of the pipe in the create command, 'gitolite git-config' and 'perms
-c' would run simultaneously. The proble...
Sitaram Chamarty

05/26/2015

11:02 pm Bug #6098 (In Progress): [API] slow search? SQL query
Radhika Chippada
07:31 pm Bug #6098: [API] slow search? SQL query
This query takes 1075ms.... Tom Clegg
05:46 pm Bug #6098: [API] slow search? SQL query
Removing the "uuid" field from the full text index (and query) moves my test query time from 1075ms to 4ms. Tom Clegg
03:18 pm Bug #6098: [API] slow search? SQL query
Definitely a full-text search problem ("explain" on the query minus the "@@" condition says 73ms). Presumably the ind... Tom Clegg
11:01 pm Story #6057: [Workbench] Provide an attractive index of public projects
Tom, thanks for the comments. I incorporated all those comments. Also, removed the integration tests and made them co... Radhika Chippada
10:58 pm Revision a2eb98fa (arvados): Merge branch 'master' into 6057-public-projects-page
Radhika Chippada
10:54 pm Revision b3e2782f (arvados): 6057: if /projects/public is accessed when anonymous config is not e...
Radhika Chippada
08:04 pm Revision 53b19718 (arvados): Merge branch 'master' into 6093-refresh-docs
Radhika Chippada
07:58 pm Bug #6149 (Resolved): [Crunch] Install step logs error message from `setup.py egg_info`
In https://cloud.curoverse.com/collections/fefe083539813d3a01108c90fa684f7c+85/qr1hi-8i9sb-74468yxq0cauxbw.log.txt
...
Bryan Cosca
07:57 pm Revision 2578f9c1 (arvados): 6093: one more
Radhika Chippada
07:51 pm Revision 7a47332f (arvados): 6093: a few more updates
Radhika Chippada
07:22 pm Task #6144 (Resolved): Review 6094-install-script-sigpipe @ e010ab7
Tom Clegg
02:31 pm Task #6144 (In Progress): Review 6094-install-script-sigpipe @ e010ab7
Brett Smith
07:21 pm Task #6122 (Resolved): Improve query performance (or prevent clients from doing it?)
Tom Clegg
07:21 pm Task #6106 (Closed): Add a performance test case that exposes the slow query
Tom Clegg
01:56 pm Task #6106 (In Progress): Add a performance test case that exposes the slow query
Tom Clegg
07:18 pm Revision 64cfaed1 (arvados): 6098: Recreate full text indexes with leading spaces, to persuade Po...
Tom Clegg
03:56 pm Revision 6bc89efd (arvados): 6093: some more doc updates.
Radhika Chippada
03:46 pm Revision b6a55136 (arvados-dev): 6035: add support for --upload as an option
refs #6035 Nico Cesar
03:46 pm Revision b6a55136 (arvados): 6035: add support for --upload as an option
refs #6035 Nico Cesar
02:31 pm Revision 9b67268f (arvados): Merge branch '6094-install-script-sigpipe' refs #6094
Tom Clegg
02:31 pm Revision d3fcdc7a (arvados): 6094: Propagate install script stderr+stdout to job log.
Tom Clegg
02:31 pm Bug #6094: [Crunch] Install script exited 141
Tom Clegg wrote:
> Brett Smith wrote:
> > this commit does not assign 1 to vec's lvalue
>
> Whoops, yes, that wa...
Brett Smith
02:26 pm Bug #6094: [Crunch] Install script exited 141
Brett Smith wrote:
> this commit does not assign 1 to vec's lvalue
Whoops, yes, that was an oversight. Amended (n...
Tom Clegg
02:14 pm Bug #6094: [Crunch] Install script exited 141
Reviewing commit:e010ab73 (literally, just that commit)
It looks like you intend to use the recipe in select's per...
Brett Smith
02:18 pm Revision c96f21fc (arvados): 6094: Propagate install script stderr+stdout to job log.
Tom Clegg
12:35 am Task #6117 (Resolved): Review 6095-arv-copy-preserve-object-order-wip
Applied in changeset arvados|commit:872cc3d79d99f5fc4b26b970c3d269ca363ea27c. Brett Smith
12:35 am Bug #6095 (Resolved): [SDKs] Make arv-copy preserve pipeline template component order
Applied in changeset arvados|commit:872cc3d79d99f5fc4b26b970c3d269ca363ea27c. Brett Smith
12:32 am Revision 872cc3d7 (arvados): Merge branch '6095-arv-copy-preserve-object-order-wip'
Closes #6095, #6117. Brett Smith
12:32 am Revision 79564b0a (arvados): 6095: arv-copy preserves order of copied JSON.
This means arv-copy no longer loses the order of pipeline template
components, which makes for a nicer presentation i...
Brett Smith

05/25/2015

10:36 pm Revision 8a2a84af (arvados): 6093: doc updates
Radhika Chippada
09:42 pm Story #6057: [Workbench] Provide an attractive index of public projects
Comments @ commit:1a0ba80
h3. Controller
If Workbench is not configured to serve public projects, I don't think...
Tom Clegg
08:43 pm Task #6115 (In Progress): Review branch: 6057-public-projects-page
Tom Clegg
08:42 pm Bug #6095: [SDKs] Make arv-copy preserve pipeline template component order
LGTM @ commit:2f64531 Tom Clegg
05:52 pm Task #6117 (In Progress): Review 6095-arv-copy-preserve-object-order-wip
Tom Clegg
05:25 pm Revision dab62e90 (arvados): 6087: Add big-manifest tests, with some finer-grained performance nu...
Tom Clegg
02:35 pm Revision ccfa3ebf (arvados): 6087: Use HTTPClient's compression feature (instead of adding the
Content-Encoding header ourselves). Rename config knob to describe
purpose instead of implementation.
Tom Clegg
02:23 pm Revision cc988c81 (arvados): 6087: Compute portable_data_hash only once during check_signatures.
Tom Clegg
02:21 pm Revision 0ded94e1 (arvados): 6087: Use app-configured key by default for blob signing and verific...
Tom Clegg

05/24/2015

10:33 pm Bug #6095: [SDKs] Make arv-copy preserve pipeline template component order
6095-arv-copy-preserve-object-order-wip is up for review. A couple of notes:
* If you look at the original JsonMo...
Brett Smith
10:02 pm Revision 2f64531d (arvados): 6095: arv-copy preserves order of copied JSON.
This means arv-copy no longer loses the order of pipeline template
components, which makes for a nicer presentation i...
Brett Smith

05/23/2015

05:36 pm Bug #6016: No pipeline to run with docker install
Mohammad Zia wrote:
> Hello,
>
> I installed the docker version of Arvados today and tried to follow the tutorial...
Albert Vilella

05/22/2015

09:43 pm Task #6144 (Resolved): Review 6094-install-script-sigpipe @ e010ab7
Tom Clegg
09:42 pm Revision 0f125dd5 (arvados): Merge branch '6094-install-script-sigpipe' (early part) refs #6094
Tom Clegg
09:41 pm Task #6111 (Resolved): Review 6094-install-script-sigpipe
Tom Clegg
09:39 pm Revision e010ab73 (arvados): 6094: Propagate install script stderr+stdout to job log.
Tom Clegg
09:29 pm Bug #6094: [Crunch] Install script exited 141
6094-install-script-sigpipe @ commit:b4c2a96 ensures the DATA section gets consumed even in that "don't need to extra... Tom Clegg
07:57 pm Bug #6094: [Crunch] Install script exited 141
I'm satisfied with that explanation, and I'm satisfied with commit:fbcc5ab if it passes testing. Thanks. Brett Smith
07:02 pm Bug #6094: [Crunch] Install script exited 141
This doesn't show the warning message "srun: error: compute32: task 0: Exited with exit code 141" but it does seem pr... Tom Clegg
06:56 pm Bug #6094 (In Progress): [Crunch] Install script exited 141
Brett Smith
03:59 pm Bug #6094: [Crunch] Install script exited 141
I created a tar archived that combines the requested Arvados SDK version and script version. That's 15144960 bytes. ... Brett Smith
09:07 pm Bug #6095 (In Progress): [SDKs] Make arv-copy preserve pipeline template component order
Brett Smith
08:56 pm Task #6136 (In Progress): review 6135-docker-git-tag
Nico César
01:28 pm Task #6136 (Resolved): review 6135-docker-git-tag
Nico César
08:53 pm Task #6135: 'docker' jenkins jobs has to tag with the git revision on push
added --tag support in 6096-docker-git-tag (arvados-dev) Nico César
08:32 pm Task #6135: 'docker' jenkins jobs has to tag with the git revision on push
I think the best place to do this is:
* add a parameter --tags to run-docker-tests
* "docker tag $ID" before t...
Nico César
07:44 pm Task #6135 (In Progress): 'docker' jenkins jobs has to tag with the git revision on push
mental note... Nico César
01:28 pm Task #6135 (Resolved): 'docker' jenkins jobs has to tag with the git revision on push
Nico César
08:52 pm Revision a1fe656e (arvados-dev): added --tags support
refs #6135 Nico Cesar
08:40 pm Revision 1a0ba80d (arvados): Merge branch 'master' into 6057-public-projects-page
Radhika Chippada
08:39 pm Revision 71f1ccba (arvados): 6057: add projects/public page, which lists publicly accessible proj...
Radhika Chippada
08:32 pm Revision a996653c (arvados-dev): 6141: Add https:// to arvados_workbench_host
Peter Amstutz
08:16 pm Bug #6142 (Resolved): [Node Manager] When canceling a SLURM shutdown, check state before resuming...
Node Manager's SLURM dispatcher always tries to resume the node in SLURM when a shutdown is canceled. However, this ... Brett Smith
08:13 pm Revision 80d57cdc (arvados): 6141: Remove hard-coded "https://" from "https://{{site.arvados_work...
Peter Amstutz
07:40 pm Revision fbcc5ab5 (arvados): 6094: Consider arvados_sdk_version (not just script_version) when
deciding there's no need to extract or install anything. Tom Clegg
07:39 pm Revision b4c2a96e (arvados): 6094: Avoid SIGPIPE by consuming DATA section even when it's not nee...
Tom Clegg
07:35 pm Bug #6141 (Resolved): [Docs] Don't hardcode https:// in links to workbench
Peter Amstutz
07:34 pm Story #6138: [Docker] Docker install uses standalone SSO server
Nico Cesar wrote:
> reviewing 6090-docker-use-local-sso commit ae3bb0033a24a38489c49ffc26e5a5e8fd93c160
>
>
> h...
Peter Amstutz
03:35 pm Story #6138: [Docker] Docker install uses standalone SSO server
So I cant really test this in my machine... I have space restrictions and SSD drive is incredible slow. Options here? Nico César
03:18 pm Story #6138: [Docker] Docker install uses standalone SSO server
reviewing 6090-docker-use-local-sso commit ae3bb0033a24a38489c49ffc26e5a5e8fd93c160
h3. docker/arvdock
* line...
Nico César
02:54 pm Story #6138 (Resolved): [Docker] Docker install uses standalone SSO server
Peter Amstutz
07:32 pm Revision a7d558d6 (arvados): 6138: Added --domain to set ARVADOS_DOMAIN. Removed useless comment...
Peter Amstutz
07:32 pm Revision 1e7b6aa1 (arvados-dev): Merge branch '6035-docker-upload-option'
closes #6035 Nico Cesar
07:32 pm Revision 1e7b6aa1 (arvados): Merge branch '6035-docker-upload-option'
closes #6035 Nico Cesar
07:28 pm Story #6057 (In Progress): [Workbench] Provide an attractive index of public projects
Radhika Chippada
05:27 pm Story #6057: [Workbench] Provide an attractive index of public projects
Discussed this with Tom and some highlights:
* Let's do the basic public projects display first. We will look into...
Radhika Chippada
07:18 pm Revision bb480826 (arvados-dev): style and fashion for classy 'if' statements
refs #6035 Nico Cesar
07:18 pm Revision bb480826 (arvados): style and fashion for classy 'if' statements
refs #6035 Nico Cesar
06:56 pm Support #5833: Cannot get Arvados installed
Check ticket_5833.txt
I was unable to reproduce the error in a brand new instance (this is in GCE ) running Ubuntu...
Nico César
06:51 pm Support #5833: Cannot get Arvados installed
By The Way, try a quick ... Nico César
06:07 pm Support #5833: Cannot get Arvados installed
Thanks Darshan for the detailed information, I'll respond inline
Darshan Singh wrote:
(..)
> darshan@Linux2:~$ ...
Nico César
05:41 pm Story #3408: [Keep] Implement Production Data Manager
Okay, I was able to run data manager against my local development instance and get a report.
Scratch one of my pri...
Peter Amstutz
03:20 pm Story #3408: [Keep] Implement Production Data Manager
Also, API server provides microsecond timestamps, but it looks like datamanager is truncating that to whole seconds. ... Peter Amstutz
03:14 pm Story #3408: [Keep] Implement Production Data Manager
I ran it without any arguments and here's what I got:... Peter Amstutz
02:16 pm Story #3408: [Keep] Implement Production Data Manager
More comments.
In @BuildPullLists@, I think the assignment @spl[destination] = pullList@ inside the @if !pullListExi...
Peter Amstutz
03:45 pm Bug #6139: [OPS] Jenkis Job arvados-api-server should create unique databases so it can ran in pa...
Current runs of Jenkins Job 'arvados-api-server' uses 'arvados-test' database. When multiple jobs are running they co... Nico César
03:39 pm Bug #6139 (Duplicate): [OPS] Jenkis Job arvados-api-server should create unique databases so it c...
Nico César
03:00 pm Task #6108 (In Progress): Review data manager
Peter Amstutz
02:57 pm Task #6099 (In Progress): Finish fixing tests for draft2-updates
Peter Amstutz
02:54 pm Task #6133 (In Progress): Review 6090-docker-use-local-sso
Peter Amstutz
02:54 pm Task #6133 (New): Review 6090-docker-use-local-sso
Peter Amstutz
02:18 pm Revision 77bf9668 (arvados-dev): 3408: Tests for datamanager
Peter Amstutz
02:03 pm Revision 3bf22ae1 (arvados): Add installation instructions for compute nodes; update the installa...
instructions for crunch dispatcher.
No issue #
Ward Vandewege
01:39 pm Task #6137 (Resolved): Fix documentation links in docker build
Peter Amstutz
01:30 pm Story #6096: [OPS] Implement a process to regularly deploy a Docker image for running jobs to Arv...
see #6135 . docker images will have tags Nico César
12:57 pm Story #6096: [OPS] Implement a process to regularly deploy a Docker image for running jobs to Arv...
mental note:
from #3847...
Nico César
04:08 am Revision 46b43471 (arvados): 6087: Add big-manifest tests, with some finer-grained performance nu...
Tom Clegg

05/21/2015

10:55 pm Story #6057: [Workbench] Provide an attractive index of public projects
Something like this should get public projects:... Tom Clegg
05:42 pm Story #6057: [Workbench] Provide an attractive index of public projects
In case it's useful, source:apps/workbench/app/views/projects/_show_featured.html.erb and source:apps/workbench/app/a... Tom Clegg
09:15 pm Revision 8b658d96 (arvados): Merge branch '6087-collection-timing' (early part) refs #6087 refs #...
Tom Clegg
09:12 pm Story #3408: [Keep] Implement Production Data Manager
Reviewing commit:5824ee2
* "DefaultReplicationLevel" should come from the "defaultCollectionReplication" key in th...
Peter Amstutz
08:58 pm Task #6133 (In Progress): Review 6090-docker-use-local-sso
Nico César
07:00 pm Task #6133 (Resolved): Review 6090-docker-use-local-sso
Peter Amstutz
08:51 pm Revision 1a2d8a73 (arvados): 6087: Get database time only once per manifest-signing/verifying eve...
Tom Clegg
08:19 pm Story #6096: [OPS] Implement a process to regularly deploy a Docker image for running jobs to Arv...
Ok .. I understand that we can do a new Jenkins job that triggers this update.
I see from "docker" Job:...
Nico César
07:34 pm Story #6096: [OPS] Implement a process to regularly deploy a Docker image for running jobs to Arv...
Nico Cesar wrote:
> As far as I understand this is as simple as:
>
> ...
Brett Smith
07:20 pm Story #6096 (In Progress): [OPS] Implement a process to regularly deploy a Docker image for runni...
As far as I understand this is as simple as:... Nico César
08:06 pm Revision 5d26f7e8 (arvados): Update tutorial pipeline page to match new definition.
No issue #. Brett Smith
07:00 pm Task #6102 (In Progress): IRC support
Peter Amstutz
07:00 pm Task #6128 (In Progress): Test Docker install
Peter Amstutz
07:00 pm Revision ae3bb003 (arvados): 6090: Docker install uses local SSO server instead of auth.curoverse...
clean up references to dev.arvados to use @@ARVADOS_DOMAIN@@. Peter Amstutz
06:52 pm Task #6056 (Resolved): Review puppet branch 3793-docker-image-cleaner-wip
Brett Smith
06:52 pm Bug #3793 (Resolved): [Crunch] Docker containers not removed consistently, filling disk
Ward Vandewege wrote:
> reviewing the puppet branch 3793-docker-image-cleaner-wip:
>
> * I think the compute_dock...
Brett Smith
06:49 pm Task #5870 (Closed): Test scaling performance on exome and whole genome
Bryan Cosca
06:36 pm Revision 6a1a627a (arvados): 6035: add support for --upload as an option
refs #6035 Nico Cesar
06:36 pm Revision 6a1a627a (arvados-dev): 6035: add support for --upload as an option
refs #6035 Nico Cesar
06:23 pm Revision 9b44bba3 (arvados): Update tutorial pipeline template definition.
This helps it run out of the box again.
No issue #.
Brett Smith
05:54 pm Bug #5850: All clients authenticate directly against SSO server without going through API server
I accidentally took a story task and treated it like a bug. Apologies to Tom. Brett Smith
05:53 pm Task #6131 (Resolved): Sort out browser and command line authentication mechanisms
Tom Clegg
04:10 pm Revision 59129909 (arvados): refs #6061
6061: set profiling_enabled to true in workbench application.default.yml for test env. Radhika Chippada
03:45 pm Story #3198: [FUSE] Writable streaming arv-mount
I think conflict filenames should avoid using ":" characters. They have a tendency to confuse programs that know abou... Tom Clegg
03:34 pm Revision 2b9864be (arvados-dev): Merge branch '6061-add-performance-plots' refs #6061
Nico Cesar
03:34 pm Revision 2b9864be (arvados): Merge branch '6061-add-performance-plots' refs #6061
Nico Cesar
03:33 pm Support #6055: [Support] Cannot log into VM
Hi Ward,
I am using the Arvados docker image and not the curoverse installation.
Thanks,
--Mohammad
Mohammad Zia
03:32 pm Revision cc7fb936 (arvados): 3793: Fix how python3 fpm arguments are added.
The previous code would add arguments such that they could be
interpreted as the package version. Avoid this by addi...
Brett Smith
03:32 pm Revision c4d584c5 (arvados-dev): Tidy up sanity checks. Add checks for fuse.h, pyconfig.h, nginx.
Suggest how to fix missing dependencies.
No issue #
Tom Clegg
03:32 pm Revision 32f2a67e (arvados): Improve packaging of third-party Python 3 packages.
* Fix the bug where we're passing in "python3" as the vendor argument
to build_and_scp_deb.
* Build more dependenci...
Brett Smith
03:32 pm Revision c4d584c5 (arvados): Tidy up sanity checks. Add checks for fuse.h, pyconfig.h, nginx.
Suggest how to fix missing dependencies.
No issue #
Tom Clegg
03:32 pm Revision cc7fb936 (arvados-dev): 3793: Fix how python3 fpm arguments are added.
The previous code would add arguments such that they could be
interpreted as the package version. Avoid this by addi...
Brett Smith
03:32 pm Revision eae444a7 (arvados): 5766: Remove "remove dependencies" step. Detect existing PyYAML faster.
Tom Clegg
03:32 pm Revision eae444a7 (arvados-dev): 5766: Remove "remove dependencies" step. Detect existing PyYAML ...
Tom Clegg
03:32 pm Revision 32f2a67e (arvados-dev): Improve packaging of third-party Python 3 packages.
* Fix the bug where we're passing in "python3" as the vendor argument
to build_and_scp_deb.
* Build more dependenci...
Brett Smith
03:32 pm Revision 31e86126 (arvados-dev): 6061: comment cleanup and final version tested. it works
closes #6061 Nico Cesar
03:32 pm Revision 31e86126 (arvados): 6061: comment cleanup and final version tested. it works
closes #6061 Nico Cesar
03:31 pm Revision 0f079375 (arvados-dev): adding 3 more test for the performance suite.
I'm guessing the names, lets see if they are good or not ...
refs #6061
Nico Cesar
03:31 pm Revision 23b42a86 (arvados-dev): 6061: plot data
added create plot data script to exit_cleanly()
3 initial test to use
Matching more generic test names
refs #6061
Nico Cesar
03:31 pm Revision 0f079375 (arvados): adding 3 more test for the performance suite.
I'm guessing the names, lets see if they are good or not ...
refs #6061
Nico Cesar
03:31 pm Revision 23b42a86 (arvados): 6061: plot data
added create plot data script to exit_cleanly()
3 initial test to use
Matching more generic test names
refs #6061
Nico Cesar
03:31 pm Revision a43a3fb3 (arvados-dev): adding the creation of plot data that jenkins plot pluging will ...
refs #6061 Nico Cesar
03:31 pm Revision 5b7e2250 (arvados): 5416: Terminate connections on the configured test database (not nec...
Tom Clegg
03:31 pm Revision a3465f97 (arvados): 5416: Run keepproxy, arv-git-httpd, and SSL proxy services during Wo...
Tom Clegg
03:31 pm Revision a43a3fb3 (arvados): adding the creation of plot data that jenkins plot pluging will be a...
refs #6061 Nico Cesar
03:31 pm Revision 4bfcd9f1 (arvados): run-tests.sh can now run the arv-git-httpd tests too.
No issue # Ward Vandewege
03:31 pm Revision 42234907 (arvados): Be consistent in our package and binary naming:
arv-git-httpd becomes arvados-git-httpd
No issue #
Ward Vandewege
03:31 pm Revision 72c34579 (arvados): 5416: Add --retry flag: when a test suite fails, prompt to try that ...
Tom Clegg
03:31 pm Revision 9a4afad1 (arvados): 3793: Build Docker cleaner service in run-build-packages.sh.
Brett Smith
03:31 pm Revision 123d3d77 (arvados): 3793: Add services/dockercleaner to run-tests.sh.
Brett Smith
03:31 pm Revision 1553f484 (arvados): 5766: Make sure it doesn't try to fetch arvados packages from PyPi, ...
we only install the local ones. Peter Amstutz
03:31 pm Revision ffa8b415 (arvados): 5416: Support testargs for go bits, like "services/arv-git-httpd_tes...
Tom Clegg
03:31 pm Revision 4bfcd9f1 (arvados-dev): run-tests.sh can now run the arv-git-httpd tests too.
No issue # Ward Vandewege
03:31 pm Revision 42234907 (arvados-dev): Be consistent in our package and binary naming:
arv-git-httpd becomes arvados-git-httpd
No issue #
Ward Vandewege
03:31 pm Revision 9a4afad1 (arvados-dev): 3793: Build Docker cleaner service in run-build-packages.sh.
Brett Smith
03:31 pm Revision 123d3d77 (arvados-dev): 3793: Add services/dockercleaner to run-tests.sh.
Brett Smith
03:31 pm Revision 1553f484 (arvados-dev): 5766: Make sure it doesn't try to fetch arvados packages from Py...
we only install the local ones. Peter Amstutz
03:31 pm Revision ffa8b415 (arvados-dev): 5416: Support testargs for go bits, like "services/arv-git-httpd...
Tom Clegg
03:31 pm Revision a3465f97 (arvados-dev): 5416: Run keepproxy, arv-git-httpd, and SSL proxy services durin...
Tom Clegg
03:31 pm Revision 5b7e2250 (arvados-dev): 5416: Terminate connections on the configured test database (not...
Tom Clegg
03:31 pm Revision 72c34579 (arvados-dev): 5416: Add --retry flag: when a test suite fails, prompt to try t...
Tom Clegg
03:31 pm Revision bf95f654 (arvados-dev): 5416: Add arv-git-httpd.
Tom Clegg
03:31 pm Revision bf95f654 (arvados): 5416: Add arv-git-httpd.
Tom Clegg
03:00 pm Revision 49d7df5e (arvados-dev): adding an empty csv in case of error
Nico Cesar
02:48 pm Revision a4b2f2a2 (arvados-dev): no more trailing spaces
Nico Cesar
02:48 pm Bug #6130 (New): [Workbench] Support searching collections by their collection name
!select_all_logs.gif!
For example, I want to remove all the log files, I would like to search for log and get ever...
Bryan Cosca
02:47 pm Revision cfb3c455 (arvados-dev): Merge branch 'master' into 6061-add-performance-plots
Nico Cesar
02:27 pm Revision 2dae0755 (arvados-dev): 6061: comment cleanup and final version tested. it works
closes #6061 Nico Cesar
02:21 pm Revision 0fbece8e (arvados): 3793: Fix how python3 fpm arguments are added.
The previous code would add arguments such that they could be
interpreted as the package version. Avoid this by addi...
Brett Smith
02:21 pm Revision 0fbece8e (arvados-dev): 3793: Fix how python3 fpm arguments are added.
The previous code would add arguments such that they could be
interpreted as the package version. Avoid this by addi...
Brett Smith
02:04 pm Revision 9fa53446 (arvados-dev): Improve packaging of third-party Python 3 packages.
* Fix the bug where we're passing in "python3" as the vendor argument
to build_and_scp_deb.
* Build more dependenci...
Brett Smith
02:04 pm Revision 9fa53446 (arvados): Improve packaging of third-party Python 3 packages.
* Fix the bug where we're passing in "python3" as the vendor argument
to build_and_scp_deb.
* Build more dependenci...
Brett Smith
01:12 pm Revision 08d02e15 (arvados-dev): Merge branch 'more-sanity-check-less-download' refs #5766
Tom Clegg
01:12 pm Revision 08d02e15 (arvados): Merge branch 'more-sanity-check-less-download' refs #5766
Tom Clegg
12:57 pm Revision 42e438d9 (arvados): 5766: Remove "remove dependencies" step. Detect existing PyYAML faster.
Tom Clegg
12:57 pm Revision 42e438d9 (arvados-dev): 5766: Remove "remove dependencies" step. Detect existing PyYAML ...
Tom Clegg
12:51 pm Revision 303a6e2a (arvados): Tidy up sanity checks. Add checks for fuse.h, pyconfig.h, nginx.
Suggest how to fix missing dependencies.
No issue #
Tom Clegg
12:51 pm Revision 303a6e2a (arvados-dev): Tidy up sanity checks. Add checks for fuse.h, pyconfig.h, nginx.
Suggest how to fix missing dependencies.
No issue #
Tom Clegg
12:24 pm Bug #5766: [SDK] setup.py install always gets latest arvados-python-sdk from PyPi, even when alre...
Peter Amstutz wrote:
> tomclegg: If I'm understanding your suggestion, you're suggesting uninstalling everything exc...
Tom Clegg

05/20/2015

09:47 pm Revision 9d3fa25b (arvados-dev): script working better
Nico Cesar
08:58 pm Task #6128 (Resolved): Test Docker install
Peter Amstutz
08:48 pm Revision cb1c4d58 (arvados-dev): Merge branch '6061-add-performance-plots' of git.curoverse.com:a...
Nico Cesar
08:47 pm Revision 63e40231 (arvados-dev): 6011: Matching more generic test names
Nico Cesar
08:47 pm Revision d16c7ef6 (arvados-dev): taking out the tests don-t make sense
Nico Cesar
08:43 pm Revision 08995325 (arvados-dev): taking out the tests don-t make sense
Nico Cesar
08:30 pm Task #6127 (New): Preserve template goal through login->UA->profile->back-to-work process, and th...
Tom Clegg
08:29 pm Task #6126 (New): Decide URL and advice for appropriate next step when in "pipeline goal" mode
Tom Clegg
08:27 pm Task #6125 (New): Enter goal mode when hitting Run on a pipeline template
Tom Clegg
08:27 pm Task #6124 (New): Add "goal mode" infrastructure
Tom Clegg
08:25 pm Task #6123 (Resolved): Determine why still NoMemoryError even with API memory-limiting feature
Tom Clegg
08:24 pm Task #6122 (Resolved): Improve query performance (or prevent clients from doing it?)
Tom Clegg
08:18 pm Task #6119 (Resolved): Review 6096-deploy-jobs-image
Nico César
08:14 pm Task #6117 (Resolved): Review 6095-arv-copy-preserve-object-order-wip
Brett Smith
08:12 pm Task #6116 (Closed): Review 5883-oauth2-update at get-evidence.git
Tom Clegg
08:11 pm Task #6115 (Resolved): Review branch: 6057-public-projects-page
Tom Clegg
08:10 pm Task #6113 (Resolved): Review branch: 6203-collection-perf-api
Tom Clegg
08:10 pm Task #6112 (Resolved): Review 6098-full-text-index 64cfaed
Tom Clegg
08:09 pm Task #6111 (Resolved): Review 6094-install-script-sigpipe
Tom Clegg
08:04 pm Task #6110 (Resolved): Review 6074-collections-index @ 22c8b63
Tom Clegg
08:04 pm Task #6052: Review branch: 5930-smalldocfix
ops... wrong ticket. Nico César
08:03 pm Bug #6086: [API] Creating new job sometimes takes too long
I am currently leaving this on the 2015-06-10 sprint. Currently we believe that #6071 will be enough of a workaround... Brett Smith
08:02 pm Task #6109 (New): Review
Tom Clegg
08:02 pm Task #6108 (Resolved): Review data manager
Peter Amstutz
07:55 pm Task #6107 (Resolved): Fix Docker image search/list
Peter Amstutz
07:54 pm Task #6106 (Closed): Add a performance test case that exposes the slow query
Tom Clegg
07:53 pm Task #6105 (Resolved): Review 5990-arv-run-default-runtime
Peter Amstutz
07:52 pm Task #6104 (Resolved): Review
Peter Amstutz
07:52 pm Task #6103 (Resolved): Review 5790-copy-most-recent-docker-image-wip
Peter Amstutz
07:51 pm Task #6102 (Resolved): IRC support
Peter Amstutz
07:50 pm Task #6100 (Resolved): Improve draft 2 spec docs
Peter Amstutz
07:50 pm Task #6099 (Resolved): Finish fixing tests for draft2-updates
Peter Amstutz
07:50 pm Bug #6098 (Resolved): [API] slow search? SQL query
... Ward Vandewege
07:50 pm Task #6097 (Resolved): Implement "id" loading
Peter Amstutz
07:35 pm Story #6096: [OPS] Implement a process to regularly deploy a Docker image for running jobs to Arv...
mental note: this is a arv-keep docker thing Nico César
07:28 pm Story #6096 (Resolved): [OPS] Implement a process to regularly deploy a Docker image for running ...

We're now in a place where we install all components via debs.
There's a script in arvados-dev called
jenki...
Brett Smith
07:31 pm Bug #5990: [SDKs] arv-run defaults to using arvados/jobs, without checking that it exists or is r...
Perhaps the default image should have no Arvados SDK, and arv-run should use the version given in the API discovery d... Tom Clegg
07:11 pm Story #5781: [API] [DRAFT] Provide API methods for manipulating and combining collections
Proposed Collection Update API:
https://arvados.org/projects/arvados/wiki/Collection_update_API
Peter Amstutz
07:09 pm Bug #5523 (New): [Crunch] crunchstat should not report errors during normal timing races
Tom Clegg
07:08 pm Bug #5748 (Resolved): [Keep] sometimes, Keep sucks up 100% cpu and becomes really slow after a while
Tom Clegg
07:04 pm Feature #5787 (Resolved): [Crunch] Add Crunch support to GATK Queue
Peter Amstutz
06:23 pm Bug #6076: [API] walk api server installations and ensure modified_at for collections is unique +...
Isn't this easy enough to handle at the client level with filters like @modified_at >= current_working_modified_time,... Brett Smith
06:23 pm Bug #3793: [Crunch] Docker containers not removed consistently, filling disk
reviewing the puppet branch 3793-docker-image-cleaner-wip:
* I think the compute_docker_image_quota default should...
Ward Vandewege
06:19 pm Task #6056 (In Progress): Review puppet branch 3793-docker-image-cleaner-wip
Ward Vandewege
06:18 pm Bug #5499: [SDKs] arv-copy fails to create collection after copying many blocks
Is it possible that this was caused by the GCE 10-minute firewall rule, and so duplicates that ticket? Brett Smith
05:58 pm Support #6055: [Support] Cannot log into VM

Hi Mohammed,
Mohammad Zia wrote:
> I have followed the instructions for logging into the VM, specifically, gene...
Ward Vandewege
05:28 pm Revision c4630bce (arvados-dev): added result checking
Nico Cesar
05:27 pm Revision 1341bb72 (arvados-dev): added result checking
Nico Cesar
03:57 pm Bug #6095 (Resolved): [SDKs] Make arv-copy preserve pipeline template component order
Implementation steps:
# Add an OrderedJsonModel class to arvados.api. This is a subclass of googleapiclient.model...
Brett Smith
03:45 pm Bug #5831: [SDKs] CollectionReader.open(filename, 'r') fails (arvados/keep.py 665 in get_from_cac...
This bug appeared again: https://workbench.su92l.arvadosapi.com/pipeline_instances/su92l-d1hrv-29d3lxak8gdlo46
Nev...
Sarah Guthrie
03:43 pm Revision 8561679d (arvados-dev): added create plot data script to exit_cleanly()
Nico Cesar
03:29 pm Revision bf0c6769 (arvados): Documentation fix (quoting) for the 'Add an Arvados repository' page.
No issue # Ward Vandewege
03:13 pm Revision 0f3a97b1 (arvados-dev): adding 3 more test for the performance suite.
I'm guessing the names, lets see if they are good or not ...
refs #6061
Nico Cesar
02:54 pm Revision 5151ef79 (arvados): 5748: Remove unused import. refs #5748
Tom Clegg
02:26 pm Bug #6094: [Crunch] Install script exited 141
More errors: qr1hi-d1hrv-oykvsg3tkkvoznt Bryan Cosca
02:23 pm Bug #6094: [Crunch] Install script exited 141
Another instance with this error: qr1hi-d1hrv-96bdqhdlhgopth1 Bryan Cosca
01:30 pm Bug #6094 (Resolved): [Crunch] Install script exited 141
Pipeline qr1hi-d1hrv-yer77lzhezepsui... Peter Amstutz
02:15 pm Revision c6dd0227 (arvados): 5748: Remove runtime.GC() obsoleted by bufferpool. refs #5748
Tom Clegg
01:48 pm Bug #4706: [Workbench] Re-connect to websockets when the connection drops; use "last_log_id" so n...
In addition, we could provide a fallback method that uses log table polling to get events. The Python SDK uses this ... Peter Amstutz
01:45 pm Task #6069 (Resolved): Review branch: 6064-collection-output-of-perf-issue
Radhika Chippada
01:45 pm Bug #6064 (Resolved): [Workbench] [Performance] Collection#show for /collections/qr1hi-4zz18-hqfx...
Applied in changeset arvados|commit:e5c66d135d453a4af4c978590b99a118f65ae12b. Radhika Chippada
01:40 pm Revision e5c66d13 (arvados): closes #6064
Merge branch '6064-collection-output-of-perf-issue' Radhika Chippada
01:10 pm Story #6057: [Workbench] Provide an attractive index of public projects
Brett Smith wrote:
> Would it maybe be a little nicer to manipulate the generated HTML directly? That way you don't...
Tom Clegg
12:13 pm Revision 35c8b0a5 (arvados): integration_performance tests: add comment describing how to run the...
since the tests in this directory will not be part of regular test run because it is not a standard rails test direct... Radhika Chippada
03:00 am Revision d144724e (arvados): 6064: rollback cache update so that it can be done in a separate bra...
Radhika Chippada
02:32 am Revision a5401e34 (arvados): Merge branch 'master' into 6064-collection-output-of-perf-issue
Radhika Chippada

05/19/2015

10:53 pm Feature #5728: ssh port 2222 is blocked on many networks
The workaround is to tunnel ssh from a non-firewalled host. You can (arguably) use a Cloud instance for that. Pjotr Prins
10:09 pm Bug #4706: [Workbench] Re-connect to websockets when the connection drops; use "last_log_id" so n...
This is still an issue, but it may be lower priority: we just realized that #6048 was happening because websockets si... Brett Smith
10:04 pm Story #5623: [Crunch] Specify Crunch2 features and APIs
Assuming no unforeseen wrinkles come up, I would really like to see this done by the end of the 2015-06-10 sprint so ... Brett Smith
09:59 pm Story #6092: [TBD] Improve the performance of the worst-performing collections component
This can't be done until we have results from #6061. Brett Smith
09:54 pm Story #6092 (Resolved): [TBD] Improve the performance of the worst-performing collections component
Brett Smith
09:50 pm Story #6091 (Resolved): Support users and triage their bugs (second half of sprint)
This includes:
* Be the front-line person to answer questions from the science team and other users
* Resolve bug...
Brett Smith
09:46 pm Story #6090 (Resolved): Support users and triage their bugs (first half of sprint)
This includes:
* Be the front-line person to answer questions from the science team and other users
* Resolve bug...
Brett Smith
09:46 pm Story #6089 (Resolved): Write development update blog (first three days of sprint)
Brett Smith
09:45 pm Story #6088 (Resolved): Contribute to CWL specification
Brett Smith
09:11 pm Story #6057: [Workbench] Provide an attractive index of public projects
Tom Clegg wrote:
> Possible first approximation:
> * Use the first non-empty line of the project description as the...
Brett Smith
07:59 pm Story #6057: [Workbench] Provide an attractive index of public projects
Possible first approximation:
* Use the first non-empty line of the project description as the "short description": ...
Tom Clegg
09:08 pm Bug #6064: [Workbench] [Performance] Collection#show for /collections/qr1hi-4zz18-hqfxewp0r3dsaov...
Good catch on the logic for caching. It looks like caching never been used. I wonder what else caching will help wi... Peter Amstutz
08:34 pm Task #6069 (In Progress): Review branch: 6064-collection-output-of-perf-issue
Peter Amstutz
02:11 am Task #6069 (Resolved): Review branch: 6064-collection-output-of-perf-issue
Radhika Chippada
08:31 pm Revision 45d885d5 (arvados): Merge branch '6071-java-sdk-timeouts' closes #6071
Peter Amstutz
08:29 pm Revision 61144008 (arvados): 6071: Bump up timeout to 120 seconds.
Peter Amstutz
08:23 pm Bug #6086 (Closed): [API] Creating new job sometimes takes too long
We have been seeing a lot of timeouts in the java sdk when it is attempting to create a job. The Java SDK uses a def... Peter Amstutz
08:01 pm Bug #5951 (In Progress): GET-Evidence login via google login no longer works - OpenID 2.0 is gone
Tom Clegg
07:49 pm Bug #6048 (Duplicate): Pipeline Instance does not automatically refresh
Duplicates #4706. Brett Smith
07:46 pm Bug #6048: Pipeline Instance does not automatically refresh
Bryan and Ward report that this is happening very consistently when pipelines run.
Sally reports it happens about ...
Brett Smith
07:31 pm Revision 974ab38a (arvados): Merge branch '6073-pycurl-version' refs #6073
Peter Amstutz
07:28 pm Revision 5824ee2e (arvados): More reformatting of unittests to improve readability.
Misha Zatsman
07:26 pm Revision 82f7fcd2 (arvados): refs #6061
Merge branch '6061-intg-perf-tests' Radhika Chippada
07:23 pm Revision 0f6a0017 (arvados): 6061: add workbench integration_performance test folder.
Radhika Chippada
07:20 pm Revision f3c45652 (arvados): 6071: Fix URL
Peter Amstutz
07:10 pm Revision 999c05af (arvados): Modified pull list code to change json output to match the specifica...
Changed PullList type
Renamed PullListEntry to PullRequest to match the type defined in services/keepstore/handlers.g...
Misha Zatsman
07:01 pm Story #3198: [FUSE] Writable streaming arv-mount
Suggest changing "Collection must be writable" error message to "Collection is not writable" or "Collection is read-o... Tom Clegg
06:44 pm Revision 8b3d7b73 (arvados-dev): adding the creation of plot data that jenkins plot pluging will ...
refs #6061 Nico Cesar
05:55 pm Revision 836999ec (arvados): 6071: Refactor project organization a bit.
Peter Amstutz
05:52 pm Revision 3c517a48 (arvados): 6071: Increase default timeout and enable retry support.
Peter Amstutz
05:26 pm Revision b2329e2c (arvados): Fixed compilation errors after merging master.
Misha Zatsman
05:16 pm Revision 6eb3d1fb (arvados): Merge branch 'master' into 3408-production-datamanager
Because it's been a long time. Misha Zatsman
05:09 pm Bug #6076 (New): [API] walk api server installations and ensure modified_at for collections is un...
Until this is done, datamanager needs to ask for groups of collections with overlap. That is not the most efficient, ... Ward Vandewege
04:55 pm Bug #6074 (Resolved): [API] collections are being returned too slowly + NoMemoryError
... Ward Vandewege
04:54 pm Revision 29009a1c (arvados): 6073: Pycurl depends on latest in Arvados SDK setup.py
Peter Amstutz
03:36 pm Task #5711 (In Progress): Requirements / API for Crunch v2
Peter Amstutz
03:19 pm Support #5833: Cannot get Arvados installed
I searched around the issues on arvados, so I ran following commands:
darshan@Linux2:~$ docker info
Containers: 1...
Darshan Singh
03:09 pm Support #5833: Cannot get Arvados installed
I got past the original error.
Now I get following error when I try to login.
Web application could not be start...
Darshan Singh
02:20 pm Feature #5829 (Resolved): [SSO] improve UI for ldap/local account login
Peter Amstutz
02:18 pm Task #6066 (In Progress): Review 3198-writable-fuse
Tom Clegg
01:57 pm Task #5965 (Resolved): Update sso install docs for plain email/password login
Peter Amstutz
01:37 pm Task #5965 (In Progress): Update sso install docs for plain email/password login
Peter Amstutz
01:36 pm Task #5936 (Resolved): Review SSO master
Peter Amstutz
01:17 pm Task #6070 (Resolved): Refactor arv-git-httpd authenticator into a package
Tom Clegg
03:30 am Feature #4717 (Resolved): [API] [SDKs] Read-only flag on keep_services table as hint to clients n...
Applied in changeset arvados|commit:4110970a34689ed526c3365e9b64a784fba7efda. Radhika Chippada
03:26 am Feature #4717: [API] [SDKs] Read-only flag on keep_services table as hint to clients not to try t...
Addressed all comments and merged into master. (Yes, the change method worked for both up and down for this case). Th... Radhika Chippada
03:29 am Task #5776 (Resolved): Review branch: 5720-ajax-loading-error
Radhika Chippada
03:28 am Task #5633 (Resolved): Update Python SDK
Radhika Chippada
03:28 am Task #5634 (Resolved): Update Go SDK
Radhika Chippada
03:27 am Task #6040 (Resolved): Review branch: 4717-read-only-keep-services-flag
Radhika Chippada
03:24 am Revision 4110970a (arvados): closes #4717
Merge branch '4717-read-only-keep-services-flag' Radhika Chippada
03:11 am Revision d5142a69 (arvados): 4717: rename writableRoots as writableLocalRoots.
Radhika Chippada
02:25 am Bug #5720 (Resolved): [Workbench] Search for pipeline templates breaks
Applied in changeset arvados|commit:f1827e2044aff826e63826880b296a59c4a17e2a. Radhika Chippada
02:22 am Revision b085f051 (arvados): Merge branch 'master' into 4717-read-only-keep-services-flag
Radhika Chippada
02:21 am Revision f1827e20 (arvados): closes #5720
Merge branch '5720-ajax-loading-error' Radhika Chippada
02:09 am Revision 100a504e (arvados): 6064: display only 5 objects for which this collection is the output...
Radhika Chippada
12:54 am Revision b9bf39b1 (arvados): Added code for generating pull lists.
Misha Zatsman
12:37 am Revision 5a662d84 (arvados): Made HostPort() an explicit method to print the HostPort, so that St...
Misha Zatsman

05/18/2015

11:54 pm Revision edd979c6 (arvados-dev): 'who-pushed' learns super-efficient 'tip search'
Sitaram Chamarty
09:20 pm Feature #4717: [API] [SDKs] Read-only flag on keep_services table as hint to clients not to try t...
At commit:e41260f
In the Go SDK, WritableRoots and writableRoots seem to mean writableLocalRoots, so should probab...
Tom Clegg
08:50 pm Revision 28e6503a (arvados): Merge branch 'master' into 3198-writable-fuse
Peter Amstutz
08:49 pm Revision 19c43ceb (arvados): 3198: Enable support for event bus based updates in arv-mount.
Peter Amstutz
08:44 pm Task #6040 (In Progress): Review branch: 4717-read-only-keep-services-flag
Tom Clegg
08:38 pm Feature #6034: [Deployment] Add sysvinit script to the keepproxy package
Yes, Go doesn't mind if you have subdirs, so that would also work -- like services/keepproxy/pkg-extras/ cf. IRC
Tom Clegg
08:11 pm Feature #6034: [Deployment] Add sysvinit script to the keepproxy package
Tom Clegg wrote:
> Nico Cesar wrote:
> > is it ok to drop the init file there too?
>
> afaik go tools ignore an...
Ward Vandewege
08:06 pm Feature #6034: [Deployment] Add sysvinit script to the keepproxy package
Nico Cesar wrote:
> is it ok to drop the init file there too?
afaik go tools ignore anything other than *.go
...
Tom Clegg
06:50 pm Feature #6034: [Deployment] Add sysvinit script to the keepproxy package
the code of keepproxy is here:... Nico César
06:59 pm Story #3198: [FUSE] Writable streaming arv-mount
3198-writable-fuse is ready for review. Notes:
* Supports creating, writing, renaming, deleting files in collecti...
Peter Amstutz
06:57 pm Bug #6064 (In Progress): [Workbench] [Performance] Collection#show for /collections/qr1hi-4zz18-h...
In IRC:
tomclegg
1:25 radhika: workbench arvados_base.rb L102 already uses the rails cache to optimize for cases ...
Radhika Chippada
04:04 pm Bug #6064 (Resolved): [Workbench] [Performance] Collection#show for /collections/qr1hi-4zz18-hqfx...
Display of /collections/qr1hi-4zz18-hqfxewp0r3dsaov takes forever.
* The issue seems to be linking all the items ...
Radhika Chippada
06:50 pm Revision faefe8fc (arvados): 3198: VWD syncs collection before loading from API server.
Peter Amstutz
06:50 pm Task #6066 (Resolved): Review 3198-writable-fuse
Peter Amstutz
06:49 pm Task #5895 (Resolved): Ensure vwd.checkin doesn't accidentally commit (or otherwise mess up) when...
Peter Amstutz
06:36 pm Bug #5662: [FUSE] cd/ls sometimes takes too long on su92l
Our current thinking is that #5748 is the main culprit here. We're very interested to hear how things look after the ... Brett Smith
06:36 pm Bug #5713: [FUSE] File access sometimes takes too long on su92l
Our current thinking is that #5748 is the main culprit here. We're very interested to hear how things look after the... Brett Smith
06:27 pm Revision 27616fe7 (arvados): 3198: Add docstring with overview of the architecture. Implement ch...
method wrapper and expand use_counter to every method containing a
llfuse.lock_released block. Add note about big_wr...
Peter Amstutz
06:26 pm Task #5785 (Resolved): Documentation and code cleanup
Peter Amstutz
06:26 pm Task #5630 (Resolved): Write tests
Peter Amstutz
05:41 pm Bug #5720: [Workbench] Search for pipeline templates breaks
5720-ajax-loading-error looks good. I pushed (without actually trying) commit:6702cda to explain what I meant about "... Tom Clegg
05:36 pm Revision 6702cdab (arvados): 5720: Leave existing content in place (and schedule another reload) ...
Tom Clegg
02:10 pm Revision fea4d2ba (arvados): 3198: writeto() notifies with WRITE instead of MOD. WRITE doesn't i...
Move llfuse.lock_released down one level into individual fusefile and fusedir methods.
Add big_writes option, improve...
Peter Amstutz
02:01 pm Story #6058 (New): [Workbench] Provide a guided workflow to new users who want to upload data and...
Imagine that I'm a bioinformatician looking at a public pipeline on an Arvados installation. I don't have an account... Brett Smith
02:00 pm Story #6057 (Resolved): [Workbench] Provide an attractive index of public projects
The index should list basic information about each project, so a user can browse the page and determine whether or no... Brett Smith
01:39 pm Task #6056 (Resolved): Review puppet branch 3793-docker-image-cleaner-wip
Brett Smith
01:39 pm Task #5997 (Resolved): Deploy Docker image cleaner via Puppet
Brett Smith
01:27 pm Support #6055 (Feedback): [Support] Cannot log into VM
Hi,
I have followed the instructions for logging into the VM, specifically, generating a public SSH key and adding...
Mohammad Zia
01:18 am Story #6054 (New): [Workbench] [Performance] Same request performance at different times varies s...
Performance of qr1hi can be quite bad at times.
Attached performance profiles show the times taken for collection#...
Radhika Chippada
01:04 am Story #6053 (Resolved): [Workbench] [Performance] Dashboard display is sluggish
Dashboard display takes several seconds to display. When looked at the workbench console log (pointing to qr1hi API s... Radhika Chippada

05/17/2015

01:40 am Bug #5930: [Documentation] Misc. documentation fixes
Review comments:
* doc/_includes/_tutorial_cluster_name.liquid
** Not sure if we should mention su12l in general ...
Radhika Chippada
01:21 am Bug #5930 (In Progress): [Documentation] Misc. documentation fixes
Radhika Chippada
01:21 am Task #6052 (Closed): Review branch: 5930-smalldocfix
Radhika Chippada
 

Also available in: Atom