06/15/2017

03:43 pm Arvados Bug #11843: Timeout attempting to create job for component gatk-haplotypecaller-cram-gvcf: Net::R...

logs table entries for a pipeline instance that had this issue are attached (logs.txt)....

06/12/2017

12:13 pm Arvados Bug #11843 (New): Timeout attempting to create job for component gatk-haplotypecaller-cram-gvcf: ...

Lately we get a lot of jobs whose sole log output is:...

04/28/2017

03:11 pm Arvados Bug #11556: occasional API server badness: 'incomplete response received from application'

as suggested by the postgres logs, am now raising the checkpoint settings:
was the default:...

03:05 pm Arvados Bug #11556: occasional API server badness: 'incomplete response received from application'

postgresql-9-3-main.log during the same time period:...

02:59 pm Arvados Bug #11556: occasional API server badness: 'incomplete response received from application'

This just happened again, and I went pretty soon after to look at the logs.
pg_stat_activity did have an autovacu...

02:35 pm Arvados Bug #11590 (Resolved): job reuse / re-use logic does not log decisions

It would be helpful if job reuse logic produced some log output to help understand why job output was or was not reus...

04/26/2017

04:40 pm Arvados Bug #11556 (New): occasional API server badness: 'incomplete response received from application'

Clients sometimes get the following response from our API server:...

04/20/2017

04:24 pm Arvados Bug #11209: stuck keep fuse mounts not cleared by crunch-job

There do generally seem to be race conditions with updating /etc/mtab by FUSE filesystems:
https://bugzilla.redhat...

04:07 pm Arvados Bug #11209: stuck keep fuse mounts not cleared by crunch-job

Another set of machines were wedged today. Did some more testing on the call with Tom:
First, on humgen-02-02 we e...

03:19 pm Arvados Bug #11209: stuck keep fuse mounts not cleared by crunch-job

kernel and system versions:...