Bug #6276
Updated by Tom Clegg almost 9 years ago
h2. summmary
Errors like "block not found" provide a block locator, but it's generally not obvious why a process was trying to read that block: "Which of the 7 collections used in my job has a missing block?"
_Ideally,_ the error message would tell you:
* collection portable_data_hash
* collection uuid and name
* filename
* (application-dependent) name of the collection (e.g., script_parameter in a crunch script)
The minimum information would be the collection's portable_data_hash, and the uuid if that's how we looked up the source collection in the first place. This is the most annoying part for the user to do manually, and in most cases it's enough information to get to the next step in troubleshooting/resolving the problem.
h2. implementation
When calling get/put-block methods, catch the relevant exceptions and raise a context-aware exception instead (incorporating the text of the original exception).
h2. background
my command:
['/tmp/crunch-job-work/bwa/bwa-0.7.5a/bwa', 'mem', '-t', '16', '-c', '100', '-M', '-R', '@RG\tID:RG_ID\tSM:RG_SM\tPL:RG_PL\tLB:RG_LB\tPU:RG_PU', u'/keep/3514b8e5da0e8d109946bc809b20a78a+5698/human_g1k_v37.fasta', u'/keep/3eee80c078a748a950a818157af6a172+1119/xaa.fastq.gz']
log message:
2015-06-10_14:51:50 qr1hi-8i9sb-i52ckct0rg38xx6 3798 1 stderr 2015-06-10 14:51:50 arvados.arvados_fuse[976] WARNING: Block not found: 984fbe6728b638cfb584f20e14026b3d+67108864+A251fa8a621e86009cc976a1f6af214b7e61d2fe6@558ac402 not found: http://keep4.qr1hi.arvadosapi.com:25107/ responded with 404 HTTP/1.1 404 Not Found\015
2015-06-10_14:51:50 qr1hi-8i9sb-i52ckct0rg38xx6 3798 1 stderr ; http://keep3.qr1hi.arvadosapi.com:25107/ responded with 404 HTTP/1.1 404 Not Found\015
2015-06-10_14:51:50 qr1hi-8i9sb-i52ckct0rg38xx6 3798 1 stderr ; http://keep2.qr1hi.arvadosapi.com:25107/ responded with 404 HTTP/1.1 404 Not Found\015
2015-06-10_14:51:50 qr1hi-8i9sb-i52ckct0rg38xx6 3798 1 stderr ; http://keep5.qr1hi.arvadosapi.com:25107/ responded with 404 HTTP/1.1 404 Not Found\015
2015-06-10_14:51:50 qr1hi-8i9sb-i52ckct0rg38xx6 3798 1 stderr ; http://keep1.qr1hi.arvadosapi.com:25107/ responded with 404 HTTP/1.1 404 Not Found\015
2015-06-10_14:51:50 qr1hi-8i9sb-i52ckct0rg38xx6 3798 1 stderr ; http://keep0.qr1hi.arvadosapi.com:25107/ responded with 404 HTTP/1.1 404 Not Found\015
2015-06-10_14:51:50 qr1hi-8i9sb-i52ckct0rg38xx6 3798 1 stderr ; http://keep6.qr1hi.arvadosapi.com:25107/ responded with 404 HTTP/1.1 404 Not Found\015
2015-06-10_14:51:50 qr1hi-8i9sb-i52ckct0rg38xx6 3798 1 stderr ; http://keep7.qr1hi.arvadosapi.com:25107/ responded with 404 HTTP/1.1 404 Not Found\015
2015-06-10_14:51:50 qr1hi-8i9sb-i52ckct0rg38xx6 3798 1 stderr
https://cloud.curoverse.com/collections/2fc686a85146e32cb0c6a707d43add28+85/qr1hi-8i9sb-i52ckct0rg38xx6.log.txt?disposition=inline&size=32378