Feature #12167
Updated by Tom Clegg over 6 years ago
When an arv-put process is slow or broken, it is useful to grep logs for related server-side activity. To make this easy, arv-put should generate a "request ID" at startup, and all components should propagate this ID to log files and other Arvados components. Specifically: * A request ID looks like "req-" + 20 random alphanum chars, e.g., @req-iNjNZUDqU35yxSXFYJ8x@ * If running in --verbose mode, arv-put should log the request ID order to stderr as soon as it's generated. * Whether running in --verbose mode or not, arv-put should log track easily the request ID in every error log. Presumably the easiest way uploads to do this is to include arvados it in _every_ log message, just like timestamp and PID. Thus, a failed arv-put will _always_ log its request ID, because every failure mode causes be good if we have a log (make sure capability like workbench's "current_request_id" for all keep/keeproxy's calls this is true even for SIGINT). * Each keepstore/keepproxy/API request made by arv-put should include a request header like @X-Request-Id: req-iNjNZUDqU35yxSXFYJ8x@ (no ETag-esque quoting). * When keepproxy receives an X-Request-Id request header, it should include that same header when it makes keepstore/API requests has to appear in the course of servicing the incoming request. It should also include the header value in the corresponding request log. * When keepstore receives an X-Request-Id request header, it should include the header value in the corresponding request log. * When API server receives an X-Request-Id request header, it should include the header value in the corresponding request log. Of course, incoming headers can be broken or malicious, so keepproxy's logs should use appropriate quoting. Related: * Workbench should include its request ID in an X-Request-Id header instead of a "current_request_id" query parameter. This way, API server will always log it. and api's logs