Idea #20521
open
Python tools emit logs from googleapiclient
Description
googleapiclient logs some useful information, like when requests are automatically retried. Any time we're writing out arvados logs, we should write googleapiclient logs to the same place.
In the main branch today, the Python SDK automatically enables logging as soon as it's imported. From sdk/python/arvados/__init__.py:
# Set up Arvados logging based on the user's configuration.
# All Arvados code should log under the arvados hierarchy.
log_format = '%(asctime)s %(name)s[%(process)d] %(levelname)s: %(message)s'
log_date_format = '%Y-%m-%d %H:%M:%S'
log_handler = logging.StreamHandler()
log_handler.setFormatter(logging.Formatter(log_format, log_date_format))
logger = logging.getLogger('arvados')
logger.addHandler(log_handler)
logger.setLevel(logging.DEBUG if config.get('ARVADOS_DEBUG')
else logging.WARNING)
The simplest version would be to add a line to the end of this stanza:
_apiclient_logger = logging.getLogger('googleapiclient')
_apiclient_logger.addHandler(log_handler)
_apiclient_logger.setLevel(logger.getLevel())
However, a library messing with global state like this, without an explicit request, is kind of antisocial. This might be a good opportunity to add an explicit "setup logging" function to the PySDK. It can function similarly to logging.basicConfig: take a level and maybe a stream or handler as arguments, then set that up for arvados and other modules we want to hear from like googleapiclient. This would give us a good place to make other changes like this in the future, like adding other modules to the logging list, without messing with the global state of all code that says import arvados.
Related issues
Updated by 
Updated by