Arvados

Idea #20521

Updated by Brett Smith 11 months ago

googleapiclient logs some useful information, like when requests are automatically retried. Any time we're writing out @arvados@ logs, we should write @googleapiclient@ logs to the same place. 

 In the main branch today, the Python SDK automatically enables logging as soon as it's imported. From @sdk/python/arvados/__init__.py@: 

 <pre class="python"> {% codeblock as python %} 
 # Set up Arvados logging based on the user's configuration. 
 # All Arvados code should log under the arvados hierarchy. 
 log_format = '%(asctime)s %(name)s[%(process)d] %(levelname)s: %(message)s' 
 log_date_format = '%Y-%m-%d %H:%M:%S' 
 log_handler = logging.StreamHandler() 
 log_handler.setFormatter(logging.Formatter(log_format, log_date_format)) 
 logger = logging.getLogger('arvados') 
 logger.addHandler(log_handler) 
 logger.setLevel(logging.DEBUG if config.get('ARVADOS_DEBUG') 
                 else logging.WARNING) 
 </pre> {% endcodeblock %} 

 The simplest version would be to add a line to the end of this stanza: 

 <pre class="python"> {% codeblock as python %} 
 _apiclient_logger = logging.getLogger('googleapiclient') 
 _apiclient_logger.addHandler(log_handler) 
 _apiclient_logger.setLevel(logger.getLevel()) 
 </pre> {% endcodeblock %} 

 However, a library messing with global state like this, without an explicit request, is kind of antisocial. This might be a good opportunity to add an explicit "setup logging" function to the PySDK. It can function similarly to @logging.basicConfig@: take a level and maybe a stream or handler as arguments, then set that up for @arvados@ and other modules we want to hear from like @googleapiclient@. This would give us a good place to make other changes like this in the future, like adding other modules to the logging list, without messing with the global state of all code that says @import arvados@.

Back