Arvados

It would be nice to have a command line tool that was able to receive log information from currently running jobs so that it could be monitored from the command line rather than going through the 'Log' page in workbench.

This should look just like crunch-job in local mode, or the Log tab of a running job in Workbench: no json formatting, etc., just the text.

For pipeline instances, this should be implemented as arv-run-pipeline-instance --log-only --instance {uuid} because a-r-p-i will need to make use of websockets soon enough anyway (but combining --log with --run-*-here is a different story, #4503).

Something like this should make it easy to share code between --log-only and the "don't poll" part of #4503:

def each_pipeline_state_change(my_instance_uuid)
  # yield when pipeline changes state. return when pipeline has finished
  subscribe_to_websocket my_instance_uuid
  listen_to_websocket do |msg|
    if {msg is a pipelineInstance update event with uuid==my_instance_uuid}
      if {new instance has different job uuids, job states, etc. than last time}
        subscribe_to_websocket {any new job uuids that have appeared}
        yield new_instance_properties
      end
      if {pipeline is not running}
        return
      end
    elsif {logging mode is enabled} and {msg is a stderr event for a job in this instance}
      print msg.properties.text
    end
  end
end

For the case of a single job, this could just as well be a standalone ruby program.

arv tail {uuid} should cause arv to invoke the appropriate program, depending on whether {uuid} is a pipeline_instance or a job uuid.