Arvados

source:apps/workbench/app/models/pipeline_instance.rb has

  def stderr_log_lines(limit=2000)
    stderr_log_query(limit).results.reverse.
      flat_map { |log| log.properties[:text].split("\n") rescue [] }
  end

and source:apps/workbench/app/views/pipeline_instances/_show_log.html.erb has

       ><%= @object.stderr_log_lines.join("\n") %></div>

I'm hoping we just need to add analogous lines to job model/view.

Alternately, in job_log_graph.js:

        $.get('/jobs/' + $(graph_div).data('object-uuid') + '/logs.json', function(data) {
            data.forEach( function( entry ) {
                processLogEventForGraph({}, entry);
            });
        });

The forEach() loop could add each entry to the log window.

Yes, using the logs we're already fetching sounds even better.

edit: ...except that -- unlike the chart -- the log window doesn't know how to deal with logs that arrive out of order. If we connect to websocket and request historical logs at the same time, we sometimes log messages will arrive twice and sometimes the current logs will arrive before the 10-minute history. Showing duplicated/out-of-order logs would be really confusing/annoying. I don't think we should go with this approach unless we also tackle the de-dup/reordering problem.

• when subscribing to websockets, use filters=['created_at', '>=', (now - 10 minutes)]
• currently we subscribe to everything, in one place: source:apps/workbench/app/assets/javascripts/event_log.js#L34 ...asking for 10 minutes worth of history (before getting to now) here would make the whole auto-update system even more wasteful/slow.
• instead we could add a new function "addEventLogSubscription" in event_log.js which sends an additional "subscribe" message to the server, and invoke this from the job log tab. We could also include the relevant job UUID so we don't get useless historical messages from other jobs. (We'd still get useless live updates, because of the "subscribe to everything" thing we already do, but at least we don't have to make it even worse.)

Great!

Just one thing, at the very top it is rendering like this:

  2016-02-24_12:40:31 wx7k5-8i9sb-4f4cixopdp1inoq 65450 0 stderr crunchstat: cpu 70.2600 user 24.5800 sys 16 cpus -- interval 10.0006 seconds 0.0000 user 0.0000 sys 
2016-02-24_12:40:31 wx7k5-8i9sb-4f4cixopdp1inoq 65450 0 stderr crunchstat: net:eth0 2625454730 tx 48905114 rx -- interval 10.0015 seconds 0 tx 0 rx

Instead of this:

2016-02-24_12:40:31 wx7k5-8i9sb-4f4cixopdp1inoq 65450 0 stderr crunchstat: cpu 70.2600 user 24.5800 sys 16 cpus -- interval 10.0006 seconds 0.0000 user 0.0000 sys 
2016-02-24_12:40:31 wx7k5-8i9sb-4f4cixopdp1inoq 65450 0 stderr crunchstat: net:eth0 2625454730 tx 48905114 rx -- interval 10.0015 seconds 0 tx 0 rx

I think you need to remove the whitespace between the closing > of the <div tag and the start of the template:

<div id="event_log_div" 
     class="arv-log-event-listener arv-log-event-handler-append-logs arv-job-log-window" 
     data-object-uuid="<%= @object.uuid %>" >
  <%= @object.stderr_log_lines(Rails.configuration.running_job_log_lines_to_fetch).join("\n") %>
</div>

<div id="event_log_div" 
     class="arv-log-event-listener arv-log-event-handler-append-logs arv-job-log-window" 
     data-object-uuid="<%= @object.uuid %>" 
  ><%= @object.stderr_log_lines(Rails.configuration.running_job_log_lines_to_fetch).join("\n") %>
</div>

Also, running_job_log_lines_to_fetch is not quite an accurate name, it is actually the number of log records (which may include multiple lines of text).

Renamed running_job_log_lines_to_fetch as running_job_log_records_to_fetch