Arvados

review at c7cbb121dec3c356bf7d72087d601c7593410b8c

I see that disable_api_methods: ["jobs.create", "pipeline_instances.create"] is short compared what we've been doing #10367#note-10. Should we include more?

From my perspective, I'd like customer to be aware that "disabling all api jobs"(like in 10367) versus "making the cluster unable to accept jobs", since users will be questioning "why is this 404/422/500 error showing up, and I can see I did this before". What I mean here: is the message oriented for the sysadmin or for the user, and if it's the later, it should be clear if the cluster is able to do the task or not, and what is the documentation for helping the user.

Nico César wrote:

review at c7cbb121dec3c356bf7d72087d601c7593410b8c

I see that disable_api_methods: ["jobs.create", "pipeline_instances.create"] is short compared what we've been doing #10367#note-10. Should we include more?

From my perspective, I'd like customer to be aware that "disabling all api jobs"(like in 10367) versus "making the cluster unable to accept jobs", since users will be questioning "why is this 404/422/500 error showing up, and I can see I did this before". What I mean here: is the message oriented for the sysadmin or for the user, and if it's the later, it should be clear if the cluster is able to do the task or not, and what is the documentation for helping the user.

Those two methods are the ones which clients (a-c-r, workbench) use to probe for whether the jobs/pipelines API is available or not. So disabling those is the minimum required to alter client behavior.

The message is a little bit tricky. The assumption is that what the user wants is to get rid of the warning and use the containers API by default. However, currently we only support that by disabling the jobs API entirely.

So another approach would be to introduce a user preferences file, so the user could do something like "arvados-cwl-runner --set-default-api=containers" and then have it remember that.

Peter Amstutz wrote:

Nico César wrote:

review at c7cbb121dec3c356bf7d72087d601c7593410b8c

I see that disable_api_methods: ["jobs.create", "pipeline_instances.create"] is short compared what we've been doing #10367#note-10. Should we include more?

From my perspective, I'd like customer to be aware that "disabling all api jobs"(like in 10367) versus "making the cluster unable to accept jobs", since users will be questioning "why is this 404/422/500 error showing up, and I can see I did this before". What I mean here: is the message oriented for the sysadmin or for the user, and if it's the later, it should be clear if the cluster is able to do the task or not, and what is the documentation for helping the user.

Those two methods are the ones which clients (a-c-r, workbench) use to probe for whether the jobs/pipelines API is available or not. So disabling those is the minimum required to alter client behavior.

Is that 100% true on workbench? the reason I brought up this is because the customer kept finding ways that workbench was failing. if you see #10367 the list was growing over time. because of this reason

The message is a little bit tricky. The assumption is that what the user wants is to get rid of the warning and use the containers API by default. However, currently we only support that by disabling the jobs API entirely.

I see, but "disabling the jobs api" is an instruction for the sysadmin. Once done, pipelines will fail to the user, without any understanding why. Just like something like 404 or MethodNotFound.

It will be good to say "hey, user!, we're deprecating this feature still supported in your cluster, here <URL FOR USERS> is a guide on how to migrate to containers API, besides you can try it out with --api=containers right away. Hey sysadmin, when you're ready to deprecate follow this guide <URL FOR SYSADMIN>". And once is disabled, "hey, user!, the feature you are requesting has been deprecated, here <URL FOR USERS> is a guide on how to migrate to containers API".

Does it make sense? maybe I'm overdoing the messages, but it seems to me that helps a LOT the communication with users, since the cluster parameters are opaque to them, they only see what the tools report.

So another approach would be to introduce a user preferences file, so the user could do something like "arvados-cwl-runner --set-default-api=containers" and then have it remember that.

Is this assuming a stateful arvados-cwl-runner running from a shell node? remember that some of our runs are from automatic pipelines and some on sporadic VMs that only have the API token. I don't think this approach is the right one, but I could be wrong.

review at e8ebb2101fa89294c91b404953cc2fbabb796e40

Looks good to me. Ready to merge.