Arvados

Ok we have this half working. (API token as password will be solved when #6384 is finished.)

But you can see progress in https://webshell.9tee4.arvadosapi.com/science/ and https://webshell.9tee4.arvadosapi.com/shell/

Radhika, try using:

https://webshell.4xphq.arvadosapi.com/shell/

it will expect username.
then it will expect the API token but it won't say anything. (no "password:" not "API token:")

if you send it with an enter should be good to go.

The '-' and '_' are not working in Firefox. Upon googling, found this URL that suggested a workaround: https://code.google.com/p/shellinabox/issues/detail?id=202

Regarding branch 6279-web-shell-client at c5e726c:

• Added /webshell as a route to workbench and it displays a modified index.html (renamed as webshell.html.erb) as a partial.

• If hostname has '.shell' suffix, this is stripped as per Nico's suggestion. Please let me know if this needs further tweaking.

• Added Login link to Virtual Machine panel in my_account page that takes the user to web shell.

• The files such as shell_in_a_box.js, css files, keyboard.html etc are added to workbench public folder. It seems like this is the only way that these files should be arranged to make them visible for one another. These are taken from shellinabox distro as is and need not be reviewed.

• The characters - and _ were not displayed in the shell window for FireFox. I added a workaround as described in note #9. Chrome did not have any issues. It now appears that all keys work in both FireFox and Chrome browsers.

• To test the implementation, point your localhost workbench to 4xphq api server and add config parameter to application.yml.

  shell_in_a_box_url: https://webshell.4xphq.arvadosapi.com  

• No automated tests are added. Only manual testing is done. My current idea is: developers will leave the config parameter as false. Please do let me know if our development environment supports local configurations, in which case I might be able to add some automated tests.

I noticed one issue after logout. I clicked on the Login link in my_account page and now I am in the shell terminal. I am able to type commands as needed. I then logged out using ^D or by typing logout. Logout happens and everything looks good. However, if I were to open the javascript console log in the browser, I see a constant flow of requests being sent to shell server.

 
[09:47:43.814] POST https://webshell.4xphq.arvadosapi.com/shell? [HTTP/1.1 400 Bad Request 828ms]
[09:47:44.643] OPTIONS https://webshell.4xphq.arvadosapi.com/shell? [HTTP/1.1 204 No Content 20ms]

Nico and I spent sometime and did not find any quick solutions or understand why this is happening. And Nico thinks that we can create a bug for this and get back to it later as time permits.

First of all, this is awesome.

Some fixes¶

I've fixed a few things and pushd to 6279-web-shell-client @ eb4fb54 (see commit message). WDYT?

Infinite network activity after disconnect¶

I too get an infinite loop of 400 requests after disconnecting. IMO we need to fix this before merging. It makes your browser look inactive but be extremely busy. (I think I'm not alone when I say I hate discovering that some web app has been silently making my whole computer slow for no reason!)

The current proxy setup on 4xphq has

        add_header 'Access-Control-Allow-Origin' '*'; ## FIXME: after integration this should be restricted only to workbench
        add_header 'Access-Control-Allow-Credentials' 'true';

Is there any reason we actually need "Allow-Credentials" here? We can't have both of these. That's such a bad idea that some browsers are smart enough to ignore the Allow-Credentials header if the server admits to having such low standards. Of the two, I'd rather have "Allow-Origin *". (We don't actually want to restrict webshell origins to "the official workbench", right?)

It looks like ShellInABox is trying to reuse (or create?) a session after it gets disconnected. We don't want this, right?

I tried commenting out the "Allow-Credentials" line in the nginx config, and everything worked equally well -- so that makes me suspect Allow-Credentials is only useful for whatever SIAB is trying to do after disconnecting.

Wish list¶

I do wish there was some feedback when login fails. Even printing "Disconnected" when disconnecting would help a lot. But I slapped a "beta" sticker on the "manage account" page and with that I'm OK to merge with the "no feedback on login failure" bug.

I wish the styling was a bit better (maybe open in a separate smaller window, more like a "real" terminal window?) -- but let's merge and get user feedback about that instead of bikeshedding amongst ourselves. :)

Tom:
Thanks much for the config updates and route updates. So much better.

That said, I see some big issues in this new revision, as listed below.

• First a simple one: You said, "maybe open in a separate smaller window, more like a "real" terminal window?"
  • Do we want to use target: "_blank" to open the webshell client in a separate window? (similar to our Help menu items)
  • I am thinking there is no need to have webshell accessible outside of workbench (regarding one of your header related comments).
  • Regarding the infinite loop issue, if we were to use _blank target, would we be able to sense a logout request
    in javascript and close the browser tab?

• I think the files need to be in /public directory; not /public/webshell directory. Some serious observations on this below.

• Infinite loop issue while logged into the shell (not after logout): Login to webshell. Open the javascript console.
  Wait for a minute. Then you will notice that the infinite chain commands are happening by looking at the console log.
  If you try to type in commands in the terminal window, yes, they work. I am able to work without any issues. However, the
  constant flow if those requests keeps going on. Few more observations about this:
  • This infinite loop issue is not present when I test with my earlier commit at f19af568
  • The infinite loop issue is not present when I directly access to webshell url in 4xphq (not via workbench)
  • Thus, it appears that this issue during non-logout is most likely due to resources not accessible in this directory structure.

• Another issue. Try this: Login to webshell.
  Right mouse click and access "Onscreen keyboard" option. This no longer works. If you look at the workbench console,
  you will notice errors such as the following. Both keyboard.gif and enabled.gif cannot be found in this setup
  (Yes, I did copy the keyboard.html, keyboard.gif, and enabled.gif into public/webshell directory).
  

  Started GET "/virtual_machines/4xphq-2x53u-kr9qek84xtdh4aw/webshell/enabled.gif" for 127.0.0.1 at 2015-07-09 20:37:59 -0400
  Processing by VirtualMachinesController#webshell as GIF
    Parameters: {"id"=>"4xphq-2x53u-kr9qek84xtdh4aw", "login"=>"enabled"}
  API client: 0.000534413 Prepare request GET https://4xphq.arvadosapi.com/arvados/v1/users/current  {"api_token"=>"1eryebw6imuqmlpvjm6u77ikgs83n2zov3z7bp7qxrmnuyoqcf", "reader_tokens"=>"[false]", "_method"=>"GET", "_profile"=>"true"}
  API client: 0.217603151 API transaction
  Content-Encoding nil, Content-Length nil, actual content size 903
  API client: 0.000397177 Parse response
  API client: 7.7144e-05 Prepare request GET https://4xphq.arvadosapi.com/arvados/v1/virtual_machines/4xphq-2x53u-kr9qek84xtdh4aw  {"api_token"=>"1eryebw6imuqmlpvjm6u77ikgs83n2zov3z7bp7qxrmnuyoqcf", "reader_tokens"=>"[false]", "_method"=>"GET", "_profile"=>"true"}
  API client: 0.062800536 API transaction
  Content-Encoding nil, Content-Length nil, actual content size 425
  API client: 0.000316958 Parse response
  #<ActionView::MissingTemplate: Missing template virtual_machines/webshell, application/webshell with {:locale=>[:en], :formats=>[:gif], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :coffee]}. Searched in:
    * "/home/radhika/arvados/apps/workbench/themes/default/views" 
    * "/home/radhika/arvados/apps/workbench/app/views" 
    * "/home/radhika/arvados/apps/workbench/vendor/bundle/ruby/2.1.0/gems/wiselinks-1.2.1/app/views" 
  

• I think it is most important to have all these features working than having the files in a separate public directory.
  If it turns out /public/webshell does not work, I think we should move the files back to /public directory and rename
  README as WEBSHELL_README ...

• Given all these issues (see below), I agree that we need to revisit the wishlist later on ...

• I have not yet paid much attention to the header thoughts you brought out and am focusing on these other issues first.

Thanks.

I logged into webshell client through my workbench (using of course my older revision at f19af568) and I noticed a few good things.

• After two hours, the webshell terminal is still available and accessible and still doing what it is supposed to do.

• The client seems to be sending a ping to the service at 45 minutes interval (just one post request every 45 seconds), which seems to be keeping the session alive.

Radhika Chippada wrote:

• First a simple one: You said, "maybe open in a separate smaller window, more like a "real" terminal window?"

Let's punt this topic for now. It's "nice to have" at best.

• I am thinking there is no need to have webshell accessible outside of workbench (regarding one of your header related comments).

The issue is that I don't want the webshell setup to depend on any system-level components trusting "the official Workbench" any more than it trusts other random web apps that the user trusts.

Otherwise, we end up in a situation where a developer asks a site admin "can you please tell your webshell to trust my customized workbench-like application so I can use it to launch shell VM sessions?" and the site admin has to say "no, some other users do not trust your customized workbench, so you must replicate the whole shell VM infrastructure yourself in order to achieve that."

• Regarding the infinite loop issue, if we were to use _blank target, would we be able to sense a logout request
  in javascript and close the browser tab?

Hm, I would find that annoying in a couple of circumstances, including "left a command running and came back the next morning to see how it ended" and "my session gets disconnected now and then" and "error occurs during login stage" (argh, the window closes by itself while I'm trying to copy the error message)... I think there's probably a good reason why "close on disconnect" is usually (always?) optional for terminal apps...

• Infinite loop issue while logged into the shell (not after logout): Login to webshell. Open the javascript console.
  Wait for a minute. Then you will notice that the infinite chain commands are happening by looking at the console log.

Hm, I don't see this behavior -- for me, the infinite loop . Difference between FF and Chrome? Perhaps fixed by assets (see below)?

• Another issue. Try this: Login to webshell.
  Right mouse click and access "Onscreen keyboard" option. This no longer works. If you look at the workbench console,
  you will notice errors such as the following. Both keyboard.gif and enabled.gif cannot be found in this setup
  (Yes, I did copy the keyboard.html, keyboard.gif, and enabled.gif into public/webshell directory).

I've pushed eb4fb54 which moves those assets into public/webshell and updates the references to them in shell_in_a_box.js. This works for me -- how about you?

• Given all these issues (see below), I agree that we need to revisit the wishlist later on ...

Right. Let's not think too hard about wishlist items until this is merged. :)

• I have not yet paid much attention to the header thoughts you brought out and am focusing on these other issues first.

Fair enough, except to the extent CORS header problems might be related to the infinite loop of XHR → 400 errors...

Try branch at 1010035

I think a4926df fixes the infinite loop of failed requests.

Tom said:

I think a4926df fixes the infinite loop of failed requests.

Yes, the infinite loop issue is resolved with this revision. Thank you. Everything looks great now.

One question. Now after logout, I see the "Connect" button which actually is clickable and brings me to the login prompt. However, assuming the user does not know how to login, it is really not helpful. How can they know that the token from the wb my account page can be copied and pasted? So my question is, when the user clicks on Connect button, would it make sense to invoke the login method as when we do with page load case?