Story #6279

[Workbench] Integrate a Web shell client

Added by Brett Smith over 6 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assigned To:
Radhika Chippada
Category:
-
Target version:
Start date:
06/22/2015
Due date:
06/22/2015
% Done:

100%

Estimated time:
(Total: 2.00 h)
Story points:
1.0

Description

  • Create a page where a user can connect to an Arvados shell they have access to. Integrate the Web client developed during #2256.
  • Link to this page at appropriate places (Tom to determine where).
  • Update our user guide to suggest using this as the primary way to connect to shell accounts. Retain the information about configuring standalone SSH clients for users who prefer that.
webshell.zip (45.8 KB) webshell.zip Nico César, 07/01/2015 03:39 PM

Subtasks

Task #6292: Review branch: 6279-web-shell-clientResolvedTom Clegg


Related issues

Follows (1 day) Arvados - Story #2256: [Workbench] Users can access an Arvados shell through the WebResolved06/18/2015

Associated revisions

Revision c47db46c
Added by Tom Clegg over 6 years ago

Merge branch '6279-web-shell-client' closes #6279

Revision 340a9ffa
Added by Radhika Chippada over 6 years ago

refs #6279
Merge branch '6279-web-shell-client'

Revision 28bdc3c6 (diff)
Added by Radhika Chippada over 6 years ago

refs #6279
Increased timeout to 2000ms; with the previous 1000ms value, the login to shell account was failing
sometimes as the token is being sent too early. With this value, it seems to work consistently.

History

#1 Updated by Brett Smith over 6 years ago

  • Description updated (diff)
  • Story points changed from 2.0 to 3.0

#2 Updated by Radhika Chippada over 6 years ago

  • Assigned To set to Radhika Chippada

#3 Updated by Nico César over 6 years ago

See webshell.zip attachment,

It should have all the minimal expression files to login.

see index.html for the USER and PASSWORD to be replaced.

also you'll notice https://shell.4xphq.arvadosapi.com/ShellInABox.js URL.. that should be replaced with the shell box that will be connecting to.

#4 Updated by Nico César over 6 years ago

Ok we have this half working. (API token as password will be solved when #6384 is finished.)

But you can see progress in https://webshell.9tee4.arvadosapi.com/science/ and https://webshell.9tee4.arvadosapi.com/shell/

#5 Updated by Radhika Chippada over 6 years ago

  • Status changed from New to In Progress

#6 Updated by Nico César over 6 years ago

Radhika, try using:

https://webshell.4xphq.arvadosapi.com/shell/

it will expect username.
then it will expect the API token but it won't say anything. (no "password:" not "API token:")

if you send it with an enter should be good to go.

#7 Updated by Brett Smith over 6 years ago

  • Target version changed from 2015-07-08 sprint to 2015-07-22 sprint

#8 Updated by Radhika Chippada over 6 years ago

  • Story points changed from 3.0 to 1.0

#9 Updated by Radhika Chippada over 6 years ago

The '-' and '_' are not working in Firefox. Upon googling, found this URL that suggested a workaround: https://code.google.com/p/shellinabox/issues/detail?id=202

#10 Updated by Radhika Chippada over 6 years ago

Regarding branch 6279-web-shell-client at c5e726c:

  • Added /webshell as a route to workbench and it displays a modified index.html (renamed as webshell.html.erb) as a partial.
  • If hostname has '.shell' suffix, this is stripped as per Nico's suggestion. Please let me know if this needs further tweaking.
  • Added Login link to Virtual Machine panel in my_account page that takes the user to web shell.
  • The files such as shell_in_a_box.js, css files, keyboard.html etc are added to workbench public folder. It seems like this is the only way that these files should be arranged to make them visible for one another. These are taken from shellinabox distro as is and need not be reviewed.
  • The characters - and _ were not displayed in the shell window for FireFox. I added a workaround as described in note #9. Chrome did not have any issues. It now appears that all keys work in both FireFox and Chrome browsers.
  • To test the implementation, point your localhost workbench to 4xphq api server and add config parameter to application.yml.
  shell_in_a_box_url: https://webshell.4xphq.arvadosapi.com  
  • No automated tests are added. Only manual testing is done. My current idea is: developers will leave the config parameter as false. Please do let me know if our development environment supports local configurations, in which case I might be able to add some automated tests.

#11 Updated by Radhika Chippada over 6 years ago

I noticed one issue after logout. I clicked on the Login link in my_account page and now I am in the shell terminal. I am able to type commands as needed. I then logged out using ^D or by typing logout. Logout happens and everything looks good. However, if I were to open the javascript console log in the browser, I see a constant flow of requests being sent to shell server.

 
[09:47:43.814] POST https://webshell.4xphq.arvadosapi.com/shell? [HTTP/1.1 400 Bad Request 828ms]
[09:47:44.643] OPTIONS https://webshell.4xphq.arvadosapi.com/shell? [HTTP/1.1 204 No Content 20ms]

Nico and I spent sometime and did not find any quick solutions or understand why this is happening. And Nico thinks that we can create a bug for this and get back to it later as time permits.

#12 Updated by Tom Clegg over 6 years ago

First of all, this is awesome.

Some fixes

I've fixed a few things and pushd to 6279-web-shell-client @ eb4fb54 (see commit message). WDYT?

Infinite network activity after disconnect

I too get an infinite loop of 400 requests after disconnecting. IMO we need to fix this before merging. It makes your browser look inactive but be extremely busy. (I think I'm not alone when I say I hate discovering that some web app has been silently making my whole computer slow for no reason!)

The current proxy setup on 4xphq has

        add_header 'Access-Control-Allow-Origin' '*'; ## FIXME: after integration this should be restricted only to workbench
        add_header 'Access-Control-Allow-Credentials' 'true';

Is there any reason we actually need "Allow-Credentials" here? We can't have both of these. That's such a bad idea that some browsers are smart enough to ignore the Allow-Credentials header if the server admits to having such low standards. Of the two, I'd rather have "Allow-Origin *". (We don't actually want to restrict webshell origins to "the official workbench", right?)

It looks like ShellInABox is trying to reuse (or create?) a session after it gets disconnected. We don't want this, right?

I tried commenting out the "Allow-Credentials" line in the nginx config, and everything worked equally well -- so that makes me suspect Allow-Credentials is only useful for whatever SIAB is trying to do after disconnecting.

Wish list

I do wish there was some feedback when login fails. Even printing "Disconnected" when disconnecting would help a lot. But I slapped a "beta" sticker on the "manage account" page and with that I'm OK to merge with the "no feedback on login failure" bug.

I wish the styling was a bit better (maybe open in a separate smaller window, more like a "real" terminal window?) -- but let's merge and get user feedback about that instead of bikeshedding amongst ourselves. :)

#13 Updated by Radhika Chippada over 6 years ago

Tom:
Thanks much for the config updates and route updates. So much better.

That said, I see some big issues in this new revision, as listed below.

  • First a simple one: You said, "maybe open in a separate smaller window, more like a "real" terminal window?"
    • Do we want to use target: "_blank" to open the webshell client in a separate window? (similar to our Help menu items)
    • I am thinking there is no need to have webshell accessible outside of workbench (regarding one of your header related comments).
    • Regarding the infinite loop issue, if we were to use _blank target, would we be able to sense a logout request
      in javascript and close the browser tab?
  • I think the files need to be in /public directory; not /public/webshell directory. Some serious observations on this below.
  • Infinite loop issue while logged into the shell (not after logout): Login to webshell. Open the javascript console.
    Wait for a minute. Then you will notice that the infinite chain commands are happening by looking at the console log.
    If you try to type in commands in the terminal window, yes, they work. I am able to work without any issues. However, the
    constant flow if those requests keeps going on. Few more observations about this:
    • This infinite loop issue is not present when I test with my earlier commit at f19af568
    • The infinite loop issue is not present when I directly access to webshell url in 4xphq (not via workbench)
    • Thus, it appears that this issue during non-logout is most likely due to resources not accessible in this directory structure.
  • Another issue. Try this: Login to webshell.
    Right mouse click and access "Onscreen keyboard" option. This no longer works. If you look at the workbench console,
    you will notice errors such as the following. Both keyboard.gif and enabled.gif cannot be found in this setup
    (Yes, I did copy the keyboard.html, keyboard.gif, and enabled.gif into public/webshell directory).
    Started GET "/virtual_machines/4xphq-2x53u-kr9qek84xtdh4aw/webshell/enabled.gif" for 127.0.0.1 at 2015-07-09 20:37:59 -0400
    Processing by VirtualMachinesController#webshell as GIF
      Parameters: {"id"=>"4xphq-2x53u-kr9qek84xtdh4aw", "login"=>"enabled"}
    API client: 0.000534413 Prepare request GET https://4xphq.arvadosapi.com/arvados/v1/users/current  {"api_token"=>"1eryebw6imuqmlpvjm6u77ikgs83n2zov3z7bp7qxrmnuyoqcf", "reader_tokens"=>"[false]", "_method"=>"GET", "_profile"=>"true"}
    API client: 0.217603151 API transaction
    Content-Encoding nil, Content-Length nil, actual content size 903
    API client: 0.000397177 Parse response
    API client: 7.7144e-05 Prepare request GET https://4xphq.arvadosapi.com/arvados/v1/virtual_machines/4xphq-2x53u-kr9qek84xtdh4aw  {"api_token"=>"1eryebw6imuqmlpvjm6u77ikgs83n2zov3z7bp7qxrmnuyoqcf", "reader_tokens"=>"[false]", "_method"=>"GET", "_profile"=>"true"}
    API client: 0.062800536 API transaction
    Content-Encoding nil, Content-Length nil, actual content size 425
    API client: 0.000316958 Parse response
    #<ActionView::MissingTemplate: Missing template virtual_machines/webshell, application/webshell with {:locale=>[:en], :formats=>[:gif], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :coffee]}. Searched in:
      * "/home/radhika/arvados/apps/workbench/themes/default/views" 
      * "/home/radhika/arvados/apps/workbench/app/views" 
      * "/home/radhika/arvados/apps/workbench/vendor/bundle/ruby/2.1.0/gems/wiselinks-1.2.1/app/views" 
    
  • I think it is most important to have all these features working than having the files in a separate public directory.
    If it turns out /public/webshell does not work, I think we should move the files back to /public directory and rename
    README as WEBSHELL_README ...
  • Given all these issues (see below), I agree that we need to revisit the wishlist later on ...
  • I have not yet paid much attention to the header thoughts you brought out and am focusing on these other issues first.

Thanks.

#14 Updated by Radhika Chippada over 6 years ago

I logged into webshell client through my workbench (using of course my older revision at f19af568) and I noticed a few good things.

  • After two hours, the webshell terminal is still available and accessible and still doing what it is supposed to do.
  • The client seems to be sending a ping to the service at 45 minutes interval (just one post request every 45 seconds), which seems to be keeping the session alive.

#15 Updated by Tom Clegg over 6 years ago

Radhika Chippada wrote:

  • First a simple one: You said, "maybe open in a separate smaller window, more like a "real" terminal window?"

Let's punt this topic for now. It's "nice to have" at best.

  • I am thinking there is no need to have webshell accessible outside of workbench (regarding one of your header related comments).

The issue is that I don't want the webshell setup to depend on any system-level components trusting "the official Workbench" any more than it trusts other random web apps that the user trusts.

Otherwise, we end up in a situation where a developer asks a site admin "can you please tell your webshell to trust my customized workbench-like application so I can use it to launch shell VM sessions?" and the site admin has to say "no, some other users do not trust your customized workbench, so you must replicate the whole shell VM infrastructure yourself in order to achieve that."

  • Regarding the infinite loop issue, if we were to use _blank target, would we be able to sense a logout request
    in javascript and close the browser tab?

Hm, I would find that annoying in a couple of circumstances, including "left a command running and came back the next morning to see how it ended" and "my session gets disconnected now and then" and "error occurs during login stage" (argh, the window closes by itself while I'm trying to copy the error message)... I think there's probably a good reason why "close on disconnect" is usually (always?) optional for terminal apps...

  • Infinite loop issue while logged into the shell (not after logout): Login to webshell. Open the javascript console.
    Wait for a minute. Then you will notice that the infinite chain commands are happening by looking at the console log.

Hm, I don't see this behavior -- for me, the infinite loop . Difference between FF and Chrome? Perhaps fixed by assets (see below)?

  • Another issue. Try this: Login to webshell.
    Right mouse click and access "Onscreen keyboard" option. This no longer works. If you look at the workbench console,
    you will notice errors such as the following. Both keyboard.gif and enabled.gif cannot be found in this setup
    (Yes, I did copy the keyboard.html, keyboard.gif, and enabled.gif into public/webshell directory).

I've pushed eb4fb54 which moves those assets into public/webshell and updates the references to them in shell_in_a_box.js. This works for me -- how about you?

  • Given all these issues (see below), I agree that we need to revisit the wishlist later on ...

Right. Let's not think too hard about wishlist items until this is merged. :)

  • I have not yet paid much attention to the header thoughts you brought out and am focusing on these other issues first.

Fair enough, except to the extent CORS header problems might be related to the infinite loop of XHR → 400 errors...

#16 Updated by Tom Clegg over 6 years ago

Try branch at 1010035

I think a4926df fixes the infinite loop of failed requests.

#17 Updated by Radhika Chippada over 6 years ago

Tom said:

I think a4926df fixes the infinite loop of failed requests.

Yes, the infinite loop issue is resolved with this revision. Thank you. Everything looks great now.

One question. Now after logout, I see the "Connect" button which actually is clickable and brings me to the login prompt. However, assuming the user does not know how to login, it is really not helpful. How can they know that the token from the wb my account page can be copied and pasted? So my question is, when the user clicks on Connect button, would it make sense to invoke the login method as when we do with page load case?

#18 Updated by Tom Clegg over 6 years ago

  • Status changed from In Progress to Resolved
  • % Done changed from 0 to 100

Applied in changeset arvados|commit:c47db46c7f2aac3ad1b099d8716c44dbb9f8bbf6.

Also available in: Atom PDF