Arvados

Here's the Go code that calculates the actual RAM request.

So we want to subtract all the overhead from the topline instance RAM to get the maximum value of "RuntimeConstraints.RAM" that the user can provide to get this instance type.

    needRAM := ctr.RuntimeConstraints.RAM + ctr.RuntimeConstraints.KeepCacheRAM
    needRAM += int64(cc.Containers.ReserveExtraRAM)
    if cc.Containers.LocalKeepBlobBuffersPerVCPU > 0 {
        // + 200 MiB for keepstore process + 10% for GOGC=10
        needRAM += 220 << 20
        // + 64 MiB for each blob buffer + 10% for GOGC=10
        needRAM += int64(cc.Containers.LocalKeepBlobBuffersPerVCPU * needVCPUs * (1 << 26) * 11 / 10)
    }
    needRAM = (needRAM * 100) / int64(100-discountConfiguredRAMPercent)

Peter Amstutz wrote in #note-24:

Here's the Go code that calculates the actual RAM request.

#20801 is tangentially related. If this ticket happens first, sure, fine, but then if we ever want to do #20801, it should probably grow to include undoing the math in Workbench 2 as well.

Changes at arvados-workbench2|038d67a8c9ebf8a462774e9b870e7f1603c01959 branch 19675-instance-types-panel
Tests developer-tests-workbench2: #1397

• All agreed upon points are implemented / addressed.
  • Added instance types panel requiring adding a few fields to the arvados config type
  • Add breadcrumbs for instance types panel
  • Add unit tests for instance types panel
  • Add missing breadcrumbs navigation handlers
  • Move process spec 'no value io parameter test' loginas into createContainerRequest then body and use admin user which is the only way I've figured out how to make that test pass on jenkins (it works fine locally for some reason)
• Anything not implemented (discovered or discussed during work) has a follow-up story.
  • Added #21253 to add MaxInstances and SupervisorFraction to the exported config so that #21256 can add them in the top box
• Code is tested and passing, both automated and manual, what manual testing was done is described
  • done
• Documentation has been updated.
  • n/a
• Behaves appropriately at the intended scale (describe intended scale).
  • intended scale is expected to be on the order of <50 instance types, beyond that the UI might be difficult to use
• Considered backwards and forwards compatibility issues between client and server.
  • n/a
• Follows our coding standards and GUI style guidelines.
  • done

I merged the changes into a branch on the main arvados repo at arvados|ee9d1e39b5d469a827be5a719c9c0860914ab2a8 branch 19675-instance-types-panel

It turns out the cypress test fix didn't actually fix the issue, it's still just flakey in probably the same way that Tom was trying to fix, so I was able to get it to pass without my fix by rerunning: developer-run-tests-services-workbench2: #84

Reviewing 19675-instance-types-panel @ ee9d1e39b5d469a827be5a719c9c0860914ab2a8

• These don't seem to be in an particular order. Can we sort them by price from lowest to highest?
• RAM should be capitalized in "Max ram request"
• Even though LocalKeepBlobBuffersPerVCPU isn't exported, I think we should calculate the maximum request assuming it is the default value. Also we should include the 5% discount (also assuming the default behavior for now). In other words, it should account for all the stuff in note-24, and just assume defaults where the configured values are not available.
  • defaults are LocalKeepBlobBuffersPerVCPU: 1 and discountConfiguredRAMPercent: 5
  • Since this is user-facing to help users set correct resource requests, it is going to be better if the values that are displayed under-shoot the maximum than show a maximum that is wrong and frustrates users because they don't get the instance type they intended. In the future, if/when we rework instance selection, we can update the formulas (or have the values provided from somewhere else).
• For CWL, the unit of memory is MiB. Let's display RAM and disk in mebibytes along with whatever units are more human-readable? e.g.
  Max disk request: 16252928 MiB (7.3 TiB)
  Max ram request: 15872 MiB (15.5 GiB)
• I realize the design sketch was a little weird, the order of the fields should be:
  • provider type
  • price
  • cores
  • RAM
  • disk
  • preemptible
  • GPU stuff
• It is currently a bunch of little cards on one big card (white on white). Can we get rid of the outer container card?
• Let's add it to the left panel, underneath "All Processes". In general I think we want to move towards phasing out the "user" menu except for user account-related stuff.
• Instead of the stack-of-servers icon, let's use the same stylized computer chip that is used for the "Resources" card of the process view.
  • Speaking of the "Resources" card, I feel like they should be formatted the same way (with the same labels, as much as possible, to emphasize the connection between the numbers), on the other hand having the field name on the top instead of to the left might needlessly waste vertical space, so I'm not sure.
• We should have some text at the top explaining what this page is and how to use it. E.g. "These are the cloud compute instance types configured for this cluster. The core count and maximum RAM request correspond to the greatest values you can put in the CWL Workflow ResourceRequest `minCores` and `minRAM` and still be scheduled on that instance type."

Changes at arvados|14a2e78ff5cd055216a9f6edc40dfb3f79122fa8 branch 19675-instance-types-panel
Tests developer-run-tests-services-workbench2: #98

• Instance types sorted by increasing cost, with an alphabetical sort of the ProviderType as a tiebreaker
• Fixed capitalization and field order
• Added keep buffer and discount ram percent accounting
• Added "CWL" size formatting and display MiB next to convenient units
• Removed container card
• Moved panel link to left panel
• Switched to resources icon
• Switched to DetailsAttribute component to better match resources panel
• Added info box to the top with the suggested text

Stephen Smith wrote in #note-32:

Changes at arvados|14a2e78ff5cd055216a9f6edc40dfb3f79122fa8 branch 19675-instance-types-panel
Tests developer-run-tests-services-workbench2: #98

• Instance types sorted by increasing cost, with an alphabetical sort of the ProviderType as a tiebreaker
• Fixed capitalization and field order
• Added keep buffer and discount ram percent accounting
• Added "CWL" size formatting and display MiB next to convenient units
• Removed container card
• Moved panel link to left panel
• Switched to resources icon
• Switched to DetailsAttribute component to better match resources panel
• Added info box to the top with the suggested text

This looks good, just one thing I noticed:

• The icon is missing from the collapsed left side panel. It appears correctly in the expanded left panel.

Rest LGTM.