Feature #19675
closedPanel that lists configured instance types
Added by Peter Amstutz about 2 years ago. Updated 9 months ago.
Description
Render the contents of config.InstanceTypes nicely for the benefit of workflow authors.
For RAM and disk, subtract out the overhead that gets added to the resource request in order to show the maximum resource request that can be made in CWL that will match that type, without bumping up to a larger size.
Should also report value of MaxInstances and SupervisorFraction.
Files
Instancetypes.png (144 KB) Instancetypes.png | Peter Amstutz, 10/27/2023 04:04 PM |
Related issues
Updated by Peter Amstutz about 2 years ago
- Description updated (diff)
- Subject changed from Panel that lists configured instance types and the largest resource request that will select that type to Panel that lists configured instance types
Updated by Peter Amstutz almost 2 years ago
- Is duplicate of Feature #12179: Panel showing available instance types on the cluster added
Updated by Peter Amstutz almost 2 years ago
- Target version changed from 2022-12-07 Sprint to 2022-12-21 Sprint
Updated by Peter Amstutz almost 2 years ago
- Story points set to 3.0
- Subject changed from Panel that lists configured instance types to Panel that lists configured instance types
Updated by Peter Amstutz almost 2 years ago
- Target version changed from 2022-12-21 Sprint to 2023-01-18 sprint
Updated by Peter Amstutz almost 2 years ago
- Target version changed from 2023-01-18 sprint to 2023-02-01 sprint
Updated by Peter Amstutz almost 2 years ago
- Release set to 59
- Target version deleted (
2023-02-01 sprint)
Updated by Peter Amstutz almost 2 years ago
- Target version set to To be scheduled
Updated by Peter Amstutz over 1 year ago
- Related to Idea #16943: WB2 Descriptions, project & collection operations added
Updated by Peter Amstutz over 1 year ago
- Target version changed from To be scheduled to Development 2023-04-26 sprint
Updated by Peter Amstutz over 1 year ago
- Target version changed from Development 2023-04-26 sprint to Development 2023-05-10 sprint
Updated by Peter Amstutz over 1 year ago
- Target version changed from Development 2023-05-10 sprint to Development 2023-05-24 sprint
Updated by Peter Amstutz over 1 year ago
- Target version changed from Development 2023-05-24 sprint to Development 2023-06-07
Updated by Peter Amstutz over 1 year ago
- Target version changed from Development 2023-06-07 to To be scheduled
Updated by Peter Amstutz over 1 year ago
- Related to Idea #18179: Better spot instance support added
Updated by Peter Amstutz about 1 year ago
- Target version changed from To be scheduled to Development 2023-11-08 sprint
Updated by Peter Amstutz about 1 year ago
- Target version changed from Development 2023-11-08 sprint to Development 2023-10-25 sprint
Updated by Peter Amstutz about 1 year ago
- Target version changed from Development 2023-10-25 sprint to Development 2023-11-08 sprint
Updated by Peter Amstutz about 1 year ago
- File Instancetypes.png Instancetypes.png added
Updated by Peter Amstutz about 1 year ago
Here's the Go code that calculates the actual RAM request.
So we want to subtract all the overhead from the topline instance RAM to get the maximum value of "RuntimeConstraints.RAM" that the user can provide to get this instance type.
needRAM := ctr.RuntimeConstraints.RAM + ctr.RuntimeConstraints.KeepCacheRAM needRAM += int64(cc.Containers.ReserveExtraRAM) if cc.Containers.LocalKeepBlobBuffersPerVCPU > 0 { // + 200 MiB for keepstore process + 10% for GOGC=10 needRAM += 220 << 20 // + 64 MiB for each blob buffer + 10% for GOGC=10 needRAM += int64(cc.Containers.LocalKeepBlobBuffersPerVCPU * needVCPUs * (1 << 26) * 11 / 10) } needRAM = (needRAM * 100) / int64(100-discountConfiguredRAMPercent)
Updated by Brett Smith about 1 year ago
Updated by Peter Amstutz 12 months ago
- Target version changed from Development 2023-11-08 sprint to Development 2023-11-29 sprint
Updated by Peter Amstutz 11 months ago
- Target version changed from Development 2023-11-29 sprint to Development 2024-01-03 sprint
Updated by Stephen Smith 11 months ago
Changes at arvados-workbench2|038d67a8c9ebf8a462774e9b870e7f1603c01959 branch 19675-instance-types-panel
Tests developer-tests-workbench2: #1397
- All agreed upon points are implemented / addressed.
- Added instance types panel requiring adding a few fields to the arvados config type
- Add breadcrumbs for instance types panel
- Add unit tests for instance types panel
- Add missing breadcrumbs navigation handlers
- Move process spec 'no value io parameter test' loginas into createContainerRequest then body and use admin user which is the only way I've figured out how to make that test pass on jenkins (it works fine locally for some reason)
- Anything not implemented (discovered or discussed during work) has a follow-up story.
- Code is tested and passing, both automated and manual, what manual testing was done is described
- done
- Documentation has been updated.
- n/a
- Behaves appropriately at the intended scale (describe intended scale).
- intended scale is expected to be on the order of <50 instance types, beyond that the UI might be difficult to use
- Considered backwards and forwards compatibility issues between client and server.
- n/a
- Follows our coding standards and GUI style guidelines.
- done
Updated by Stephen Smith 11 months ago
I merged the changes into a branch on the main arvados repo at arvados|ee9d1e39b5d469a827be5a719c9c0860914ab2a8 branch 19675-instance-types-panel
It turns out the cypress test fix didn't actually fix the issue, it's still just flakey in probably the same way that Tom was trying to fix, so I was able to get it to pass without my fix by rerunning: developer-run-tests-services-workbench2: #84
Updated by Peter Amstutz 11 months ago
Reviewing 19675-instance-types-panel @ ee9d1e39b5d469a827be5a719c9c0860914ab2a8
- These don't seem to be in an particular order. Can we sort them by price from lowest to highest?
- RAM should be capitalized in "Max ram request"
- Even though LocalKeepBlobBuffersPerVCPU isn't exported, I think we should calculate the maximum request assuming it is the default value. Also we should include the 5% discount (also assuming the default behavior for now). In other words, it should account for all the stuff in note-24, and just assume defaults where the configured values are not available.
- defaults are
LocalKeepBlobBuffersPerVCPU: 1
anddiscountConfiguredRAMPercent: 5
- Since this is user-facing to help users set correct resource requests, it is going to be better if the values that are displayed under-shoot the maximum than show a maximum that is wrong and frustrates users because they don't get the instance type they intended. In the future, if/when we rework instance selection, we can update the formulas (or have the values provided from somewhere else).
- defaults are
- For CWL, the unit of memory is MiB. Let's display RAM and disk in mebibytes along with whatever units are more human-readable? e.g.
Max disk request: 16252928 MiB (7.3 TiB)
Max ram request: 15872 MiB (15.5 GiB) - I realize the design sketch was a little weird, the order of the fields should be:
- provider type
- price
- cores
- RAM
- disk
- preemptible
- GPU stuff
- It is currently a bunch of little cards on one big card (white on white). Can we get rid of the outer container card?
- Let's add it to the left panel, underneath "All Processes". In general I think we want to move towards phasing out the "user" menu except for user account-related stuff.
- Instead of the stack-of-servers icon, let's use the same stylized computer chip that is used for the "Resources" card of the process view.
- Speaking of the "Resources" card, I feel like they should be formatted the same way (with the same labels, as much as possible, to emphasize the connection between the numbers), on the other hand having the field name on the top instead of to the left might needlessly waste vertical space, so I'm not sure.
- We should have some text at the top explaining what this page is and how to use it. E.g. "These are the cloud compute instance types configured for this cluster. The core count and maximum RAM request correspond to the greatest values you can put in the CWL Workflow ResourceRequest `minCores` and `minRAM` and still be scheduled on that instance type."
Updated by Stephen Smith 11 months ago
Changes at arvados|14a2e78ff5cd055216a9f6edc40dfb3f79122fa8 branch 19675-instance-types-panel
Tests developer-run-tests-services-workbench2: #98
- Instance types sorted by increasing cost, with an alphabetical sort of the ProviderType as a tiebreaker
- Fixed capitalization and field order
- Added keep buffer and discount ram percent accounting
- Added "CWL" size formatting and display MiB next to convenient units
- Removed container card
- Moved panel link to left panel
- Switched to resources icon
- Switched to DetailsAttribute component to better match resources panel
- Added info box to the top with the suggested text
Updated by Peter Amstutz 11 months ago
Stephen Smith wrote in #note-32:
Changes at arvados|14a2e78ff5cd055216a9f6edc40dfb3f79122fa8 branch 19675-instance-types-panel
Tests developer-run-tests-services-workbench2: #98
- Instance types sorted by increasing cost, with an alphabetical sort of the ProviderType as a tiebreaker
- Fixed capitalization and field order
- Added keep buffer and discount ram percent accounting
- Added "CWL" size formatting and display MiB next to convenient units
- Removed container card
- Moved panel link to left panel
- Switched to resources icon
- Switched to DetailsAttribute component to better match resources panel
- Added info box to the top with the suggested text
This looks good, just one thing I noticed:
- The icon is missing from the collapsed left side panel. It appears correctly in the expanded left panel.
Rest LGTM.
Updated by Stephen Smith 11 months ago
- Status changed from In Progress to Resolved
Added the icon to the collapsed left menu, tests developer-run-tests-services-workbench2: #103