Let container requests specify whether they'd rather run fast or run cheap
Most complete implementation would be something like:
- There's a field somewhere on container requests that expresses whether it would be better for the container to finish ASAP (run fast) or whether it's willing to wait to reduce costs (run cheap)
- Configuration lets the administrator set a default policy for container requests that come in without a preference
- Configuration lets the administrator set an optional override for all container requests, putting the preference under the control of the administrator and out of the hands of individual users
This would let us make more sophisticated scheduling decisions like:
- If preemptible instances are not available, should we switch to reserved instances (run fast) or wait until the spot price cools (run cheap)?
- If the cheapest instance type to accommodate a container is not immediately available, should we request a bigger instance (run fast) or wait until it becomes available (run cheap)?