Arvados

A few additional thoughts

For me, the hard part of this story isn't so much packing containers onto instances as deciding when it is appropriate to start up a larger instance in order to have capacity to schedule multiple containers. This isn't HPC where we have a static set of nodes we work with.

One approach is to look ahead in the queue and pick out containers that fit together nicely, then schedule them together.

However, during the start-up phase of a batch, workflows are being submitted every 6-12 seconds. At this point, it's possible we may not have a substantial backlog of containers that would allow us to look at the queue and pack 4 or 8 jobs to an instance.

So another strategy would be that, when a small container doesn't have an instance, we prospectively start somewhat larger instance, e.g. 4 or 8 cores, and then additional containers in the queue get scheduled onto the instance.

Right now the cloud dispatcher has a pretty strong model of 1:1 containers to instances. Perhaps we could use a strategy where we subdivide an instance into 4 or 8 fractional instances, and schedule 1:1 on these fractional instances.

This would mean requesting a fractional instance would start up the full size instance but then add 4-8 fractional instances to the pool.

Scheduling to a fractional instance would mean finding an instance with the least spare capacity greater than 0, the running it there (this is to discourage situations where you have a bunch of full size instances that are each running 1 container).

When all the fractional slots of a full size instance are idle, the full size instance can be shut down.

Another thought:

We have the notion of "supervisor" containers.

We could have a configured instance type specifically for "supervisor" containers, along with a maximum number of containers we're willing to run -- so we could take a 4 gigabyte instance and specify that we want to schedule 6 or 8 workflow runners on it on the assumption that they won't all use their full RAM or CPU quota.