Arvados

See Containers API

Implementation has two parts:

Check for past expiry when creating/updating container requests¶

Filter CRs on expires_at is null or expires_at>now

This will stop a container when its requesting CR is updated to expires_at=now, for example.

Check expiry periodically¶

This can be implemented as a Rails script that runs as a cron job (or systemd timer).

A single postgres query should be able to return the containers whose priority has gone out of sync due to expired requests. Something like this:

select c.uuid from containers c
 join container_requests cr on (cr.container_uuid=c.uuid and cr.expires_at is null or cr.expires_at>now)
 where c.priority > 0 and c.state in ('Queued', 'Locked', 'Running')
 having cr.container_uuid is null or max(cr.priority)<c.priority

Run c.update_priority! on the selected containers.