[Crunch] Jobs do not re-run with latest docker image if job is unchanged but docker image is updated
I successfully ran a job with a docker image of my own making called 'arvados/dax-node'. I did not alter the job but updated the docker image and re-uploaded it via 'arv-keepdocker' (with the same name 'arvados/dax-node'). After the new docker image was uploaded and the job re-run, the job completed immediately (successfully) presumably because it thought nothing had changed.
Looking at the docker image used in the job, it was the old docker image and not the new one.
Specifically, the pipeline instance run was 'tb05z-d1hrv-hbgo72pe87pj7er'. The new docker image ('arvados/dax-node', 'latest') was 'tb05z-4zz18-nslhjho8iyq29am' but the job used an older docker image (also named 'arvados/dax-node', 'latest') with UUID 'tb05z-4zz18-yog9g7dk10ooxjk'.
The workaround is easy enough, just 'dirty' the pipeline in some way, either by forcing a re-run or updating the git repository.