Bug #15045
Updated by Ward Vandewege about 5 years ago
As of commit:115cbd6482632c47fdcbbbe4abc9543e7e8e30ec arvados-cloud-dispatch fails with:
<pre>
Mar 29 13:29:00 c97qk.arvadosapi.com systemd[1]: Starting arvados-dispatch-cloud...
Mar 29 13:29:00 c97qk.arvadosapi.com arvados-dispatch-cloud[115048]: {"PID":115048,"level":"warning","msg":"SystemRootToken missing from cluster config, falling back to ARVADOS_API_TOKEN environment variable","time":"2019-03-29T13:29:00.695994395Z"}
Mar 29 13:29:00 c97qk.arvadosapi.com arvados-dispatch-cloud[115048]: {"PID":115048,"level":"warning","msg":"Services.Controller.ExternalURL missing from cluster config, falling back to ARVADOS_API_HOST(_INSECURE) environment variables","time":"2019-03-29
Mar 29 13:29:01 c97qk.arvadosapi.com arvados-dispatch-cloud[115048]: {"Listen":"[::]:9006","PID":115048,"Service":"arvados-dispatch-cloud","level":"info","msg":"listening","time":"2019-03-29T13:29:01.406605587Z"}
Mar 29 13:29:01 c97qk.arvadosapi.com arvados-dispatch-cloud[115048]: {"PID":115048,"level":"error","msg":"error updating queue: parse https://{https c97qk.arvadosapi.com false }/arvados/v1/api_client_authorizations/current: invalid character \"{\" i
Mar 29 13:29:01 c97qk.arvadosapi.com arvados-dispatch-cloud[115048]: {"PID":115048,"level":"info","msg":"waiting 1s before retry","time":"2019-03-29T13:29:01.407880564Z"}
Mar 29 13:29:01 c97qk.arvadosapi.com systemd[1]: Started arvados-dispatch-cloud.
Mar 29 13:29:01 c97qk.arvadosapi.com arvados-dispatch-cloud[115048]: {"N":0,"PID":115048,"level":"info","msg":"loaded initial instance list","time":"2019-03-29T13:29:01.573110101Z"}
Mar 29 13:29:02 c97qk.arvadosapi.com arvados-dispatch-cloud[115048]: {"PID":115048,"level":"error","msg":"error updating queue: parse https://{https c97qk.arvadosapi.com false }/arvados/v1/api_client_authorizations/current: invalid character \"{\" i
Mar 29 13:29:02 c97qk.arvadosapi.com arvados-dispatch-cloud[115048]: {"PID":115048,"level":"info","msg":"waiting 1s before retry","time":"2019-03-29T13:29:02.406034481Z"}
Mar 29 13:29:03 c97qk.arvadosapi.com arvados-dispatch-cloud[115048]: {"PID":115048,"level":"error","msg":"error updating queue: parse https://{https c97qk.arvadosapi.com false }/arvados/v1/api_client_authorizations/current: invalid character \"{\" i
Mar 29 13:29:03 c97qk.arvadosapi.com arvados-dispatch-cloud[115048]: {"PID":115048,"level":"info","msg":"waiting 1s before retry","time":"2019-03-29T13:29:03.406277629Z"}
...
</pre>
That is with the API host and token defined as config values. When I add it to the config.yml file (new style):
<pre>
Mar 29 13:53:29 c97qk.arvadosapi.com systemd[1]: Starting arvados-dispatch-cloud...
Mar 29 13:53:29 c97qk.arvadosapi.com arvados-dispatch-cloud[33313]: {"Listen":"[::]:9006","PID":33313,"Service":"arvados-dispatch-cloud","level":"info","msg":"listening","time":"2019-03-29T13:53:29.842080633Z"}
Mar 29 13:53:29 c97qk.arvadosapi.com arvados-dispatch-cloud[33313]: {"PID":33313,"level":"error","msg":"error updating queue: parse https://{https c97qk.arvadosapi.com false }/arvados/v1/api_client_authorizations/current: invalid character \"{\" in
Mar 29 13:53:29 c97qk.arvadosapi.com arvados-dispatch-cloud[33313]: {"PID":33313,"level":"info","msg":"waiting 1s before retry","time":"2019-03-29T13:53:29.845262273Z"}
Mar 29 13:53:29 c97qk.arvadosapi.com systemd[1]: Started arvados-dispatch-cloud.
Mar 29 13:53:29 c97qk.arvadosapi.com arvados-dispatch-cloud[33313]: {"N":0,"PID":33313,"level":"info","msg":"loaded initial instance list","time":"2019-03-29T13:53:29.965732628Z"}
Mar 29 13:53:30 c97qk.arvadosapi.com arvados-dispatch-cloud[33313]: {"PID":33313,"level":"error","msg":"error updating queue: parse https://{https c97qk.arvadosapi.com false }/arvados/v1/api_client_authorizations/current: invalid character \"{\" in
Mar 29 13:53:30 c97qk.arvadosapi.com arvados-dispatch-cloud[33313]: {"PID":33313,"level":"info","msg":"waiting 1s before retry","time":"2019-03-29T13:53:30.845396279Z"}
...
</pre>
After downgrading to version 1.3.1.20190320192402-1:
<pre>
Mar 29 13:54:36 c97qk.arvadosapi.com systemd[1]: Starting arvados-dispatch-cloud...
Mar 29 13:54:37 c97qk.arvadosapi.com arvados-dispatch-cloud[35662]: {"Listen":"[::]:9006","PID":35662,"Service":"arvados-dispatch-cloud","level":"info","msg":"listening","time":"2019-03-29T13:54:37.711254903Z"}
Mar 29 13:54:37 c97qk.arvadosapi.com systemd[1]: Started arvados-dispatch-cloud.
Mar 29 13:54:37 c97qk.arvadosapi.com arvados-dispatch-cloud[35662]: {"N":0,"PID":35662,"level":"info","msg":"loaded initial instance list","time":"2019-03-29T13:54:37.854359307Z"}
Mar 29 13:54:38 c97qk.arvadosapi.com arvados-dispatch-cloud[35662]: {"ContainerUUID":"c97qk-dz642-6v7kierfqpq71zi","InstanceType":"Standard_D1_v2","PID":35662,"Priority":562948399697602265,"State":"Queued","level":"info","msg":"adding container to queue"
Mar 29 13:54:38 c97qk.arvadosapi.com arvados-dispatch-cloud[35662]: {"ContainerUUID":"c97qk-dz642-g53p2qka5uji43t","InstanceType":"Standard_D1_v2","PID":35662,"Priority":562948399577667391,"State":"Queued","level":"info","msg":"adding container to queue"
Mar 29 13:54:38 c97qk.arvadosapi.com arvados-dispatch-cloud[35662]: {"ContainerUUID":"c97qk-dz642-kpcru2t7ze6bsqx","InstanceType":"Standard_D1_v2","PID":35662,"Priority":562948399556037107,"State":"Queued","level":"info","msg":"adding container to queue"
Mar 29 13:54:38 c97qk.arvadosapi.com arvados-dispatch-cloud[35662]: {"ContainerUUID":"c97qk-dz642-5spp83v4tsw4l1i","InstanceType":"Standard_D1_v2","PID":35662,"Priority":562948399667883208,"State":"Queued","level":"info","msg":"adding container to queue"
Mar 29 13:54:38 c97qk.arvadosapi.com arvados-dispatch-cloud[35662]: {"ContainerUUID":"c97qk-dz642-przvbj7f98shcl4","InstanceType":"Standard_D2_v2","PID":35662,"Priority":562948399681383698,"State":"Queued","level":"info","msg":"adding container to queue"
Mar 29 13:54:38 c97qk.arvadosapi.com arvados-dispatch-cloud[35662]: {"ContainerUUID":"c97qk-dz642-wmsn71xeu1t5ndr","InstanceType":"Standard_D1_v2","PID":35662,"Priority":562948399685000499,"State":"Queued","level":"info","msg":"adding container to queue"
Mar 29 13:54:38 c97qk.arvadosapi.com arvados-dispatch-cloud[35662]: {"ContainerUUID":"c97qk-dz642-hba1b9dyy6nig8w","InstanceType":"Standard_D1_v2","PID":35662,"Priority":562948399588516543,"State":"Queued","level":"info","msg":"adding container to queue"
Mar 29 13:54:38 c97qk.arvadosapi.com arvados-dispatch-cloud[35662]: {"ContainerUUID":"c97qk-dz642-1pops5ei6rx8llp","InstanceType":"Standard_D1_v2","PID":35662,"Priority":562948399697616780,"State":"Queued","level":"info","msg":"adding container to queue"
...
</pre>