Arvados: Issueshttps://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422022-06-17T13:29:29ZArvados
Redmine Arvados - Bug #19201 (Resolved): [documentation] when viewing old versions, warn the userhttps://dev.arvados.org/issues/192012022-06-17T13:29:29ZWard Vandewegeward@curii.comArvados - Task #19176 (Resolved): review 19175-doc-refactor-multi-host-installationhttps://dev.arvados.org/issues/191762022-06-02T15:36:36ZWard Vandewegeward@curii.comArvados - Feature #19175 (Resolved): [Documentation] refactor the multihost installer pagehttps://dev.arvados.org/issues/191752022-06-02T15:36:15ZWard Vandewegeward@curii.comArvados - Task #19165 (Resolved): review 19164-add-flagshttps://dev.arvados.org/issues/191652022-05-25T19:15:40ZWard Vandewegeward@curii.comArvados - Feature #19164 (Resolved): [compute image builder] [AWS} add flag for public IP associa...https://dev.arvados.org/issues/191642022-05-25T17:09:57ZWard Vandewegeward@curii.com
<p>Add a flag that can be used to disable the association of a public IP address for the node that Packer uses to build a new images, when building on AWS. A public IP is not needed when that node is reachable via the internal IP, and in some environments public IPs are not allowed by policy.</p>
<p>The option should default to "true": in the common case a public IP is desired.</p>
<p>Likewise, add a flag for ENA support. In some environments this feature is not available. Default it to "true".</p> Arvados - Task #19130 (Resolved): review 19126-nginx-proxy-settings-changehttps://dev.arvados.org/issues/191302022-05-12T15:32:06ZWard Vandewegeward@curii.comArvados - Task #19128 (Resolved): review 19127-remove-warningshttps://dev.arvados.org/issues/191282022-05-11T21:06:56ZWard Vandewegeward@curii.comArvados - Bug #19127 (Resolved): spurious config warnings when using local keepstore on computehttps://dev.arvados.org/issues/191272022-05-11T19:14:11ZWard Vandewegeward@curii.com
<p>Observed on Arvados 2.4.0 on pirca, e.g. <a href="https://arvadosapi.com/pirca-xvhdp-m1mdkxyqugein4m">pirca-xvhdp-m1mdkxyqugein4m</a></p>
<p>These warnings get logged in the container logs under these circumstances:</p>
<p>Config:</p>
<pre>
Containers:
PreemptiblePriceFactor: 1
LocalKeepLogsToContainerLog: errors
</pre>
Two questions/bugs:
<ul>
<li>these are warnings, why are they showing up when <code>LocalKeepLogsToContainerLog</code> is set to <code>errors</code> ? Does <code>errors</code> also somehow include warnings? That's confusing.</li>
<li>the warnings seem wrong. They don't show up when checking the config with <code>arvados-server config-check</code>.</li>
</ul>
<p>Warnings:</p>
<pre>
2022-05-11T19:12:59.961853108Z starting keepstore on http://localhost:42569
2022-05-11T19:12:59.961889139Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c52xlarge.Scratch","time":"2022-05-11T19:12:59.374860700Z"}
2022-05-11T19:12:59.961898898Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c52xlarge.Name","time":"2022-05-11T19:12:59.374935708Z"}
2022-05-11T19:12:59.961908852Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c54xlarge.Name","time":"2022-05-11T19:12:59.374964424Z"}
2022-05-11T19:12:59.961918521Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c54xlarge.Scratch","time":"2022-05-11T19:12:59.374988872Z"}
2022-05-11T19:12:59.961927672Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g52xlarge.preemptible.Name","time":"2022-05-11T19:12:59.375745729Z"}
2022-05-11T19:12:59.961935857Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g52xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.375953678Z"}
2022-05-11T19:12:59.961957863Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g54xlarge.Name","time":"2022-05-11T19:12:59.376123687Z"}
2022-05-11T19:12:59.961964834Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g54xlarge.Scratch","time":"2022-05-11T19:12:59.376285332Z"}
2022-05-11T19:12:59.961973470Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.inf16xlarge.preemptible.Name","time":"2022-05-11T19:12:59.376450316Z"}
2022-05-11T19:12:59.961980428Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.inf16xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.376619218Z"}
2022-05-11T19:12:59.961989254Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c5large.Scratch","time":"2022-05-11T19:12:59.376796323Z"}
2022-05-11T19:12:59.961998393Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c5large.Name","time":"2022-05-11T19:12:59.376989255Z"}
2022-05-11T19:12:59.962004546Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g58xlarge.preemptible.Name","time":"2022-05-11T19:12:59.377145316Z"}
2022-05-11T19:12:59.962012511Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g58xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.377301156Z"}
2022-05-11T19:12:59.962034722Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5xlarge_extradisk.Name","time":"2022-05-11T19:12:59.377478606Z"}
2022-05-11T19:12:59.962043258Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5xlarge_extradisk.Scratch","time":"2022-05-11T19:12:59.377632135Z"}
2022-05-11T19:12:59.962078491Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c54xlarge.preemptible.Name","time":"2022-05-11T19:12:59.377820339Z"}
2022-05-11T19:12:59.962102477Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c54xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.377991277Z"}
2022-05-11T19:12:59.962109769Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m52xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.378155300Z"}
2022-05-11T19:12:59.962117943Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m52xlarge.preemptible.Name","time":"2022-05-11T19:12:59.378307037Z"}
2022-05-11T19:12:59.962124702Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5large.preemptible.Scratch","time":"2022-05-11T19:12:59.378462819Z"}
2022-05-11T19:12:59.962134576Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5large.preemptible.Name","time":"2022-05-11T19:12:59.378615162Z"}
2022-05-11T19:12:59.962146387Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5xlarge_extradisk.preemptible.Name","time":"2022-05-11T19:12:59.378769767Z"}
2022-05-11T19:12:59.962155085Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5xlarge_extradisk.preemptible.Scratch","time":"2022-05-11T19:12:59.378926964Z"}
2022-05-11T19:12:59.962161833Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.t3small.Name","time":"2022-05-11T19:12:59.379080403Z"}
2022-05-11T19:12:59.962183550Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.t3small.Scratch","time":"2022-05-11T19:12:59.379286475Z"}
2022-05-11T19:12:59.962190470Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c52xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.379332414Z"}
2022-05-11T19:12:59.962204018Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c52xlarge.preemptible.Name","time":"2022-05-11T19:12:59.379378580Z"}
2022-05-11T19:12:59.962211510Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g52xlarge.Name","time":"2022-05-11T19:12:59.379405404Z"}
2022-05-11T19:12:59.962219782Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g52xlarge.Scratch","time":"2022-05-11T19:12:59.379426468Z"}
2022-05-11T19:12:59.962227280Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m54xlarge.Scratch","time":"2022-05-11T19:12:59.379448616Z"}
2022-05-11T19:12:59.962237904Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m54xlarge.Name","time":"2022-05-11T19:12:59.379467450Z"}
2022-05-11T19:12:59.962260826Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c5large.preemptible.Scratch","time":"2022-05-11T19:12:59.379512398Z"}
2022-05-11T19:12:59.962269791Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c5large.preemptible.Name","time":"2022-05-11T19:12:59.379537204Z"}
2022-05-11T19:12:59.962276352Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g54xlarge.preemptible.Name","time":"2022-05-11T19:12:59.379558744Z"}
2022-05-11T19:12:59.962285306Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g54xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.379578922Z"}
2022-05-11T19:12:59.962293162Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m54xlarge.preemptible.Name","time":"2022-05-11T19:12:59.379622906Z"}
2022-05-11T19:12:59.962308322Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m54xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.379646674Z"}
2022-05-11T19:12:59.962315887Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g5xlarge.Name","time":"2022-05-11T19:12:59.379668849Z"}
2022-05-11T19:12:59.962339655Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g5xlarge.Scratch","time":"2022-05-11T19:12:59.379712975Z"}
2022-05-11T19:12:59.962346432Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g5xlarge.preemptible.Name","time":"2022-05-11T19:12:59.379738887Z"}
2022-05-11T19:12:59.962359248Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g5xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.379758653Z"}
2022-05-11T19:12:59.962364123Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5large.Name","time":"2022-05-11T19:12:59.379780002Z"}
2022-05-11T19:12:59.962381293Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5large.Scratch","time":"2022-05-11T19:12:59.379833750Z"}
2022-05-11T19:12:59.962388598Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.t3small.preemptible.Name","time":"2022-05-11T19:12:59.379855657Z"}
2022-05-11T19:12:59.962396513Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.t3small.preemptible.Scratch","time":"2022-05-11T19:12:59.379905486Z"}
2022-05-11T19:12:59.962417432Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.inf16xlarge.Name","time":"2022-05-11T19:12:59.379930223Z"}
2022-05-11T19:12:59.962427236Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.inf16xlarge.Scratch","time":"2022-05-11T19:12:59.379955265Z"}
2022-05-11T19:12:59.962434348Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.379976380Z"}
2022-05-11T19:12:59.962444912Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5xlarge.preemptible.Name","time":"2022-05-11T19:12:59.379997807Z"}
2022-05-11T19:12:59.962452483Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c5xlarge.Scratch","time":"2022-05-11T19:12:59.380044466Z"}
2022-05-11T19:12:59.962460825Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c5xlarge.Name","time":"2022-05-11T19:12:59.380099065Z"}
2022-05-11T19:12:59.962468509Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c5xlarge.preemptible.Name","time":"2022-05-11T19:12:59.380150390Z"}
2022-05-11T19:12:59.962494344Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.c5xlarge.preemptible.Scratch","time":"2022-05-11T19:12:59.380172697Z"}
2022-05-11T19:12:59.962502201Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g58xlarge.Name","time":"2022-05-11T19:12:59.380194002Z"}
2022-05-11T19:12:59.962511181Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.g58xlarge.Scratch","time":"2022-05-11T19:12:59.380238818Z"}
2022-05-11T19:12:59.962518448Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m52xlarge.Name","time":"2022-05-11T19:12:59.380263034Z"}
2022-05-11T19:12:59.962526997Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m52xlarge.Scratch","time":"2022-05-11T19:12:59.380282939Z"}
2022-05-11T19:12:59.962534002Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5xlarge.Scratch","time":"2022-05-11T19:12:59.380326160Z"}
2022-05-11T19:12:59.962543591Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.InstanceTypes.m5xlarge.Name","time":"2022-05-11T19:12:59.380349136Z"}
2022-05-11T19:12:59.962550634Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.Users.UserNotifierEmailBcc.info@curii.com","time":"2022-05-11T19:12:59.380390762Z"}
2022-05-11T19:12:59.962573121Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.Users.NewUserNotificationRecipients.playground@curii.com","time":"2022-05-11T19:12:59.380438381Z"}
2022-05-11T19:12:59.962592406Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.Users.NewUserNotificationRecipients.sysadmin+PIRCA@curii.com","time":"2022-05-11T19:12:59.380458961Z"}
2022-05-11T19:12:59.962601848Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.Users.NewInactiveUserNotificationRecipients.playground@curii.com","time":"2022-05-11T19:12:59.380487479Z"}
2022-05-11T19:12:59.962610289Z {"level":"warning","msg":"deprecated or unknown config entry: Clusters.pirca.Users.NewInactiveUserNotificationRecipients.sysadmin+PIRCA@curii.com","time":"2022-05-11T19:12:59.380541614Z"}
2022-05-11T19:12:59.962622666Z {"ClusterID":"pirca","PID":1682,"level":"info","msg":"keepstore 2.4.0 (go1.17.1) starting, pid 1682","time":"2022-05-11T19:12:59.391936777Z"}
2022-05-11T19:12:59.962646332Z {"ClusterID":"pirca","PID":1682,"level":"info","msg":"started volume <a href="https://arvadosapi.com/pirca-nyw5e-000000000000000">pirca-nyw5e-000000000000000</a> (s3-bucket:\"<a href="https://arvadosapi.com/pirca-nyw5e-000000000000000">pirca-nyw5e-000000000000000</a>-volume\"), ReadOnly=false","time":"2022-05-11T19:12:59.392114368Z"}
2022-05-11T19:12:59.962663470Z {"ClusterID":"pirca","Listen":"127.0.0.1:42569","PID":1682,"Service":"keepstore","URL":"http://localhost:42569/","level":"info","msg":"listening","time":"2022-05-11T19:12:59.538837708Z"}
2022-05-11T19:12:59.961214904Z using local keepstore process (pid 1682) at http://localhost:42569, writing logs to keepstore.txt in log collection
2022-05-11T19:13:02.946896108Z crunch-run 2.4.0 (go1.17.1) started
2022-05-11T19:13:02.947553420Z crunch-run process has uid=0(root) gid=0(root) groups=0(root)
2022-05-11T19:13:02.947571669Z Executing container '<a href="https://arvadosapi.com/pirca-dz642-mufa2uvwonyri0s">pirca-dz642-mufa2uvwonyri0s</a>' using docker runtime
2022-05-11T19:13:02.947581564Z Executing on host 'ip-10-254-254-115'
2022-05-11T19:13:03.054213584Z container token "v2/pirca-gj3su-07dfbqhvkvb9ksh/2nms9f8otf3x4jcfqd6h79eycble41tf6ynfeactoyb8r032st/pirca-dz642-mufa2uvwonyri0s"
...
</pre> Arvados - Bug #19126 (Resolved): Disable nginx proxy caching for controllerhttps://dev.arvados.org/issues/191262022-05-11T17:01:36ZWard Vandewegeward@curii.com
<p>When deployed behind nginx for tls termination, controller's responses can get truncated when nginx's reverse proxy tries to cache responses which exceed its temp file size. We don't really need nginx to do any caching in this scenario.</p>
<ul>
<li>update documentation accordingly</li>
<li>update the arvados-server boot template (also used in our test suite)</li>
<li>update our internal salt configs</li>
</ul> Arvados - Task #19078 (Resolved): review 19015-rvm-gemset-passenger-fixhttps://dev.arvados.org/issues/190782022-04-28T15:38:52ZWard Vandewegeward@curii.comArvados - Bug #19057 (Resolved): [controller] should not allow adding the same user+login to a VM...https://dev.arvados.org/issues/190572022-04-26T19:53:30ZWard Vandewegeward@curii.com
<p>As observed in <a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: VM admin interface rough edges (Resolved)" href="https://dev.arvados.org/issues/19049">#19049</a>, it is currently possible to attach the same user to a VM multiple times, which does not make sense and should not be allowed.</p>
<p><img src="https://dev.arvados.org/attachments/download/2995/20220426-virtual-machine-duplicate-link.png" alt="" /></p>
<p>The link has the same head and tail uuid, so presumably we could filter out duplicates in controller and return an error.</p> Arvados - Task #19055 (Resolved): review 19054-local-keepstore-aws-doc-updatehttps://dev.arvados.org/issues/190552022-04-25T19:35:28ZWard Vandewegeward@curii.comArvados - Bug #19054 (Resolved): [documentation] clarify AWS credentials needed for local keepsto...https://dev.arvados.org/issues/190542022-04-25T19:34:40ZWard Vandewegeward@curii.comArvados Workbench 2 - Bug #19049 (Resolved): VM admin interface rough edgeshttps://dev.arvados.org/issues/190492022-04-25T12:39:15ZWard Vandewegeward@curii.com
<p>Observed on 9tee4 at v2.5.0~dev20220419201321:</p>
<ul>
<li>when adding a user to a shell machine, the "Add login permission" dropdown shows double smaller/greater than symbols around the e-mail address</li>
</ul>
<ul>
<li>when adding a user to a shell machine, only the name and e-mail is shown. If there are users with identical name/e-mail addresses (e.g. on 9tee4), it is impossible to tell which is which. I suggest the UUID for the user is added at the end, perhaps between parentheses.</li>
</ul>
<ul>
<li>when adding a user to a shell machine, all users show up in the search list, even if they have already been added to the shell machine. Suggest limiting the list to users not yet on the shell machine.</li>
</ul>
<ul>
<li>when adding a user to a shell machine, if a user is selected that is already on the shell machine, the user gets added again (see also previous point, if they are no longer selectable, this one will also be fixed automatically). See also <a class="issue tracker-1 status-3 priority-4 priority-default closed parent" title="Bug: [controller] should not allow adding the same user+login to a VM more than one time (Resolved)" href="https://dev.arvados.org/issues/19057">#19057</a>, controller should not allow this.</li>
</ul>
<ul>
<li>when adding a user to a shell machine, for some users (federated users?) I get a "Not Found" popup, and also an empty red box at the bottom left: <img src="https://dev.arvados.org/attachments/download/2994/20220425-user-add-404.png" alt="" /> I think the user in the example is uuid <a href="https://arvadosapi.com/ce8i5-tpzed-xo2k4i24bjzwedw">ce8i5-tpzed-xo2k4i24bjzwedw</a>, but it's hard to be sure until the second bullet on this list is fixed.</li>
</ul> Arvados - Task #19013 (Resolved): review 18995-code-cleanup-3https://dev.arvados.org/issues/190132022-04-15T17:54:14ZWard Vandewegeward@curii.com