Arvados: Issueshttps://dev.arvados.org/https://dev.arvados.org/favicon.ico?15576888422022-01-06T16:06:36ZArvados
Redmine Arvados - Task #18616 (New): Reviewhttps://dev.arvados.org/issues/186162022-01-06T16:06:36ZPeter Amstutzpeter.amstutz@curii.comArvados - Task #18609 (New): Reviewhttps://dev.arvados.org/issues/186092022-01-06T15:56:12ZLucas Di Pentimalucas.dipentima@curii.comArvados - Bug #18278 (New): [k8s] start using an ingresshttps://dev.arvados.org/issues/182782021-10-19T19:52:53ZWard Vandewegeward@curii.com
<p>As reported in <a class="external" href="https://forum.arvados.org/t/deploy-arvados-on-gke">https://forum.arvados.org/t/deploy-arvados-on-gke</a>, the GKE k8s setup needs some work.</p> Arvados - Task #18235 (New): Reviewhttps://dev.arvados.org/issues/182352021-09-29T15:33:42ZPeter Amstutzpeter.amstutz@curii.comArvados - Task #18234 (New): Reviewhttps://dev.arvados.org/issues/182342021-09-29T15:33:32ZPeter Amstutzpeter.amstutz@curii.comArvados - Feature #17313 (New): [Documentation] improve federation documentationhttps://dev.arvados.org/issues/173132021-02-02T14:45:51ZWard Vandewegeward@curii.com
<p>The Architecture page at <a class="external" href="https://doc.arvados.org/v2.1/architecture/federation.html">https://doc.arvados.org/v2.1/architecture/federation.html</a> and the admin section at <a class="external" href="https://doc.arvados.org/v2.1/admin/federation.html">https://doc.arvados.org/v2.1/admin/federation.html</a> have considerable overlap, but neither clearly lays out the types of federation and what some example use cases are.</p>
<p>We should upgrade the Architecture page to an entire Federation section, with these pages:</p>
<ul>
<li>Overview</li>
<li>Peer federation</li>
<li>Login federation</li>
<li>public-data access federation (need a new term for this type!)</li>
<li>Federation mechanics (move the "federation and signatures" section from <a class="external" href="https://doc.arvados.org/v2.1/architecture/manifest-format.html#federationsignatures">https://doc.arvados.org/v2.1/architecture/manifest-format.html#federationsignatures</a> here; also this is where some of the text from <a class="external" href="https://doc.arvados.org/v2.1/architecture/federation.html">https://doc.arvados.org/v2.1/architecture/federation.html</a> will go)</li>
</ul>
<p>For each federation type the page needs to clarify:</p>
<ul>
<li>when this type of federation is appropriate</li>
<li>how to configure the clusters involved</li>
<li>what the functional impact is (e.g. multisite search works in this particular way, login works like this, what happens when a user on cluster a requests a uuid that lives on another cluster, what happens when a workflow does that, etc)</li>
<li>security considerations</li>
</ul>
<p>Needless to say, it's fine to say "not yet implemented" for stuff that does not work.</p> Arvados - Task #17283 (New): Reviewhttps://dev.arvados.org/issues/172832021-01-20T16:58:06ZPeter Amstutzpeter.amstutz@curii.comArvados - Task #17277 (New): Make sure machine is configured to respond for that virtual hosthttps://dev.arvados.org/issues/172772021-01-20T16:47:16ZPeter Amstutzpeter.amstutz@curii.comArvados - Bug #16832 (New): replace linkchecker in our build environmenthttps://dev.arvados.org/issues/168322020-09-10T19:34:47ZWard Vandewegeward@curii.com
<p>It's Python 2 only, and appears unmaintained upstream.</p>
<p>When done, make sure to update </p>
<pre><code>lib/install/deps.go</code></pre>
<p>so that </p>
<pre><code>arvados-server install -type test</code></pre>
<p>keeps doing the right thing.</p> Arvados - Idea #13484 (New): Support multiple load-balanced API server nodeshttps://dev.arvados.org/issues/134842018-05-14T21:07:29ZTom Cleggtom@curii.com
<p>Background: the Rails API server application becomes a performance bottleneck during heavy load, e.g., when hundreds of containers/nodes are running. There are some ways to respond to this -- use a bigger/faster machine, adjust logging configs, move postgresql to a different machine -- but it would be much better if the operator could add API server nodes to increase capacity. However, there are some parts of the code base that assume there's only one API server.</p>
<p>In this issue, we remove those barriers, so a site admin can safely add and remove additional API servers and route traffic to them with a load balancer.</p>
<p>(However, multi-API-server installations are not expected to support crunch1 jobs.)</p>
Known/suspected issues:
<ul>
<li>Job validation code assumes git repositories are stored in the local filesystem (todo: confirm this only affects crunch1)</li>
<li>Audit log cleanup code uses flock() to avoid wastefully running concurrent cleanup threads (todo: confirm concurrent cleanup threads are harmless, and/or use a database lock instead)</li>
<li>Sample DNS update scripts (triggered by "node ping") assume the API host is the DNS server (todo: offer a sample DNS update strategy suitable for multiple nodes).</li>
</ul> Tapestry - Task #6925 (New): Review 6924-survey-field-idhttps://dev.arvados.org/issues/69252015-08-06T17:26:14ZTom Cleggtom@curii.comGET-Evidence - Task #6072 (New): Setup OAuth2 credentials with Googlehttps://dev.arvados.org/issues/60722015-05-19T15:04:51ZAbram Connellyabram.connelly@gmail.com
<p>The OAuth2 needs credentials setup with Google.</p>
<p>From Tom's post: <a class="external" href="https://arvados.org/blogs/30">https://arvados.org/blogs/30</a></p>
<a name="Step-1-Tell-Google-Developer-Console-how-to-recognize-your-web-application"></a>
<h3 >Step 1. Tell Google Developer Console how to recognize your web application.<a href="#Step-1-Tell-Google-Developer-Console-how-to-recognize-your-web-application" class="wiki-anchor">¶</a></h3>
<ul>
<li> Visit <a class="external" href="https://console.developers.google.com/project">https://console.developers.google.com/project</a></li>
<li> Click “Create Project”. Give it a name and ID.</li>
<li> In the new project, click “APIs” in the “APIs & auth” section in the left nav.</li>
<li> Find “Google+ API” in the big list and enable it.</li>
<li> Click “Credentials” in the “APIs & auth” section in the left nav.</li>
<li> Under OAuth, click the “Create a new Client ID” button. In the dialog:</li>
<li> Application type: Web application</li>
<li> Authorized Javascript origins: <a class="external" href="https://your-app.example.com/">https://your-app.example.com/</a></li>
<li> Authorized redirect URIs: <a class="external" href="https://your-app.example.com/google-oauth2.php">https://your-app.example.com/google-oauth2.php</a></li>
<li> Yes, you can use stuff like “localhost:1234” in both of these fields, for testing.</li>
<li> Click “Create Client ID”</li>
<li> You should have a “Client ID for web application” table on the right.</li>
<li> Copy the client ID and client secret. You’ll need those soon.</li>
</ul> Tapestry - Task #5229 (New): Review 4766https://dev.arvados.org/issues/52292015-02-17T16:09:15ZPhil Hodgsonphil@curoverse.comTapestry - Task #3225 (New): Review 2930https://dev.arvados.org/issues/32252014-07-10T20:39:53ZPhil Hodgsonphil@curoverse.com
<p>Okay so I dug in and I've finally pursuaded all of the tests to pass. I did not go out of my way to create new tests, but I did fix every existing test and deprecation that was possible to fix, and in many controller classes the test coverage was significantly increased (from default generated and completely failing tests to at least a reasonably complete set of RESTful tests). It was a pretty big job, but it probably fixed at least a couple security holes and bugs in Tapestry. Mostly I just wanted to finally get the tests into a condition where we'd <em>notice</em> if something went awry, going forward. It's hard to tell the difference between 376 tests failing and 379 tests failing, but the difference between 0 and 3 is huge!</p>
<p>I did notice thought that I probably spent too much time fixing controllers and their tests that may not even still be in use. There are so many dark corners to Tapestry!</p>
<p>In any case, now we can choose to increase test coverage - the safer but more tedious option - or move on to upgrading to ruby 1.9.3 and 2 and also upgrading Rails upwards.</p>
<p>I have not looked into tools for analyzing test coverage of code - perhaps this might be a good way to measure it. But certainly it would be good to know whether a lot of the more obscure controllers that don't seem to have active enrollment steps associated with them should still be maintained - or removed.</p>
<p>(There are still some deprecation notices about some certain shoulda directives but it's just because we're caught between an old version of Ruby and newer gems, of course.)</p> Tapestry - Feature #2231 (New): add flag to choose locale https://dev.arvados.org/issues/22312014-02-26T12:34:38ZWard Vandewegeward@curii.com