Arvados

13996-new-api-config @ 5f08fc8fcda25dbaa184dfc63a06cb6633af5877

https://ci.curoverse.com/view/Developer/job/developer-run-tests/1152/

https://ci.curoverse.com/view/CWL/job/arvados-cwl-conformance-tests/69/

• Loads new config object
• Loads legacy config, migrates keys into new config object
• Performs typechecking/coercion for Durations, URIs and essential file paths
• "rake config:dump" to get active config
• "rake config:diff" to get new config migrated from legacy config
• Update all the API server code to use new config parameters
• Fix all the tests

I realized this doesn't cover database.yml so I need to look into that.

Comments/questions:

• If I try to start the console or do a "rake config:dump” from 023015bacfded08503d0240d3f71838204d8dcb7 I get an the error 'missing Git.Repositories’, but if I run it from 6e9fcde0422b33d081e2985975e3104eb2434957, it works ok.
• When running "rake config:check" from 6e9fcde0422b33d081e2985975e3104eb2434957, I get the following error: "NoMethodError: undefined method `sort' for nil:NilClass" (from config_check.rake line 9)
• When dumping the new config, some keys are listed the old way. Example: 'whiny_nils: true'. Are these rails-only configs needed in the dump?
• The secret_token key is not being translated on config:dump as RailsSessionSecretToken (on the TO-DO list of the google doc), is that on purpose?
• Maybe some upgrade notes describing the existence of the new rake commands would be needed?

Lucas Di Pentima wrote:

Comments/questions:

• If I try to start the console or do a "rake config:dump” from 023015bacfded08503d0240d3f71838204d8dcb7 I get an the error 'missing Git.Repositories’, but if I run it from 6e9fcde0422b33d081e2985975e3104eb2434957, it works ok.

This was due to a missing symlink to config.default.yml. Fixed (and now also reports if the defaults file is missing.)

• When running "rake config:check" from 6e9fcde0422b33d081e2985975e3104eb2434957, I get the following error: "NoMethodError: undefined method `sort' for nil:NilClass" (from config_check.rake line 9)

config:check is gone, because the configuration is always checked on load, and most of the code in that task doesn't make sense for the new system.

• When dumping the new config, some keys are listed the old way. Example: 'whiny_nils: true'. Are these rails-only configs needed in the dump?

There are now three commands:

rake config:diff                               # Print values from application.yml that differ from system config.yml
rake config:dump                               # Print configuration as accessed through Rails.configuration
rake config:migrate                            # Print config.yml after merging with legacy application.yml

config:dump is intended for debugging so it includes any other Rails configuration options we know about, which are not settable in config.yml but may differ between test, development and production configurations.

• The secret_token key is not being translated on config:dump as RailsSessionSecretToken (on the TO-DO list of the google doc), is that on purpose?

Only that it was in the TO-DO section. Added it to API.

• Maybe some upgrade notes describing the existence of the new rake commands would be needed?

Yep, added two new documentation pages.

13996-new-api-config @ b05be6e2d1db4e63074fc28e978c40a271376f1f

https://ci.curoverse.com/view/Developer/job/developer-run-tests/1187/

Fix ERB error (hopefully):

8b5dff30937a7c12dcf7cd71153770393266500d

https://ci.curoverse.com/view/Developer/job/developer-run-tests/1188/

Pretty neat the usage of OrderedOptions! This LGTM, but I had some issues trying to start arvbox, but I'm not sure if it's just my local instance.

I've just tried to render the documentation, this is what I got:

[lucas:~/Devel/git/arvados/doc] 8b5dff309+ 9s ± rake run
/Users/lucas/.rvm/gems/ruby-2.3.8@deleteme/gems/liquid-2.6.1/lib/liquid/htmltags.rb:43: warning: key "index0" is duplicated and overwritten on line 46
[2019-04-11 18:58:54] INFO  WEBrick 1.3.1
[2019-04-11 18:58:54] INFO  ruby 2.3.8 (2018-10-18) [x86_64-darwin18]
[2019-04-11 18:58:54] INFO  WEBrick::HTTPServer#start: pid=11719 port=8000
[2019-04-11 18:58:55] ERROR bad URI `=\x0F��܅qZK��,�����HF���^2m����p\x004\x13\x03\x13\x01\x13\x02�,�+�$�#�'.
127.0.0.1 - - [11/Apr/2019:18:58:55 -03] "\x16\x03\x01\x02\x00\x01\x00\x01�\x03\x03u��\x15�I�\x14�FK���F\x1C\x04\x1E1��\x18|��f\x17��� =\x0F��܅qZK��,�����HF���^2m����p\x004\x13\x03\x13\x01\x13\x02�,�+�$�#�" 400 340
- -> =\x0F��܅qZK��,�����HF���^2m����p\x004\x13\x03\x13\x01\x13\x02�,�+�$�#�
[2019-04-11 18:58:55] ERROR bad Request-Line `\x16\x03\x01\x00�\x01\x00\x00�\x03\x03��I\x02n=�z��\\�5\x1A�\x0E\x13��t\eV�8#��6�x\x00\x00<�,�+�$�#�'.
127.0.0.1 - - [11/Apr/2019:18:58:55 -03] "\x16\x03\x01\x00�\x01\x00\x00�\x03\x03��I\x02n=�z��\\�5\x1A�\x0E\x13��t\eV�8#��6�x\x00\x00<�,�+�$�#�" 400 358
- ->
[2019-04-11 18:58:55] ERROR bad Request-Line `\x16\x03\x01\x00�\x01\x00\x00�\x03\x01\x1C~��\f \x14دx\r�o/��y�\x04V5Nٶ\x19DŽ�Y�5\x00\x00\x14�'.
127.0.0.1 - - [11/Apr/2019:18:58:55 -03] "\x16\x03\x01\x00�\x01\x00\x00�\x03\x01\x1C~��\f \x14دx\r�o/��y�\x04V5Nٶ\x19DŽ�Y�5\x00\x00\x14�" 400 347
- ->

...I also get test failure when running doc tests, but didn't failed on jenkins. I'm not sure what's happening.

Lucas Di Pentima wrote:

I've just tried to render the documentation, this is what I got:

[...]

That looks to me like you tried to connect with HTTPS to a plain HTTP server.

...I also get test failure when running doc tests, but didn't failed on jenkins. I'm not sure what's happening.

I think that's #15043 ? I don't think the new docs add any new errors.

Thanks, you were right about HTTPS vs HTTP.

The docs looks good, the only addition I think would be useful is a reference to the migration section on the upgrade notes, "current master branch" section, so that it gets included on the future release upgrade notes.

With that, LGTM.