Arvados

17267-broken-schema-links @ 29a8298caf922a2651fbd51baf7b97b58ae25840

developer-run-tests: #2265

• Given that this issue is IMO pretty critical, I think it would be convenient to have it tested.
• The changes on pathmapper.py's visit() seems to be applicable for other callers than the one processing the $schema, maybe it would need an additional argument so that the caller can ask to ignore broken links on certain cases?
• Related Q: Would a broken schema link make validation impossible, or the schemas have other uses?

Lucas Di Pentima wrote:

• Given that this issue is IMO pretty critical, I think it would be convenient to have it tested.

Yes, I wasn't quite sure if a unit test would make sense, but I could add an integration test.

• The changes on pathmapper.py's visit() seems to be applicable for other callers than the one processing the $schema, maybe it would need an additional argument so that the caller can ask to ignore broken links on certain cases?

So, this only applies to things that are referenced in the CWL document with http links, that need to be downloaded on the fly into Arvados.

The prior behavior was that, if a http download failed, it would log an error, throw a fatal exception.

The new behavior is that if a http download fails, it logs an error, does not add an entry in the "mapper". If something tries to look the mapping without checking to see if it exists first, it will throw a fatal exception.

So I think the behavior is functionally the same.

• Related Q: Would a broken schema link make validation impossible, or the schemas have other uses?

No, they are entirely optional. The only thing they are really used for is to check if your annotations appear in the vocabulary of the extension schema, and warn if not found. It is more to assist with spellchecking than anything.

17267-broken-schema-links @ b01ff2414daaf5fd8ff7f0e78ed49e63d431ccd3

Added integration test.

This LGTM, thanks!