Idea #19977
open
Update tests to run with a strict vocabulary
Description
- Write a
vocubalary.jsonfor the tests in the right place (FIXME: where is that?). It should define at least a couple of tags and"strict_tags": true. - Update the test configuration in
run_test_server.pyto setAPI.VocabularyPathto that file path. - Update tests broken by this change to continue passing while exercising the same behavior as before.
We want to make this change to help us catch system tags that we add without documentation or adding them to systemTagKeys in vocabulary.go.
Files
Updated by Brett Smith almost 2 years ago
- Story points set to 1.0
- Category set to Tests
- Tracker changed from Bug to Idea
- File developer-run-tests-sdk-python-ruby-2778.log developer-run-tests-sdk-python-ruby-2778.log added
- File developer-run-tests-remainder-3635.log developer-run-tests-remainder-3635.log added
- File developer-run-tests-apps-workbench-units-3478.log developer-run-tests-apps-workbench-units-3478.log added
- File developer-run-tests-apps-workbench-integration-3718.log developer-run-tests-apps-workbench-integration-3718.log added
- File developer-run-tests-apps-workbench-functionals-3518.log developer-run-tests-apps-workbench-functionals-3518.log added
I made a throwaway branch to define a strict vocabulary for the tests and run that on Jenkins to see how much effort would be involved. I just used the example vocabulary from our documentation, with strict_tags set true. I've attached the logs from all failing tests.
As far as I can tell, all of these are little one-offs that are just checking property propagation generally, and it would be no problem to update them to use properties defined in our test vocabulary. s3_test.go records a lot of failures, but that's because the property is in a common fixture and checked on every test. We just need to update one fixture and one assertion to get them all passing again.
Updated by Peter Amstutz almost 2 years ago
- Target version changed from Future to To be scheduled
Updated by Peter Amstutz almost 2 years ago
- Target version changed from To be scheduled to 2023-03-01 sprint
Updated by Peter Amstutz almost 2 years ago
- Target version changed from 2023-03-01 sprint to Development 2023-03-15 sprint
Updated by Peter Amstutz over 1 year ago
- Target version changed from Development 2023-03-15 sprint to Development 2023-03-29 Sprint
Updated by Peter Amstutz over 1 year ago
- Target version changed from Development 2023-03-29 Sprint to To be scheduled
Updated by Peter Amstutz 9 months ago
- Target version changed from To be scheduled to Future