Update tests to run with a strict vocabulary
- Write a
vocubalary.jsonfor the tests in the right place (FIXME: where is that?). It should define at least a couple of tags and
- Update the test configuration in
API.VocabularyPathto that file path.
- Update tests broken by this change to continue passing while exercising the same behavior as before.
We want to make this change to help us catch system tags that we add without documentation or adding them to
Updated by Brett Smith 8 months ago
- Story points set to 1.0
- Category set to Tests
- Tracker changed from Bug to Story
- File developer-run-tests-sdk-python-ruby-2778.log developer-run-tests-sdk-python-ruby-2778.log added
- File developer-run-tests-remainder-3635.log developer-run-tests-remainder-3635.log added
- File developer-run-tests-apps-workbench-units-3478.log developer-run-tests-apps-workbench-units-3478.log added
- File developer-run-tests-apps-workbench-integration-3718.log developer-run-tests-apps-workbench-integration-3718.log added
- File developer-run-tests-apps-workbench-functionals-3518.log developer-run-tests-apps-workbench-functionals-3518.log added
I made a throwaway branch to define a strict vocabulary for the tests and run that on Jenkins to see how much effort would be involved. I just used the example vocabulary from our documentation, with
strict_tags set true. I've attached the logs from all failing tests.
As far as I can tell, all of these are little one-offs that are just checking property propagation generally, and it would be no problem to update them to use properties defined in our test vocabulary.
s3_test.go records a lot of failures, but that's because the property is in a common fixture and checked on every test. We just need to update one fixture and one assertion to get them all passing again.