Open
Conversation
New section to encompass definitive documentation of our CI environment for all four repos.
vyvle
reviewed
Feb 12, 2026
|
|
||
| A meta umbrella term in this project is End to End or E2E testing, which broadly speaking means Integration (and therefore functional) testing. | ||
|
|
||
| == Test Types |
Contributor
There was a problem hiding this comment.
Since we are explaining what we do here, maybe we can write a bit more info here about the tests, so we arent vague. Hopefully this isnt an overkill, so let me know what you think.
- Linters: Style, syntax, formatting
- Sanity Tests (ansible-test sanity) Import tests
- Compilation checks
- Documentation validation
- Argument spec validation
- Code quality checks (pylint, pep8)
- we dont write these, we conform them. they come as standardization for ansible.
- Unit Tests
- Individual functions/methods tested in isolation
- Integration Tests
- Module integration tests (test modules against real/fake APIs)
- Role integration tests (test roles in test environments)
- Can use real systems OR mocks (like srv.go fake OpenStack)
- Functional tests
- For role testing, meaning testing the POV of personas admin and tenant, we are ensuring migration works regardless of which permission level is used. we are testing it via ansible playbooks. in our project, it is categorized as functional, we are testing a lot of stuff from the perspective, but not full workflow as with e2e.
- E2E Tests
- Full workflow tests from user perspective
- Typically against real systems
then.... i am not sure where to put Module testing as you are describing it now (meaning: separate headers/topics). I would personally categorize module testing under either unit (testing single small targeted feature in module) or integration (stuff is working together/ idempotency).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New section to encompass definitive documentation of our CI environment for all four repos.