Skip to content

dev: add start of CI section#34

Open
davidjpeacock wants to merge 1 commit intoos-migrate:mainfrom
davidjpeacock:ci-docs
Open

dev: add start of CI section#34
davidjpeacock wants to merge 1 commit intoos-migrate:mainfrom
davidjpeacock:ci-docs

Conversation

@davidjpeacock
Copy link
Copy Markdown
Contributor

New section to encompass definitive documentation of our CI environment for all four repos.

New section to encompass definitive documentation of our CI environment for all four repos.

A meta umbrella term in this project is End to End or E2E testing, which broadly speaking means Integration (and therefore functional) testing.

== Test Types
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are explaining what we do here, maybe we can write a bit more info here about the tests, so we arent vague. Hopefully this isnt an overkill, so let me know what you think.

  1. Linters: Style, syntax, formatting
  2. Sanity Tests (ansible-test sanity) Import tests
  • Compilation checks
  • Documentation validation
  • Argument spec validation
  • Code quality checks (pylint, pep8)
  • we dont write these, we conform them. they come as standardization for ansible.
  1. Unit Tests
  • Individual functions/methods tested in isolation
  1. Integration Tests
  • Module integration tests (test modules against real/fake APIs)
  • Role integration tests (test roles in test environments)
  • Can use real systems OR mocks (like srv.go fake OpenStack)
  1. Functional tests
  • For role testing, meaning testing the POV of personas admin and tenant, we are ensuring migration works regardless of which permission level is used. we are testing it via ansible playbooks. in our project, it is categorized as functional, we are testing a lot of stuff from the perspective, but not full workflow as with e2e.
  1. E2E Tests
  • Full workflow tests from user perspective
  • Typically against real systems

then.... i am not sure where to put Module testing as you are describing it now (meaning: separate headers/topics). I would personally categorize module testing under either unit (testing single small targeted feature in module) or integration (stuff is working together/ idempotency).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants