Skip to content

Continuous Integration (CI) and Test Automation#

Continuous Integration (CI) ensures consistent code quality through automated testing, reporting, and deployments.
Our setup combines GitHub Actions and tox to streamline testing across environments, making the process robust and efficient.

GitHub Actions: Workflows#

GitHub Actions automates key workflows like testing, linting, and code quality checks. The CI pipeline includes the following tasks:
- Unit Testing: Using pytest to ensure comprehensive test coverage.
- Code Formatting: Validating syntax with ruff.
- Linting: Verifying code style and docstrings with ruff.lint. - Import Sorting: Organizing imports with ruff.lint.isort.

To set up GitHub Actions workflows for the repository, follow the official GitHub guide.
The GitHub Actions workflow is triggered on pull requests or commits to the develop or production branches.
1. Setup: Prepares a Python environment and installs dependencies.
2. Testing: Executes the tox automation suite to run the defined tests.
3. Reporting: Generates detailed failure reports for debugging.

GitHub Actions also automates documentation updates, ensuring the latest changes from the develop branch are reflected. For more details, see the official GitHub Actions guide.

Tox: Automating Testing#

tox is a versatile tool for managing test environments and automating testing tasks across Python versions and operating systems (Linux, macOS, Windows).
It enhances reproducibility and reliability by creating isolated environments for testing:

  • Virtual Environments: Isolates dependencies for each Python version.
  • Automated Testing: Runs tests and style checks (for example, pytest, ruff).
  • Cross-Version/Platform Testing: Ensures compatibility across Python versions and operating systems.
  • Dependency Management: Customizes dependencies for diverse test scenarios.
  • Reproducibility: Maintains consistent workflows by way of 📝 tox.ini.
  • Extensibility: Supports plugins for custom functionality.

Install#

Install the required package in a python environment.
💻 pip install tox Install tox
💻 tox Run tox locally

Adding and Managing Tests#

New tests should be placed in the 📝 test/ directory. For example:
- Add a test to validate new functionality.
- Use pytest for unit tests, or extend 📝 tox.ini for additional test configurations.

By combining tox and GitHub Actions, our CI pipeline ensures robust, reproducible, and scalable testing workflows.

Test Patterns#

  1. Unit and Integration Test
  2. Purpose: Test individual functions or methods.
  3. Framework: unittest, pytest

  4. Example: Simple function test and external API interactions.

  5. Functional Testing

  6. Purpose: Test complete functionality (end-to-end).
  7. Framework: pytest, selenium

  8. Example: Test user login in a web app.

  9. Mocking

  10. Purpose: Simulate external dependencies.
  11. Framework: unittest.mock, pytest-mock

  12. Example: Mock API calls in tests.

  13. Acceptance Testing

  14. Purpose: Verify the software meets requirements.
  15. Framework: pytest, Behave

  16. Example: Verify login form functionality.

  17. Performance Testing

  18. Purpose: Measure performance under load.
  19. Framework: pytest-benchmark, locust

  20. Example: Measure function execution time.

  21. Security Testing

    • Purpose: Test for security vulnerabilities.
    • Framework: bandit
    • Example: Scan for hardcoded passwords.

  22. Snapshot Testing

    • Purpose: Compare outputs to saved snapshots.
    • Framework: pytest-snapshot
    • Example: Compare function output snapshots.

  23. Property-Based Testing

    • Purpose: Test properties of functions with random inputs.
    • Framework: hypothesis
    • Example: Test that a sorting function always returns a sorted list.

Examples#

The file 📝 test/test_example.py contains basic examples for the functions in
📝 super_repo/test_calculator.py.

In Python, the assert statement is used to test if a condition is true.
If the condition is false, an AssertionError is raised, indicating the test failed.

result = add(3, 4)
assert result == 7

The raises context manager from the pytest library ensures that a ValueError is raised.
The match argument specifies that the error message matches the expected error pattern.

with raises(ValueError, match=r"Cannot divide by zero"):
    divide(15, 0)

We encourage contributions of additional tests and examples to help improve coverage and showcase different use cases—your input is valuable to enhancing the project!

Used Icons

🐙 GitHub | 💠 git | 📝 File | 💻 Command Line