Python API Testing Project

This project serves as a comprehensive example of building a robust testing suite for a Python Flask API. It demonstrates various testing methodologies, including unit, behavior-driven development (BDD), integration, end-to-end (E2E), performance, and contract testing, along with static application security testing (SAST) and dependency scanning.

Project Structure

python_test_harness/
├── .github/workflows/  # GitHub Actions CI/CD workflows
│   └── main.yml
├── features/           # Behave BDD feature files and step definitions
│   ├── steps/
│   └── user_management.feature
├── performance/        # Locust performance test scripts
│   └── locustfile.py
├── src/                # Flask application source code
│   └── app.py
├── tests/              # Pytest unit, integration, E2E, and contract tests
│   ├── contract/
│   │   ├── test_consumer.py
│   │   └── test_provider.py
│   ├── e2e/
│   │   └── test_user_journey.py
│   ├── integration/
│   │   └── test_db_integration.py
│   ├── factories.py
│   ├── conftest.py
│   └── test_app.py
├── Dockerfile          # Dockerfile for building the Flask application image
├── docker-compose.yml  # Docker Compose for orchestrating services (app, DB, Pact Broker)
├── Makefile            # Utility for running tests and development tasks
└── requirements.txt    # Python dependencies

Technologies Used

• Flask: Web framework for the API.
• pytest: Python testing framework for unit, integration, E2E, and contract tests.
• behave: BDD framework for writing human-readable tests.
• locust: Open-source load testing tool for performance testing.
• Flask-SQLAlchemy: ORM for database interactions.
• PostgreSQL: Relational database (used via Docker).
• Faker: Library for generating realistic fake data for tests.
• Factory Boy: Test data factories for SQLAlchemy models.
• Bandit: Static Application Security Testing (SAST) tool.
• pip-audit: Dependency vulnerability scanner.
• Pact: Consumer-Driven Contract testing framework (used with pact-python). While consumer-side tests are robust, provider-side verification within Docker Compose has proven challenging with pact-python due to its interaction with Docker networking and file systems. For a production setup, consider alternative Python contract testing libraries or a dedicated Pact Broker setup in your CI/CD pipeline.
• Docker & Docker Compose: For containerization and orchestration of the application and its dependencies.
• GitHub Actions: For Continuous Integration and Continuous Delivery.

Setup and Installation

1. Clone the repository:

   git clone https://github.com/your-username/python-api-testing.git
   cd python-api-testing

2. Install Python dependencies:

   pip install -r requirements.txt

3. Install Docker and Docker Compose: Ensure Docker and Docker Compose (or the docker compose plugin) are installed and running on your system. Refer to the official Docker documentation for installation instructions.

4. Build and start Docker services: This will build the Flask application image and start the PostgreSQL database and Pact Broker containers.

   docker compose up --build -d

Running Tests

This project uses a Makefile to simplify running various test suites. You can run individual test types or all tests at once.

Individual Test Suites

• Unit Tests (pytest): Verifies individual components of the application.

  make unit-tests

• Behavior-Driven Development (BDD) Tests (behave): Tests features from a user's perspective.

  make bdd-tests

• Integration Tests (pytest): Tests interactions with the PostgreSQL database.

  make integration-tests

• Consumer Pact Tests (pytest): Defines the contract from the consumer's perspective and generates a pact file.

  make contract-consumer-tests

• Provider Pact Tests (pytest): Verifies the API against the consumer contract.

  make contract-provider-tests

• End-to-End (E2E) Tests (pytest): Simulates a full user journey through the API.

  make e2e-tests

• Static Application Security Testing (SAST) (Bandit): Scans for common security issues in the Python code.

  make security-sast

• Dependency Scanning (pip-audit): Identifies known vulnerabilities in project dependencies.

  make security-deps

Running All Tests

To run all test suites sequentially:

make all-tests

Cleaning Up

To remove temporary files and Docker containers:

make clean
docker compose down -v # Removes containers and volumes

CI/CD with GitHub Actions

The project includes a GitHub Actions workflow (.github/workflows/main.yml) that automatically runs all test suites on every push and pull request to the main branch. This ensures continuous quality assurance and early detection of issues.

Future Enhancements

• More comprehensive E2E tests: Integrate a headless browser (e.g., Playwright) for full UI-driven E2E tests if a frontend is added.
• Advanced performance testing: Explore distributed Locust runs and integration with monitoring tools like Prometheus and Grafana.
• Enhanced security testing: Incorporate dynamic application security testing (DAST) and more in-depth security audits.
• Test reporting: Integrate tools like Allure Report for rich, interactive test reports.
• Pact Broker integration: Fully leverage a Pact Broker for contract management and verification in a distributed microservices environment.

Contributing

Feel free to fork this repository, open issues, and submit pull requests. Contributions are welcome!