Hands-On API Contract Testing with OpenAPI + Schemathesis (Catch Bugs Your Unit Tests Miss)

Pictalingo

·

Hands-On API Contract Testing with OpenAPI + Schemathesis (Catch Bugs Your Unit Tests Miss)

Most junior/mid devs learn API testing as “write a few happy-path tests.” That’s useful, but it leaves gaps: weird inputs, missing fields, edge-case query params, and unexpected status codes. Contract testing flips the approach: you declare the API contract (usually OpenAPI), then automatically generate and run many requests that try to break your assumptions.

In this tutorial you’ll build practical, repeatable API contract tests using OpenAPI + Schemathesis (a property-based API testing tool) + pytest. You’ll end up with a test suite that:

  • Validates responses match your OpenAPI schema
  • Generates inputs to probe edge cases automatically
  • Supports auth headers and state setup (create → fetch → delete)
  • Runs locally and in CI with one command

What You’ll Need

  • An API that exposes an OpenAPI schema (e.g., /openapi.json)
  • Python 3.10+ and a virtual environment

Install the tools:

python -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate pip install schemathesis pytest requests

Tip: If your API is local, start it in another terminal. For this guide, assume the base URL is http://localhost:8000 and your schema is at /openapi.json.

Step 1: Confirm Your OpenAPI Schema Is Reachable

Your contract tests depend on the schema being correct and accessible. Quickly check:

curl -s http://localhost:8000/openapi.json | head -n 20

If you don’t have an OpenAPI endpoint, fix that first. Many frameworks can generate it automatically (FastAPI, NestJS with Swagger, Springdoc, etc.).

Step 2: Run a Zero-Code “Smoke Contract Test”

Schemathesis can run directly against your schema without you writing any tests. This is the fastest way to find obvious issues.

schemathesis run http://localhost:8000/openapi.json \ --base-url http://localhost:8000

What this does:

  • Reads endpoints and schemas from OpenAPI
  • Generates requests (query params, JSON bodies, path params)
  • Validates responses against the schema (types, required fields, etc.)

If it fails, it will show the endpoint, a failing example, and the mismatch. Common early wins include:

  • Response says 200 but schema documents 201 (or vice versa)
  • A field marked required is sometimes missing
  • A response type is wrong (string vs number)

Step 3: Turn It into a Real pytest Suite

CLI runs are great, but most teams want tests integrated with pytest (markers, selective runs, CI, reports). Create tests/test_contract.py:

import schemathesis from schemathesis import Case SCHEMA_URL = "http://localhost:8000/openapi.json" BASE_URL = "http://localhost:8000" schema = schemathesis.from_uri(SCHEMA_URL, base_url=BASE_URL) @schema.parametrize() def test_api_contract(case: Case): # Executes the request generated from your OpenAPI schema response = case.call() # Validates: # - response matches documented status codes # - JSON schema types/required fields # - headers / content-type where applicable case.validate_response(response)

Run it:

pytest -q

You now have contract tests that scale with your API. Add endpoints? Update OpenAPI? Your tests adapt automatically.

Step 4: Add Authentication (Headers / Bearer Tokens)

Many APIs require auth. You can attach headers globally using a pytest fixture or by configuring a “hook.” Here’s a simple approach: read a token from an environment variable and add it to every request.

import os import schemathesis from schemathesis import Case SCHEMA_URL = "http://localhost:8000/openapi.json" BASE_URL = "http://localhost:8000" schema = schemathesis.from_uri(SCHEMA_URL, base_url=BASE_URL) def auth_headers(): token = os.getenv("API_TOKEN") if not token: return {} return {"Authorization": f"Bearer {token}"} @schema.parametrize() def test_api_contract(case: Case): response = case.call(headers=auth_headers()) case.validate_response(response)

Run with:

export API_TOKEN="your_token_here" pytest -q

Pro move: Use a short-lived test token or run your API in a test mode where a known token is accepted.

Step 5: Focus on “Interesting” Endpoints First (Don’t Boil the Ocean)

On a large API, generating tests for everything can be noisy at first. Start with a subset (e.g., /users, /orders) using a filter. One practical way is to skip endpoints by path or tag.

import schemathesis from schemathesis import Case schema = schemathesis.from_uri( "http://localhost:8000/openapi.json", base_url="http://localhost:8000", ) # Keep only endpoints matching a prefix schema = schema.filter(lambda endpoint: endpoint.path.startswith("/v1/users")) @schema.parametrize() def test_users_contract(case: Case): response = case.call() case.validate_response(response)

As your schema quality improves, widen coverage.

Step 6: Make Failures Reproducible (Save the Failing Example)

Contract tests can generate unusual payloads. When something fails, you want a stable reproduction. Schemathesis shows failing cases, but you can also increase clarity by printing the request details on failure:

import schemathesis from schemathesis import Case schema = schemathesis.from_uri( "http://localhost:8000/openapi.json", base_url="http://localhost:8000", ) @schema.parametrize() def test_api_contract(case: Case): response = case.call() try: case.validate_response(response) except Exception: # Useful debug info for juniors on the team print("METHOD:", case.method) print("PATH:", case.path) print("HEADERS:", case.headers) print("QUERY:", case.query) print("BODY:", case.body) print("STATUS:", response.status_code) print("RESPONSE:", response.text[:2000]) raise

This turns a “mystery failure” into something you can reproduce with curl or Postman quickly.

Step 7: Add a Simple “State Setup” for Create → Fetch Flows

Some endpoints require existing data (e.g., you can’t GET /users/{id} unless a user exists). Contract tests are strongest when the API is in a known-good state.

A practical pattern: write one small helper that creates a resource via a real request, then feed its ID into tests for dependent endpoints.

import requests import schemathesis from schemathesis import Case BASE_URL = "http://localhost:8000" schema = schemathesis.from_uri(f"{BASE_URL}/openapi.json", base_url=BASE_URL) def create_user() -> str: payload = {"email": "[email protected]", "name": "Contract Test"} r = requests.post(f"{BASE_URL}/v1/users", json=payload, timeout=10) r.raise_for_status() data = r.json() return data["id"] # Filter to a dependent endpoint user_detail = schema.filter(lambda e: e.path == "/v1/users/{id}" and e.method == "GET") @user_detail.parametrize() def test_user_detail_contract(case: Case): user_id = create_user() # Provide path params explicitly response = case.call(path_parameters={"id": user_id}) case.validate_response(response)

This is “good enough” state management for many teams: create a record, test the dependent endpoint, optionally clean up in teardown.

Step 8: Put It in CI (One Command)

Even if you’re not ready for a full pipeline, aim for a single command that runs locally and in CI:

pytest -q

If your CI starts the API container/service before tests, you’re set. Keep these CI-friendly habits:

  • Use environment variables for BASE_URL, tokens, and timeouts
  • Fail fast: start with a smaller endpoint subset, then expand
  • Keep schemas honest: update OpenAPI as part of feature work

Example environment-driven config:

import os import schemathesis BASE_URL = os.getenv("BASE_URL", "http://localhost:8000") SCHEMA_URL = os.getenv("SCHEMA_URL", f"{BASE_URL}/openapi.json") schema = schemathesis.from_uri(SCHEMA_URL, base_url=BASE_URL)

Common Gotchas (And How to Avoid Them)

  • Schema drift: If OpenAPI is outdated, tests “fail” even though the API works. Fix by generating the schema from code or treating the schema as a first-class artifact.
  • Over-strict required fields: Teams mark fields required that are actually optional in some responses. Contract tests will expose this quickly—decide whether to change code or schema.
  • Unstable endpoints: If an endpoint depends on time, randomness, or external services, expect flaky results. Use mocks, test environments, or mark those endpoints to skip in contract runs.
  • Too much too soon: Don’t start with 200 endpoints. Start with a core set, get green, then expand.

Where This Fits in Your Testing Strategy

Contract tests are not a replacement for unit tests or a few carefully written integration tests. They’re a multiplier:

  • Unit tests: verify business logic fast and precisely
  • Integration tests: verify key flows you care about (login, checkout, etc.)
  • Contract tests: continuously validate the API surface for surprises and regressions

If you maintain an OpenAPI schema, contract testing is one of the highest ROI ways to raise API reliability—especially for junior/mid teams—because it catches schema mismatches and edge cases before your users do.

Next step: Add contract tests for your top 5 endpoints, run them in CI, and treat OpenAPI updates as part of your definition of done.

Leave a Reply

Your email address will not be published. Required fields are marked *