Practical API Testing with Pytest: Smoke, Auth, and Contract Validation (With JSON Schema)

Practical API Testing with Pytest: Smoke, Auth, and Contract Validation (With JSON Schema)

When you’re building (or consuming) APIs, bugs rarely show up as “the server crashed.” They show up as silent contract breaks: a field disappears, a status code changes, pagination shifts, or auth stops working. Automated API tests are the cheapest way to catch those problems before your users do.

This hands-on guide shows how to build a small but powerful API test suite using pytest, httpx, and jsonschema. You’ll cover:

• Configurable base URLs (local/staging/prod)
• Fast smoke tests for health and core endpoints
• Auth flows (token-based) without hardcoding secrets
• Contract testing with JSON Schema (catch breaking response changes)
• Negative tests (validate error responses too)

1) Project setup

Create a dedicated folder for API tests. A clean structure keeps things maintainable as your test suite grows.

api-tests/ requirements.txt pytest.ini tests/ conftest.py test_health.py test_users.py schemas/ user.json error.json 

Install dependencies:

# requirements.txt pytest==8.3.2 httpx==0.27.0 python-dotenv==1.0.1 jsonschema==4.23.0 

Then:

python -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate pip install -r requirements.txt 

Add basic pytest configuration (optional but helpful):

# pytest.ini [pytest] testpaths = tests addopts = -q 

2) Configure base URL and a reusable HTTP client

Hardcoding https://staging.example.com inside tests is a pain. Instead, read configuration from environment variables. You can keep a .env locally, and set real env vars in CI.

# .env (local only, do not commit secrets) API_BASE_URL=http://localhost:3000 API_TOKEN= 

Create a shared client fixture in tests/conftest.py:

import os import httpx import pytest from dotenv import load_dotenv load_dotenv() def _base_url() -> str: url = os.getenv("API_BASE_URL", "").strip() if not url: raise RuntimeError("API_BASE_URL is required (e.g. http://localhost:3000)") return url.rstrip("/") @pytest.fixture(scope="session") def api_base_url() -> str: return _base_url() @pytest.fixture(scope="session") def api_token() -> str: # Keep empty if your API has public endpoints; tests can skip auth when missing return os.getenv("API_TOKEN", "").strip() @pytest.fixture(scope="session") def client(api_base_url: str) -> httpx.Client: # A single session improves performance and keeps headers consistent with httpx.Client(base_url=api_base_url, timeout=10.0) as c: yield c 

This gives you:

• client for requests
• api_base_url for debugging logs
• api_token for authenticated calls (when available)

3) Start with a “health” smoke test

Smoke tests should be fast and stable. They don’t check everything—they answer: “Is the API up and responding correctly?”

# tests/test_health.py def test_health_endpoint(client): # Common patterns: /health, /status, /ping resp = client.get("/health") assert resp.status_code == 200 data = resp.json() # Keep assertions minimal to reduce flakiness assert data.get("status") in ("ok", "healthy", "up") 

If your API doesn’t have /health, pick the lightest endpoint (like GET /version or GET /ping) and adjust the test.

4) Add authenticated requests without leaking secrets

For token-based auth, attach the Authorization header only when a token is present. This lets developers run public tests locally without needing access to staging tokens.

# tests/test_users.py import pytest def _auth_headers(token: str) -> dict: return {"Authorization": f"Bearer {token}"} if token else {} @pytest.mark.skipif(True is False, reason="placeholder to show pattern") def test_placeholder(): pass def test_get_current_user(client, api_token): if not api_token: pytest.skip("API_TOKEN not set; skipping auth-only test") resp = client.get("/me", headers=_auth_headers(api_token)) assert resp.status_code == 200 data = resp.json() assert "id" in data assert "email" in data 

Tip: Keep auth tests separate from public endpoint tests. That way a missing token doesn’t hide failures in unauthenticated routes.

5) Contract testing with JSON Schema (catch breaking changes)

Most API regressions are “shape” regressions: a field changes type, a nested object becomes null, or a list turns into a dictionary. JSON Schema helps you lock down the response structure.

Create a schema file for a user response:

// tests/schemas/user.json { "$schema": "https://json-schema.org/draft/2020-12/schema", "type": "object", "required": ["id", "email", "createdAt"], "properties": { "id": { "type": ["string", "integer"] }, "email": { "type": "string", "format": "email" }, "name": { "type": ["string", "null"] }, "createdAt": { "type": "string" } }, "additionalProperties": true } 

Now validate responses in tests:

# tests/test_users.py import json from pathlib import Path from jsonschema import validate SCHEMAS_DIR = Path(__file__).parent / "schemas" def load_schema(name: str) -> dict: return json.loads((SCHEMAS_DIR / name).read_text(encoding="utf-8")) def test_list_users_contract(client, api_token): if not api_token: # If listing users is public in your API, remove this skip import pytest pytest.skip("API_TOKEN not set; skipping auth-only test") resp = client.get("/users?limit=5", headers={"Authorization": f"Bearer {api_token}"}) assert resp.status_code == 200 data = resp.json() assert isinstance(data, list) user_schema = load_schema("user.json") for user in data: validate(instance=user, schema=user_schema) 

This test will fail immediately if the API suddenly removes email, renames createdAt, or returns something that isn’t a list.

6) Negative tests: verify error responses are consistent

APIs should be predictable even when they fail. Negative tests are great at preventing accidental changes to status codes and error formats.

Create an error schema:

// tests/schemas/error.json { "$schema": "https://json-schema.org/draft/2020-12/schema", "type": "object", "required": ["error", "message"], "properties": { "error": { "type": "string" }, "message": { "type": "string" }, "details": {} }, "additionalProperties": true } 

Test an unauthorized request:

# tests/test_users.py from jsonschema import validate def test_me_requires_auth(client): resp = client.get("/me") # Your API may use 401 or 403; pick what your backend promises assert resp.status_code in (401, 403) data = resp.json() error_schema = load_schema("error.json") validate(instance=data, schema=error_schema) 

If your backend accidentally starts returning HTML error pages or inconsistent JSON, this catches it fast.

7) Make tests easy to run locally and in CI

Run tests with a base URL:

API_BASE_URL=http://localhost:3000 pytest 

Or with a token (example):

API_BASE_URL=https://staging.example.com API_TOKEN="your-token" pytest 

If you want a simple GitHub Actions workflow, here’s a minimal example. It runs tests against a staging URL and uses repository secrets for the token.

# .github/workflows/api-tests.yml name: api-tests on: push: pull_request: jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: "3.11" - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt - name: Run API tests env: API_BASE_URL: ${{ vars.API_BASE_URL }} API_TOKEN: ${{ secrets.API_TOKEN }} run: pytest 

Keep the suite fast. If tests take minutes, developers won’t run them. Start small, then expand coverage around high-impact routes.

Common pitfalls (and how to avoid them)

• Flaky tests due to data dependencies: avoid assuming specific IDs exist. Create data via the API (setup) or test only general invariants (types, required fields).

• Over-asserting: smoke tests should be minimal. Use schema validation for structure, not strict full payload equality unless needed.

• Testing only “happy paths”: add negative tests for auth failures, validation errors, and missing resources (404).

• Ignoring pagination and sorting: when endpoints paginate, assert that response is stable (e.g., limit respected, fields present) rather than exact ordering unless contract guarantees it.

Next steps

Once this foundation is in place, your next upgrades are straightforward:

• Add coverage for key workflows (login → create resource → fetch → update → delete).

• Centralize schemas and reuse them across endpoints.

• Track response times for a few critical endpoints (basic performance checks).

• Run the same suite against multiple environments by switching API_BASE_URL.

With a few focused tests and contract validation, you’ll catch breaking API changes early, keep your endpoints consistent, and make releases less stressful—especially as teams and codebases grow.