Contract testing a nutrition API with millions of messy records

A schema can remain valid while every client still breaks. Changing search from a JSON array to {"results": [...]}, converting null to zero, or renaming carbs_g is enough to break a mobile release that cannot update immediately.

Freeze the consumer-visible shape

An OpenAPI document is necessary, but a small executable fixture catches accidental differences between the document and the running service. Store one representative response and assert keys, types and nullability.

EXPECTED_KEYS = {

    "id", "name", "brand", "barcode", "category",

    "serving_size_g", "serving_desc", "calories_kcal",

    "protein_g", "fat_g", "carbs_g", "fiber_g", "sugar_g",

    "sodium_mg", "source", "confidence", "static_url",

}



def test_food_contract(client):

    food = client.get("/food/1068319").json()

    assert EXPECTED_KEYS <= food.keys()

    assert isinstance(food["id"], int)

    assert food["brand"] is None or isinstance(food["brand"], str)

Test the envelope separately

The most damaging change is often outside the object. Assert that search returns a bare array if that is the published contract, a food lookup returns one object, and a missing ID or barcode returns 404.

def test_endpoint_envelopes(client):

    assert isinstance(client.get("/search?q=yogurt").json(), list)

    assert isinstance(client.get("/food/1068319").json(), dict)

    assert client.get("/barcode/00000000").status_code == 404

Nullability deserves dedicated fixtures

Clean example data hides real failures. Keep fixtures for a complete branded product, a sparse community record, a zero-calorie item, a missing serving size, Unicode text and a duplicate barcode. Verify that unknown nutrients remain null while reported zero stays numeric zero.

Separate contract tests from ranking tests

A contract test asks whether clients can parse the response. A ranking test asks whether useful foods appear in the right order. Freeze a set of queries and expected top IDs or relevance bands, but do not make every exact rank immutable: data refreshes legitimately add products.

Probe production read-only

Local tests cannot prove that the deployed reverse proxy, database and serializer agree. Run a small read-only probe after deployment: health, one search, one food ID, one known barcode, one 404 and one rate-limit header check. Never mutate community data during a smoke test.

def test_live_search_contract(session, base):

    r = session.get(f"{base}/search", params={"q": "oat milk", "limit": 2})

    r.raise_for_status()

    assert r.headers["content-type"].startswith("application/json")

    assert len(r.json()) <= 2

Classify changes before shipping

• Additive: a new nullable field is usually safe.




• Behavioral: ranking or rate-limit changes need release notes and tests.




• Breaking: renamed fields, changed types or envelopes require a version or migration window.




• Data: corrected values should not require a schema version, but may affect snapshots.

Keep a frozen production-baseline JSON file in the repository and review diffs. That turns “the API probably stayed compatible” into evidence.

Use producer tests and consumer tests together

Producer-side tests verify that the API implementation follows its declared schema. Consumer-driven tests capture assumptions made by real clients: search is an array, a particular header exists, unknown sugar remains null and a 404 body can be parsed. Neither view is sufficient alone. The server may satisfy OpenAPI while changing an undocumented behavior on which every released mobile client depends.

Collect consumer expectations deliberately rather than recording all current behavior forever. Protect the parts required for compatibility, and allow internal implementation details to change. Give each expectation an owner and an explanation so obsolete constraints can be retired safely.

Avoid brittle full-response snapshots

Nutrition values and product names legitimately change as upstream records are corrected. A snapshot of an entire live response will create noisy failures and encourage developers to approve changes without reading them. Assert schema, invariants and a few controlled fixtures instead. When values matter, seed a test database with records owned by the test suite.

Useful invariants include nutrient values being numeric or null, IDs being positive integers, result limits being respected, confidence staying in its documented range and barcode misses returning 404 rather than a fabricated object. Property-based tests can generate combinations of nullable fields and serving sizes that hand-written examples overlook.

Version data contracts independently from deployments

A server can deploy every day while its public contract remains version one. Track contract changes in release notes and compare the generated OpenAPI document to a reviewed baseline in continuous integration. A diff that removes a property, narrows a type or changes required fields should fail until someone classifies it.

When a breaking change is necessary, prefer an explicit endpoint or media-type version and run both contracts during a migration window. Monitor which version clients use before retiring the old one. Announcing a date is not enough if telemetry shows active clients cannot upgrade.

Test operational behavior as part of the contract

Rate-limit responses, authentication failures, cache headers and content types affect integrations as much as JSON fields. Verify that 429 includes a useful Retry-After, that anonymous endpoints remain anonymous if promised, and that privileged keys never appear in logs or error bodies. Confirm that proxies preserve status codes instead of converting every failure into HTML.

Run probes from outside the origin network. An internal health endpoint can be green while DNS, TLS or the CDN is failing for customers. Keep production probes small, read-only and rate-aware, and distinguish origin processing time from customer-visible end-to-end latency.

Make compatibility review part of code review

When a pull request touches response models, serializers, SQL column aliases or middleware, require a contract impact note. Reviewers should see the before-and-after schema and relevant fixture changes. Generated clients can be compiled in CI to expose changes that look harmless in JSON but break a strongly typed language.

The goal is not to prevent evolution. It is to make breaking changes intentional, observable and survivable. A boring stable contract is a product feature when customers build apps that outlive your latest deployment.

Dietly contract: search remains a bare JSON array; nutrient fields retain nullability; ID and barcode lookups return one object or 404. The OpenAPI specification documents the public shape.

Originally published at getdietly.com. Data from the Dietly Nutrition API — 4.7M+ indexed foods, free tier available.