Detect Breaking API Changes Before Consumers Do

A client release compiles on Monday and starts returning 400 on Tuesday. The server still accepts the JSON it received last week, except the new OpenAPI description now marks locale as required. A generated SDK took that declaration literally and made locale a mandatory constructor argument. Another consumer keeps running but no longer sees suspended accounts because SUSPENDED disappeared from a response enum.

A line diff saw punctuation move around. A response snapshot saw one ordinary ACTIVE account. Neither check answered the compatibility question: can a consumer that followed yesterday's contract still perform its work against tomorrow's service?

Compare releases, not whichever two files happen to be in the checkout

The baseline must be the description for the version consumers can use now. The revision must be the resolved description you intend to release. Comparing a generated file with an ungenerated source file turns bundling, key order and moved $ref targets into noise. Comparing two files from the same branch can miss the whole release boundary.

Store the inputs beside the report. That gives a reviewer three artefacts: the baseline, the candidate, and the semantic findings. It also makes the verdict reproducible after the default branch has moved.

#!/usr/bin/env bash
set -euo pipefail

baseline="artifacts/openapi-released.yaml"
candidate="artifacts/openapi-candidate.yaml"

# Fetch or generate these through your own release process. Do not compare
# an unresolved authoring file with a bundled release artefact.
test -s "$baseline"
test -s "$candidate"

oasdiff validate "$baseline"
oasdiff validate "$candidate"

# `--` prevents a path beginning with '-' from being parsed as a flag.
oasdiff changelog --format markdown -- "$baseline" "$candidate" \
  > artifacts/api-changelog.md

oasdiff breaking --format json --fail-on ERR -- "$baseline" "$candidate" \
  > artifacts/api-breaking.json

This uses the breaking and changelog commands for different jobs. The breaking report contains definite and potential compatibility findings; --fail-on ERR returns a failing status for definite breaks. The changelog includes non-breaking consumer-visible changes as well. Pin the oasdiff release in the CI image rather than installing “latest”; this article's commands were checked against oasdiff 1.30.0.

Resolving the description does not mean flattening every composition keyword by reflex. In oasdiff, --flatten-allof is opt-in because it transforms comparison semantics. Without it, a change inside allOf may be softened to a warning when a sibling branch could still supply the constraint. If such a warning appears, rerun that comparison with the flag and keep both results. Do not globally enable a transform to silence one awkward schema.

The same schema edit changes meaning when data changes direction

JSON Schema defines which instances satisfy a schema. Compatibility adds time and direction. For a request, the old consumer produces data and the new service consumes it. For a response, the new service produces data and the old consumer consumes it. A safe change preserves the valid values that cross that boundary.

The word usually in that table is deliberate. OpenAPI 3.1 uses JSON Schema 2020-12 vocabularies, and JSON Schema says omitted additionalProperties behaves like an empty schema, which accepts every additional property's value. A consumer configured to reject unknown fields has adopted a narrower contract than the API declared. That may be a valid local choice, but an OpenAPI diff cannot infer it.

format needs the same care. Under JSON Schema 2020-12, the default format vocabulary collects an annotation; format assertion is a separate vocabulary and implementation support varies. A change from format: uuid to an unconstrained string can still break code that parses directly into a UUID type. Record whether code generation, a gateway, or a runtime validator treats format as an assertion. Do not promote or suppress every format finding with one global rule.

Two edits produce three different verdicts

Consider the account API below. The released request allows callers to omit locale. Its response can contain either account state:

# released.yaml
components:
  schemas:
    CreateAccount:
      type: object
      required: [email]
      properties:
        email: { type: string, format: email }
        locale: { type: string, enum: [en-IN, en-GB] }
    Account:
      type: object
      required: [id, status]
      properties:
        id: { type: string, format: uuid }
        status: { type: string, enum: [ACTIVE, SUSPENDED] }

The candidate makes locale required, removes one response state, and adds an optional display name:

# candidate.yaml
components:
  schemas:
    CreateAccount:
      type: object
      required: [email, locale]
      properties:
        email: { type: string, format: email }
        locale: { type: string, enum: [en-IN, en-GB] }
    Account:
      type: object
      required: [id, status]
      properties:
        id: { type: string, format: uuid }
        status: { type: string, enum: [ACTIVE] }
        displayName: { type: string }

The required request property is a deterministic break. Under JSON Schema, every name in required must appear in the instance. A default would not repair the contract: the request without locale remains invalid even if this particular server supplies a fallback.

The response enum removal needs two labels. At the structural boundary it narrows the set of values the new producer may send, so an old decoder that accepts ACTIVE and SUSPENDED still accepts every declared new response. At the product boundary it can remove an observable state. If a billing export, fraud queue or audit feed waits for SUSPENDED, that workflow broke even though deserialization did not. Mark it as a policy break only after naming that consumer and proving the dependency.

The optional displayName is additive under the declared open schema. A mobile client using a strict decoder may still reject it. So the verdict has two halves. The contract permits the addition, and the consumers that decode strictly have to be exercised before it ships.

Send each finding to the consumer that can disprove it

A central compatibility job should not run every consumer suite. Maintain a small ownership file that connects operations and schema paths to the contracts that matter:

# contracts/consumers.yaml
consumers:
  - name: legacy-account-sdk
    owner: identity-platform
    operations: [createAccount]
    contract: pnpm playwright test contracts/legacy-account.spec.ts
  - name: risk-export
    owner: fraud-operations
    schemas: [Account.status]
    contract: pnpm playwright test contracts/risk-export.spec.ts
  - name: android-account-cache
    owner: mobile-foundation
    schemas: [Account]
    contract: ./gradlew accountContractTest

The mapping is reviewable evidence. A regex that scans repositories for the word Account is only a discovery aid; it cannot decide ownership or whether generated code reaches production.

The legacy request test captures the caller that would fail first. It deliberately omits the new field:

import { test, expect } from '@playwright/test';

test('released client shape can still create an account', async ({ request }) => {
  const response = await request.post('/v1/accounts', {
    data: { email: '[email protected]' },
    headers: { 'X-Client-Version': 'web-4.8.2' },
  });

  expect(response.status()).toBe(201);
  const account = await response.json();
  expect(account).toEqual(expect.objectContaining({
    id: expect.any(String),
    status: 'ACTIVE',
  }));
});

During a staged migration, that test can remain green while the published next-version schema requires locale. This is not permission to lie in one contract. Keep the existing operation compatible, or publish a new version boundary where the required field is truthful.

The response enum needs a behaviour-producing test, not another schema assertion:

test('suspended accounts remain present in the risk export', async ({ request }) => {
  const created = await request.post('/test-support/accounts', {
    data: { status: 'SUSPENDED' },
  });
  expect(created.status()).toBe(201);

  const { id } = await created.json();
  const exported = await request.get(`/v1/risk/accounts/${id}`);

  expect(exported.status()).toBe(200);
  expect(await exported.json()).toEqual({ id, status: 'SUSPENDED' });
});

This test answers the question the enum diff could not: does the consumer-visible workflow still produce the state? If the state is intentionally retired, replace the test with migration evidence from the owning team rather than deleting it in the same pull request.

An exception should read like a migration record

A permanent ignore file turns the compatibility gate into archaeology. Approve an intentional break only when the review contains:

• the exact operation, direction and JSON path from the semantic report;
• the affected consumers and their owners, including “none found” with the search evidence;
• the replacement contract or versioned endpoint;
• a removal date for the old behaviour; and
• an issue or change record that survives after the pull request closes.

Scope the exception to a check and endpoint. oasdiff accepts error and warning ignore files, but the ignore line itself cannot carry all the evidence above. Keep the policy record next to it and make expiry a separate CI check. An approval with no deadline is a new compatibility rule written accidentally.

When the gate is noisy, diagnose the noise

Every pull request is red after regeneration

You are comparing authoring shapes rather than released contracts. Bundle references through the same path on both sides, discard documentation-only changes from the breaking decision, and keep the full diff separately when reviewers need it. Use the JSON Diff tool to inspect a suspicious generated fragment, but do not substitute a value diff for OpenAPI semantics.

The diff is green but a client crashes on a new field

The client is stricter than the declared schema. Decide whether the API should close the object with additionalProperties: false or whether the client should tolerate declared additions. Until that decision is shipped, map the client to the response schema and run its decoder contract on additive changes. The JSON Schema Generator can create a starting schema from a payload, but representative data cannot tell you which additions the contract should permit.

A business-breaking enum edit passes structural checks

The dependency is behavioural, not syntactic. Add the consumer that waits for the removed state to the ownership file and make its contract produce that state. The opposite edit, adding a response enum member, is structurally dangerous because old closed decoders may receive an unknown value; test a fallback branch rather than only the currently observed values.

Approved breaks never leave the exception file

The gate checks syntax but no process checks expiry. Require a migration owner and a machine-readable removal date, then fail when it passes. Inspect old Postman collections with the Postman Collection Viewer when mapping consumers: request examples, environment variables and saved response examples often reveal a caller that repository search missed.

Questions teams ask once the first real diff lands

Is changing a response status code always breaking?

Changing or removing a documented response can break a consumer that branches on it, but status codes need operation context. Replacing a documented 202 with 200 changes an asynchronous completion promise even though both are successful. Adding a documented error response may only describe behaviour that already existed. Use the HTTP Status Code Reference to check the protocol meaning, then test the caller's branch.

Can response snapshots replace an OpenAPI breaking-change check?

No. A snapshot records one observed instance. It cannot prove that an optional request property became required, that an unobserved enum member was removed, or that a numeric constraint narrowed. Keep snapshots for examples where the exact representation matters. Use the API description for the declared set of inputs and outputs, and executable consumer contracts for dependencies the description cannot express.

Should every warning fail CI?

Start by failing definite errors and requiring review for warnings. A warning means the tool lacks enough information for a deterministic verdict, often because composition or serialization is ambiguous. Turning every warning into an error can be appropriate after you classify your own specifications, but doing it on day one trains people to bypass the gate. Record each severity override by rule identifier and the local contract assumption that justifies it.

Where should the released baseline come from?

Use the immutable artefact delivered with the running API version: a release attachment, registry object or versioned repository path. Do not fetch a mutable default-branch URL during the comparison. Record its digest with the candidate and report. The goal is to reproduce the verdict after both the service repository and the consumer repositories have advanced.

Does a major version bump make a breaking change safe?

No. A version boundary makes the incompatibility explicit; it does not migrate a caller. Keep the old operation available for the promised overlap, publish the candidate description under the new version, and run contracts against both until mapped consumers move. The evidence for removal is zero remaining owners on the old contract plus telemetry or an inventory appropriate to your environment, not the new number in info.version.

Primary references

• OpenAPI Specification 3.1.1: OpenAPI Description semantics, Schema Object dialect behaviour, composition, and the annotation status of readOnly, writeOnly, and default format
• OpenAPI 3.1 JSON Schema dialect: the base vocabulary used by OpenAPI 3.1 Schema Objects
• JSON Schema 2020-12 Validation: the exact validation behaviour of enum, required, numeric constraints, and format annotation versus assertion
• JSON Schema 2020-12 Core: applicator behaviour and omitted additionalProperties acting as an empty schema
• oasdiff — Breaking Changes and Changelog: ERR, WARN, and INFO levels, --fail-on, report formats, ignore files, and contract-direction rules
• oasdiff — Diff engine: opt-in transformations, allOf handling, reference matching, and comparison limits
• oasdiff 1.30.0 release: the tool version against which the example commands were checked
• OWASP API Security Top 10 — 2023: API inventory and unsafe consumption as wider operational reasons to keep producer and consumer knowledge current

Apply this now

Take the OpenAPI artefact from your last release and the candidate from the next one. Run a pinned semantic changelog and breaking check, saving both inputs and both reports. For each finding, write the data direction beside the path: old consumer to new service for requests, new service to old consumer for responses.

Choose the highest-risk request and response finding. Add one executable contract for the oldest supported caller shape and one for the business behaviour the schema cannot express. Assign an owner before allowing any exception. Then look at what the run left behind: a pinned tool version, a digest for each input, the rule identifier and JSON path, the consumer that path maps to, that consumer's test result, a migration issue, and a removal date. A gate that cannot produce all of those has not recorded enough to defend its own verdict six months later, which is when someone will ask.