Turn a Postman Collection into Maintainable API Tests

The failure that usually starts this work looks like a CI problem and is not one. A team has an eight-request collection that passes every time somebody clicks Run in the Postman app. It goes into the pipeline under Newman. The first run of the day fails at request 5 with a 404 on /orders/{{orderId}}, and the next four runs pass. Nobody can reproduce it locally.

The cause is two documented facts that are jointly invisible. Scripts write a variable's local value, never its shared one; and scheduled runs, monitors and the Postman CLI send the shared value. On a developer's machine orderId holds whatever the last manual run put there, so request 5 passes against a real order created hours earlier. In CI only the shared value ships, and request 5 asserts against whatever placeholder was committed. The suite was never testing what it appeared to test; moving it to CI only made that legible.

Read the export before you port a single request

Convert nothing by hand until you know what is in the file. Collection v2.1 is a JSON Schema document whose root has exactly six properties — info, item, event, variable, auth, protocolProfileBehavior — of which only info and item are required.

Two definitions matter for a migration. An Item requires a request and is a single call. An ItemGroup requires an item array and is a folder, carrying its own variable, event and auth. They sit in the same array and are told apart only by which key is present.

{
  "info": { "name": "Orders API", "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json" },
  "variable": [{ "key": "baseUrl", "value": "https://staging.example.com", "type": "string" }],
  "event": [
    { "listen": "prerequest", "script": { "type": "text/javascript", "exec": ["// runs before every request in the collection"] } }
  ],
  "item": [
    {
      "name": "Checkout",
      "item": [
        {
          "name": "Create order",
          "request": {
            "method": "POST",
            "header": [{ "key": "Content-Type", "value": "application/json" }],
            "body": { "mode": "raw", "raw": "{\"sku\":\"SKU-9\",\"qty\":2}" },
            "url": {
              "raw": "{{baseUrl}}/v1/orders?expand=lines",
              "protocol": "https",
              "host": ["{{baseUrl}}"],
              "path": ["v1", "orders"],
              "query": [{ "key": "expand", "value": "lines" }]
            }
          },
          "event": [
            { "listen": "test", "script": { "exec": [
              "pm.test('created', function () { pm.response.to.have.status(201); });",
              "pm.collectionVariables.set('orderId', pm.response.json().id);"
            ] } }
          ],
          "response": []
        }
      ]
    }
  ]
}

Three shapes in there bite a naive converter. host and path are each either a string or an array of segments, so host: ["{{baseUrl}}"] is a single-element array holding a variable reference rather than a domain. script.exec is either a string or an array of strings, one line each. And url.raw duplicates the exploded parts, so a tool reading only raw and a tool reading only path can disagree about the same request.

Take an inventory before you take a position. This walks the tree and reports how much of the collection is assertions and how much is plumbing:

// scripts/collection-inventory.mjs
// Usage: node scripts/collection-inventory.mjs orders.postman_collection.json
import { readFileSync } from 'node:fs';

const doc = JSON.parse(readFileSync(process.argv[2], 'utf8'));
const lines = (script) =>
  Array.isArray(script?.exec) ? script.exec : (script?.exec ?? '').split('\n');

const rows = [];
const walk = (items, trail) => {
  for (const entry of items ?? []) {
    const path = [...trail, entry.name];
    // An ItemGroup has `item`; an Item has `request`. Nothing else distinguishes them.
    if (Array.isArray(entry.item)) { walk(entry.item, path); continue; }

    const scripts = Object.fromEntries(
      (entry.event ?? []).map((e) => [e.listen, lines(e.script).join('\n')]),
    );
    const body = scripts.test ?? '';
    rows.push({
      name: path.join(' / '),
      method: entry.request?.method ?? 'GET',
      // Writes to shared state are the migration's real work.
      writes: [...body.matchAll(/pm\.(collectionVariables|environment|globals)\.set\(\s*['"]([^'"]+)/g)]
        .map((m) => `${m[1]}:${m[2]}`),
      assertions: (body.match(/pm\.test\(/g) ?? []).length,
      chains: /setNextRequest/.test(body) || /setNextRequest/.test(scripts.prerequest ?? ''),
      examples: (entry.response ?? []).length,
    });
  }
};
walk(doc.item, []);

for (const r of rows) {
  console.log(
    `${r.method.padEnd(6)} ${r.name}\n` +
    `        ${r.assertions} assertion(s)` +
    `${r.writes.length ? `, writes ${r.writes.join(', ')}` : ''}` +
    `${r.chains ? ', CHAINS' : ''}` +
    `${r.examples ? `, ${r.examples} saved example(s)` : ''}`,
  );
}

The output is the migration plan. Requests with zero pm.test calls are exploration and should not become tests. Requests that write to collectionVariables are setup steps that become fixtures. Requests flagged CHAINS are where run order is load-bearing, and they need reading in full first.

Run order is a dependency the format never declares

By default Postman runs every request in the order it appears, with requests inside folders running first and requests at the collection root after them. That order is a UI artefact: dragging requests around in the Collection Runner changes that run configuration, not the collection, so the sequence a developer validated by hand is not necessarily the sequence committed to source control. On top of it sits an explicit control-flow primitive that behaves unlike anything in a test runner:

// Post-response script on "Create order"
pm.test('created', function () {
  pm.response.to.have.status(201);
});

pm.collectionVariables.set('orderId', pm.response.json().id);

if (pm.response.json().requiresReview) {
  pm.execution.setNextRequest('Approve order');   // jumps
} else {
  pm.execution.setNextRequest(pm.info.requestId); // re-runs this request: an infinite loop
}
// This line still executes. setNextRequest takes effect only when the
// script finishes, no matter where in the script it was called.

Four documented properties of pm.execution.setNextRequest make it hard to reason about and harder to port:

• It takes effect at the completion of the current request, wherever it sits in the script. Code after the call still runs.
• If it is assigned more than once, the last value set wins.
• It has no effect when you send a single request — only in the Collection Runner, the Postman CLI or Newman. The behaviour you are testing does not exist in the mode you develop in.
• Its scope is the source of the run: run one folder and you cannot reach requests in other folders or at the root.

Passing null stops the workflow after the current request. There is no loop guard; the docs tell you to add your own exit condition or force-close the runner. Its sibling pm.execution.skipRequest() goes in a Pre-request tab, and when it is hit the request is not sent, remaining pre-request scripts are skipped, and no tests run for it — a test that silently does not exist.

The variable ladder is global state with five rungs

Postman resolves variables across five scopes, ordered broadest to narrowest: global, collection, environment, data, local. The narrowest scope wins, so pm.variables.get('score') returns the environment value when the same key also exists on the collection. Each rung has a different lifetime, and that decides how it migrates:

• pm.globals — workspace-wide, reachable from any collection. Nothing in a test suite corresponds to this; whatever lives here is configuration your CI should be setting.
• pm.collectionVariables — travels with the export, so it is the rung most likely to hold a stale id committed by accident. Creating one from a script needs Editor access.
• pm.environment — per-target configuration, and the only rung that maps cleanly onto baseURL and process environment variables.
• pm.iterationData — one iteration per row of the data file, columns exposed as variables. That is parameterisation, and it becomes a loop over a table of cases.
• pm.variables.set — a local variable that overrides every other scope and is gone when the run completes. Useful precisely because it does not persist.

Now the fact from the opening failure, stated plainly: a script writes only the local value of a variable, never the shared one, and promoting one to the other is a manual action in the UI. Scheduled runs, monitors and the Postman CLI send the shared value. That is why a collection can be green in the app for months and wrong the first time a machine runs it — the two execution modes read different values out of the same named variable.

Assertions written against one captured response

pm.test takes a name and a function; pm.expect exposes ChaiJS expect BDD syntax; pm.response exists only in post-response scripts and carries code, status, headers, responseTime and responseSize. If any assertion inside a pm.test callback fails, the whole test fails. The API is fine. What goes wrong is what gets asserted, because the assertion is usually written by eyeballing one captured response:

// Post-response script on "Get order" — the version that ships in most collections
pm.test('order looks right', function () {
  pm.response.to.have.status(200);
  const body = pm.response.json();
  pm.expect(body.id).to.eql(pm.collectionVariables.get('orderId'));
  pm.expect(body.total).to.eql(4198);          // whatever it was the day this was written
  pm.expect(body.status).to.eql('AWAITING_PAYMENT');
  pm.expect(pm.response.responseTime).to.be.below(800);
});

Every line after the status check asserts on data this request did not create. total is a constant copied out of a response. status is whatever state that particular order happened to be in. The responseTime bound turns network jitter into a failure. And the id comparison means nothing unless the request that set orderId ran, in this process, first. Here is the same coverage where the test owns its data, so every assertion checks a value the test itself chose:

// tests/orders.spec.ts
import { test, expect } from '@playwright/test';

test('a newly created order is awaiting payment and totals its lines', async ({ request }) => {
  const created = await request.post('/v1/orders', {
    data: { sku: 'SKU-9', qty: 2 },
  });
  await expect(created).toBeOK();          // asserts status is in 200..299

  const order = await created.json();
  const fetched = await request.get(`/v1/orders/${order.id}`);
  await expect(fetched).toBeOK();

  const body = await fetched.json();
  expect(body.id).toBe(order.id);
  expect(body.status).toBe('AWAITING_PAYMENT');
  // The total is derived from what this test asked for, not from a captured number.
  const expected = body.lines.reduce((sum, l) => sum + l.unitPrice * l.qty, 0);
  expect(body.total).toBe(expected);
});

The request fixture is built in and already respects baseURL and extraHTTPHeaders from your config, which is where the environment file's contents belong. toBeOK() is awaited because it may read the body to build the failure message.

Fixtures replace the requests that only existed to set up other requests

Every request in the inventory that writes to collectionVariables and asserts nothing is setup wearing a test's clothes. In Playwright, setup is a fixture and teardown is the code after await use(...). Dependent fixtures are set up before and torn down after the fixtures they depend on, so ordering becomes a declaration rather than a position in a list.

// tests/fixtures/api.ts
import { test as base, expect } from '@playwright/test';
import type { APIRequestContext } from '@playwright/test';

type Fixtures = { authed: APIRequestContext; order: { id: string } };

export const test = base.extend<Fixtures>({
  // Replaces the collection pre-request script that fetched a token.
  authed: async ({ playwright, baseURL }, use) => {
    const anon = await playwright.request.newContext({ baseURL });
    const login = await anon.post('/v1/tokens', {
      data: { user: process.env.API_USER, secret: process.env.API_SECRET },
    });
    await expect(login).toBeOK();
    const { token } = await login.json();
    await anon.dispose();

    // Isolated cookie storage: this context does not share cookies with a browser.
    const ctx = await playwright.request.newContext({
      baseURL,
      extraHTTPHeaders: { Authorization: `Bearer ${token}` },
    });
    await use(ctx);
    await ctx.dispose();
  },

  // Replaces "Create order" + the collectionVariables.set that followed it.
  order: async ({ authed }, use) => {
    const created = await authed.post('/v1/orders', { data: { sku: 'SKU-9', qty: 2 } });
    await expect(created).toBeOK();
    const body = await created.json();

    await use({ id: body.id });

    // Teardown runs even when the test failed, which the collection had no way to express.
    await authed.delete(`/v1/orders/${body.id}`);
  },
});

export { expect };

// tests/approval.spec.ts — the chained requests, unchained
import { test, expect } from './fixtures/api';

test('an order over the review threshold cannot be paid until approved', async ({ authed, order }) => {
  const paid = await authed.post(`/v1/orders/${order.id}/payments`, {
    data: { method: 'card' },
  });
  expect(paid.status()).toBe(409);

  const approved = await authed.post(`/v1/orders/${order.id}/approvals`);
  await expect(approved).toBeOK();

  const retried = await authed.post(`/v1/orders/${order.id}/payments`, {
    data: { method: 'card' },
  });
  await expect(retried).toBeOK();
});

Compare that against the setNextRequest branch it replaces. The ordering that is genuinely business logic — approval precedes payment — is still there, but as three statements on one screen rather than a jump target resolved at run time. The ordering that was never business logic has moved into a fixture that creates an order and deletes it afterwards. Test-scoped fixtures tear down after each test; a worker-scoped one tears down only when the worker process does, which is right for a token and wrong for a row you intend to mutate.

What Newman keeps, and what it will not give you

You do not have to migrate to get a collection into CI, and for a small smoke collection you should not bother. Newman exits 0 when everything runs without exceptions, so CI can gate on the exit code directly.

# Note the explicit `cli`: naming any other reporter turns the CLI reporter off.
newman run orders.postman_collection.json \
  --environment ci.postman_environment.json \
  --env-var "baseUrl=https://staging.example.com" \
  --env-var "apiKey=$API_KEY" \
  --folder Checkout \
  --bail failure \
  --reporters cli,junit \
  --reporter-junit-export results/newman-junit.xml

--bail stops on the first test script error and exits 1; its failure modifier makes the stop graceful, after the current test script finishes. The built-in reporters are cli, json, junit, progress and emojitrain. -x suppresses the exit code, which is the flag to grep for when somebody claims the collection has been passing.

What it cannot do is remove the properties the collection already had. It runs one ordered sequence; -n repeats it and --folder narrows it. There is no isolation boundary between requests, so anything that mutates a shared record still affects everything after it. And --env-var is a flat key-value list, so CI is feeding values into the same namespace whose precedence rules produced the original problem.

Keep the collection for the jobs the test suite cannot do

Deleting the collection after migrating is the wrong ending, because two of its capabilities have no equivalent in a test suite and both rest on the same asset: saved examples. An example is a request-and-response pairing — method, URL, parameters, headers and body on one side; status code, headers and body on the other. One request can hold several, which is how you show the 200, the 409 and the 422 for the same endpoint.

Postman generates documentation for every collection automatically, and it includes the examples: edit one and the docs update, delete it and it disappears from both. A mock server answers calls by matching the incoming request to the closest saved example in the associated collection — the algorithm ignores the request body and headers unless you turn that matching on, and the x-mock-response-name, x-mock-response-code and x-mock-response-id headers let a caller pin a specific example, so give every example a unique name.

The split is therefore clean. The collection is the artefact you hand a consumer: browsable, self-documenting, able to stand in for the service before it exists. The test suite is the artefact that gates a merge. Asking one to be the other is what produced the eight ordered requests in the first place.

Apply this now

Export the collection you most want to trust and run the inventory script over it. You are after three counts: how many requests contain no pm.test at all, how many write to collectionVariables or environment, and how many mention setNextRequest.

Then migrate the smallest useful slice. Take one request with real assertions, find its dependencies by following the variables it reads, and write those as a single fixture that creates what it needs and deletes it after use. Move the environment file's contents to baseURL and extraHTTPHeaders in the Playwright config, and read secrets from the process environment rather than any variable scope. Leave the collection in place with its examples intact. The evidence that this worked is a run in a random order against a clean database that passes, and the same tests passing individually — neither a property the original collection could have had.

Frequently asked questions

Can I convert the collection automatically?

Partially, and only the boring half. Method, headers, URL parts and body.mode are structured data in the v2.1 JSON and convert mechanically. The event entries cannot: script.exec is arbitrary JavaScript against the pm sandbox, and the semantics that matter — which variable scope a write lands in, when setNextRequest takes effect — have no counterpart to translate into. Generate the skeletons, write the assertions by hand.

Where does the auth token live once the collection pre-request script is gone?

In a fixture or a setup project, not a variable. Playwright's documented pattern is a setup project that authenticates through APIRequestContext and writes storageState to a file, declared as a dependency of the test projects. Storage state is interchangeable between BrowserContext and APIRequestContext, so an API login can seed a browser session. Write it under outputDir, which is cleaned before every run, unless it needs to survive.

Should API calls go through page.request or a separate context?

It depends on whether you want cookie sharing. page.request is a shortcut for page.context().request: it fills the Cookie header from the browser context and updates browser cookies from Set-Cookie. Use it when an API call should observe or affect the logged-in session. For pure API tests, apiRequest.newContext() gives isolated cookie storage and no accidental coupling to a browser.

What replaces setNextRequest for a genuine multi-step flow?

The steps go in one test, in order, as ordinary statements. A checkout that must create, approve and pay is one behaviour, not three tests, and splitting it into three ordered tests is how you get back to where you started. If it is long enough to want separate reporting, wrap each phase in test.step.

Should response-time assertions carry over?

No. A single-sample latency bound inside a functional test fails on unrelated infrastructure noise and passes on a genuinely slow endpoint that happened to be warm. Measure latency where you have a distribution.

Primary references

• Postman Collection Format v2.1.0 — JSON Schema: the six root properties, Item versus ItemGroup, event.listen values, and the script.exec and URL shapes
• Postman — Store and reuse values using variables: the five-scope precedence order and the narrowest-scope-wins rule
• Postman — Reference variables in scripts: per-scope accessors, scripts writing only local values, and the Editor-access requirement
• Postman — Customize request order in a collection run: default run order, setNextRequest timing and scope, and skipRequest
• Postman — Use scripts to add logic and tests to requests: collection, then folder, then request order for both script phases
• Postman — Writing tests and assertions in scripts: pm.test, pm.expect as ChaiJS BDD, and multiple assertions per test
• Newman — command-line collection runner: --bail modifiers, the reporter list, the CLI reporter being disabled once others are named, and -x
• Playwright — API testing: the built-in request fixture honouring baseURL and extraHTTPHeaders
• Playwright — APIRequest: newContext() options, and isolated cookie storage versus page.request
• Playwright — Fixtures: teardown after await use(), dependency ordering, and fixture scopes
• Playwright — Test: test.step for reported steps inside a single test
• Playwright — APIResponseAssertions: toBeOK() asserting a status in the 200–299 range
• Playwright — Authentication: the setup project writing storageState, interchangeable between browser and API contexts
• Postman — How a mock server matches requests to saved examples: closest-example matching, body and header matching off by default, and x-mock-response-*
• Postman — Create examples of request responses: what an example contains and how it feeds generated documentation