Skip to main content
automationtester.in
Engineering publication

The Automation Brief

Practical quality-engineering guides for people building reliable tests, delivery systems, developer tools, and AI-assisted workflows.

Featured Playlists

All series (5)
8 parts live

API Contracts in Practice

Eight practical guides to OpenAPI, JSON Schema, consumer contracts, negative cases, API security, and AI-assisted drift analysis.

Start Reading Series
8 parts live

JVM Quality Architecture

Eight guides to JUnit architecture, Gradle test suites, Testcontainers, parallel isolation, virtual-thread behavior, and AI-generated test review.

Start Reading Series
8 parts live

Playwright Reliability Lab

Eight evidence-first guides for building Playwright suites that survive UI change, parallel CI, network variability, and AI-assisted authoring.

Start Reading Series

Latest articles

51 published
SRShashank Rawlani
Automation Tutorials

Understand Appium 3 Drivers, Plugins, and Boundaries

A global Appium install is not an environment: drivers, plugins, the device-side servers they ship and the platform toolchain all version separately. This guide covers what Appium 3 changed for setups (Node 20.19, mandatory insecure-feature prefixes, session discovery), why a project package.json with a lockfile beats APPIUM_HOME alone, why installed plugins stay inactive until named, a loopback-bound .appiumrc.json with scoped allow and deny lists, per-platform capability files, and a CI gate that parses appium driver list --installed --json and runs driver doctor.

15 min readSign-in article
Dark technical illustration read left to right. Near the left, five orange pills float at uneven heights, each linked by a faint dashed line to a dashed orange outline of itself sitting at a different height. An orange curve runs from each pill into one row of a tall green panel in the middle, where every row has a ringed green dot pierced by a short vertical green bar, followed by a green pill. Five green curves leave the panel and fan into a dashed green boundary on the right that holds a stack of five green layered panels, each a little narrower than the one above, each with a pill and a small dot. A dashed orange path climbs out of the top of the boundary and ends at a crossed-out orange ring; a green curve leaves the middle panel of the stack, crosses the boundary and ends in an arrowhead at a large ringed green dot. A grey rail enters from the left edge and stops at a small grey ringed dot. Grey rules and faint pills run along the top and bottom edges.
SRShashank Rawlani
Automation Tutorials

Build a Mobile Test Strategy by Fidelity and Cost

A mobile suite that runs every check on a cloud phone pays the most for the least evidence. This guide splits one sign-in journey across host tests, Compose and SwiftUI screen tests with fakes, Espresso and XCUITest on managed emulators and simulators, and a few release-build journeys on physical devices. It covers what Robolectric and simulators cannot observe as documented, why out-of-process drivers like UI Automator and Appium matter for minified builds, a placement table with triggers and flake notes, and how to give every nightly device run an owner and a gate.

17 min read
Dark technical illustration read left to right. A grey rail enters from the left edge into a tall green phone-shaped panel holding a stack of seven green pills. To its right, four long green rounded lanes are stacked from top to bottom and run off the right edge, each lower lane starting further left than the one above. Green curved connectors leave the pills and end in arrowheads at the lanes: three reach the bottom lane, which is filled with rows of small green dots; two reach the lane above it, which holds a row of small screen panels containing pills; one reaches a lane of dashed phone outlines; and one reaches the short top lane at a ringed green dot beside three solid green phones. A dashed orange arc leaves the lowest pill, climbs above every lane and levels out above the top lane and ends at a crossed-out orange ring, short of the phones. Faint pills and grey rules sit at the top and bottom edges over a dashed vertical grid.
SRShashank Rawlani
AI in Testing

Validate AI Agent Tool Calls in TypeScript: Schema, Authorization, Approval

Strict tool use and MCP SDK validation check the shape of a model's arguments; neither knows who is asking. This guide builds a draft-only MCP tool with the TypeScript SDK v2 and Zod 4: strict schemas with no identity fields, the user taken from the verified request, per-call ownership checks on model-supplied ids, publication turned into a human-approved request, and results and errors bounded so nothing secret reaches the model. It covers why tool annotations are untrusted hints, why z.object silently drops a forged authorId, how a thrown exception's message becomes model-visible text, and a Vitest suite that asserts every denial by state as well as response.

21 min readSign-in article
A dim grey panel on the left holds four horizontal pill rows. The top three are green and curve right, converging into one thick green line that ends in an arrow at a large ringed green node in the centre; the bottom row is orange and stops at a crossed-out orange ring just outside the panel. A second green line rises into the central node from below, out of a small green panel holding a ringed dot and a pill. From the node a green arrow runs right into a green panel of stacked pills wrapped in a dashed green boundary. A dashed orange curve climbs from the node toward a grey panel in the top right and ends at a crossed-out orange ring before reaching it, while a green line from the right edge ends at a ringed green node beside that panel with an arrow pointing into it. Along the bottom, beneath the small panel, a green lane leaves the right-hand panel, runs left through a small dashed box holding three short green pills, where a long grey bar fades out past the box's edge, and turns up into the bottom of the left panel with an arrow.
SRShashank Rawlani
Automation Tutorials

Make TypeScript Type-Level Tests Earn Their Place

A type-level test earns its place when a type-only change could break a caller you cannot see. Using a typed API client whose endpoint map infers params from the path, this guide covers expectTypeOf and *.test-d.ts files under vitest --typecheck, why a plain vitest run skips them, an @ts-expect-error that stays green after the contract breaks, how a leaked any passes assertType and toExtend but not toEqualTypeOf, the type test that mirrors its implementation, and when not to write type tests at all.

16 min readSign-in article
Dark technical illustration over a faint dashed vertical grid. Five pills enter from the left edge: three green ones run level and then curve together into a tall dashed green gate holding a ringed green dot, while two orange ones between them stop at crossed-out orange rings well short of the gate. Above everything, a sixth orange pill starts a dashed orange arc that sweeps over the top of the gate and ends at another crossed-out ring. From the gate three green arrows fan out to three rounded green panels stacked on the right, each with a small dot and a different arrangement of green bars, and faint lines continue from each panel to arrowheads at the right edge. Below a thin rule, a separate grey lane runs across the full width with a small grey panel holding two grey bars.
SRShashank Rawlani
Automation Tutorials

Test TypeScript Error Handling Without Losing Causes

Rethrowing new Error(e.message) turns a refused connection into two words and a stack that starts in your own catch block. This guide covers what the cause option actually installs, the subclass field that silently resets cause to undefined, an invoice client that classifies validation, unauthorized, timeout and unknown failures with fixed user copy and retry flags, a single redacting log boundary, and Vitest tests that assert each classification, including why toThrow with an error instance cannot see a lost cause.

22 min readSign-in article
Dark technical illustration read left to right over a faint dashed vertical grid. On the left, three small orange rings linked in a row trail behind a larger ringed orange node, and an orange line carries them into a dashed green rounded box. Inside the box the same three rings reappear in green, linked in a row and enclosed by one large green circle. From the bottom of the box a dashed orange curve carrying a single thin orange pill drops to a crossed-out orange ring. From the right side of the box four green curves fan out to four stacked green panels, each holding a ringed dot and a pill; the lower two panels also hold a small circular green arrow at their right end. All four panels send curves that converge on one ringed green node on the right. Beneath that node a short orange pill runs into a small crossed-out orange ring, while a single green line leaves the node and ends in an arrowhead at the right edge.
SRShashank Rawlani
Automation Tutorials

Test React Server and Client Boundaries in Next.js

An admin page that redirects editors can still let them import data, because its Server Action is a separate POST endpoint. Using a Next.js 16 page with a client CSV import form, this guide covers testing the access decision and the action as plain functions, why server-only throws under Vitest and how to alias it, checking props for React's serialization rules, testing the form in Vitest browser mode with the action passed as a prop, and a Playwright contract that replays the admin's action request with an editor's session.

22 min readSign-in article
Dark technical illustration over a faint dashed vertical grid. Two grey rails enter from the left edge and end at ringed dots, a green one above and an orange one below. The green dot's curve reaches a ringed green gate and continues with an arrowhead into a large green panel that holds a heading pill, four short rows, and on its right a dashed inner container with a small file panel, three short pills and a wide solid pill. The orange dot also curves up to that gate, where a dashed orange line drops to a crossed-out orange ring. From the bottom of the dashed inner container a green curve falls to a second ringed green gate lower down, and the orange dot's other path runs straight along the bottom of the frame, under the panel, to the same gate. From that gate a green line with an arrowhead enters a tall green panel of stacked pills on the right, and a grey rail carries on to the right edge, while a dashed orange line drops from the gate to a second crossed-out orange ring.
SRShashank Rawlani
Automation Tutorials

Test TypeScript Discriminated Unions as State Machines

A discriminated union rules out impossible shapes, but not illegal moves between legal ones or unknown statuses from the server. Using a payment flow, this guide covers deriving the union from a Zod discriminated-union schema, the exact issue Zod reports for an unknown discriminant, why a default branch absorbs a new variant and what assertNever and switch-exhaustiveness-check do instead, a transition table typed with satisfies, generated tests for all sixteen pairs, and why rejected events should return the same object.

18 min readSign-in article
Dark technical illustration over a faint dashed vertical grid. On the left, four green pills enter from the edge and their curves converge on a tall dashed green gate holding a ringed green dot; below them an orange pill runs into a crossed-out orange ring at the foot of the gate. From the gate an arrow leads into a dashed green frame around a four-by-four grid of rounded cells: four cells glow green with a dot inside, while the other twelve are dim with a short grey pill. From the grid a line reaches a small ringed dot, then an arrow to a larger one that forks up to one ringed dot and down to another. A solid green curve loops from the lower dot back to the fork, and a dashed orange curve arcs from the upper dot back toward the fork, crossed out midway by an orange ring. A green line leaves the upper dot and ends in an arrowhead at the right edge.
SRShashank Rawlani
Automation Tutorials

Debug Vitest Module Mocking and Cache Leaks

When a Vitest test passes with -t and fails in the full file, a module cached by an earlier test usually holds the state. This guide reproduces the failure with a fixed shuffle seed, shows what per-file isolation covers in each pool (including process.env surviving between files in the vm pools), fixes an import-time environment read with vi.stubEnv, a scoped vi.resetModules and onTestFinished, and separates late mocks, module singletons and leaked fake timers by their symptoms. It also shows why clearMocks, mockReset, restoreMocks and unstubEnvs leave the module cache alone.

16 min readSign-in article
Dark technical illustration of two dashed green rounded containers side by side across the middle of the frame, both hanging by short grey stems from a heavy grey rail that runs the full width near the bottom, with faint grey pills resting along it; fainter grey dashed containers bleed off the left and right edges. In the left container an orange pill feeds a curved orange arrow into a grey-outlined panel filled with four orange pills, and three small green dots with green pills sit below it. A green line runs from the bottom of that panel to a large ringed green dot on the container's right edge. From the panel's right side a dashed orange arc rises over the top of the gap between the containers and ends at a large crossed-out orange ring in the gap, with only a faint dashed trace continuing toward the right container. In the right container a green pill feeds a green arrow into a green-outlined panel filled with four green pills, with green dotted rows below; a green line leaves the panel's bottom, reaches a ringed green dot on the container's right edge and continues as a solid green line to an arrowhead at the right edge of the frame. Thin grey rules and faint pills sit at the very top and bottom over a faint dashed vertical grid.
SRShashank Rawlani
Automation Tutorials

Choose Vitest, Browser Mode, or Playwright by Risk

Each test layer can observe different things: Node sees pure logic and Blobs, jsdom sees DOM structure but no layout, Vitest browser mode sees real geometry, input and object URLs, and only Playwright sees what the browser does outside the page. This guide splits one file-download component across all four, gives a capability-to-layer decision table, shows the vacuous jsdom layout test and its fix, and prices each layer in feedback time, CI browser installs and diagnostic artefacts on Vitest 5 and Playwright 1.63.

17 min readSign-in article
Dark technical illustration over a faint dashed grid. On the left, a green rounded pane with three grey dots across its header, three green text pills and a green button holds the start of a single thick green horizontal line that runs to the right. The line passes through four nested dashed green rectangles, each larger than the one inside it, and inside each band between two rectangles it meets one ringed green dot. Beside the first dot are short stacked pills; beside the second, a small three-node tree; beside the third, pills of clearly unequal widths over a dotted measuring line; the fourth and largest dot, outside the third rectangle, drops a curve into a green page icon with a folded corner holding three pills. From the first dot a dashed orange arc climbs over all the rectangles to the top right and ends at a crossed-out orange ring. Grey rules and faint pills fill the top, bottom and outer edges.
SRShashank Rawlani
Automation Tutorials

Type Safety Stops at Runtime: Test the TypeScript Boundary

A type assertion compiles away, and JSON.parse and Response.json() are typed any, so a payload missing a required role or carrying an unlisted status reaches your domain code unchecked. This guide covers why hand-written type predicates repeat the problem, parsing unknown through a Zod 4 schema into a separate domain type, when to strip or reject undeclared keys, why a missing enum field and an unknown enum value report the same issue code, and a Vitest suite that asserts path, code and a sanitised received value without leaking personal data into failures or test names.

20 min read
A dim grey panel on the left holds six horizontal rows: three green pills, one empty dashed orange outline, one solid orange pill and one grey pill. Each green row runs right through a ringed green checkpoint on a tall dashed green vertical gate in the centre and lands as a matching green pill inside a green panel on the right. The dashed outline and the orange pill each end at a crossed-out orange ring on the gate. The grey row peels off downward, passes beneath the bottom of the gate along a faint dotted line and fades into two small grey pills. Above everything, a dashed orange arc leaps from the top of the left panel over the gate and points down into the top of the right panel. A green line leaves the right panel toward the right edge, and faint rules and pills fill the margins.
SRShashank Rawlani
AI in Testing

Review AI-Generated JUnit Tests Before They Raise Coverage

Two JUnit suites for the same pricing rule produce identical JaCoCo rows, and only one of them fails when the threshold or the rounding mode changes. This guide covers mapping each generated test to a clause of the ticket, BigDecimal oracle traps (the double constructor, scale-sensitive equals), why stubbing the class that owns the rule shows up as NO_COVERAGE and inflates PIT's test strength, which mutants PIT's current DEFAULTS and STRONGER groups actually generate, and how to seed by hand the RoundingMode mutant no PIT operator produces.

17 min readSign-in article
Dark technical illustration read from left to right across two side-by-side rounded lanes separated by a dashed green vertical line with a ringed dot at each end. Above each lane sits the same fully filled green gauge bar, identical in both. Inside each lane a thick green line runs level, then steps down to a lower level partway across. At each step a dashed orange segment continues along the upper level a little further before dropping, marking the same step shifted to the right. Beneath both lanes a grey axis with small ticks crosses the full width, and dashed green stems rise from ringed dots on the axis to ringed dots sitting on the green step. In the left lane the two stems are far apart, one near the start of the upper level and one near the end of the lower level, and a dashed orange line runs beneath the second and ends in an orange arrowhead at the lane's right edge. In the right lane the two stems sit tightly either side of the step; the second lands beneath the orange shifted segment, and above it a crossed-out orange ring marks that segment struck out, while a green line continues from the lane to a green arrowhead on the right.
SRShashank Rawlani
Automation Tutorials

Make JUnit Failures Useful in CI

A red CI job that prints AssertionFailedError and a line number forces the next engineer to rebuild the scenario by hand. Using a parameterized contract test for GET /orders/{id}, this guide covers why Gradle's LIFECYCLE console omits assertion messages, keeping timestamps and generated ids out of test names, path/expected/actual messages with Supplier and assertAll, TestReporter entries and sanitized file attachments that Gradle 9.4 now carries into HTML and XML, redacting before anything reaches stdout, and why mergeReruns can make a flaky failure read as an ordinary pass.

17 min readSign-in article
Dark technical illustration read left to right. On the left, five tall grey panels stand side by side, the two leftmost fading off the edge, each holding a column of short horizontal pills. A thick straight green line runs through all five panels at the same height, with a green dot where it crosses each one. In the three brighter panels a single orange pill sits at a different height in each, and dashed orange segments zig-zag between them before curving down to a crossed-out orange ring near the bottom. The green line ends in an arrowhead at a large ringed green dot in the centre. Above the dot, a green panel holds three stacked pills of shrinking length; below it, a dotted tether drops to a larger green panel of paired pills in which two of the right-hand pills are replaced by solid grey blocks. To the right, the green line enters a dashed green rounded boundary containing an orange ringed dot and a green ringed dot side by side, each with a pill beneath, then leaves the boundary and runs to an arrowhead at the right edge. Below the boundary, four small green dots each lead a fading green pill, and thin grey rules and faint pills sit at the top and bottom of the frame.
SRShashank Rawlani
Automation Tutorials

Test Java Virtual-Thread Code for Pinning and Cancellation

A test that submits a thousand virtual-thread tasks and waits for them proves only that they ended. This guide measures the ten connections behind them instead: a probe that asserts the concurrency bound is reached and never exceeded, a 500 ms deadline that has to reach the wait for a connection, the Semaphore release bug that inflates permits on cancellation, JDK 27 structured concurrency timeouts, and a JFR RecordingStream check for the pinning that survives JEP 491.

17 min readSign-in article
Dark technical illustration read left to right. A dense field of small green dots fills the left third, brighter toward the middle. Thin green curves run from the field into a tall dashed green gate holding exactly ten stacked rounded slots, each with a short bright green bar inside. A few grey curves leave the bottom of the field, pass beneath the gate and stop at small grey dots against a vertical dashed green line topped by a ringed marker. Below the ten slots an eleventh slot is outlined in dashed orange, fed by a dashed orange curve and followed by a crossed-out orange ring. On the right, lines from all ten slots converge on one large ringed green dot, which sends a thick green line and arrowhead off the right edge, with three green pills beneath it.
SRShashank Rawlani
Automation Tutorials

Build Trustworthy Database Tests with Testcontainers

A disposable database guarantees availability and a known image; it says nothing about whether your schema matches production or which test owns which rows. This guide covers separating container lifetime from data isolation, the singleton pattern the JUnit 5 extension deliberately does not provide, running your real migration set rather than letting an ORM invent a schema, choosing between a schema, a rollback and an owned key range, what the jdbc:tc URL mode is actually for, and a concurrent-insert test that asserts SQLSTATE 23505 rather than a message.

17 min readSign-in article
Dark technical illustration inside one large dashed green rounded boundary that extends past the left and right edges of the frame, over a faint dashed vertical grid. In its upper left, seven green horizontal bars of increasing length are stacked in order, each preceded by a small dot; a curved line runs from the stack to an arrowhead at a large ringed green dot. From that dot another curve rises to the right to a smaller ringed dot with two green pills beside it. On the far left a heavy grey rail enters from off the edge and ends at a ringed green dot with a short vertical green bar clamped across it. A heavy grey horizontal line crosses the whole width below, and a green line drops from the large ringed dot to meet it. Three vertical stems hang from that line, each ending at a ringed dot above its own smaller dashed green container. The outer two containers each hold three rows of a ringed green dot with a green pill beside it. The middle container instead holds a green ringed dot at its top left and an orange one at its bottom left, with a green curve and an orange curve converging on a single large green ringed circle at its centre. From that circle one green curve leaves and sweeps upward across the top-left corner of the third container, levelling out and ending in an arrowhead at the right edge; a dashed orange curve leaves the same circle, sweeps downward across that container's bottom-left corner and ends at a large crossed-out orange ring near the bottom right. Thin grey rules and faint pills sit at the very top and bottom of the frame.
SRShashank Rawlani
Automation Tutorials

Model JVM Test Layers with Gradle Test Suites

When integration tests live in src/test/java behind a filename filter, nothing stops a slow dependency entering the fast feedback path — and under JUnit Platform the filename is not even a discovery mechanism. This guide covers what a JvmTestSuite creates for you, why only the built-in test suite inherits the production code's dependencies and what implementation project() does and does not bring with it, wiring each suite into check with an explicit ordering, keeping suite identity in the JUnit XML and the aggregation variant, and the three costs of the split.

13 min readSign-in article
Dark technical illustration read from left to right. On the left, nine faint grey lines and short grey pills enter from off the left edge and converge on one tall green-outlined rounded panel that also runs off the edge. Inside it, nine horizontal pills of varying length are stacked; six are green and three are a brighter orange, interleaved with them. Four curved grey lines leave that panel, pass through a dashed green vertical line that has a ringed dot at each end, and continue as green curves ending in arrowheads at four separate dashed green containers stacked down the right two-thirds of the frame. Each container has three short stubs entering its left edge from small pills, and holds two or three rows made of a ringed dot followed by a long pill and a shorter one beneath it. Three of the containers are green throughout; the second one from the top is orange throughout, including its entry pills. From every container a straight green line leaves the right edge, crosses a faint vertical grey rail that runs off the top and bottom of the frame, and ends at a large ringed green dot with a small grey pill beneath it. Thin grey rules and faint pills sit at the very top and bottom.
SRShashank Rawlani
Automation Tutorials

Debug JUnit Parallel-Execution Flakes

A test that passes alone and fails in the suite, with the failure moving between classes, is an ownership bug that parallel execution revealed rather than caused. This guide covers pinning the configuration so the flake reproduces, the documented reason your fork-join pool can run more tests at once than the parallelism you configured, a probe that shows which threads two failing tests were on, when the answer is @TempDir rather than a lock, what @ResourceLock's READ and READ_WRITE modes actually schedule, and the check that tells Jupiter's threads apart from Gradle's forked JVMs.

14 min readSign-in article
Dark technical illustration of seven horizontal lanes running left to right across the frame. The top two lanes are thick and bright green and enter from off the left edge; the lower five are thinner and much fainter and begin only in the left third, each preceded by a short dashed lead-in and marked by a ringed dot where it starts. A dashed green vertical line with a ringed dot at each end passes through all seven at exactly that point. Short green pills of varying length sit along every lane like segments of work. Right of centre, an orange-outlined rounded panel holding three orange pills interrupts the picture: two of the lanes, one bright and one faint, stop short at a ringed orange dot and curve out of their tracks toward it, converging on its left side and ending in a single arrowhead at the panel's edge; neither of those two lanes continues past that point. A thick dashed orange line leaves the panel on the right and stops at a large crossed-out orange ring. The five uninvolved lanes continue past the panel and end in green arrowheads at the right edge. A row of short tick marks runs across the top of the frame, and thin grey rules with faint pills sit at the very top and bottom.
SRShashank Rawlani
Automation Tutorials

Use JUnit Parameterized Tests Without Matrix Explosion

A parameterized test that runs 288 times reports its failures as a coordinate rather than a claim, and by default Jupiter throws away any column your method forgot to declare — silently, with the test still passing. This guide covers choosing rows by the rule each one defends rather than by crossing dimensions, switching argument count validation to strict, why a CSV value is quoted in the display name before it is converted, keeping the expected value out of production code, and the three signals that a table should have been two tests.

15 min readSign-in article
Dark technical illustration split by a dashed green vertical line with a small ringed dot at each end. The left two-thirds are a dense lattice of small rounded grey squares, thirteen columns by fifteen rows, sitting on a grid of thin rules that run off the top, bottom and left edges, with a grey dot centred in every square. One whole column of that lattice is orange instead: brighter squares and dots threaded by a thick dashed orange vertical line, capped at top and bottom by a crossed-out orange ring. On the right of the dashed boundary, a thick green line rises left to right in four square steps and continues off the right edge; the area beneath it is filled with a faint green wash and thin vertical hatching down to a grey horizontal baseline that also runs off the right edge. A large ringed green dot sits on each of the four step corners with two green pills of decreasing length beside it, and a faint dashed green line drops from each corner to the baseline. Below the baseline, four short green pills sit in a row. Two faint green pills sit at the top of the right band, and thin grey rules cross the top and bottom of the whole frame.
SRShashank Rawlani
Automation Tutorials

Structure JUnit 5 Tests Around Behavior, Not Classes

Under JUnit Platform a test class name is not a discovery filter — Gradle states outright that scanForTestClasses has no effect there — so the one-test-class-per-production-class convention has no mechanical support behind it. This guide covers what the engine actually builds, grouping by the precondition a set of assertions shares rather than by package, turning the container tree into readable report prose with IndicativeSentences, the way inherited @Test methods silently multiply across subclasses, and the point at which a group stops being a container and becomes its own suite.

12 min read
Dark technical illustration in three vertical bands. On the left, six grey rounded panels stacked in a column run off the left edge, each holding two short grey pills. A dashed grey link joins every one of them to a matching green-outlined panel in a second column of six, each of those holding three green pills. Three of the second column's panels also contain an orange rounded square, at different heights and different horizontal offsets, and a dashed orange curve loops between the three squares from top to bottom. A dashed green vertical line with a ringed dot at each end stands to the right of that column, and faint curved grey lines fan from each panel through it. On the right sits a large dashed green rounded container with a green pill and a small ringed dot at its top; inside it are two smaller dashed green containers, each also headed by a pill and a dot. The upper inner container holds three grey pills on short stubs. The lower one holds three green rounded squares, each with a green pill beside it, all gathered together. A curved green line leaves the large container on the right, ends in an arrowhead near the right edge, and two faint green pills sit beneath it. Thin grey rules run edge to edge at the top and bottom.
SRShashank Rawlani
AI in Testing

Review AI-Generated pytest Tests with Mutation Checks

A generated test file can execute every branch of a pricing rule and still pass when the threshold moves by one, because coverage records that a line ran and nothing records that an assertion depended on it. This guide covers scoping a mutmut run to a single function, telling a surviving mutant apart from a mutant that any calling test kills, the two review patterns — a mock placed one layer too low and an assertion that restates its own fixture — that make every mutant survive, and the pragmas and settings that quietly remove code from the run.

13 min readSign-in article
Dark technical illustration read from left to right. Ten green horizontal bars enter from off the left edge, stacked evenly inside a large faint rounded panel crossed by a regular dashed vertical grid. A dashed green vertical line runs top to bottom through the middle of the stack with a ringed dot at each end, and a small rounded square sits on it at the height of every bar; eight of those squares are green and two are orange. From each green square a thin line runs rightward and ends in an arrowhead beside a thick green vertical segment, then a ringed green dot. Those thick segments stack into a broken vertical rail, and at the two heights where the rail has no segment there are instead two short orange dashes marking the gap. At each of those two heights an orange line runs unbroken from the square, straight through the gap and off the right edge of the frame, ending in an arrowhead. Beside each orange line, to the right of a faint grey vertical divider, sits a pair of orange pills of decreasing length; the other rows in that region carry much fainter grey pills. Thin grey rules run edge to edge across the top and bottom of the frame.
SRShashank Rawlani
Automation Tutorials

Test Python Retry Logic Without Sleeping

In tenacity the sleeper is a constructor argument, so a retry test never needs to patch time.sleep or wait real seconds — pass a callable that records the delay and the list it collects is the backoff schedule. This guide covers asserting the schedule instead of the call count, bounding jittered waits using the documented formula rather than pinning the random source, enforcing a deadline against a clock the test owns, and why the exception your caller catches is not the one your code raised.

14 min readSign-in article
Dark technical illustration built on a horizontal time axis that runs off both edges of the frame, over a faint dashed vertical grid. Six vertical stems rise from the axis, each topped by a rounded square box and marked by a dot where it meets the axis; the first five stems and boxes are green and the sixth, on the right, is orange. Below the axis, five horizontal dimension bars with vertical caps at both ends measure the gaps between consecutive stems: the first three roughly double in length, and the last two are equal, each carrying two short vertical tally marks at its centre. A short dashed bracket steps up past the right cap of every bar. Beneath those, five solid bars all start at the left edge and end at successively later points, forming a staircase; four are green and the longest is orange. A dashed orange vertical line with a dot at each end crosses the entire composition, passing between the fifth and sixth stems and through the orange staircase bar near its right end. Above the sixth stem's box sits a crossed-out orange ring. Along the bottom, a row of five rounded bars of increasing width rests on a horizontal rule, four green and the last orange.
SRShashank Rawlani
Automation Tutorials

Build Deterministic Python API Test Data Factories

A duplicate-key violation blamed on parallelism, an assertion broken by a Faker patch bump, and a 41-line payload diff with two relevant fields all come from the same cause: generating data without deciding what is fixed, what varies, and what the test asserts. This guide covers the reseed-to-zero behaviour of the faker fixture, why generated values must never appear as expected outputs, deriving reproducible unique identity from the test node id with uuid5, and the jsonschema format checker that does nothing unless you switch it on.

13 min readSign-in article
Dark technical illustration. Along the top, an orange line enters from the left edge of the frame and passes through four orange bars of steadily decreasing length, ending at a solid orange dot; a short orange connector then drops from it toward the top of a large green card below. The card has a solid outer outline and a dashed inner one, and holds eight horizontal rows. The first row is an orange pattern of four slots, three filled and one nearly empty. Of the remaining seven rows, two are bright green bars spanning the card's full inner width with a small green dot beside each, and five are shorter mid-grey bars about half that width. To the right stand three narrower cards built the same way, each holding a green four-slot pattern at the top above five short grey bars; the filled slots differ in every one of the three patterns and differ again from the orange pattern in the large card. Across the bottom of the frame, running off both edges, is a row of eleven rounded boxes on a horizontal rule: ten contain a green tick, and the seventh from the left is orange and contains a cross instead.
SRShashank Rawlani
Automation Tutorials

Property-Based Testing in Python: Start with Invariants

A property test asserting isinstance(encode(s), str) can run sixty thousand inputs and establish nothing, because no input in the domain could falsify it. This guide covers the three families of claim that can actually fail, why the default text alphabet is a decision rather than a default, constraining the domain in the strategy instead of with assume(), the correctness health check that catches function-scoped fixtures not resetting per input, and how to pin every shrunk counterexample so it never has to be rediscovered.

16 min readSign-in article
Dark technical illustration. A regular grid of small dots covers the entire frame; each dot outside a central region is dim grey and circled faintly, while the dots inside that region are larger and bright green. A thick green closed outline with a slightly irregular, blob-like shape encloses the green dots, marking them off from the grey field around them. To the right of that outline, out among the grey dots, sits a large orange dot inside a filled orange circle, itself surrounded by a wider dashed orange ring. A dashed orange line runs from the right edge of the green outline toward that orange circle but stops well short of it, ending in a small orange dot in open space. Along the lower part of the green region, four orange circles of steadily decreasing size are joined left to right by short solid orange lines, ending at a small solid orange dot held between two short vertical orange brackets.
SRShashank Rawlani
Automation Tutorials

Run pytest in Parallel Without Shared-State Flakes

Adding workers does not create shared state — it reveals that the suite only ever worked because it ran one test at a time. This guide starts from the fact that every xdist worker collects independently and runs its own copy of each session-scoped fixture, then covers deriving a reclaimable namespace from worker_id, one PostgreSQL schema per worker, why collection order has to agree across workers, and why -s cannot work under xdist at all.

15 min readSign-in article
Dark technical illustration of four tall green columns standing side by side across the frame, each outlined twice and holding its own arrangement of short green blocks — two rows near the top and two near the bottom, with a different number of blocks per row in each column so no two columns match. A short green line runs down the centre of each column from its upper blocks and ends in a green dot. Cutting horizontally through all four columns is a single wide orange band that runs off both edges of the frame, drawn as a filled bar inside a fainter orange outline. Where each column's centre line meets the band it stops; below the band each column resumes in orange rather than green, ending in an orange dot. At all four points where the columns cross the band there is a crossed-out orange ring.
SRShashank Rawlani
Automation Tutorials

Debug pytest-asyncio Event-Loop and Fixture Failures

pytest-asyncio gives one event loop per pytest collector, so every async resource is bound to the loop of whatever created it. That single fact explains attached to a different loop, Event loop is closed, ScopeMismatch on a fixture you never wrote, and the AttributeError on an async_generator that mentions no loop at all. This guide separates the five causes, gives a check that tells them apart, and starts with the two configuration defaults that do not match.

15 min readSign-in article
Dark technical illustration of nested rings. One very wide green ellipse spans the frame and runs off both edges, ringed with evenly spaced outward tick marks and shadowed by two faint dashed inner ellipses. Inside it sit three green circles of equal size, each also ringed with ticks and each shadowed by a dashed inner circle. Inside each of those sit two smaller tick-ringed circles, six in total, and every one carries a small green dot on its edge with a short green stub pointing inward, staying within its own circle. Five of the six small circles are green; the left-hand one in the middle group is drawn in orange instead, ticks and fill included. From the top of the wide outer ellipse a solid orange line runs down and slightly left into that orange circle, ending in an orange dot at its centre. Partway along, where the line crosses out of the middle green circle's boundary, it is struck through by a crossed-out orange ring.
SRShashank Rawlani
Automation Tutorials

Use pytest Parametrization Without Hiding Test Intent

The generated ID is the only part of a parametrized case that reaches the report, the selection flag and the failure history — so an auto-generated one costs you both diagnosis and reproduction. This guide covers the exact rule pytest uses to build IDs, the default that silently renumbers collisions into positions, the effects that fire at collection time because a parameter table is just an expression in a decorator, and how to choose equivalence classes instead of Cartesian products.

12 min readSign-in article
Dark technical illustration in two halves. The upper half is a wide grid of sixty identical rounded square outlines, four rows deep, each drawn in the same green at the same weight with a single green dot at its centre; one square near the middle right is orange instead, but is otherwise indistinguishable in size and shape from its neighbours. Below the grid a horizontal rule separates the halves, and five curved lines descend from spread-out positions in the grid, four dashed green and one solid orange. They arrive at a row of five much larger rounded rectangles, each containing a different arrangement of short paired green tick marks in four possible slots, so no two of the five look alike. The third rectangle and its ticks are orange. From its lower edge a single solid orange line drops to a solid orange dot resting on a wide, faint orange bar near the bottom of the frame.
SRShashank Rawlani
Automation Tutorials

Design pytest Fixtures as Explicit Resource Contracts

A test that passes alone and errors in the full suite is almost never a flaky test — it is a fixture whose lifetime is wider than the state it holds, requested by nothing the test names. This guide works from what pytest actually reads when it orders fixtures, why one autouse fixture pulls its whole dependency graph into every test in scope, and why teardown after yield covers only the fixtures that already succeeded.

15 min read
Dark technical illustration of nested lifetimes drawn as horizontal bars of decreasing length, stacked from top to bottom against a faint dashed vertical time grid. The top two bars are green and each is closed by a short vertical bracket at both its left and right end. The third bar down is orange and has a bracket only on its left end; past where its right end should close, it breaks into five progressively shorter and fainter orange fragments that fade out. The fourth bar is green and closed at both ends. Below them, eight short green bars sit in a row, each bracketed at both ends and arranged in two groups of four with a gap between the groups. Beneath the whole row runs a single long orange bar with a solid dot at each end, and a dashed orange riser climbs from it into the underside of every one of the eight short bars, crossing the gap to reach the right-hand group as well. Thin solid vertical connectors join each stacked bar to the one below it near its left edge, green between the closed bars and orange where they meet the unclosed one.
SRShashank Rawlani
AI in Testing

Review an AI-Generated OpenAPI Description Before You Publish It

Structural validity and truthfulness about a service are unrelated properties: a generated description parses, resolves and lints while inventing a DELETE endpoint and quietly making the refund route public. This guide starts with the security override that removes authentication in two characters of YAML, reconciles operations against a route inventory taken from the booted router, and shows why the specification's SHOULD on examples means nothing is obliged to reject an invented field.

11 min readSign-in article
Dark technical illustration reading left to right across six rows. A green panel on the left holds five stacked pills, four green and one orange, with the bottom row left empty. A matching panel on the right also holds five pills, four green and one orange, but its empty row is the fifth rather than the sixth. Three rows connect the two panels with green arrows passing through small dashed rectangular gates, each marked with a green dot. A fourth row crosses through a gate drawn only as two dashed orange horizontal lines with no sides, and its connecting arrows are orange rather than green. From the orange pill on the left a dashed orange curve falls away and ends at a crossed-out circle in the lower middle of the frame. From the orange pill on the right a second dashed orange curve reaches back toward the left panel and stops at a small orange dot in open space, connected to nothing.
SRShashank Rawlani
Automation Tutorials

Verify Pagination, Filtering, and Sorting Contracts

Asserting page lengths and spot-checking page one is true of an endpoint that loses records. This guide builds a reference set whose total order you control, walks it by following the Link header instead of assembling URLs, and asserts membership, uniqueness and sequence across the union — then inserts and deletes rows mid-traversal, which is exactly where offset pagination skips and duplicates.

15 min readSign-in article
Dark technical illustration reading left to right. A green panel on the left holds seven stacked pills of slightly varying length. A green arrow leads to the first of three dashed vertical windows, each containing three pills, joined left to right by green arrows. Above the second window an orange pill sits alone, and a solid orange line curves down from it into the top of that window, whose middle pill is orange rather than green. A dashed orange line falls from the bottom of the same window and ends at a crossed-out circle near the bottom of the frame. A green arrow leaves the third window for a second panel on the right holding the same stack of pills, except that the fourth position is an empty dashed orange outline instead of a filled pill.
SRShashank Rawlani
Automation Tutorials

Test API Authorization Beyond Happy-Path Tokens

A valid token answers who is calling and nothing else. This guide separates the object, property and function questions the way OWASP's 2023 edition does, reads the security declarations most OpenAPI descriptions get wrong, and builds a deny-by-default grid whose denials are proved by comparison against a fabricated ID and whose rejected writes are read back by an identity that can actually see the field.

14 min readSign-in article
Dark technical illustration reading left to right. A small green panel on the left holds one bright pill above two grey ones, and a line leaves it for a row of four dashed vertical gates, each containing a ringed node — the first node larger than the other three. Green arrows carry the path from gate to gate. From the second gate an orange line curves down to a crossed-out circle near the bottom of the frame, from the third an orange line curves up to a crossed-out circle near the top, and from the fourth another curves down to a third crossed-out circle. Past the last gate a green arrow reaches a panel holding a four-by-five lattice of small rounded cells: three are lit bright green and the remaining seventeen are dim grey.
SRShashank Rawlani
Automation Tutorials

Test API Idempotency Under Retries and Timeouts

A sequential double-submit test passes on services that double-charge in production, because the dangerous retry follows an outcome the client never observed. This guide constructs that ambiguity deliberately with an aborted in-flight request, separates the four replays that must produce four different answers, and shows why the durable ledger count is the assertion that fails when a cached response body would have passed.

14 min readSign-in article
Dark technical illustration. On the left a green panel holds two request groups, each beginning with a short pill of identical length. A line from the upper group runs right into a large dashed rectangle containing a big ringed node; from that rectangle an orange line curves back to the upper left, ends in an arrowhead and a crossed-out circle, and fades away as a dashed orange trail toward the left edge. A line from the lower group meets a small green ringed node and continues into the same dashed rectangle. A dashed orange curve also leaves that small node, sweeps down beneath the rectangle and terminates at a second crossed-out circle. A single green line leaves the rectangle on the right and reaches a green panel holding four equal slots: the top one filled green with a small ring beside it, the three below it grey and empty.
SRShashank Rawlani
Automation Tutorials

Design Negative API Tests from the Contract

A negative test that asserts only a 4xx status is satisfied by a crashed pod, a rewritten gateway response, and a rejection that already decremented stock. This guide derives one invalid request per contract rule, places each mutation on the boundary the rule actually defines, and gives every case a three-part oracle: the exact status code, the machine-readable location of the violation, and a probe proving no forbidden side effect survived the rejection.

16 min readSign-in article
Dark technical illustration reading left to right. A green request panel of stacked pills sits on the left, one of its pills half green and half orange. A line runs from that pill to a tick-marked ruler crossed by a dashed vertical line: an orange crossed-out circle sits far to the left of the dashed line, while an orange ring and a green ring sit close together on either side of it. The ruler feeds a dashed rectangular gate with a large ringed node. Two branches leave the gate: an orange branch reaching an orange panel where one pill is drawn in green with a small ring resting on it, and a green branch reaching a green panel holding three pills of identical length each ending in a dot. A dashed orange curve drops from the gate and terminates at a crossed-out circle below the second panel.
SRShashank Rawlani
Automation Tutorials

Detect Breaking API Changes Before Consumers Do

A line diff cannot tell whether an OpenAPI edit invalidates an old request, widens a response beyond a client's decoder, or removes behaviour a consumer depends on. This guide builds a deterministic compatibility gate around a released baseline, a semantic report, consumer ownership, and two focused executable contracts, then shows what evidence an intentional break needs before approval.

11 min readSign-in article
Dark technical illustration. Two stacked document panels on the left feed into a tall dashed gate with a large ringed node at its centre; the lower document has one line highlighted in orange. Three paths leave the gate to the right: an orange path rising to a crossed-out circle that continues only as a faint dashed line, a straight green path entering three small dashed consumer cells side by side, and a green path descending to a ringed node that carries on to the right edge as a solid rail. A separate dashed orange line drops straight down out of the bottom of the gate into its own bordered box holding an orange node.
SRShashank Rawlani
Automation Tutorials

Turn a Postman Collection into Maintainable API Tests

A Postman collection is an exploration artefact: its run order is an undeclared dependency, its variables are five rungs of shared mutable state, and its assertions are usually copied from one captured response. This walks the Collection v2.1 JSON to inventory what you actually have, shows the Postman script beside the migrated Playwright test, and argues for keeping the collection for documentation and mock servers rather than deleting it.

15 min readSign-in article
Dark technical illustration split in two. On the left, six orange request cards are stacked in a single column and welded head to tail by short connectors, forming one rigid ordered chain; two long dashed orange arcs loop out past the left edge of the frame from an early card down to a much later one, showing a hidden dependency on state an earlier request wrote. On the right, nine green cells sit in a grid, each sealed inside its own dashed boundary and carrying a bright setup badge at the top and a teardown badge at the bottom, with no connectors of any kind running between them. A small green arrow between the two halves marks the reorganisation.
SRShashank Rawlani
Automation Tutorials

API Contract Testing with Playwright and JSON Schema

Playwright's request fixture gives you a browser-free API suite, but a schema check only catches a renamed or vanished field if it sets additionalProperties: false and keeps the name in required. This walks through the open-schema trap, the Ajv settings that decide whether failures are readable, generating schemas from an OpenAPI document, why allOf silently reopens a closed schema, and the honest boundary between schema validation and consumer-driven contract testing with Pact.

14 min read
Dark technical illustration of a JSON payload being checked field by field. Eight horizontal lanes run edge to edge, each carrying a rounded key pill and a longer value pill from the left. In the centre stands a tall green schema wall with rectangular slots cut into it, one per declared property; most lanes drop into their slot and continue to the right as green result pills. One lane is drawn hollow and dashed with a crossed-out orange circle at its slot, marking a required property that never arrived. In the middle of the wall there is no slot at all, only a gap with torn orange edges, through which a bright orange lane passes uninspected and carries on to the right edge.
SRShashank Rawlani
AI in Testing

Reviewing AI-Generated Playwright Tests: Eight Tells in the Diff

A model writing Playwright reaches for the same constructs every time, so reviewing generated specs is a short checklist rather than an audit — and every item is visible in the diff without running anything. The eight tells: waitForTimeout standing in for a wait condition, an await inside expect() that kills retrying, .first() silencing a strict-mode violation, assertions that cannot fail, an over-broad page.route(), module-level state that breaks under fullyParallel, hardcoded URLs that belong in baseURL and storageState, and toHaveScreenshot() baselines generated on someone's laptop. It ends with the greps, lint rules and repeat-each run that make most of the pass mechanical.

14 min readSign-in article
A dark schematic of a review gate: on the left, eight identical machine-generated spec cards queue on a vertical feeder spine, each with the same abstract lines of code. Every lane runs right into a tall dashed gate column where three rounded check cells and a ringed inspection node sit on each lane. Four lanes emerge green, run across the frame and merge into a large ringed node that continues off the right edge; the other four are stopped at crossed-out orange circles just past the gate and their dashed orange paths curve down into a smaller orange rework node at the lower right.
SRShashank Rawlani
Tools & Reviews

Build a k6 Load Model from Real Traffic, Not a VU Count

In k6's closed-model executors the next iteration waits on the previous one, so a degrading system quietly receives less load and the run still reports green. This walks through converting peak-hour analytics into an arrival rate, driving it with constant-arrival-rate and ramping-arrival-rate, sizing preAllocatedVUs from median iteration duration times rate, and reading dropped_iterations as a capacity finding rather than a misconfiguration. It closes on thresholds — per-flow tag sub-metrics, p(95) and p(99) instead of avg, abortOnFail with delayAbortEval, and the duplicate-key trap that silently discards a gate.

14 min readSign-in article
A dark technical illustration read as one left-to-right timeline. Along the top, a bright green curve ramps out of the left edge and holds a plateau to the right, with vertical ticks hanging from it that grow denser as it rises. Below, a wide panel of twelve horizontal lanes fills with green segments whose idle gaps shrink until, past a dashed orange vertical line, every lane is a solid unbroken bar. To the right of that line the ticks turn orange, fall as dashed lines straight past the full lanes, and accumulate as a rising stack of small orange squares along the bottom, continuing a flat green baseline that runs back to the left edge.
SRShashank Rawlani
Tools & Reviews

Test Cron Schedules by Their Fire Times, Not Their Strings

Reviewing a five-field string cannot tell you when a job runs: the two day fields are combined with OR in cron but AND in systemd, the spring gap deletes a run and the autumn fold duplicates one, and Kubernetes then decides separately whether a matched run may start. This shows you how to compute fire times from a pinned base time using the parser your scheduler actually vendors — robfig/cron for CronJobs, systemd-analyze calendar --base-time for timers — and how startingDeadlineSeconds, concurrencyPolicy and the 100-missed-schedule ceiling interact. It ends with the evidence to keep: the cronjob-scheduled-timestamp annotation diffed against actual start times, so a run that never happened shows up as a gap.

14 min readSign-in article
Dark technical illustration of two schedule timelines running edge to edge. The upper lane shows wall-clock fire times as a row of green ticks hanging from a rule; at one highlighted transition column a tick is missing, drawn as a dashed outline with a crossed-out circle above it, and at a second transition column a single slot carries two overlapping orange ticks ringed together. Curved droppers connect every wall-clock tick to the lower lane, where the same schedule in UTC is a perfectly even row of green ticks with no gap and no duplicate, the fold's two instants landing as separate ringed nodes. A strip of small cells along the bottom repeats the pattern, one cell empty and one doubled.
Newsletter

The Automation Brief

Test suites, agents, workflows and pipelines — and how you prove any of it actually works. Practical notes from the tools being built here.

No spam, no sending cadence promised. Unsubscribe any time.