< to <=, or a threshold literal by one, and rerun only the tests that touch it. Any generated test that still passes is asserting something other than the rule it claims to protect, and the review note writes itself: the mutant, the test that should have caught it, and the boundary value it never used.Here is a pull request that a reviewer approves in forty seconds. A pricing rule, and two tests for it:
\n\n# pricing.py\nfrom decimal import Decimal\n\nBULK_THRESHOLD = 10\n\ndef discount_for(quantity: int, unit_price: Decimal) -> Decimal:\n if quantity < BULK_THRESHOLD:\n return Decimal(\"0\")\n return (unit_price * quantity * Decimal(\"0.15\")).quantize(Decimal(\"0.01\"))# test_pricing.py — generated\ndef test_discount_applied_for_bulk_order():\n assert discount_for(50, Decimal(\"2.00\")) == Decimal(\"15.00\")\n\n\ndef test_no_discount_for_small_order():\n assert discount_for(5, Decimal(\"2.00\")) == Decimal(\"0\")Both branches execute. Statement and branch coverage of discount_for are both 100%. And the pair cannot detect the single most likely defect in the function, which is that the threshold is off by one.
Move BULK_THRESHOLD to 11. Both tests still pass: 5 is below either threshold and 50 is above either. Change < to <=. Both tests still pass, for the same reason. The rule these tests exist to defend, \"bulk pricing starts at ten units,\" is the one thing they never look at.
mutate_only_covered_lines enabled it also skips anything your coverage configuration excludes, and # pragma: no mutate plus do_not_mutate_patterns can remove more. Read those settings as part of the diff before you read the score. Checked against mutmut 3.7.0 and coverage.py 7.16.0 on 2026-09-07.Mutation testing makes that argument mechanical. A mutation tool edits your source, runs the tests, and records whether anything failed. A mutant that gets caught is killed; one that does not is a survivor. Every survivor is a change to production behaviour that your suite would have merged.
\n\nmutmut 3.7 keeps its mutation catalogue deliberately small and deliberately subtle. The three you will meet on almost every review are stated outright in its documentation:
\n\n< is changed to <=.break is changed to continue and vice versa.Those are exactly the edits a tired author makes by accident, which is the point: \"in general the idea is that the mutations should be as subtle as possible.\" Run it against the function above and you get two survivors and one kill:
\n\n$ mutmut run \"pricing.discount_for*\"\n$ mutmut browsepricing.discount_for\n x_discount_for__mutmut_1 BULK_THRESHOLD -> 11 survived\n x_discount_for__mutmut_2 < -> <= survived\n x_discount_for__mutmut_3 \"0.15\" -> \"XX0.15XX\" killedThe kill is the one worth ignoring. mutmut mutates string literals by wrapping them. The documentation's own example is logging.info(\"Foo\") becoming logging.info(\"XXFooXX\"), so Decimal(\"XX0.15XX\") raises InvalidOperation before any assertion runs. Any test that calls the function at all kills that mutant. It tells you the function was invoked, which is what coverage already told you.
Sort survivors and trivial kills apart before you write a single review comment. A suite can have a high mutation score built almost entirely from mutants that crash on contact, and the two survivors above are worth more than fifty of those.
\n\nMutation runs are usually abandoned because someone started with mutmut run across a whole service and watched it for an hour. Reviewing a pull request does not need that. mutmut takes Unix filename patterns, and the unit can be a single function:
mutmut run \"pricing*\" # one module\nmutmut run \"pricing.discount_for*\" # one functionScope it to the functions the pull request touched. That turns a mutation run from an overnight job into something that finishes while you read the diff, and it keeps the result attributable: a survivor in a function nobody changed is not this author's problem.
\n\nTwo configuration choices decide whether the run is usable at all. Both live in pyproject.toml:
[tool.mutmut]\nsource_paths = [ \"src/\" ]\npytest_add_cli_args_test_selection = [ \"tests/unit\" ]\n# Only mutate lines coverage.py saw execute. Default is false, which means\n# mutmut falls back to \"any function that gets called\".\nmutate_only_covered_lines = true\n# Stop counting a test as relevant to a function once the call is this deep.\nmax_stack_depth = 8max_stack_depth is the one people miss, and mutmut is explicit about the trade it makes: a lower value \"will increase mutation speed and lead to more localized tests, but will also lead to more surviving mutants that would otherwise have been caught.\" Without it, a utility called incidentally from three hundred integration tests drags all three hundred into every mutant's verdict, and a survivor stops telling you which test was supposed to catch it.
The other number to know before you start is the timeout. mutmut kills a slow mutant after (duration_of_original_tests + timeout_constant) * timeout_multiplier seconds, defaulting to a constant of 1.0 and a multiplier of 15.0. That formula is multiplicative in your suite's own runtime, so a selection containing one test that sleeps for two real seconds does not cost two seconds per mutant. It moves the ceiling for every mutant in the run. Point pytest_add_cli_args_test_selection at the fast unit tests and leave the sleeping ones out.
When every mutant in a function survives, not one, all of them, the function almost certainly never ran. The usual cause in generated tests is a mock placed one layer too low:
\n\n# Wrong. checkout.discount_for is the behaviour this test claims to cover,\n# and this line replaces it. Every mutant inside discount_for survives.\ndef test_checkout_applies_bulk_discount(monkeypatch):\n monkeypatch.setattr(\"checkout.discount_for\", lambda q, p: Decimal(\"15.00\"))\n\n assert checkout_total(50, Decimal(\"2.00\")) == Decimal(\"85.00\")# Right. Mock the boundary the test does not own — the tax service — and let\n# the pricing rule execute. Mutants inside discount_for now have somewhere\n# to show up.\ndef test_checkout_applies_bulk_discount():\n tax = create_autospec(TaxService, instance=True)\n tax.rate_for.return_value = Decimal(\"0\")\n\n assert checkout_total(50, Decimal(\"2.00\"), tax) == Decimal(\"85.00\")Use create_autospec or autospec=True rather than a bare Mock, because the standard library is blunt about what a bare one costs: \"if you refactor some of your code, rename members and so on, any tests for code that is still using the old api but uses mocks instead of the real objects will still pass. This means your tests can all pass even though your code is broken.\" Autospec fixes that by limiting the mock to the real object's API and giving mocked methods \"the same call signature as the original so they raise a TypeError if they are called incorrectly.\"
The related failure is a patch that silently does nothing. unittest.mock's rule is that \"you patch where an object is looked up, which is not necessarily the same place as where it is defined\". If checkout.py says from pricing import discount_for, then patching pricing.discount_for has no effect, because checkout already holds a reference to the real function. The test passes, the mock is never consulted, and nothing in the output says so. A mutation run does say so: the mutants get killed by a test the author believed was stubbed out.
The second pattern that produces uniform survival is an assertion whose expected value came from the setup two lines above it:
\n\ndef test_order_totals(order_factory):\n order = order_factory(subtotal=Decimal(\"100.00\"), currency=\"USD\")\n\n assert order.subtotal == Decimal(\"100.00\") # the factory said so\n assert order.currency == \"USD\" # the factory said so\n assert order.total > 0 # true for every non-empty orderNothing here depends on the pricing rule, so no mutation of it can fail this test. The discriminating question at review time is not \"is this assertion true\" but \"what value would make it false\". If the answer is \"only a broken factory\", the test belongs to the factory, not to the order.
\n\nRewrite it with a value the test computed independently of the code path it is checking:
\n\n@pytest.mark.parametrize(\n (\"quantity\", \"expected\"),\n [\n pytest.param(9, Decimal(\"0\"), id=\"one-below-threshold\"),\n pytest.param(10, Decimal(\"3.00\"), id=\"exactly-at-threshold\"),\n pytest.param(11, Decimal(\"3.30\"), id=\"one-above-threshold\"),\n ],\n)\ndef test_bulk_discount_starts_at_ten_units(quantity, expected):\n assert discount_for(quantity, Decimal(\"2.00\")) == expectedThose three rows kill both survivors from earlier. The 9/10 pair kills the threshold-literal mutant, and the same pair kills the <-to-<= mutant, because moving the comparison moves the boundary between them. The 11 row is not redundant padding: it is what distinguishes \"the boundary is right\" from \"the boundary is right and the arithmetic still scales\".
Note that the expected figures were derived from the written rule: fifteen per cent of quantity times unit price, rounded to the cent, and not from running the function and pasting what it returned. An oracle copied from the implementation cannot disagree with the implementation, which is the failure mutation testing is least able to help with.
\n\nA clean mutation report is only as strong as the set of mutants that were generated, and mutmut skips more than most people expect.
\n\nModule-level code is not mutated at all in mutmut 3. The documentation points you at mutmut 2, \"which has a different execution model\", if you need it. Constants defined at import time, dataclass field defaults and configuration dictionaries therefore produce no mutants, and a threshold that lives at module scope rather than inside the function is invisible to the run.
\n\nWith mutate_only_covered_lines = true, mutmut \"also honours the lines coverage.py is told to leave out of its measurement, so code marked with # pragma: no cover, or matched by your exclude_lines/exclude_also settings, is not mutated either.\" An exclusion added to make a coverage gate green silently removes that code from mutation as well.
And mutation can be turned off in the source. # pragma: no mutate suppresses a line, # pragma: no mutate block suppresses a whole indentation block including an entire function when placed inline on the def, and do_not_mutate_patterns suppresses by regex across the codebase. All three are legitimate: the documented use is logging calls and break-versus-continue optimisations. All three are also a way to make a report look clean. Read them as part of the diff.
# Worth a look before trusting any mutation number in a pull request.\ngit diff --unified=0 origin/main -- '*.py' | grep -E 'pragma: no (mutate|cover)'\ngit diff origin/main -- pyproject.toml setup.cfg | grep -E 'do_not_mutate|only_mutate|max_stack_depth'None of this retires coverage. Branch coverage answers a question mutation testing does not: which paths were never taken at all. Coverage.py tracks, for each line that can jump to more than one next line, which destinations were actually visited, and reports the missing transition by number: in its own if/else example, a flag that is always true reports the missing branch as 2->5.
That is the right first pass on a generated test file, and it is cheap. Run it with --branch, look for partial branches in the code the pull request touched, and treat each one as a case the author did not think of. Then mutate the branches that were taken, because that is the set coverage has already declared satisfied and cannot say anything more about. The two tools disagree about different things, which is why running one is not a reason to skip the other.
Take the next pull request containing generated tests and pick the one function in it that encodes a business rule with a number in it: a threshold, a limit, a retention window, a rate band. Run mutmut run \"<module>.<function>*\" against it with the fast unit selection only.
Then split the report in two. Trivial kills, where the mutated source raises before an assertion runs, go in one pile and are not evidence. Survivors go in the other, and each one becomes a review comment of exactly three lines: the mutant mutmut printed, the test whose name claims to cover that rule, and the input value that would have separated them.
\n\nWhat you should have at the end is a diff that adds boundary rows rather than test functions, and a short list of rules whose thresholds are now pinned by a test that fails when they move. If the mutation run produced no survivors and no interesting kills either, check source_paths and the pragmas before believing it.
Not as a global gate, at least not first. A repository-wide percentage is dominated by trivial kills and by code nobody disputes, so it moves for reasons unrelated to test quality and teams learn to ignore it. mutmut does support publishing one. mutmut export-cicd-stats feeds mutmut badge, which emits Shields endpoint JSON, and that is genuinely useful as a trend on a module you are actively hardening. Enforce it on a package you have already cleaned, not on the whole tree on day one.
They find opposite things. Property-based generation searches the input space for a case your assertions reject; mutation testing holds the inputs fixed and searches the source for a change your assertions ignore. A property test with a weak property will happily explore ten thousand inputs and kill nothing, and a mutation run will tell you so. Where they meet is useful: a surviving mutant often names the input dimension your generator was never given a strategy for.
\n\nThe test selection, not the mutation settings. mutmut runs your chosen tests once per mutant, so anything slow is multiplied by the mutant count, and the timeout ceiling scales with your suite's own runtime as well. Point pytest_add_cli_args_test_selection at unit tests only, add max_stack_depth, and scope the run with a pattern. Reach for parallelism after those three, because they change the size of the problem rather than the speed of the machine.
Yes, and the survivors tell you which. A test that kills no mutant that another test does not already kill, and whose assertions all restate its own setup, is costing you runtime and a maintenance obligation in exchange for nothing. Deleting it is not a reduction in coverage of behaviour, because it was covering none. The ones to keep and repair are those aimed at a real rule with the wrong assertion attached.
\n\nThe technique does, at the cost of doing it by hand. Change the threshold literal by one in your working copy, run the tests, and change it back. If nothing fails, the review comment is the same one you would have written from a report. That is worth doing on the spot for a single suspicious rule, and it is how you convince a team that the tool is worth adding: a live demonstration that the suite they trust does not notice the change is more persuasive than a percentage.
\n\n< changed to <=, break and continue swapped, with the design goal that \"the mutations should be as subtle as possible\". Also the filename-pattern scoping (mutmut run \"my_module.my_function*\"), the source_paths and pytest_add_cli_args_test_selection configuration, mutate_only_covered_lines and its interaction with # pragma: no cover, max_stack_depth and its stated trade-off, the no mutate pragmas and do_not_mutate_patterns, the timeout formula (duration_of_original_tests + timeout_constant) * timeout_multiplier with defaults 1.0 and 15.0, the string-literal mutation example logging.info(\"XXFooXX\"), that mutmut 3 does not mutate code outside functions, and export-cicd-stats with mutmut badge.TypeError if they are called incorrectly\". Also that a spec turns an attribute typo into an AttributeError rather than a silently passing assertion.from a import SomeClass case where patching the defining module has no effect at all.2->5 in the if/else example. Version 7.16.0.@pytest.mark.parametrize signature and pytest.param(..., id=...), which is what lets each boundary row report under a name a reviewer can read in the failure output.