Why 80% coverage mandates produce worse tests

The metric that eats itself

Here's a pattern I've watched play out at more than a few teams. Leadership sets a coverage mandate — 80% line coverage before merge, enforced in CI. Engineers, who are not slow, immediately figure out the fastest path to green. Within a quarter the repo is full of tests that look like this:

def test_process_payment():
    process_payment(order_id="abc123", amount=99.00)
    # TODO: add assertions

Or worse — snapshot tests generated wholesale against UI components nobody has looked at in two years, auto-updated every time something changes, committed back to the repo, and checked in as "snapshot updated." The coverage number climbs. Trust in the test suite erodes. Flaky tests start accumulating because the snapshots are sensitive to anything — a timestamp, a font-render difference, a locale setting — and engineers start hitting "re-run" instead of investigating.

You set a coverage policy to improve quality. You got a compliance checkbox instead.

What coverage actually measures

Line coverage (and its cousins: branch coverage, statement coverage) measures one thing: which lines of production code were executed by at least one test. It says nothing about whether the test checked anything meaningful when those lines ran.

This is not a niche edge case. A test that calls your payment processor, swallows the result, and asserts nothing will happily push coverage from 72% to 81%. The CI gate turns green. The code ships. You now have production code with zero behavioral verification, wearing the costume of a tested codebase.

The number isn't lying — those lines really were executed. The metric just doesn't have an opinion about whether executing them means anything.

How the incentive structure breaks

When a number is the gate, engineers optimize for the number. That's not a character flaw — it's rational behavior. A few patterns that show up consistently:

Assertion-free "smoke" tests. Calling functions without asserting return values or side effects. Cheap to write, excellent coverage yield, worthless for catching regressions.

Over-reliance on snapshot tests. Snapshots aren't inherently bad. Snapshots auto-generated against five hundred React components to pad a coverage report are a maintenance disaster. They fail for the wrong reasons (whitespace, ordering, rendered timestamps) and get silently updated because the diff is too large to review. That's a flakiness factory — and once your flaky-test rate climbs, your CI signal degrades for everyone.

Testing implementation instead of behavior. To hit a branch-coverage number, engineers write tests that exercise private methods or internal state transitions that shouldn't be public API at all. When the implementation changes — even correctly — the tests break. Now you have brittle tests that produce false negatives and teach engineers that "tests lie."

Dead-code coverage inflation. Legacy code that's actually unreachable in production gets a thin integration test bolted on just to bump the percentage. The test is probably fine in isolation, but it's protecting code the product doesn't use.

Each of these degrades the CI signal in a slightly different way, but they all lead to the same place: a team that doesn't trust its own test suite and compensates by re-running failures until they pass.

What to set policy on instead

If raw line coverage is the wrong lever, what's the right one? A few things that actually correlate with test quality:

Mutation score

Mutation testing tools (Stryker for JS/TS, mutmut for Python, PIT for Java) automatically introduce small bugs into your production code — flip a > to >=, delete a return statement, change a constant — and then run your test suite. If your tests don't catch the mutation, it "survives." Mutation score is the percentage of mutations your tests killed.

You cannot game mutation score with assertion-free tests. A test that calls a function without checking its output will let every mutation survive. This is the property you actually want from a coverage policy: it's hard to fake.

Mutation testing is slower than line coverage. Run it on a weekly schedule or gate it on changed files only, not on every PR. The signal is worth the compute.

Flaky test rate

A test suite that passes 100% of the time, every run, on every branch, is giving you reliable signal. A suite where 8% of runs have at least one non-deterministic failure is not — regardless of its coverage percentage. Flakiness is a direct measure of how much you can trust your CI gate.

Track flaky test rate as a first-class metric. If it's climbing, your test quality is declining, and no coverage number will tell you that. BuildPulse's flaky test detection surfaces this by analyzing test result patterns across CI runs, so you're not relying on engineers to manually report intermittent failures they've learned to ignore.

Critical-path coverage

Instead of a global line-coverage floor, identify the code paths where a bug has the highest business consequence — payment flows, authentication, data-export logic, anything that touches PII or financial records — and enforce coverage specifically there. A 95% mandate on your checkout service is meaningful. An 80% mandate on your entire monorepo including generated protobuf clients is noise.

This is harder to configure but not very hard. In most coverage toolchains you can exclude paths and set per-directory thresholds:

# jest.config.js (example)
coverageThreshold:
  global:
    lines: 60        # floor, not a target
  "./src/payments/":
    lines: 90
    branches: 85
  "./src/auth/":
    lines: 90
    branches: 85

A low global floor plus high thresholds on critical paths stops coverage from becoming a vanity metric while still protecting the parts of the system where a regression costs real money.

Assertion density (informal, but useful)

Some teams do lightweight PR review heuristics: flag tests above a certain line count that have fewer than N assertions, or require that any test touching a public API boundary assert on the return value or a side effect. This doesn't scale to a CI gate easily, but as a code-review norm it surfaces the assertion-free test problem before it metastasizes.

The compliance angle

If you're in a SOC 2 or ISO 27001 shop, your auditors will ask about test coverage. The temptation is to point at the CI gate and say "we require 80% before merge" because it's simple and legible.

The problem is that auditors are asking the underlying question: do you have controls that catch regressions in sensitive code paths before they reach production? A coverage gate on a suite full of assertion-free tests does not actually answer that question. It answers a proxy question, badly.

A more defensible posture: document your critical-path coverage thresholds, your flaky test rate trend, and your mutation score on sensitive modules. That's a richer evidence package that demonstrates you're actually managing test quality, not just optimizing a number. It also happens to be harder for an auditor to poke holes in, because it's honest about what you're measuring.

CI reliability as a change-management control deserves a separate post, but the short version is: a flaky CI gate provides weaker change-management guarantees than a slow, reliable one. The coverage percentage printed on the gate is not the variable that determines how much you can trust it.

What a better coverage policy looks like in practice

Pull the pieces together and a reasonable policy looks something like this:

• Global line coverage floor: 60–65%. Enough to block PRs that delete tests without replacing them. Low enough that engineers aren't incentivized to pad it.
• Critical-path coverage thresholds: 85–90% lines and branches on modules you've explicitly designated as high-consequence.
• Flaky test rate SLO: tracked per-week, alarmed when it exceeds a threshold you've agreed on (1% of test runs containing a non-deterministic failure is a reasonable starting point).
• Mutation score: run on a schedule on your critical-path modules. Use it as a quality signal, not a hard gate — at least until your team trusts the tooling.
• PR review norm: any test file that adds more than 20 lines without a proportional number of assertions gets a comment. Enforce culturally before you enforce mechanically.

None of this is complicated. It's just a slightly different set of questions than "did we hit 80%?"

The underlying bet

Every coverage policy is a bet on what behavior you want to incentivize. A raw line-coverage gate incentivizes execution. What you actually want is verification — tests that will fail when something breaks, pass when it doesn't, and do so deterministically.

Optimizing for execution and hoping verification follows is how you end up with a green CI badge on a codebase you don't trust. The badge is the goal, not the test suite behind it.

Set policy on the things that are actually hard to fake.