You inherit a codebase with almost no tests, and the obvious plan is to raise coverage before anyone changes more code. For most teams, that is the wrong default. A broad test campaign can consume months while the next release still breaks the same customer path. Start instead with a few executable checks at the boundaries where a change can do costly, observable harm.
A coverage target is the wrong first contract
Line coverage answers which code ran during tests, not whether a customer received the right result. That distinction matters most in inherited software: a test can execute a payment handler without noticing that a retried event creates two receipts. I would not set an initial coverage percentage for the whole repository, because it rewards tests of easy code while the most consequential behavior may remain unverified.
Inherited Untested Code Is It Actually Getting Faster asks a reasonable question, but speed is the wrong first constraint when a release can silently create bad data. A team can shorten its test run and still spend days finding which deployment caused a duplicate transaction. The first useful question is narrower: which customer-visible mistake could the next plausible change introduce without anyone noticing?
Look for evidence rather than nominating the scariest-looking module. Search incident tickets for repeated symptoms, inspect recent reverts with Git, and ask support which outcomes required manual repair. Trace one such outcome backward through the HTTP endpoint, queue consumer, database write, and any external callback. OpenTelemetry traces can help connect those steps if the service emits them; Sentry errors may identify failures that never reached a ticket. Neither tool can establish correctness on its own, because a successful response can still contain the wrong value.
For an illustrative baseline, suppose you observe 18 production changes over 4 weeks, with 3 reverts linked to the same order flow. Those are example counts to collect locally, not a benchmark to claim as your result. Record the change, the failure mode, and how someone detected it. A ratio such as change failure rate becomes useful only after the team agrees what counts as a failed change; otherwise, a relabeled incident can make the trend look better without making releases safer.
Choose a pilot scope of 3 customer paths as a value to adjust with your team. That limit forces a decision about exposure and consequence. It also gives developers a reviewable reason for each proposed test: the test protects a named outcome, rather than helping a dashboard reach a round number.
Tests at change boundaries catch more than tests around old functions
In a codebase without tests, internal functions often encode years of accidental behavior. Freezing every output in a snapshot makes refactoring harder because the test cannot distinguish a required result from an implementation detail. A boundary test instead specifies what a caller may rely on: an HTTP response, a persisted record, or one message emitted after a retry.
Take a payment-event consumer that may receive the same event twice. The valuable assertion is not that its private parsing helper returns a particular object; it is that two deliveries produce one receipt. The following pytest 8.x example runs locally and expresses that invariant with SQLite. It is a small executable model, not evidence that your inherited consumer already satisfies it:
import sqlite3
def test_replayed_event_creates_one_receipt():
db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE receipts (event_id TEXT PRIMARY KEY, cents INT)")
def handle(event_id, cents):
db.execute("INSERT OR IGNORE INTO receipts VALUES (?, ?)", (event_id, cents))
handle("evt_42", 1900)
handle("evt_42", 1900)
result = db.execute("SELECT COUNT(*), SUM(cents) FROM receipts").fetchone()
assert result == (1, 1900)
db.close()
Run it with python -m pytest -q. Then replace the local handle function with an adapter that invokes the real consumer in an isolated test environment. Until that replacement happens, the snippet tests SQLite’s uniqueness behavior, not production behavior. If production uses PostgreSQL, exercise its actual transaction boundary too: an application-level “check, then insert” can behave differently under concurrent deliveries even when a single-threaded model passes.
Use contracts where another system depends on the boundary. OpenAPI 3.1 can describe an HTTP request and response, while JSON Schema 2020-12 can constrain a message payload. Pact provider verification can show whether a changed service still meets a consumer’s recorded expectations. These checks are worth maintaining when a consumer actually relies on the fields; a schema that declares everything optional cannot catch a missing required value.
Do not preserve a bad response merely because today’s service returns it. If the current endpoint returns HTTP 200 after failing to save a receipt, write down the intended behavior with the service owner before making that response a permanent fixture. Characterization is a way to learn what exists; acceptance criteria are a decision about what must continue to exist.
A selective release gate is safer than an indiscriminate one
The popular next step is to put every available test in the required CI job. That sounds cautious, but an inherited suite may contain tests that fail with timing, shared data, or external service availability. When a required job produces false alarms, developers have an incentive to rerun it until it turns green; that habit makes a real failure easier to dismiss. A gate earns authority through repeatable results, not through the size of its test count.
Put the small boundary suite in GitHub Actions or your existing CI system with a documented fixture, isolated credentials, and a clear owner. Use git diff –check as a cheap preliminary check, but do not mistake it for behavior verification. Run affected contract and database tests on pull requests; run slower end-to-end paths against a staging build before a release. Keep failures that are still under investigation visible in a separate job rather than quietly deleting them.
Playwright documents a default test timeout of 30 seconds; that vendor-published default is a starting point, not a reason to give every slow test another minute. When a browser test exceeds it, inspect where the time went before increasing the timeout. A wait for an unknown background request, for example, needs a better completion signal rather than a longer sleep. For your pilot, use 5 minutes as a tunable ceiling for the required boundary job, then revise it from observed CI runs. The aim is feedback soon enough that a failed check is still connected to the change that caused it.
Flaky tests deserve their own incident trail. Record the failing assertion, environment, commit, and whether a clean rerun passed. Do not use automatic retries to declare the underlying behavior safe, because a race that appears in CI may also appear in a customer request. If a test must be temporarily removed from the gate, name an owner and an expiry date in the pull request; otherwise, “temporary” quarantine tends to become an invisible reduction in protection.
A release gate also needs a way to detect what the suite missed. Compare deployments with Sentry events, a Prometheus counter for rejected or duplicate operations, and support reports tied to the protected paths. Grafana can display the counter, but the alert should be based on an outcome with a clear interpretation, because a graph of total requests cannot reveal whether a receipt was duplicated.
Selective boundary tests beat blanket coverage until change patterns shift
There are two credible options: blanket coverage expansion and selective boundary protection. Blanket expansion wins when a stable component is about to undergo a broad rewrite and the team can define expected behavior across most of it. Its cost is substantial fixture work and the risk of preserving accidental implementation details. Selective protection wins when changes cluster around a few exposed flows and releases cannot pause for a repository-wide campaign. Its cost is that an unselected path can still fail, so the team must revisit its choices as incidents and changes move.
I would choose selective protection first for most inherited codebases because QA usually has limited evidence about which internal behavior is intentional. That position is not an argument against unit tests. Once a boundary failure is understood, a focused unit test can locate the rule more precisely and run more cheaply than an end-to-end test. The sequence matters: establish the customer-visible rule, find the smallest seam that enforces it, and keep both tests only when they answer different questions.
Inherited an untested codebase How do you prove QA pays off raises a fair budget question, but a coverage increase alone is weak evidence of value because it says nothing about prevented harm. For the pilot, compare similar changes before and after the new gate: which protected-path defects reached production, how often releases needed a revert, and how long it took to restore service. Report the number of relevant changes alongside each count; a quiet month with few changes is not proof that the gate worked.
Keep one record per caught failure: the proposed change, the failing assertion, the incorrect outcome it exposed, and the fix made before release. That record supports a modest claim—this check caught this defect at this point—not an invented estimate of revenue saved. If defects move to an unprotected area, change the selected paths. If the same boundary stays quiet while engineers repeatedly modify it, consider a smaller unit-level suite and spend the released CI time where uncertainty has grown.
The decision is therefore reversible. A coverage campaign may become the right investment after the team understands the contracts and has evidence that broad change is coming. Starting there, however, asks QA to specify the whole legacy system before learning which promises anyone needs it to keep.
Your first test should follow a real failure
Tomorrow, pick one recent production defect that required repair. Reproduce its input in an isolated environment, write down the customer-visible result that should have occurred, and make one test fail against the current behavior before fixing it. Ask the service owner to confirm that result. You will have a defensible release check—and a concrete reason to write the next one.
