The QA Metrics That Actually Matter (and Which to Drop)

The QA metrics that actually matter measure quality that reached your users, not activity inside your process. The four worth tracking are escaped defect rate (bugs found in production vs. before release), defect reopen rate (fixes that didn't hold), mean time to resolution segmented by severity, and test flakiness rate. Metrics like raw bug count, total test count, and headline test pass rate reward motion instead of outcomes, and should be demoted or dropped.

Why most QA dashboards measure the wrong thing

Most QA dashboards fill up with numbers that are easy to collect: how many bugs were filed, how many tests exist, what percentage passed. These are activity metrics. They go up when the team is busy and look reassuring in a status meeting, but none of them answer the only question that matters — is the product getting better for the people using it?

The problem is that activity and outcome can move in opposite directions. A team can write 500 new tests, ship a 99% pass rate, close 80 bugs a sprint, and still have users hitting broken checkout every week — because the tests cover the paths QA already knew about, and the real defects live in the paths nobody tested. Good QA metrics are the ones that stay honest when the process looks busy.

The four QA metrics worth tracking

Each of these measures an outcome, is hard to game, and points to a specific fix when it moves in the wrong direction.

1. Escaped defect rate. The share of defects discovered in production rather than before release. Compute it as production defects divided by total defects (pre-release + production) for the period, times 100. This is the single best proxy for "how well is testing actually working," because it counts only the bugs that got past every gate. A rising trend means your coverage or environment parity is slipping — not that users got unlucky.
2. Defect reopen rate. The percentage of resolved bugs that get reopened because the fix was incomplete or wrong. A healthy fix stays fixed. A climbing reopen rate signals rushed fixes, missing regression tests, or bugs that were closed as "cannot reproduce" without ever being understood. It's a direct measure of fix quality, which raw close-count completely hides.
3. Mean time to resolution (MTTR), by severity. How long a bug takes from report to verified fix. The key is to segment it: a blended average lets a fast pile of typo fixes mask slow critical-bug resolution. Track critical, high, and low separately and drive each trend down. In practice, most of MTTR is not coding time — it's the round-trips spent asking the reporter for steps, logs, and environment that should have been captured up front.
4. Test flakiness rate. The percentage of test runs that fail and then pass on retry without any code change. Flaky tests quietly poison every other metric: they inflate your "failures," get silenced with blanket retries, and eventually train the team to ignore red builds. Measure flakiness explicitly and quarantine offenders so a real regression never hides behind a test everyone has learned to re-run.

Which QA metrics to drop (or demote)

These aren't useless, but as headline numbers they mislead more than they inform:

• Raw bug count. More bugs found can mean better testing or a worse product — the number alone can't tell you which. Watch escaped defect rate instead, and use bug count only as supporting context.
• Total number of tests. Test count measures how much you wrote, not how much you covered. A thousand assertions on a settings page while checkout goes untested is a big number and a bad safety net. Coverage of critical user journeys matters; the raw total does not.
• Headline test pass rate. Pass rate reflects only the checks you already thought to write, and it's easily gamed by flaky retries and shallow tests. A green suite is necessary, not sufficient. Pair it with escaped defect rate so you can see when "all green" and "actually working" have drifted apart.
• Bugs closed per person. Rewarding close-count encourages closing tickets, not fixing problems — and it punishes the engineer who takes an extra day to fix the root cause instead of the symptom. Reopen rate is the honest counterweight.

How to make these metrics easy to collect

The reason teams fall back on vanity metrics is that the good ones feel expensive to measure. They don't have to be, if the data is captured at the source:

1. Tag where each defect was found. Add a required "found in" field (production vs. staging vs. pre-release) to every bug so escaped defect rate falls out of a query instead of a manual audit.
2. Capture evidence at report time. Most MTTR bleeds out in the gap between "a bug was filed" and "an engineer can reproduce it." If every report ships with exact steps, console output, network activity, and environment attached, that gap collapses. In-app tools like Klavity Snap capture that context on the first report, which is what actually moves MTTR — and it makes reopen rate meaningful, because closed bugs were genuinely understood, not guessed at.
3. Instrument flakiness in CI. Record which tests fail-then-pass on retry and surface the list every week. AutoSim's self-healing tests reduce one common flakiness source — brittle selectors that break when the UI shifts — so your flakiness rate reflects real instability, not maintenance debt.
4. Review trends, not snapshots. Every metric here is a trend line, not a target to hit once. Look at direction over the last several releases; a single month's number is noise.

Once escaped defects, reopen rate, MTTR, and flakiness are on one dashboard, retros change character: instead of arguing about how many bugs were closed, the team asks why a defect escaped and how to close the gap. For more on turning a noisy backlog into signal, see our guides on bug triage and reproducibility.