How to Decide Which Tests to Automate First: A Framework

Automate the tests that run most often, stay stable across releases, are slow or tedious to verify by hand, and guard your highest-risk user flows — in roughly that order. Prioritizing this way front-loads the return on your automation effort, because a test's value is approximately its manual execution cost multiplied by how often you would otherwise run it, divided by what it costs to build and maintain. Score every candidate test on those factors and automate from the top of the list down.

Why you can't automate every test at once

Automation is not free. Each test costs engineering time to write, and — often the bigger number — ongoing time to maintain as the product changes. If you try to automate everything, you spend that budget on low-value checks and slow suites before you cover the flows that would actually catch a costly regression.

The goal is not maximum coverage. It is maximum return per hour of automation effort. That means treating your list of candidate tests as a backlog to be ranked, not a checklist to be exhausted.

What makes a test worth automating first

Four properties raise a test's priority. The first three raise its value; the fourth lowers its cost.

1. Run frequency. How often would you execute this check by hand if it weren't automated? A smoke test you run on every deploy pays back far faster than one you run once a quarter. Frequency is the strongest single predictor of automation value.
2. Manual cost per run. A check that takes 20 seconds to eyeball is cheap to keep manual. One that requires setting up test data, walking through eight screens, and comparing values is expensive every single time — and expensive-per-run is exactly what automation is good at eliminating.
3. Business risk. What breaks if this path fails silently? Login, checkout, payment, and data-integrity flows carry outsized risk; a tooltip's wording does not. Weight the flows where a bug costs revenue, trust, or data.
4. Stability of the behavior. A flow whose UI and logic change every sprint will break its test every sprint, driving maintenance cost up. Stable, settled behavior is cheap to automate; churny, half-designed features are not — automate them once they stop moving.

A simple scoring framework for automation priority

Turn the four properties into a rough score you can sort by. You do not need precision — you need a defensible order.

1. List the candidates. Pull every repeatable check your team performs: regression passes, smoke tests, critical user journeys, edge cases you keep re-verifying after fixes.
2. Score each factor 1–5. Rate frequency, manual cost, and business risk from low (1) to high (5). Then rate maintenance cost from low (1) to high (5) separately — this one works against automation.
3. Compute a priority number. Use (frequency × manual cost × business risk) ÷ maintenance cost. High-value, low-upkeep tests float to the top; expensive-to-maintain checks sink even if their value is decent.
4. Automate down the list. Start at the top and work down until you exhaust your automation budget for this cycle. Everything below the line stays manual — on purpose, and revisited next cycle.

A worked example: an end-to-end checkout test scored frequency 5 (every deploy), manual cost 5 (multi-step, needs test data), risk 5 (revenue), maintenance 2 (stable flow) lands at 62.5 — automate it now. A settings-page label check at frequency 2, manual cost 1, risk 1, maintenance 1 lands at 2 — leave it manual. The gap between them is the whole point.

Which tests to automate last — or never

Some checks are legitimately better left to a human, and forcing them into a suite wastes budget and adds flakiness:

• One-off verifications. If you'll run it once to confirm a fix, just run it. Convert it to an automated regression test only if the bug is likely to recur.
• Rapidly changing UI. Automating a feature that's still being redesigned means rewriting the test weekly. Wait for it to settle.
• Subjective and exploratory checks. Visual polish, tone, and "does this feel right" are human judgments. Exploratory testing finds the bugs you didn't think to script — automation can't replace that.
• Rarely-hit, low-risk paths. A test that runs twice a year to guard a cosmetic corner rarely earns its maintenance cost.

How to keep the priority list from going stale

The ranking is a snapshot, not a law. A feature that shipped last quarter is now high-frequency; a flow you deprecated is now zero-risk. Two habits keep the list honest:

1. Re-score each release. Frequency and business risk drift as usage and the product change. A quick pass every release keeps you automating what matters now, not what mattered six months ago.
2. Drive down maintenance cost so more tests qualify. The denominator in the score is not fixed. Resilient selectors and self-healing end-to-end tests that auto-repair broken locators when the UI shifts lower maintenance cost across the board — which pulls more tests above the automation line without extra upkeep.

One more compounding source of candidates: the bugs your users actually hit. When a real report comes in through in-app bug reporting with the steps, console, and network state attached, you get a reproducible case that's often worth locking in as an automated regression test — a data-driven way to keep the top of your priority list pointed at the paths that break in the wild.