A test suite grows one fix at a time, each test landing in whatever layer was quickest to reach. A year on, it takes twenty minutes to run and fails often enough that people have stopped reading the output. The tests were rarely the problem; what was missing is a strategy: which behavior is worth testing, at which level, and how far to take each part.
What a testing strategy actually decides#
Strip away the jargon and a testing strategy answers three questions:
- What to test. You cannot test everything to the same depth. Risk decides priority: the checkout flow matters more than the footer.
- At what level. The same behavior can be checked by a fast unit test, a slower integration test, or a full browser test. Picking the right level is most of the skill.
- How much. Depth costs time and maintenance. A strategy sets where deep coverage earns its keep and where a smoke check is enough.
Get these three right and the suite stays fast, trustworthy, and cheap to maintain. Get them wrong and you end up with a slow suite that misses the bugs that matter.
The testing pyramid: the default shape#
The testing pyramid is the starting point for most strategies. It describes the ideal balance across levels; see the testing pyramid explained for a closer look at the model itself.
| Level | What it checks | Speed | How many |
|---|---|---|---|
| Unit | A small piece of logic, isolated from the database, network, and filesystem | Very fast | The most |
| Integration | Several units wired together against a real database, API, or service | Medium | Fewer |
| End-to-end | A complete user flow through the whole running app, from the UI inward | Slow | The fewest |
The shape matters. A healthy suite is wide at the bottom (many fast unit tests) and narrow at the top (a few high-value end-to-end tests). The common failure mode is the inverted "ice cream cone": most coverage sits in slow, brittle UI tests because that is what gets written first. It runs for twenty minutes, breaks on every redesign, and still misses logic bugs a unit test would have caught in milliseconds.
The rule of thumb: push every test as far down the pyramid as it can go while still proving what you need.
The core strategies#
These are not mutually exclusive. A real strategy combines several.
Risk-based testing#
Spend testing effort in proportion to risk. Rank features by likelihood of breaking and cost of failure, then test the high-risk ones deeply and the low-risk ones lightly. Payment, auth, and data integrity earn deep coverage; a marketing banner does not.
Shift-left testing#
Test as early as possible. A bug caught by a unit test on the developer's machine is cheap; the same bug caught in production is not. Shift-left means moving checks earlier: linting and unit tests on commit, integration tests in the pull request, end-to-end on the deploy. The earlier the gate, the cheaper the fix. This is quality assurance in action, the preventive side of QA versus QC.
Regression testing#
Every fixed bug and shipped feature should stay fixed and shipped. A regression strategy keeps a suite that re-runs on every change to catch things that used to work and quietly stopped. See regression testing for the main types and how to build a suite that stays useful. This is where automation pays for itself, because re-running these checks by hand on every release does not scale.
Exploratory testing#
Scripted tests only catch what you thought to check. Exploratory testing is a human deliberately probing the app without a script, looking for the weird edge cases automation never anticipates. See exploratory testing for how it works and when it beats a scripted suite. It does not replace automated coverage; it complements it. Most real strategies keep a slice of human exploratory time even on heavily automated products.
Balancing manual and automated#
Automate the repetitive, deterministic, high-frequency checks (regression, smoke, critical paths). Keep humans on the judgment-heavy work (exploratory, usability, one-off verification). Automating everything is as wrong as automating nothing; the strategy is knowing which is which.
Static, structural, and behavioral testing#
A separate axis from level and risk: how much the test knows about the code it is checking.
Static testing never runs the program at all. Code review, linting, and static analysis catch mistakes before a single test executes, the cheapest point in the pipeline to catch one.
Structural testing, also called white-box testing, is written with the code open. The author sees the branches, loops, and internal state, and aims tests at the paths the logic actually takes. Most unit tests are structural, and so is much of what sits at the bottom of the testing pyramid.
Behavioral testing, also called black-box testing, is written from the outside. The author works only from what the software is supposed to do, not how it does it. End-to-end and acceptance tests are almost always behavioral, since nobody writing them needs to know how a button click ends up calling a database.
Most suites use both. Structural tests catch logic bugs early and cheaply; behavioral tests confirm the system works correctly from a user's point of view, independent of how the internals happen to be built. Neither replaces the other, the same way unit and end-to-end tests do not replace each other in the pyramid.
Types of testing strategies#
The pyramid and the practices above answer "which level" and "how much." A separate classification, closer to how ISTQB teaches it, answers a different question: how do you decide what to test in the first place? Most real strategies borrow from more than one of these.
- Analytical. Start from a formal risk or requirements analysis and derive tests from it directly, so every test traces back to a documented risk or a line in the spec. Common on regulated software, where an auditor will later ask why a given path was tested.
- Model-based. Build a model of the system, a state diagram, a decision table, a flowchart, and generate tests from the model rather than from intuition. Pays off when the behavior itself is complex enough that a model catches combinations a person would miss.
- Methodical. Work through a pre-defined checklist or a known set of quality attributes (boundary values, error guessing, common failure modes) rather than deriving tests fresh each time. Cheaper to run than analytical or model-based work, and it is the default for teams without the time to build a formal model.
- Process- or standard-compliant. The test process itself follows an external standard, ISO 29119, a domain-specific regulation, an internal QA handbook, so the tests are whatever that document requires rather than what any one tester would choose.
- Dynamic and heuristic. Skip the upfront model and let the current state of the software guide the next test, the way exploratory testing works. A tester adapts each session based on what the last one found, which surfaces bugs a fixed script would never reach.
- Consultative and expert-based. Lean on input from people outside the test team, domain experts, support engineers, the users themselves, to decide what is worth testing. Useful when the biggest risk is not a code defect but a misunderstanding of what the software is supposed to do.
None of these replace the pyramid or the core strategies above. They answer what gets tested at all; the pyramid answers at what level, and the core strategies answer how the effort gets spent.
How to build a strategy for your team#
A workable strategy in six steps:
- Map the risk. List the features and rank them by impact and likelihood of breaking. This is your priority order.
- Assign a level. For each risk, pick the lowest level that can prove it. Push logic into unit tests, contracts into integration tests, and reserve end-to-end for the few flows that must work for a user.
- Decide manual versus automated. Automate the repeatable checks; schedule human time for exploratory and usability passes.
- Set a regression baseline. Decide which suite runs on every change so nothing silently regresses.
- Wire it into CI. A strategy that depends on people remembering to run tests is not a strategy. Gate merges and deploys on the suites that matter.
- Measure and adjust. Track what actually catches bugs and what only adds runtime. Cut tests that never fail for a real reason; add depth where bugs keep escaping.
A worked example#
Take a simple SaaS product with a free trial and a paid checkout, and walk it through the six steps above.
- Map the risk. Checkout and billing rank highest: a broken payment path costs revenue and trust directly. The marketing pages and account settings rank lower; a visual bug there is annoying, not costly.
- Assign a level. The checkout logic itself, discount codes, tax calculation, plan math, gets unit tests, since it is pure logic that does not need a browser. The handoff to the payment provider gets an integration test against a sandbox account. Only the full click-through, land on pricing, pick a plan, pay, reach the dashboard, gets an end-to-end test.
- Decide manual versus automated. The happy path through checkout is automated and runs on every change. A human still walks the signup flow on major releases, looking for what a script would not think to check: broken copy, a confusing error state, a button that is technically clickable but visually wrong.
- Set a regression baseline. The end-to-end checkout test and the unit tests around discount and tax logic run on every pull request. A slower suite covering edge-case plans and currencies runs nightly instead.
- Wire it into CI. The pull-request suite blocks merge. The nightly suite reports to the team but does not block, since one flaky edge-case test should not stop every merge.
- Measure and adjust. If a tax-calculation bug ships despite the suite, that is a gap in step 2: the level chosen did not cover the input that broke. The fix is adding that input to the unit suite, not running the same end-to-end happy path a second time.
Common pitfalls#
- Everything through the UI. Slow, flaky, and expensive. Most of what people test in the browser belongs lower in the pyramid.
- No regression plan. Without one, the same bug ships twice.
- Chasing a coverage number. 100 percent line coverage of trivial code proves little. Coverage of the risky paths is what counts.
- Ignoring flakiness. A suite people do not trust gets ignored, and an ignored suite is worse than no suite, because it gives false confidence.
Where automation fits#
The hardest part of any strategy is not writing the first tests, it is keeping the automated layer green as the app changes. Brittle selectors and constant rework are why teams abandon end-to-end coverage. This is where AI test tooling helps: ObserveOne's Autopilot generates Playwright suites from a URL and self-heals selectors when the UI shifts, so the regression and critical-path layers survive redesigns instead of rotting. The strategy still decides what to cover; the tooling lowers the cost of keeping it covered.
The short version#
A testing strategy is three decisions: what to test, at what level, and how much. Anchor it on the testing pyramid, weight your effort by risk, test early, keep a regression baseline, and leave room for human exploration. The goal is not maximum tests. It is the smallest suite that reliably catches the bugs that would actually hurt. If you are picking the tools to execute it, the Cypress vs Selenium comparison and the AI testing guide are good next reads.