Last year we had a client come to us after they'd fired their entire manual QA team. They'd invested six months into a Cypress suite with 400+ tests, hired two automation engineers, and felt confident they had testing covered.
Three weeks after the manual testers left, their support tickets tripled.
The automated suite was passing. Every single run: green. And their users were reporting bugs that no script had ever thought to check for.
I've seen this play out at BetterQA more times than I can count. A team gets excited about automation, treats it as a silver bullet, and then learns the hard way that a green CI pipeline is not the same thing as a working product.
This is the story of what actually happens when you go all-in on automation and abandon manual testing entirely.
The green suite problem
Here's the thing nobody tells you about a 100% automated test suite: it only checks for things you already thought of.
Every automated test starts as a human decision. Someone sat down, considered a scenario, and wrote a script to verify it. That script will faithfully run that same check forever. It will never wonder "what happens if I click this button twice really fast?" or "does this flow still make sense after the last redesign?"
On this particular project, the client's suite covered login flows, CRUD operations, payment processing, and a handful of API contract tests. Solid coverage on paper. But nobody was testing:
- What happens when a user fills out a form, leaves for 20 minutes, and comes back
- Whether the new dashboard layout actually makes sense to someone seeing it for the first time
- How the mobile experience feels on a slow 3G connection
- Whether the error messages help users recover or just confuse them
These are the kinds of things a manual tester catches in the first five minutes of an exploratory session. No script in the world is looking for them.
Automation is a regression tool, not a testing strategy
I need to be blunt about this because the industry has muddied the water: automation testing and software testing are not the same thing.
Automation is phenomenal at regression. You fixed a bug? Write a test so it never comes back. You have a critical payment flow? Automate it so every deploy verifies it still works. You need to run the same checks across 12 browser/device combinations? Automation saves you days of repetitive work.
But regression is only one slice of testing. Exploratory testing, usability evaluation, edge case discovery, accessibility review, "does this feature actually solve the user's problem" testing: none of that can be scripted. Not because the tools aren't good enough, but because the value of those activities comes from human judgment and creativity.
When our client killed their manual team, they didn't just lose testers. They lost the people who understood how real users interact with the product.
What broke (specifically)
Let me walk through the actual failures we saw on this project, because abstract arguments are easy to dismiss.
Usability regressions went unnoticed for weeks. The dev team shipped a redesigned settings page. The automation suite verified that every button and input worked. What it couldn't tell them was that the new layout was confusing: users couldn't find the save button because it was below the fold on most screens. Support tickets piled up. A manual tester would have caught this in one session.
Edge cases multiplied. The suite tested the happy path and a few known error states. But real users do unpredictable things. They paste formatted text from Word documents into plain text fields. They open the app in two tabs and edit the same record simultaneously. They use browser autofill in ways that break client-side validation. The automation engineers couldn't write scripts fast enough to cover the edge cases that a curious manual tester would stumble into organically.
False confidence from false negatives. The suite had several tests that were passing but not actually verifying what they claimed to verify. A selector had drifted after a UI update, so the test was clicking a different element and asserting on stale data. Green check mark, zero value. When we audited the suite, about 8% of the tests were essentially testing nothing. A manual tester running the same scenarios would have noticed immediately that the behavior was wrong.
Dynamic content broke silently. The app served personalized dashboards with data-driven layouts. The automation suite used hardcoded selectors and fixed test data. Every time the personalization engine changed what was displayed, tests either broke (noisy failures that got ignored) or passed incorrectly (silent failures that hid real issues). The team spent more time maintaining flaky tests than they saved by automating.
The maintenance tax nobody budgets for
This is the part that surprises teams the most. Automation isn't "write it once and forget it." It's a living codebase that needs maintenance, refactoring, and debugging just like your production code.
On this project, the two automation engineers were spending roughly 60% of their time maintaining existing tests and only 40% writing new coverage. Every UI change, every feature flag toggle, every API response format update meant updating test scripts.
Compare that to a manual tester who can adapt on the fly. The button moved? They find it. The API response changed shape? They notice the UI looks different and investigate. The feature flag is on? They test the new behavior. No script updates required.
I'm not saying maintenance is a reason to avoid automation. I'm saying that if you don't budget for it, your "cost savings" from firing the manual team evaporate fast.
Complex scenarios resist automation
Some testing scenarios are genuinely hard to automate well. Multi-step workflows that span multiple systems, tests that depend on timing or environmental conditions, scenarios that require judgment calls about whether the output "looks right."
We had one case where the client needed to test a document generation feature. The automation could verify that a PDF was produced and that it contained certain text strings. But it couldn't tell whether the formatting was correct, whether the layout was readable, or whether the generated content actually made sense in context. A human looks at the PDF and immediately knows if something is off.
This isn't a tooling limitation that better frameworks will solve. It's a fundamental constraint: some quality attributes require human perception.
What we actually recommend
When we onboarded this client, we didn't tell them to throw away their automation suite. That would have been equally wrong in the other direction.
We helped them build a balanced approach:
Automate regression and smoke tests. The things that need to pass on every deploy, the critical paths that must always work, the repetitive checks across environments and devices. This is where automation earns its keep.
Keep manual testers for exploratory work. Dedicate time for testers to explore new features without a script. Let them break things creatively. Give them the freedom to follow their instincts when something feels off. This is where you find the bugs that matter most to users.
Use manual testing for usability evaluation. Before any major release, have a real human go through the key flows and ask: does this make sense? Is this intuitive? Would I be frustrated if I were a customer? No automated tool can answer these questions.
Rotate who does exploratory testing. Don't limit it to QA. Developers, product managers, designers: fresh eyes catch things that familiar eyes skip. The person who built the feature is the worst person to evaluate whether it's intuitive.
Review your automation suite quarterly. Audit for false negatives, outdated selectors, tests that pass but don't verify anything meaningful. Prune ruthlessly. A smaller suite that actually catches bugs is worth more than a massive suite that gives you false confidence.
The AI question
I'll address the elephant in the room because everyone asks. With AI-powered testing tools getting better every month, does this change the equation?
Our founder Tudor Brad has a line I keep coming back to: "AI will replace development before it replaces QA."
His reasoning is sound. AI can generate code, but the act of evaluating whether that code does what users actually need requires human judgment. And with AI accelerating development speed (features that used to take months now take hours), the volume of things that need testing is exploding. You don't need less QA in an AI-accelerated world. You need more.
AI tools are great at generating test cases, identifying patterns in bug reports, and even doing some basic visual regression checking. But the core question of "does this product work for real humans in real situations" still requires a human in the loop.
The real lesson
The client I mentioned at the start? After we helped them rebuild their testing approach with a mix of automation and manual testing, their support ticket volume dropped back to pre-automation-only levels within six weeks. They kept their Cypress suite. They also brought back two manual testers.
The lesson isn't that automation is bad. The lesson is that automation is a tool, not a strategy. And the teams that treat it as the entire strategy are the ones who end up with a green pipeline and angry users.
If your test suite is passing and your users are still finding bugs, the suite isn't the problem. What's missing is the human who would have found those bugs first.
More on testing strategy and QA methodology on the BetterQA blog.
SOCIAL SHARE CARD GENERATOR