Is The Testing Pyramid Broken?

“One of the inventions most important to a class of highly skilled workers (engineers) would be a small motive power — ranging perhaps from the force of from half a man to that of two horses, which might commence as well as cease its action at a moment’s notice.”

Twenty years before the first practical electric generator, Babbage saw the need for the electric motor. And as electrification took off starting in 1900, everyone envisioned major improvements to industrial productivity.

Those improvements, though, didn’t appear right away. Factories that swapped steam drive systems for electric motors saw almost no immediate increase in productivity. The reasons are complex, but the principle is general: Even when an improvement has been sought after, planned for and universally praised, its benefits don’t come until we rethink how we work.

The testing and QA process are a final step of a huge automated process for releasing modern software. And while . Let’s look at how we used to test software, how microservices architectures have broken that process and how we can get back to a world where we confidently release working code to production, without abandoning the benefits of microservices.

The History of the Testing Pyramid

Multiple explanations of testing as a pyramid, with fewer more expensive tests at the top and many small cheap tests at the bottom, have been proposed, but most recognize the simple three layer pyramid from Mike Cohn’s “ at key points during the development cycle, such as after significant changes or features have been added. They ensure that individual units interact correctly within the system.

Example: In the same project management tool, integration tests might check how the task management system interacts with the calendar. They ensure that new tasks appear on the correct dates and that updates to tasks are reflected accurately in the calendar. While the datastore might be updated as part of an integration test, we might not test the UI or third-party services like a payment provider to create a subscription.

3. End-to-End (E2E) Tests

Description: E2E tests, at the top of the pyramid, validate the entire application flow from the user’s perspective. These tests are comprehensive and simulate real user scenarios.

When to Use: E2E tests should be run at significant milestones, such as before a major release. Due to their complexity and resource demands, they are executed less frequently but provide critical validation of the application’s overall functionality.

Example: For the project management tool, E2E tests might involve creating a project, adding team members, managing tasks and ensuring the entire project workflow functions correctly. Tests would be run using an automated browser controlled with a test framework like Playwright. These tests mimic real-world usage to verify that the application meets user expectations, and everything from visual updates of buttons to complete authentication, account updates and complex application interactions would be tested.

How the Testing Pyramid Is Broken for Microservices

Microservices have come to be the dominant architecture for production services, in general offering the benefits of:

• Clear separation of concerns


• Smooth scaling — demand on single services is easy to identify, and services can be replicated without scaling roadblocks


• Faster debugging of failures — isolated to a single service, it’s easier to find a root cause


• Easier maintainability of code — a small team becomes an “expert” on their few microservices

These benefits are irrefutable, and this article doesn’t seek to advocate for older-style monolithic software. However, the process of testing in a microservice architecture is broken in a few key ways.

Before I dive in, let me say that these breaking issues involve the way microservices are implemented in the real world. In an ideal world, some of the problems below wouldn’t exist, but I’d ask the reader to ask whether some or all of the product architectures they have seen had these issues. In the order of pyramid layers:

Unit testing tends to break in microservice architectures for one reason: interdependence. When developers are tasked with creating unit tests, they tend to create tests that are easy to implement, and by design, they don’t create tests that require any other services. In the example above I mentioned a simple backend action like “creating a new task.” However, in reality, even such a simple task involves at least two services.

Extensive mocking of outside components

End-to-end testing breaks in two significant ways: cost and quality of feedback. In order to run complete end-to-end tests, there are significant costs in terms of time and infrastructure costs for a high-quality environment and test runner to store results. Quality of feedback is another issue, partly as an outgrowth of costs: first off, a failure observed from the frontend doesn’t have a clear pointer to where the request actually failed, then if QA teams are running E2E to meet our community.