Skip to content
EncryptTechBD
Home Industries Our Work Blog Pricing About Contact Discuss Your Project
বাংলা (BN)
Software Engineering 6 min read

Automated testing pyramid problems and fixes

Why test suites become slow, flaky and ignored, and how to rebalance unit, integration and end-to-end tests without chasing a coverage number.

E
EncryptTechBD · 0 comments
Developer writing code
Photo: Negative Space / StockSnap

The test pyramid is a useful idea that is often adopted as a rule and then blamed when it fails. Teams are told to write many unit tests, fewer integration tests and a handful of end-to-end tests. Six months later the suite is slow, the end-to-end tests fail unpredictably, and developers have started merging without waiting for the result.

The pyramid is not the problem. The problem is usually that the layers are being used for the wrong purpose. This troubleshooting guide covers the symptoms a team actually sees, the causes behind them, and a diagnostic sequence that restores confidence without doubling the test count.

Symptom 1: the pipeline takes too long

Likely cause: end-to-end tests are carrying logic that belongs in a unit or integration test. A browser test that checks a validation rule, a currency calculation or a permission matrix is slow because it starts a browser, a database and a server to prove something that could be proved in milliseconds.

Fix: list every end-to-end test and write the business rule it protects. If the rule can be tested below the browser, move it down. Keep the browser test only for the journey that crosses the UI, the API and the database together, such as checkout or login. Run the full browser suite on a schedule and a smaller smoke set on every pull request.

Symptom 2: tests fail intermittently

Likely cause: shared state, timing assumptions or test data that changes between runs. Two tests using the same customer record, a fixed sleep instead of a condition, a background job that runs during the suite, or a third-party sandbox that is not deterministic.

Fix: give each test its own data, created and removed by the test. Replace sleeps with explicit waits for a visible state. Isolate external services behind a stub or a contract test, and keep one small set of tests that hit the real sandbox. Record the failure rate by test, not just by suite, so the worst offenders are visible.

Symptom 3: everything passes but production breaks

Likely cause: the tests mock the parts where the defects live. If every integration is replaced by a mock that returns the shape the developer expected, the suite proves the code agrees with itself. It does not prove the API contract, the database constraint or the payment callback.

Fix: add contract tests at the boundaries: request and response schemas, status codes, error formats and authentication. Run migrations against a real database in the pipeline. Keep a small end-to-end set for the revenue-critical paths, and add a production smoke check after deployment.

Symptom 4: refactoring breaks hundreds of tests

Likely cause: tests are coupled to implementation details: private methods, internal class names, DOM selectors that change with styling, or the exact sequence of calls. The suite is testing the code's structure rather than its behaviour.

Fix: test through the public interface and assert on outcomes a user or caller can observe. Replace brittle CSS selectors with accessible roles and labels. Delete tests that only assert a call was made when the same behaviour is covered by an integration test. A refactor should change tests only when the behaviour changes.

Symptom 5: coverage is high and confidence is low

Likely cause: coverage measures lines executed, not decisions protected. A test that calls a function and asserts nothing can raise coverage. A suite can execute every line and still miss the permission check, the empty state or the failed payment.

Fix: stop reporting coverage as a target. Track coverage of critical modules as a floor, and track behaviour coverage through a short list of risks: who can see which record, what happens when a payment fails, what happens when an API times out. Write tests for the risks, not for the number.

Symptom 6: nobody owns the failures

Likely cause: the suite is treated as infrastructure rather than product. When it fails, the person who wrote the last change assumes someone else will look, or the test is skipped to unblock a release. Skipped tests accumulate quietly.

Fix: assign an owner for the suite, publish a weekly failure summary and make a red pipeline a stop-the-line event for the team. Set a rule for skipping: a skip needs a ticket, an owner and an expiry date. Review skipped tests every sprint.

Diagnostic sequence

  1. Measure the suite by layer. Record runtime, failure rate and flake rate for unit, integration and end-to-end tests separately. Do this before changing anything.
  2. Classify recent failures. For the last twenty failures, mark each as a real defect, a test defect, an environment problem or a data problem. The pattern tells you where to invest.
  3. Map tests to risks. List the ten things that would hurt the business most, then find the test that protects each one. Gaps matter more than counts.
  4. Remove or rewrite the worst tests. Start with the slowest and flakiest end-to-end tests. Move logic down, delete duplicates and fix data isolation.
  5. Add guardrails. A runtime budget for the pull-request suite, a smoke set for deployment and a monthly review of skips and failures.

Common mistakes

  • Treating the pyramid as a percentage target rather than a design heuristic.
  • Writing end-to-end tests for every acceptance criterion, then wondering why the suite is slow.
  • Mocking the integration that caused the last production incident.
  • Adding a coverage gate without deleting low-value tests.
  • Letting the suite run only after merge, so failures arrive too late to influence the change.
  • Buying a test management tool before fixing test data and ownership.

Metrics to report

  • Pull-request suite runtime and the percentage of runs that pass first time.
  • Flake rate by test, with the top ten named.
  • Escaped defects found in production, by severity.
  • Percentage of critical risks with an automated check.
  • Number of skipped tests and their age.
  • Time from a red pipeline to a fix.

Where this fits in our work

Test strategy and pipeline work sit inside Custom Software & Integrations, usually alongside API and application delivery. The pricing page explains how a test-hardening engagement is scoped. For the interface layer that often causes boundary failures, read the REST API design checklist.

Sources

Services that deliver this

Want this implemented properly?

Pricing →

Every engagement starts with a written scope. If a requirement is outside it, we quote it separately before starting.

Responses (0)

No comments yet. Add the first one below.

Leave a response

Your email address will not be published. Required fields are marked *.

Keep reading

More field notes