An integration test that passes on a laptop and fails in CI is not a testing problem. It is a design problem that testing exposed. The same is true of a suite that fails one run in five, or a Postman collection that works until the token expires.
This guide covers the failures we see most often in API integration work, what causes them, how to fix each one, and a diagnostic sequence to follow when a suite has stopped being trusted.
Symptoms worth recognising
- The suite passes locally and fails in the pipeline.
- The same test passes and fails without a code change.
- Tests pass against the sandbox but production behaves differently.
- Authentication works in a manual tool and fails in the test runner.
- A provider changes a field and nothing alerts you.
- Tests pass in isolation and fail when run together.
- Webhook or callback tests never receive the event.
- The suite takes forty minutes, so people skip it.
Cause and fix, one by one
1. Environment drift
Cause: tests depend on a developer machine's variables, database state or installed services. Fix: define the test environment as code, including container images, migrations, seed data and environment variables in the repository. If a test needs a database, start one as part of the run instead of pointing at a shared instance.
2. Shared state and ordering
Cause: a test creates data that survives into the next test. Fix: each test owns its fixture, uses a unique tenant or identifier and cleans up. Prefer transactional rollback where the framework supports it, and never depend on execution order.
3. Time, randomness and identifiers
Cause: assertions compare against the current clock or a generated ID. Fix: inject a clock, freeze time in tests, and assert on shape rather than on generated values. Run at least one test in a non-UTC offset so timezone logic is exercised.
4. Token and credential handling
Cause: tokens cached outside the test, expired secrets, or scopes granted by hand in the sandbox. Fix: acquire tokens inside setup, assert the expected scopes, and keep credentials in a secret store with a rotation date. Add a test for the expired-token path, because that is the path users hit.
5. Sandbox and production mismatch
Cause: the sandbox accepts payloads that production rejects. Fix: treat the provider's documentation as a contract, validate payloads against its schema before sending, and keep a small set of sanitised real responses as fixtures. Contract tests catch field renames that mocks cannot.
6. Mock fidelity
Cause: mocks return the shape the developer expected, not the shape the provider sends. Fix: generate fixtures from real responses, version them, and re-record on a schedule. A mock that never changes is no longer measuring anything.
7. Asynchronous and callback flows
Cause: the test posts a request and immediately asserts a final state that arrives later by callback. Fix: test the state machine, poll with a bounded timeout, and expose a reachable callback URL in CI. Assert the intermediate pending state as well, because that is what the customer sees.
8. Rate limits and retries
Cause: parallel tests trip the provider's limit, retries amplify load, or a retry hides a real failure. Fix: cap concurrency below the documented limit, add jitter, and make retries visible in logs. A retry that succeeds silently should still emit a warning.
9. Schema drift without contracts
Cause: the provider adds or removes a field and your parser quietly ignores it. Fix: validate responses against a schema and fail on breaking changes. For your own APIs, publish an OpenAPI document and diff it in the pipeline.
10. Slow suites
Cause: every test starts a full stack, or the suite waits on fixed sleeps. Fix: replace sleeps with readiness checks, parallelise by isolation rather than by hope, and keep a fast smoke suite on every commit with the full suite running nightly.
Diagnostic sequence
- Reproduce with the exact pipeline command, not the IDE runner.
- Capture request and response including headers, status, correlation ID and timing.
- Diff the environments: variables, versions, timezone, locale, database state and network egress.
- Run the test alone, then with its neighbours, to separate isolation faults from logic faults.
- Check the provider's status page and changelog before assuming your code changed.
- Inspect retries and queue depth for hidden amplification.
- Write the failing case as a permanent test before fixing it, so the fix carries evidence.
Local provider constraints
Local payment gateways and SMS providers often have thinner sandbox environments and slower support cycles than global platforms. Build the test strategy around that: record real responses early, keep a named contact who can confirm behaviour changes, and avoid designing a test that depends on a sandbox feature the provider may never implement.
What good looks like
- A smoke suite that finishes in under five minutes on every pull request.
- Contract tests against recorded provider responses, refreshed on a schedule.
- Every external dependency containerised, mocked with recorded fixtures, or explicitly marked as an integration test that can fail independently.
- Test data created per test, with no shared accounts.
- Failure output that shows the request, the response and the correlation ID.
Metrics
- Flake rate: percentage of runs with a test that failed and then passed on retry.
- Suite duration at the 90th percentile.
- Escaped integration defects per release.
- Contract coverage: percentage of provider endpoints covered by a schema check.
- Mean time from red pipeline to green.
Where this fits in our work
If your integration suite has stopped being trusted, start with Custom Software & Integrations; the pricing page explains how a test-hardening engagement is scoped. For the design side, read the REST API design checklist.
Sources
- RFC 9110, HTTP Semantics, accessed 17 November 2026.
- RFC 9457, Problem Details for HTTP APIs, accessed 17 November 2026.
- OWASP, API Security Top 10 (2023), accessed 17 November 2026.
- OWASP, Application Security Verification Standard, accessed 17 November 2026.
Responses (0)
No comments yet. Add the first one below.
Leave a response
Your email address will not be published. Required fields are marked *.