Industry practices · Practice 4 of 10

Remove the causes of UI test flakiness

Most UI test flakiness traces back to the causes Fowler listed, and the standard fixes are specific to each one.

Replace fixed sleeps with waiting on observable conditions. Playwright and Cypress auto-wait on element state, and both provide network-idle and request-interception hooks for synchronising with the app.
Stub backend calls at the network boundary for module-level UI tests, which today run as system tests against deployed services, using Playwright's page.route, Cypress intercept or Mock Service Worker. The test then exercises the module's own behaviour and cannot fail because a service is down or redeploying.
Give each test its own data, created through an API at the start of the test and keyed by a unique identifier, so parallel tests never share records.
Freeze time and randomness in the test harness and pin the browser version used in CI.
Run tests in ephemeral, per-pipeline environments (containers or namespaces created and destroyed with the run) when real backends are required, so that pipelines do not contend with each other.
Make each UI suite set up the data and state it needs, so that no suite depends on what another suite's pipeline left behind and the UI pipelines that wait on other UI pipelines today can drop that dependency.

In GoCD

Elastic agent plugins for Kubernetes, Docker and Amazon ECS start a fresh agent for each job and remove it afterwards, so no state carries over between UI runs.
Agent resources and elastic profiles pin the container image, which fixes the browser version used in CI.
Job timeouts end hung UI jobs, and tasks with run_if set to any, plus on_cancel tasks, run the teardown for a per-pipeline environment even when the tests fail or the job is cancelled.
Environment variables such as GO_PIPELINE_COUNTER give each run a unique key for its test data and environment names.

Creating and destroying a per-pipeline environment, such as a Kubernetes namespace with the services a UI suite needs, is left to scripts in the job.
GoCD has no service containers or sidecars attached to a job, so backend stubs and test databases have to be started by the job itself or by the elastic profile.

The fixes remove flakiness at its cause, so the gains do not depend on tooling or policy.
Condition-based waits and stubbed networks usually make tests faster as well as more reliable.
Module UI tests that stub the network keep passing while services are down or redeploying.
Fixing the cause of a flake keeps the test in the suite, which commenting it out does not.

The work is spread across about 100 suites and competes with feature work in every team.
Stubbed responses can drift from what the services actually return, which contract tests (practice 5) have to catch.
A stubbed UI suite stops being a system test, so a small set of journeys has to keep running against real services to check the system as a whole.
Ephemeral per-pipeline environments cost infrastructure and need automation to create and destroy them reliably.

Addresses non-deterministic tests, shared long-lived environments, integrated tests used as the gate.

Sources Martin Fowler, "Eradicating Non-Determinism in Tests" (2011); Microsoft Playwright documentation on auto-waiting and network mocking; Cypress documentation for cy.intercept; Mock Service Worker.