Flaky Tests

Flaky tests fail or change without a meaningful code change, eroding trust in your test suite. Argos tackles flakiness from two angles: it detects unstable tests so you can decide with confidence, and it gives you tools and strategies to address the root causes.

How Argos helps

• Flaky test detection: Argos flags unstable tests with a flaky badge, a stability score, and a detailed history view. See Flaky Test Detection for details.

• Ignore noisy changes: Dismiss a specific change directly from the build or test page so the same change is no longer reported.

• Auto-ignore recurring flaky changes: Configure Argos to automatically ignore changes that recur over the last 7 days, filtering out noise while keeping real regressions visible.

These features let you separate signal from noise without losing the ability to catch real regressions.

Common causes

Visual flakiness usually stems from one of the following:

• Dynamic content: Ads, user-generated content, or other data that changes between runs.

• Asynchronous loading: Elements or styles that render at inconsistent times.

• Rendering differences: Browser, OS, or device variations that alter visual output.

• External dependencies: Reliance on third-party systems that introduce variability.

• Animations and transitions: Unhandled motion captured mid-frame.

• Resolution and scaling: Screen size or pixel density differences between environments.

Fix the root cause

Ignoring noise is useful, but the most reliable suite is one where flakiness is addressed at the source. The Playbook covers practical techniques:

• Wait for Loading — capture screenshots only after the page is fully loaded.

• Stabilize Date & Time — hide or freeze dynamic dates and times.

• Browser Glitches — standardize environments and handle rendering quirks.

• Argos Helpers — use data-visual-test attributes to control how dynamic elements are captured.

Emphasis on accessibility

At Argos, we advocate for accessibility-focused end-to-end testing. Improving accessibility (semantic markup, aria-busy, predictable focus states) not only supports users with disabilities but also makes tests more deterministic and less prone to flakiness.