Visual testing

A journey to image stabilization

March 25, 2025

|

Greg Bergé

Image stabilization is a real challenge in visual testing. Even after images finish loading, subtle rendering differences often sneak in and trigger false positives. After years of chasing down these inconsistencies, we finally cracked one of the most elusive causes: responsive images using srcset.

In this article, we’ll walk you through how we discovered the issue, why it happens, and how we fixed it as part of our ongoing effort to make visual tests more stable and trustworthy.

Problem: inconsistent images causing layout shift

At Argos, we continuously interact with our customers. One recurring complaint from users was that images inside their pages were rendering inconsistently, leading to layout shift and unreliable visual regression results. Small differences between browsers are expected, but in these cases, users noticed variations in CI on the same setup. That suggested something deeper was going on.

After investigating, we found the issue often appeared when using the viewports feature to test different screen sizes. Even in stable environments, image rendering wasn’t behaving as expected.

Analysis

We discovered that when the viewport size changed, some images were resized slightly differently between runs. For example, one image might be rendered at 288×359.81 pixels, while another appears at 288×359.64.

These small differences came from changes in the aspect ratio when the browser window was resized. For example:

• On initial load: The aspect ratio is 800:999, resulting in 288 * 999 / 800 = 359.64
• After resize: The aspect ratio is 1536:1919, which gives a computed height of 288 * 1919 / 1536 = 359.8125

Additionally, the images differ in resolution: the first is high-resolution (1536x1919 for 2.4MB) and the second is low-resolution (800x999 for 1.1MB). The culprit here is the use of the srcset and sizes attributes in the <img> tag.

The role of srcset and sizes

The srcset attribute lets you specify different image sources for varying screen sizes. The browser uses the sizes attribute to determine which image to load.

For instance:

<img
  src="image.jpg"
  srcset="image-800.jpg 800w, image-1536.jpg 1536w"
  sizes="(max-width: 800px) 100vw, 800px"
  alt="A beautiful image"
/>

If the viewport is below 800px, the browser loads image-800.jpg. Above that, it loads image-1536.jpg. During resizing, the browser may reuse a cached high-resolution image instead of reloading the correct one, which can lead to visual differences between renders.

Our solution: forcing a re-evaluation of srcset

We needed a method to force the browser to reload the image when the viewport is resized. A naive approach might be to reset the srcset attribute; however, browsers don’t re-evaluate it unless a reflow is triggered.

Code to force the reload

const forceReload = (img: HTMLImageElement) => {
  const srcset = img.getAttribute("srcset");
  img.setAttribute("srcset", "");
  // Force reflow
  void img.offsetWidth;
  img.setAttribute("srcset", srcset);
};

By resetting srcset, triggering a reflow (using offsetWidth), and restoring the original value, we force the browser to re-evaluate which image to load.

Listening for resize events

We then detect viewport changes using the resize event:

const onResize = () => {
  const images = document.querySelectorAll("img[srcset]");
  images.forEach((img) => {
    forceReload(img as HTMLImageElement);
  });
};

window.addEventListener("resize", onResize);

This ensures every image with a srcset is reloaded appropriately upon window resize.

Integrating with Playwright

When running tests in Playwright, simply reloading images isn’t enough. Since the browser takes a screenshot before the new image loads, we must wait for the images to complete loading:

// Reload srcset on resize
await page.evaluate(() => {
  const forceReload = (img: HTMLImageElement) => {
    const srcset = img.getAttribute("srcset");
    img.setAttribute("srcset", "");
    void img.offsetWidth;
    img.setAttribute("srcset", srcset);
  };

  const onResize = () => {
    const images = document.querySelectorAll("img[srcset]");
    images.forEach((img) => {
      forceReload(img as HTMLImageElement);
    });
  };

  window.addEventListener("resize", onResize);
});

// Resize the viewport
await page.setViewportSize({ width: 800, height: 600 });

// Wait for all images to load
await page.waitForFunction(() =>
  Array.from(document.images).every((img) => img.complete),
);

// Take a screenshot
await page.screenshot({ path: "screenshot.png" });

This approach guarantees that screenshots capture the correctly reloaded images.

Stabilization in Argos

While image instability is one issue, visual testing can also be disrupted by:

• Fonts not loading
• Visible loaders
• Unwanted spellcheck markers

Argos addresses these with a stabilization algorithm. This system auto-detects such issues and applies fixes, ensuring consistent results across testing engines like Cypress, Playwright, Puppeteer, and WebdriverIO.

Plugin for image stabilization

To encapsulate the image stabilization, we implemented a plugin:

import type { Plugin } from "..";

/**
 * Force the reload of srcset on resize.
 * To ensure that if the viewport changes, it's the same behaviour
 * as if the page was reloaded.
 */
export const plugin: Plugin = {
  name: "image-srcset",
  beforeAll() {
    const handleResize = () => {
      Array.from(document.images).forEach((img) => {
        const srcset = img.getAttribute("srcset");
        if (srcset) {
          img.setAttribute("srcset", "");
          // Force reflow
          void img.offsetWidth;
          img.setAttribute("srcset", srcset);
        }
      });
    };

    window.addEventListener("resize", handleResize);

    return () => {
      window.removeEventListener("resize", handleResize);
    };
  },
};

This plugin adds a resize event listener before tests run and cleans it up afterward to prevent memory leaks.

Conclusion

Visual testing should be rock-solid, not a guessing game. Argos brings determinism to the chaos by eliminating flakiness at the source: unstable images, layout shifts, flickering fonts, and even sneaky spellcheck artifacts. Our stabilization engine rewrites the rules with deep viewport awareness and precise control over srcset behavior, delivering unmatched reliability across browsers and test runners.

Skip the false positives, ship with confidence, and let Argos handle the visual noise.