Tuning the threshold Parameter in the Visual Diff API to Avoid False Positives

The fastest way to make a team stop trusting visual regression checks is to let them cry wolf. If a diff flags "changed" on a page that a human can look at and see is identical, people learn to ignore the red check — and then they miss the one time it is right.

Most of that noise does not come from real changes. It comes from anti-aliasing and font rendering producing slightly different pixels between two otherwise-identical captures. The threshold parameter on the Snapshot Site Compare API exists specifically to filter that out.

What threshold Actually Controls

POST /api/v3/compare accepts a threshold value between 0 and 1, defaulting to 0.1. It sets how sensitive the pixel comparison is: lower values flag smaller differences, higher values require a bigger visual change before a pixel counts as "mismatched."

curl --request POST \
  --url https://api.prod.ss.snapshot-site.com/api/v3/compare \
  --header 'Content-Type: application/json' \
  --header 'x-snapshotsiteapi-key: YOUR_API_KEY' \
  --data '{
    "before": {"url": "https://example.com/pricing", "width": 1440, "fullSize": true, "hideCookie": true},
    "after": {"url": "https://staging.example.com/pricing", "width": 1440, "fullSize": true, "hideCookie": true},
    "threshold": 0.1
  }'

The response includes mismatchPixels, mismatchPercentage, and totalPixels — so the number you actually act on in CI or a review checklist is mismatchPercentage, not the raw pixel count.

Where the Noise Comes From

Two captures of the exact same page, taken moments apart, are rarely byte-for-byte identical:

• Font rendering can differ by a pixel or two at glyph edges, especially at small sizes.
• Anti-aliasing on rounded corners, gradients, and diagonal lines varies slightly between renders.
• Sub-pixel layout rounding shifts elements by a hair depending on exact viewport width.

At a low threshold, all of that shows up as "mismatched" pixels even though nothing meaningfully changed. Raising the threshold tells the comparison to treat small color deltas as the same pixel, so only genuine visual differences — a moved button, a missing section, a broken layout — surface in mismatchPercentage.

A Practical Way to Find Your Number

Rather than guessing, run the same page against itself once:

1. Capture the page as before.
2. Capture the exact same page again a few seconds later as after.
3. Run the compare with your default threshold and note the mismatchPercentage.

Whatever percentage comes back on an unchanged page is your noise floor. Set your CI failure budget comfortably above that number — if an identical page reports 0.6% mismatch, do not fail builds at anything under roughly 2-3%.

Different Pages Need Different Budgets

A mostly-static marketing page and a data-dense dashboard do not behave the same way:

• Marketing and pricing pages are stable between loads, so a tight budget (2-4%) catches real regressions without much noise.
• Dashboards with live charts, timestamps, or user avatars naturally vary between loads even with nothing "broken," so they need a wider budget (8-12%) or a hide selector to blank out the genuinely dynamic regions before comparing.

If you are wiring this into a pull request check, see Automating Visual Regression Checks in GitHub Actions with the Compare API for the full workflow this threshold value plugs into. And if your diffs still look noisier than expected after tuning threshold, check the format you are comparing — PNG vs JPEG vs WebP: Choosing the Right Screenshot Format for Speed and Quality explains why lossy formats make a noisy baseline even worse.

Getting the threshold right is a one-time calibration per page type, and it is the difference between a visual diff check people trust and one they learn to click past — which is exactly why Snapshot Site exposes it as a plain, tunable number instead of a fixed black box.