Visual Regression Testing with Scrnify and Jest (JavaScript)
3/14/2025
Hey there! Laura and Heidi here from SCRNIFY! ๐
Do you find yourself manually checking your web applications for visual bugs? Are you tired of discovering that a CSS change broke your UI layout only after it's been deployed to production? We've been there too! ๐คฆโโ๏ธ
As developers who've built a screenshot service, we know firsthand how challenging it can be to catch visual regressions before they reach your users. That's why today we're diving into visual regression testing with Jest and our Scrnify API.
In this tutorial, you'll learn how to set up automated visual comparison tests that will catch even the slightest pixel changes in your UI. No more "it looked fine on my machine" moments! Let's get started and get free access to the SCRNIFY API during our open beta to follow along.
What is Visual Regression Testing and Why is it Important? ๐
Visual regression testing is a technique that helps you detect unintended visual changes in your UI. Unlike functional tests that check if your code works, visual tests ensure your application still looks right.
Here's why it matters:
- โ Catches what functional tests miss: Unit and integration tests don't verify visual elements
- โ Prevents CSS regressions: Small CSS changes can have big visual impacts
- โ Ensures cross-browser consistency: Identifies rendering differences across browsers
- โ Improves user experience: Maintains consistent design and branding
- โ Reduces manual testing time: Automates the tedious process of visual inspection
As applications grow more complex, it becomes increasingly difficult to manually catch every visual issue. Automated visual testing gives you confidence that your UI elements appear as expected, even after significant code changes.
Prerequisites ๐ ๏ธ
Before we dive in, make sure you have the following:
- Node.js (v14 or higher) installed
- A basic understanding of JavaScript and Jest testing
- A Scrnify API key (you can sign up for the beta)
- A project with Jest installed or a willingness to set one up
Let's set up our project:
# Create a new project directory
mkdir visual-regression-tests
cd visual-regression-tests
# Initialize a new Node.js project
npm init -y
# Install required dependencies
npm install --save-dev jest jest-image-snapshot node-fetch@2 dotenv
Note: We're using
node-fetch@2
because it provides a simpler CommonJS import pattern. If you're using ES modules, you can use the latest version.
Setting up a Jest Test ๐งช
First, let's create a basic Jest configuration. Create a file named jest.config.js
in your project root:
// jest.config.js
module.exports = {
testTimeout: 30000, // Screenshots might take time, so we increase the timeout
testMatch: ['**/*.test.js'],
};
Next, let's create a .env
file to store our Scrnify API key:
SCRNIFY_API_KEY=your_api_key_here
Don't forget to add this file to your .gitignore
to avoid committing your API keys!
Now, let's create a simple utility to interact with the Scrnify API. Create a file named screenshot.js
:
// screenshot.js
const fetch = require('node-fetch');
const dotenv = require('dotenv');
dotenv.config();
const API_KEY = process.env.SCRNIFY_API_KEY;
const BASE_URL = 'https://api.scrnify.com/capture';
/**
* Takes a screenshot of a webpage using Scrnify API
* @param {string} url - The URL to capture
* @param {Object} options - Screenshot options
* @returns {Promise<Buffer>} - The screenshot as a buffer
*/
async function takeScreenshot(url, options = {}) {
// Default options
const defaultOptions = {
type: 'image',
format: 'png',
width: 1920,
height: 1080,
fullPage: false,
waitUntil: 'networkIdle',
};
// Merge default options with user-provided options
const mergedOptions = { ...defaultOptions, ...options };
// Create query parameters
const params = new URLSearchParams({
key: API_KEY,
url: encodeURIComponent(url),
...mergedOptions,
});
try {
// Make the API request
const response = await fetch(`${BASE_URL}?${params.toString()}`);
if (!response.ok) {
const error = await response.text();
throw new Error(`Screenshot API error (${response.status}): ${error}`);
}
// Return the image as a buffer
return await response.buffer();
} catch (error) {
console.error('Failed to take screenshot:', error);
throw error;
}
}
module.exports = { takeScreenshot };
Taking a Baseline Screenshot ๐ธ
Now that we have our utility set up, let's create our first test that will take a baseline screenshot. Create a file named homepage.test.js
:
// homepage.test.js
const { toMatchImageSnapshot } = require('jest-image-snapshot');
const { takeScreenshot } = require('./screenshot');
// Extend Jest's expect
expect.extend({ toMatchImageSnapshot });
describe('Visual regression tests', () => {
it('should match homepage screenshot', async () => {
// Take a screenshot of our test page
const screenshot = await takeScreenshot('https://example.com', {
width: 1280,
height: 800,
fullPage: true,
});
// Compare with the baseline or create one if it doesn't exist
expect(screenshot).toMatchImageSnapshot();
});
});
Run the test with:
npx jest
The first time you run this test, Jest will create a directory called __image_snapshots__
with your baseline screenshot. Since there's no existing baseline to compare against, the test will pass and save the current state as the reference.
This is what happens behind the scenes:
- Scrnify API captures a high-quality screenshot of the webpage
- Jest saves this screenshot as the baseline in
__image_snapshots__
- In future test runs, new screenshots will be compared against this baseline
Making a Change ๐
To demonstrate how visual regression testing works, let's simulate a scenario where we've made changes to our website. Instead of actually modifying a website, we'll use a different URL for our "changed" version.
Update the test to include a second test case:
// homepage.test.js
const { toMatchImageSnapshot } = require('jest-image-snapshot');
const { takeScreenshot } = require('./screenshot');
expect.extend({ toMatchImageSnapshot });
describe('Visual regression tests', () => {
it('should match homepage screenshot', async () => {
const screenshot = await takeScreenshot('https://example.com', {
width: 1280,
height: 800,
fullPage: true,
});
expect(screenshot).toMatchImageSnapshot();
});
it('should detect changes on a different page', async () => {
// Using a different page to simulate changes
const screenshot = await takeScreenshot('https://example.org', {
width: 1280,
height: 800,
fullPage: true,
});
expect(screenshot).toMatchImageSnapshot();
});
});
If you run the tests now, the first test should pass (as we're using the same URL as before), but the second test will fail because example.org
looks different from the baseline image of example.com
.
Comparing Screenshots ๐
Jest-image-snapshot is doing the heavy lifting of comparing the screenshots for us. When it detects differences, it:
- Creates a visual diff highlighting the changes
- Saves the diff image in a
__diff_output__
directory - Fails the test with information about the differences
The diff images use a red/green color scheme to highlight the differences:
- Red pixels show elements that were removed or changed
- Green pixels show elements that were added or changed
- Unchanged areas retain their original appearance
This makes it easy to spot even subtle changes in your UI.
Handling Differences โ๏ธ
Not all visual differences are bugs. Sometimes you intentionally change your UI. Jest-image-snapshot provides options to handle these scenarios:
Updating Baselines for Intentional Changes
When you make intentional UI changes, you can update your baselines by running:
npx jest -u
This tells Jest to replace the old baselines with the new screenshots.
Setting Custom Thresholds
Sometimes minor pixel differences are acceptable. Let's modify our test to allow for some flexibility:
// homepage.test.js
it('should match homepage screenshot with customized threshold', async () => {
const screenshot = await takeScreenshot('https://example.com', {
width: 1280,
height: 800,
fullPage: true,
});
expect(screenshot).toMatchImageSnapshot({
customDiffConfig: {
threshold: 0.1, // Higher threshold means less sensitive to minor changes
},
failureThreshold: 0.01, // The test will fail only if more than 1% of pixels are different
failureThresholdType: 'percent',
});
});
This configuration:
- Sets the per-pixel threshold to 0.1 (pixels must be significantly different to count)
- Sets the overall threshold to 0.01 (test fails only if more than 1% of pixels differ)
- Uses percentage as the failure threshold type (you can also use 'pixel')
Ignoring Specific Areas
Sometimes your page contains dynamic content like dates or ads that will always cause differences. You can create a customized matcher to ignore specific regions:
// Custom matcher that masks a specific region
it('should ignore dynamic content areas', async () => {
const screenshot = await takeScreenshot('https://example.com', {
width: 1280,
height: 800,
fullPage: true,
});
// Define a rectangle to mask (x, y, width, height)
const maskConfig = {
diffMaskColor: '#FF0000', // Red mask for visualization
createMask: (ctx, image) => {
// Mask a specific region (e.g., a date display area)
ctx.fillRect(50, 100, 300, 50);
}
};
expect(screenshot).toMatchImageSnapshot({
customSnapshotIdentifier: 'homepage-with-masked-area',
customDiffConfig: { threshold: 0.1 },
comparisonMethod: 'ssim', // Uses structural similarity, better for masked areas
customSnapshotsDir: './__image_snapshots__/masked',
...maskConfig,
});
});
This approach:
- Creates a mask that ignores a specific region (x: 50, y: 100, width: 300, height: 50)
- Uses a red color to highlight the masked area in the diff image
- Switches to the SSIM comparison method, which is better for masked comparisons
- Stores these special snapshots in a separate directory
Example Test Report ๐
When a visual regression test fails, Jest provides a clear report in your terminal. Here's an example of what you might see:
FAIL ./homepage.test.js
Visual regression tests
โ should match homepage screenshot (1244ms)
โ should detect changes on a different page (1357ms)
โ Visual regression tests โบ should detect changes on a different page
Expected image to match or be a close match to snapshot but was 35.24% different from snapshot (threshold: 0.01%).
See diff for details: /path/to/project/__image_snapshots__/__diff_output__/homepage-test-js-visual-regression-tests-should-detect-changes-on-a-different-page-1-diff.png
The diff output is a visual image that shows exactly what changed:
[Diff Image: Red areas show removed content, green areas show added content]
This makes it easy to:
- Quickly identify what changed
- Decide if the changes were intentional
- Update the baseline if needed (
jest -u
) or fix the regression
Integrating with CI/CD ๐
Visual regression tests are most valuable when integrated into your CI/CD pipeline. Here's a simple example for GitHub Actions:
# .github/workflows/visual-tests.yml
name: Visual Regression Tests
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
visual-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Node.js
uses: actions/setup-node@v3
with:
node-version: '16'
- name: Install dependencies
run: npm ci
- name: Run visual regression tests
run: npm test
env:
SCRNIFY_API_KEY: ${{ secrets.SCRNIFY_API_KEY }}
- name: Upload diff images on failure
uses: actions/upload-artifact@v3
if: failure()
with:
name: visual-test-diffs
path: __image_snapshots__/__diff_output__
This workflow:
- Runs on push to main and on pull requests
- Sets up Node.js and installs dependencies
- Runs the visual tests with the API key from GitHub secrets
- If tests fail, uploads the diff images as artifacts for review
Advanced Features ๐
Testing Responsive Designs
Test how your site looks at different screen sizes:
// responsive.test.js
const { toMatchImageSnapshot } = require('jest-image-snapshot');
const { takeScreenshot } = require('./screenshot');
expect.extend({ toMatchImageSnapshot });
describe('Responsive design tests', () => {
const devices = [
{ name: 'mobile', width: 375, height: 667 },
{ name: 'tablet', width: 768, height: 1024 },
{ name: 'desktop', width: 1440, height: 900 }
];
devices.forEach(device => {
it(`should look correct on ${device.name}`, async () => {
const screenshot = await takeScreenshot('https://example.com', {
width: device.width,
height: device.height,
fullPage: false,
});
expect(screenshot).toMatchImageSnapshot({
customSnapshotIdentifier: `homepage-${device.name}`,
});
});
});
});
Testing User Interactions
You can use Scrnify's API with other tools to test how your application looks after user interactions:
// interaction.test.js
const puppeteer = require('puppeteer');
const { toMatchImageSnapshot } = require('jest-image-snapshot');
const { takeScreenshot } = require('./screenshot');
expect.extend({ toMatchImageSnapshot });
describe('User interaction tests', () => {
it('should show dropdown menu correctly', async () => {
// First get a baseline screenshot
const beforeScreenshot = await takeScreenshot('https://your-site.com', {
width: 1280,
height: 800,
});
// Launch Puppeteer locally to interact with the page
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.setViewport({ width: 1280, height: 800 });
await page.goto('https://your-site.com');
// Perform an interaction (click a dropdown menu)
await page.click('#dropdown-trigger');
await page.waitForSelector('#dropdown-menu', { visible: true });
// Take a screenshot of the interactive state using Scrnify
// (In a real scenario, you might need to use a custom implementation
// to capture the state after the interaction)
const afterUrl = page.url();
await browser.close();
const afterScreenshot = await takeScreenshot(afterUrl, {
width: 1280,
height: 800,
// You might need custom JS to trigger the dropdown
// For example with the evaluateBeforeCapture option (if available)
});
// Compare with baseline
expect(beforeScreenshot).toMatchImageSnapshot({
customSnapshotIdentifier: 'before-dropdown',
});
expect(afterScreenshot).toMatchImageSnapshot({
customSnapshotIdentifier: 'after-dropdown',
});
});
});
Conclusion ๐ฌ
Visual regression testing with Scrnify and Jest offers a powerful way to catch UI bugs before they reach production. By automating the process of comparing screenshots, you can:
- Spend less time on manual testing
- Catch visual regressions early in the development cycle
- Maintain consistent UI across your application
- Have confidence when refactoring CSS or making design changes
The combination of Scrnify's high-quality screenshot API and Jest's powerful testing capabilities gives you a robust solution for detecting even the smallest visual changes in your application.
Get free access to the SCRNIFY API during our open beta and start generating screenshots today! scrnify.com
Have you implemented visual regression testing in your projects? What challenges did you face? Let us know in the comments below!
Cheers, Laura & Heidi ๐ฆ๐น
P.S. Interested in exploring more testing techniques and browser automation? Check out our documentation at scrnify.com/docs and follow us on Twitter @scrnify for the latest updates and tips on screenshot automation!