Visual Regression Testing with Scrnify and Jest (JavaScript)

Hey there! Laura and Heidi here from SCRNIFY! 👋

Do you find yourself manually checking your web applications for visual bugs? Are you tired of discovering that a CSS change broke your UI layout only after it's been deployed to production? We've been there too! 🤦‍♂️

As developers who've built a screenshot service, we know firsthand how challenging it can be to catch visual regressions before they reach your users. That's why today we're diving into visual regression testing with Jest and our Scrnify API.

In this tutorial, you'll learn how to set up automated visual comparison tests that will catch even the slightest pixel changes in your UI. No more "it looked fine on my machine" moments! Let's get started and get free access to the SCRNIFY API during our open beta to follow along.

What is Visual Regression Testing and Why is it Important? 🔍

Visual regression testing is a technique that helps you detect unintended visual changes in your UI. Unlike functional tests that check if your code works, visual tests ensure your application still looks right.

Here's why it matters:

• ✅ Catches what functional tests miss: Unit and integration tests don't verify visual elements
• ✅ Prevents CSS regressions: Small CSS changes can have big visual impacts
• ✅ Ensures cross-browser consistency: Identifies rendering differences across browsers
• ✅ Improves user experience: Maintains consistent design and branding
• ✅ Reduces manual testing time: Automates the tedious process of visual inspection

As applications grow more complex, it becomes increasingly difficult to manually catch every visual issue. Automated visual testing gives you confidence that your UI elements appear as expected, even after significant code changes.

Prerequisites 🛠️

Before we dive in, make sure you have the following:

• Node.js (v14 or higher) installed
• A basic understanding of JavaScript and Jest testing
• A Scrnify API key (you can sign up for the beta)
• A project with Jest installed or a willingness to set one up

Let's set up our project:

# Create a new project directory
mkdir visual-regression-tests
cd visual-regression-tests

# Initialize a new Node.js project
npm init -y

# Install required dependencies
npm install --save-dev jest jest-image-snapshot node-fetch@2 dotenv

Note: We're using node-fetch@2 because it provides a simpler CommonJS import pattern. If you're using ES modules, you can use the latest version.

Setting up a Jest Test 🧪

First, let's create a basic Jest configuration. Create a file named jest.config.js in your project root:

// jest.config.js
module.exports = {
  testTimeout: 30000, // Screenshots might take time, so we increase the timeout
  testMatch: ['**/*.test.js'],
};

Next, let's create a .env file to store our Scrnify API key:

SCRNIFY_API_KEY=your_api_key_here

Don't forget to add this file to your .gitignore to avoid committing your API keys!

Now, let's create a simple utility to interact with the Scrnify API. Create a file named screenshot.js:

// screenshot.js
const fetch = require('node-fetch');
const dotenv = require('dotenv');

dotenv.config();

const API_KEY = process.env.SCRNIFY_API_KEY;
const BASE_URL = 'https://api.scrnify.com/capture';

/**
 * Takes a screenshot of a webpage using Scrnify API
 * @param {string} url - The URL to capture
 * @param {Object} options - Screenshot options
 * @returns {Promise<Buffer>} - The screenshot as a buffer
 */
async function takeScreenshot(url, options = {}) {
  // Default options
  const defaultOptions = {
    type: 'image',
    format: 'png',
    width: 1920,
    height: 1080,
    fullPage: false,
    waitUntil: 'networkIdle',
  };

  // Merge default options with user-provided options
  const mergedOptions = { ...defaultOptions, ...options };

  // Create query parameters
  const params = new URLSearchParams({
    key: API_KEY,
    url: encodeURIComponent(url),
    ...mergedOptions,
  });

  try {
    // Make the API request
    const response = await fetch(`${BASE_URL}?${params.toString()}`);

    if (!response.ok) {
      const error = await response.text();
      throw new Error(`Screenshot API error (${response.status}): ${error}`);
    }

    // Return the image as a buffer
    return await response.buffer();
  } catch (error) {
    console.error('Failed to take screenshot:', error);
    throw error;
  }
}

module.exports = { takeScreenshot };

Taking a Baseline Screenshot 📸

Now that we have our utility set up, let's create our first test that will take a baseline screenshot. Create a file named homepage.test.js:

// homepage.test.js
const { toMatchImageSnapshot } = require('jest-image-snapshot');
const { takeScreenshot } = require('./screenshot');

// Extend Jest's expect
expect.extend({ toMatchImageSnapshot });

describe('Visual regression tests', () => {
  it('should match homepage screenshot', async () => {
    // Take a screenshot of our test page
    const screenshot = await takeScreenshot('https://example.com', {
      width: 1280,
      height: 800,
      fullPage: true,
    });

    // Compare with the baseline or create one if it doesn't exist
    expect(screenshot).toMatchImageSnapshot();
  });
});

Run the test with:

npx jest

The first time you run this test, Jest will create a directory called __image_snapshots__ with your baseline screenshot. Since there's no existing baseline to compare against, the test will pass and save the current state as the reference.

This is what happens behind the scenes:

1. Scrnify API captures a high-quality screenshot of the webpage
2. Jest saves this screenshot as the baseline in __image_snapshots__
3. In future test runs, new screenshots will be compared against this baseline

Making a Change 🔄

To demonstrate how visual regression testing works, let's simulate a scenario where we've made changes to our website. Instead of actually modifying a website, we'll use a different URL for our "changed" version.

Update the test to include a second test case:

// homepage.test.js
const { toMatchImageSnapshot } = require('jest-image-snapshot');
const { takeScreenshot } = require('./screenshot');

expect.extend({ toMatchImageSnapshot });

describe('Visual regression tests', () => {
  it('should match homepage screenshot', async () => {
    const screenshot = await takeScreenshot('https://example.com', {
      width: 1280,
      height: 800,
      fullPage: true,
    });

    expect(screenshot).toMatchImageSnapshot();
  });

  it('should detect changes on a different page', async () => {
    // Using a different page to simulate changes
    const screenshot = await takeScreenshot('https://example.org', {
      width: 1280,
      height: 800,
      fullPage: true,
    });

    expect(screenshot).toMatchImageSnapshot();
  });
});

If you run the tests now, the first test should pass (as we're using the same URL as before), but the second test will fail because example.org looks different from the baseline image of example.com.

Comparing Screenshots 🔍

Jest-image-snapshot is doing the heavy lifting of comparing the screenshots for us. When it detects differences, it:

1. Creates a visual diff highlighting the changes
2. Saves the diff image in a __diff_output__ directory
3. Fails the test with information about the differences

The diff images use a red/green color scheme to highlight the differences:

• Red pixels show elements that were removed or changed
• Green pixels show elements that were added or changed
• Unchanged areas retain their original appearance

This makes it easy to spot even subtle changes in your UI.

Handling Differences ⚙️

Not all visual differences are bugs. Sometimes you intentionally change your UI. Jest-image-snapshot provides options to handle these scenarios:

Updating Baselines for Intentional Changes

When you make intentional UI changes, you can update your baselines by running:

npx jest -u

This tells Jest to replace the old baselines with the new screenshots.

Setting Custom Thresholds

Sometimes minor pixel differences are acceptable. Let's modify our test to allow for some flexibility:

// homepage.test.js
it('should match homepage screenshot with customized threshold', async () => {
  const screenshot = await takeScreenshot('https://example.com', {
    width: 1280,
    height: 800,
    fullPage: true,
  });

  expect(screenshot).toMatchImageSnapshot({
    customDiffConfig: {
      threshold: 0.1, // Higher threshold means less sensitive to minor changes
    },
    failureThreshold: 0.01, // The test will fail only if more than 1% of pixels are different
    failureThresholdType: 'percent',
  });
});

This configuration:

• Sets the per-pixel threshold to 0.1 (pixels must be significantly different to count)
• Sets the overall threshold to 0.01 (test fails only if more than 1% of pixels differ)
• Uses percentage as the failure threshold type (you can also use 'pixel')

Ignoring Specific Areas

Sometimes your page contains dynamic content like dates or ads that will always cause differences. You can create a customized matcher to ignore specific regions:

// Custom matcher that masks a specific region
it('should ignore dynamic content areas', async () => {
  const screenshot = await takeScreenshot('https://example.com', {
    width: 1280,
    height: 800,
    fullPage: true,
  });

  // Define a rectangle to mask (x, y, width, height)
  const maskConfig = {
    diffMaskColor: '#FF0000', // Red mask for visualization
    createMask: (ctx, image) => {
      // Mask a specific region (e.g., a date display area)
      ctx.fillRect(50, 100, 300, 50);
    }
  };

  expect(screenshot).toMatchImageSnapshot({
    customSnapshotIdentifier: 'homepage-with-masked-area',
    customDiffConfig: { threshold: 0.1 },
    comparisonMethod: 'ssim', // Uses structural similarity, better for masked areas
    customSnapshotsDir: './__image_snapshots__/masked',
    ...maskConfig,
  });
});

This approach:

1. Creates a mask that ignores a specific region (x: 50, y: 100, width: 300, height: 50)
2. Uses a red color to highlight the masked area in the diff image
3. Switches to the SSIM comparison method, which is better for masked comparisons
4. Stores these special snapshots in a separate directory

Example Test Report 📊

When a visual regression test fails, Jest provides a clear report in your terminal. Here's an example of what you might see:

FAIL  ./homepage.test.js
  Visual regression tests
    ✓ should match homepage screenshot (1244ms)
    ✕ should detect changes on a different page (1357ms)

  ● Visual regression tests › should detect changes on a different page

    Expected image to match or be a close match to snapshot but was 35.24% different from snapshot (threshold: 0.01%).

    See diff for details: /path/to/project/__image_snapshots__/__diff_output__/homepage-test-js-visual-regression-tests-should-detect-changes-on-a-different-page-1-diff.png

The diff output is a visual image that shows exactly what changed:

[Diff Image: Red areas show removed content, green areas show added content]

This makes it easy to:

1. Quickly identify what changed
2. Decide if the changes were intentional
3. Update the baseline if needed (jest -u) or fix the regression

Integrating with CI/CD 🔄

Visual regression tests are most valuable when integrated into your CI/CD pipeline. Here's a simple example for GitHub Actions:

# .github/workflows/visual-tests.yml
name: Visual Regression Tests

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  visual-test:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Set up Node.js
      uses: actions/setup-node@v3
      with:
        node-version: '16'

    - name: Install dependencies
      run: npm ci

    - name: Run visual regression tests
      run: npm test
      env:
        SCRNIFY_API_KEY: ${{ secrets.SCRNIFY_API_KEY }}

    - name: Upload diff images on failure
      uses: actions/upload-artifact@v3
      if: failure()
      with:
        name: visual-test-diffs
        path: __image_snapshots__/__diff_output__

This workflow:

1. Runs on push to main and on pull requests
2. Sets up Node.js and installs dependencies
3. Runs the visual tests with the API key from GitHub secrets
4. If tests fail, uploads the diff images as artifacts for review

Advanced Features 🚀

Testing Responsive Designs

Test how your site looks at different screen sizes:

// responsive.test.js
const { toMatchImageSnapshot } = require('jest-image-snapshot');
const { takeScreenshot } = require('./screenshot');

expect.extend({ toMatchImageSnapshot });

describe('Responsive design tests', () => {
  const devices = [
    { name: 'mobile', width: 375, height: 667 },
    { name: 'tablet', width: 768, height: 1024 },
    { name: 'desktop', width: 1440, height: 900 }
  ];

  devices.forEach(device => {
    it(`should look correct on ${device.name}`, async () => {
      const screenshot = await takeScreenshot('https://example.com', {
        width: device.width,
        height: device.height,
        fullPage: false,
      });

      expect(screenshot).toMatchImageSnapshot({
        customSnapshotIdentifier: `homepage-${device.name}`,
      });
    });
  });
});

Testing User Interactions

You can use Scrnify's API with other tools to test how your application looks after user interactions:

// interaction.test.js
const puppeteer = require('puppeteer');
const { toMatchImageSnapshot } = require('jest-image-snapshot');
const { takeScreenshot } = require('./screenshot');

expect.extend({ toMatchImageSnapshot });

describe('User interaction tests', () => {
  it('should show dropdown menu correctly', async () => {
    // First get a baseline screenshot
    const beforeScreenshot = await takeScreenshot('https://your-site.com', {
      width: 1280,
      height: 800,
    });

    // Launch Puppeteer locally to interact with the page
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.setViewport({ width: 1280, height: 800 });
    await page.goto('https://your-site.com');

    // Perform an interaction (click a dropdown menu)
    await page.click('#dropdown-trigger');
    await page.waitForSelector('#dropdown-menu', { visible: true });

    // Take a screenshot of the interactive state using Scrnify
    // (In a real scenario, you might need to use a custom implementation
    // to capture the state after the interaction)
    const afterUrl = page.url();
    await browser.close();

    const afterScreenshot = await takeScreenshot(afterUrl, {
      width: 1280,
      height: 800,
      // You might need custom JS to trigger the dropdown
      // For example with the evaluateBeforeCapture option (if available)
    });

    // Compare with baseline
    expect(beforeScreenshot).toMatchImageSnapshot({
      customSnapshotIdentifier: 'before-dropdown',
    });

    expect(afterScreenshot).toMatchImageSnapshot({
      customSnapshotIdentifier: 'after-dropdown',
    });
  });
});

Conclusion 🎬

Visual regression testing with Scrnify and Jest offers a powerful way to catch UI bugs before they reach production. By automating the process of comparing screenshots, you can:

1. Spend less time on manual testing
2. Catch visual regressions early in the development cycle
3. Maintain consistent UI across your application
4. Have confidence when refactoring CSS or making design changes

The combination of Scrnify's high-quality screenshot API and Jest's powerful testing capabilities gives you a robust solution for detecting even the smallest visual changes in your application.

Get free access to the SCRNIFY API during our open beta and start generating screenshots today! scrnify.com

Have you implemented visual regression testing in your projects? What challenges did you face? Let us know in the comments below!

Cheers, Laura & Heidi 🇦🇹

P.S. Interested in exploring more testing techniques and browser automation? Check out our documentation at scrnify.com/docs and follow us on Twitter @scrnify for the latest updates and tips on screenshot automation!

Resources

• Jest Documentation
• jest-image-snapshot Documentation
• Scrnify API Documentation