Automated Website Monitoring with Scrnify and Node.js: Complete Tutorial

3/18/2025

Hey there, Laura and Heidi here from SCRNIFY! 👋

Ever found yourself frantically refreshing your website to check if it's still up and running properly? Or maybe you've needed to track visual changes on your site over time but didn't want to manually take screenshots every hour? We've all been there! 🤦‍♀️

As developers who've built a screenshot service, we understand the challenges of reliable website monitoring. While traditional uptime monitors tell you if a site responds, they don't show you what users actually see. This is where visual monitoring comes in - capturing screenshots at regular intervals to detect visual changes, broken layouts, or unexpected content.

In this tutorial, we'll build a robust automated website monitoring system using Node.js and SCRNIFY's screenshot API. You'll learn how to:

Set up a Node.js project for website monitoring
Use SCRNIFY's API to capture high-quality screenshots
Schedule regular screenshots with cron jobs
Save screenshots with organized, date-based filenames
Send alerts when issues are detected

Let's dive in! 🏊‍♀️

Get free access to the SCRNIFY API during our open beta and start generating screenshots today! Sign up here: scrnify.com

Prerequisites

Before we begin, make sure you have:

Node.js installed (version 14 or higher)
npm or yarn for package management
A SCRNIFY API key (get yours free during the open beta)
Basic JavaScript knowledge
A website you want to monitor

Setting Up the Project

Let's start by creating our project structure. We'll need to set up a Node.js application and install the necessary dependencies.

# Create a new directory for our project
mkdir website-monitor
cd website-monitor

# Initialize a new Node.js project
npm init -y

# Install required dependencies
npm install node-cron axios fs-extra dotenv nodemailer

These packages will help us with:

node-cron: Scheduling our screenshot tasks
axios: Making HTTP requests to the SCRNIFY API
fs-extra: Enhanced file system operations
dotenv: Managing environment variables
nodemailer: Sending email alerts

Next, create a .env file in your project root to store your configuration:

# .env
SCRNIFY_API_KEY=your_api_key_here
WEBSITE_URL=https://example.com
SCREENSHOT_INTERVAL=0 */1 * * *
SCREENSHOT_DIR=./screenshots
EMAIL_USER=your-email@gmail.com
EMAIL_PASS=your-app-password
EMAIL_TO=alerts@your-domain.com
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/xxx/yyy/zzz

Make sure to replace the placeholder values with your actual credentials. The SCREENSHOT_INTERVAL follows cron syntax - the example above runs every hour.

Now let's create our main application file:

// index.js
require('dotenv').config();
const cron = require('node-cron');
const axios = require('axios');
const fs = require('fs-extra');
const path = require('path');
const { sendEmailAlert, sendSlackAlert } = require('./alerts');

// Ensure screenshots directory exists
fs.ensureDirSync(process.env.SCREENSHOT_DIR);

console.log('Website monitoring service starting...');
console.log(`Target website: ${process.env.WEBSITE_URL}`);
console.log(`Screenshot interval: ${process.env.SCREENSHOT_INTERVAL}`);

// Start the monitoring process
startMonitoring();

function startMonitoring() {
  // Take initial screenshot
  takeScreenshot()
    .then(() => {
      console.log('Initial screenshot captured successfully');
      
      // Schedule regular screenshots
      cron.schedule(process.env.SCREENSHOT_INTERVAL, () => {
        console.log('Running scheduled screenshot task...');
        takeScreenshot()
          .then(result => {
            if (result.changed) {
              console.log('Changes detected in website appearance!');
              sendAlerts(result.currentPath, result.previousPath);
            } else {
              console.log('No significant changes detected');
            }
          })
          .catch(error => {
            console.error('Screenshot task failed:', error);
            sendAlerts(null, null, error);
          });
      });
    })
    .catch(error => {
      console.error('Failed to take initial screenshot:', error);
    });
}

This sets up our basic monitoring framework. Now let's implement the core functionality.

Taking Timed Screenshots with Scrnify

The heart of our monitoring system is capturing screenshots. Let's implement the takeScreenshot function using SCRNIFY's API:

// Continued from index.js

async function takeScreenshot() {
  try {
    // Generate filename with date
    const timestamp = new Date();
    const formattedDate = formatDateForFilename(timestamp);
    const filename = `screenshot_${formattedDate}.png`;
    const filepath = path.join(process.env.SCREENSHOT_DIR, filename);
    
    // Build the SCRNIFY API URL
    const apiUrl = buildScrnifyApiUrl(process.env.WEBSITE_URL);
    
    console.log(`Taking screenshot of ${process.env.WEBSITE_URL}`);
    
    // Call the SCRNIFY API and save the image
    const response = await axios({
      method: 'get',
      url: apiUrl,
      responseType: 'arraybuffer'
    });
    
    await fs.writeFile(filepath, response.data);
    console.log(`Screenshot saved to ${filepath}`);
    
    // Compare with previous screenshot (if available)
    const result = {
      currentPath: filepath,
      previousPath: await findPreviousScreenshot(filepath),
      changed: false
    };
    
    if (result.previousPath) {
      result.changed = await compareScreenshots(result.currentPath, result.previousPath);
    }
    
    return result;
  } catch (error) {
    console.error('Error taking screenshot:', error.message);
    throw error;
  }
}

function buildScrnifyApiUrl(websiteUrl) {
  // Construct the SCRNIFY API URL with parameters
  const params = new URLSearchParams({
    key: process.env.SCRNIFY_API_KEY,
    url: websiteUrl,
    type: 'image',
    format: 'png',
    width: 1920,
    height: 1080,
    fullPage: true,
    waitUntil: 'networkIdle'
  });
  
  return `https://api.scrnify.com/capture?${params.toString()}`;
}

function formatDateForFilename(date) {
  // Format: YYYY-MM-DD_HH-MM-SS
  return date.toISOString()
    .replace('T', '_')
    .replace(/:/g, '-')
    .split('.')[0];
}

This function does several important things:

Creates a timestamp-based filename for organization
Constructs the SCRNIFY API URL with proper parameters
Makes the API request and saves the screenshot
Prepares to compare with the previous screenshot

The SCRNIFY API offers numerous configuration options - we're using fullPage: true to capture the entire webpage and waitUntil: 'networkIdle' to ensure the page is fully loaded before taking the screenshot.

Setting Up a Cron Job

We've already set up our cron job in the starter code using node-cron. Let's examine the cron syntax in more detail:

// The cron schedule format:
// ┌──────────── second (optional)
// │ ┌────────── minute
// │ │ ┌──────── hour
// │ │ │ ┌────── day of month
// │ │ │ │ ┌──── month
// │ │ │ │ │ ┌── day of week
// │ │ │ │ │ │
// * * * * * *

// Examples:
// '*/5 * * * *'       - Every 5 minutes
// '0 */1 * * *'       - Every hour (at minute 0)
// '0 9-18 * * 1-5'    - Every hour from 9 AM to 6 PM, Monday to Friday
// '0 0 * * *'         - Once a day at midnight

You can adjust the SCREENSHOT_INTERVAL in your .env file based on your monitoring needs:

Every 5 minutes: */5 * * * *
Every hour: 0 */1 * * *
Every day at 8 AM: 0 8 * * *
Every Monday at 9 AM: 0 9 * * 1

This flexibility allows you to balance between monitoring frequency and resource usage. For critical websites, more frequent checks might be necessary, while for less critical ones, daily or weekly checks may suffice.

Saving Images with Date

We're already generating date-based filenames in our code, but let's add a function to find the previous screenshot for comparison:

// Continued from index.js

async function findPreviousScreenshot(currentPath) {
  try {
    // List all files in the screenshots directory
    const files = await fs.readdir(process.env.SCREENSHOT_DIR);
    
    // Filter only screenshot files and sort by date (newest first)
    const screenshots = files
      .filter(file => file.startsWith('screenshot_') && file.endsWith('.png'))
      .sort()
      .reverse();
    
    // Find the index of the current screenshot
    const currentFileName = path.basename(currentPath);
    const currentIndex = screenshots.indexOf(currentFileName);
    
    // If there's a previous screenshot, return its path
    if (currentIndex !== -1 && currentIndex < screenshots.length - 1) {
      return path.join(process.env.SCREENSHOT_DIR, screenshots[currentIndex + 1]);
    }
    
    return null;
  } catch (error) {
    console.error('Error finding previous screenshot:', error);
    return null;
  }
}

This function helps us organize our screenshots chronologically and access the previous one for comparison.

For more advanced organization, you could create a folder structure like:

screenshots/
├── 2025-03-11/
│   ├── screenshot_2025-03-11_08-00-00.png
│   ├── screenshot_2025-03-11_09-00-00.png
│   └── ...
├── 2025-03-12/
│   ├── screenshot_2025-03-12_08-00-00.png
│   └── ...
└── ...

Here's how you could implement this:

// Enhanced version of formatDateForFilename
function getScreenshotPath() {
  const now = new Date();
  const dateStr = now.toISOString().split('T')[0]; // YYYY-MM-DD
  const timeStr = now.toISOString().replace('T', '_').replace(/:/g, '-').split('.')[0]; // YYYY-MM-DD_HH-MM-SS
  
  const dailyDir = path.join(process.env.SCREENSHOT_DIR, dateStr);
  fs.ensureDirSync(dailyDir);
  
  const filename = `screenshot_${timeStr}.png`;
  return {
    path: path.join(dailyDir, filename),
    filename: filename,
    date: now
  };
}

This approach makes it easier to manage large numbers of screenshots over extended monitoring periods.

Comparing Screenshots and Detecting Changes

To make our monitoring system complete, we need to implement the compareScreenshots function. There are several approaches:

Pixel-by-pixel comparison: Most accurate but susceptible to minor changes
Image hashing: Creates a "fingerprint" of the image for efficient comparison
Visual difference threshold: Determines a percentage of change that's significant

For this tutorial, we'll use the simple approach of comparing image file sizes, but you could integrate with an image comparison library for more sophisticated detection:

// Continued from index.js

async function compareScreenshots(currentPath, previousPath) {
  try {
    // Get file stats for both screenshots
    const currentStats = await fs.stat(currentPath);
    const previousStats = await fs.stat(previousPath);
    
    // Calculate file size difference as a percentage
    const sizeDiff = Math.abs(currentStats.size - previousStats.size);
    const sizePercentage = (sizeDiff / previousStats.size) * 100;
    
    console.log(`Screenshot size difference: ${sizePercentage.toFixed(2)}%`);
    
    // If the difference is more than 5%, consider it a significant change
    // This threshold can be adjusted based on your needs
    return sizePercentage > 5;
  } catch (error) {
    console.error('Error comparing screenshots:', error);
    return false;
  }
}

This is a simplistic approach but works well for detecting major changes. For a production system, you might want to use a proper image comparison library like pixelmatch or resemblejs.

Report and Alert

When our monitoring system detects a significant change, we want to send alerts. Let's implement email and Slack notifications:

// alerts.js
const nodemailer = require('nodemailer');
const axios = require('axios');
const fs = require('fs-extra');
const path = require('path');

// Configure email transporter
const transporter = nodemailer.createTransport({
  service: 'gmail',
  auth: {
    user: process.env.EMAIL_USER,
    pass: process.env.EMAIL_PASS
  }
});

async function sendEmailAlert(currentScreenshot, previousScreenshot, error) {
  try {
    const subject = error 
      ? 'ALERT: Website monitoring error' 
      : 'ALERT: Website appearance changed';
    
    const text = error 
      ? `Error monitoring ${process.env.WEBSITE_URL}: ${error.message}` 
      : `Changes detected on ${process.env.WEBSITE_URL} at ${new Date().toISOString()}.`;
    
    const mailOptions = {
      from: process.env.EMAIL_USER,
      to: process.env.EMAIL_TO,
      subject: subject,
      text: text,
      html: `<p>${text}</p><p>Please check the monitoring system for details.</p>`,
      attachments: []
    };
    
    // Attach screenshots if available
    if (currentScreenshot && await fs.exists(currentScreenshot)) {
      mailOptions.attachments.push({
        filename: 'current.png',
        path: currentScreenshot
      });
    }
    
    if (previousScreenshot && await fs.exists(previousScreenshot)) {
      mailOptions.attachments.push({
        filename: 'previous.png',
        path: previousScreenshot
      });
    }
    
    // Send email
    const info = await transporter.sendMail(mailOptions);
    console.log('Email alert sent:', info.messageId);
    return true;
  } catch (error) {
    console.error('Failed to send email alert:', error);
    return false;
  }
}

async function sendSlackAlert(currentScreenshot, previousScreenshot, error) {
  // Skip if no Slack webhook configured
  if (!process.env.SLACK_WEBHOOK_URL) {
    return false;
  }
  
  try {
    const message = error 
      ? `🚨 Error monitoring ${process.env.WEBSITE_URL}: ${error.message}` 
      : `🔍 Changes detected on ${process.env.WEBSITE_URL} at ${new Date().toISOString()}.`;
    
    // Send the text notification
    await axios.post(process.env.SLACK_WEBHOOK_URL, {
      text: message,
      blocks: [
        {
          type: "section",
          text: {
            type: "mrkdwn",
            text: message
          }
        },
        {
          type: "context",
          elements: [
            {
              type: "mrkdwn",
              text: `*Monitor:* Website Screenshot Monitor\n*Target:* ${process.env.WEBSITE_URL}`
            }
          ]
        }
      ]
    });
    
    console.log('Slack alert sent');
    return true;
  } catch (error) {
    console.error('Failed to send Slack alert:', error);
    return false;
  }
}

// Combined function to send all alerts
async function sendAlerts(currentScreenshot, previousScreenshot, error) {
  const emailSent = await sendEmailAlert(currentScreenshot, previousScreenshot, error);
  const slackSent = await sendSlackAlert(currentScreenshot, previousScreenshot, error);
  
  return { emailSent, slackSent };
}

module.exports = {
  sendEmailAlert,
  sendSlackAlert,
  sendAlerts
};

And in our main file, let's add:

// Back in index.js, update the startMonitoring function

const { sendAlerts } = require('./alerts');

// Then use it in the appropriate place:
if (result.changed) {
  console.log('Changes detected in website appearance!');
  sendAlerts(result.currentPath, result.previousPath);
} else {
  console.log('No significant changes detected');
}

You can also extend this with Discord notifications, SMS alerts, or any other notification method that suits your workflow.

Putting It All Together

Let's make sure our project structure is organized:

website-monitor/
├── .env                 # Configuration variables
├── index.js             # Main application
├── alerts.js            # Alert notification functions
├── package.json         # Project dependencies
├── screenshots/         # Where screenshots are saved
└── README.md            # Project documentation

To run the monitoring system:

node index.js

For production use, you might want to run it with a process manager like PM2:

# Install PM2
npm install -g pm2

# Start the monitoring service
pm2 start index.js --name website-monitor

# Make it restart on server reboot
pm2 save
pm2 startup

Advanced Features

Here are some ideas to enhance your website monitoring system:

Visual diff highlighting: Generate images that highlight the differences between screenshots
Dashboard: Create a simple web interface to view recent screenshots and alerts
Multiple website monitoring: Extend the system to monitor multiple URLs
Performance tracking: Measure load times and report performance degradation
Custom element monitoring: Track specific elements on the page instead of the entire page
Machine learning: Train a model to identify "normal" variations vs. actual issues

Conclusion

Congratulations! 🎉 You've built a complete website monitoring system using Node.js and SCRNIFY. This system helps you:

Automatically capture screenshots of your website at regular intervals
Detect visual changes that might indicate problems
Receive immediate alerts when issues occur
Maintain a visual history of your website

This approach to monitoring goes beyond simple uptime checks by showing you exactly what your users see. It's particularly valuable for:

E-commerce sites where product availability and pricing are critical
News sites that need to verify content is displaying correctly
Applications where visual layout is important
Detecting unauthorized changes that might indicate a security breach

Now you can sleep better at night knowing your website is being monitored around the clock! ⏰