site-screenshot-script

Crawl a list of URLs and capture a full-page desktop + mobile screenshot of every page, plus a basic design-token audit (every distinct color and font-family actually rendered on the page).

Useful for visual QA before a migration/redesign, or for archiving what a site looked like at a point in time.

Requirements

• Node.js 20+
• No other tools required — Puppeteer downloads its own headless Chromium on npm install.

Install

git clone <this-repo-url>
cd site-screenshot-script
npm install

Usage

node crawl-and-screenshot.js --input <path> [options]

You need a list of URLs to feed it. Two input formats are supported:

Option A: plain text file (easiest, no extra tools)

Create a file with one URL per line (blank lines and # comments are ignored):

# urls.txt
https://example.com/
https://example.com/about/
https://example.com/pricing/

node crawl-and-screenshot.js --input urls.txt

Option B: Screaming Frog CSV export

If you already crawl the site with Screaming Frog SEO Spider (free for up to 500 URLs), export the Internal → HTML tab as internal_all.csv. The script auto-detects this format and filters to pages that are Status Code 200, Content Type text/html, and Indexability = Indexable.

node crawl-and-screenshot.js --input internal_all.csv

Options

Flag	Default	Description
-i, --input <path>	(required)	CSV or plain URL list
-o, --out <dir>	./output	Output directory
-d, --delay <ms>	500	Delay between page loads (politeness)
-s, --skip <regex>	(none, repeatable)	Skip URLs matching this regex, e.g. -s "/blog/"
--desktop <WxH>	1440x900	Desktop viewport size
--mobile <WxH>	390x844	Mobile viewport size
-h, --help		Show usage

Some file types and /api/ paths are always skipped by default (.pdf, .jpg, .png, .zip, etc.) even in plain-list mode.

Examples

# Basic run
node crawl-and-screenshot.js --input urls.txt

# Custom output dir, slower delay, skip anything under /blog/
node crawl-and-screenshot.js --input urls.txt --out ./shots --delay 1000 --skip "/blog/"

# Wider desktop viewport
node crawl-and-screenshot.js --input urls.txt --desktop 1920x1080

Output

output/
  routes.json                 <- every screenshotted URL + HTTP status
  design-tokens.json          <- every color/font-family seen across all pages
  screenshots/
    <slug>/
      desktop.png
      mobile.png

Each URL's path becomes its <slug> folder name, e.g. /about/ → about.

License

MIT