crawldiff's Introduction

crawldiff

crawlldiff will help with migrating websites from one system to another. Will crawl a website and compare it to another, reporting on any differences.

Installation

You can optionally choose to install this tool locally, or just execute it using npx and npm will automatically download it on first use.

To install it globally, use:

  npm install -g crawldiff

Or as part of a specific project, use:

  npm install --save-dev crawldiff

Configuration

No configuration necessary.

Usage

Usage: npx crawldiff [options] <start> <comparison>

Crawls a website and compares each page to a different host.

Arguments:

Argument	Description
start	the initial page to start crawling (e.g. https://www.example.com/)
comparison	the secondary host to compare against (e.g. https://new.example.com/ )

Options:

Option	Description
-v, --version	output the version number
-s, --similarity <0.01-1.0>	specifiy the minimum required similarity between hosts (default: 1)
-t, --timeout	the maximum time in seconds to run the crawler (default: 60)
-r, --resources	include static resources (such as scripts and styles) in comparison
-d, --detailed	report full detailed diffs for each problem
-m, --mobile	identify as a mobile device via user-agent
-w, --warmup-cache	makes two requests to each URL and compares only the second. This ensures that you are always comparing the warmed (cached) version of the page
-V, --verbose	provide more detailed output while running
-D, --debug	provide debug level output while running
-h, --help	display help for command