Giter VIP home page Giter VIP logo

amazon-reviews-crawler's Introduction

Amazon Reviews Crawler

Crawls product reviews from Amazon.

Installation

Via npm:

npm install amazon-reviews-crawler

Or Yarn:

yarn add amazon-reviews-crawler

Usage

Load the module

var reviewsCrawler = require('amazon-reviews-crawler')

Get reviews by a product ASIN

reviewsCrawler('0062472100')
	.then(function(results){
		console.log(results)
	})
	.catch(function(err){
		console.error(err)
	})

This will return an object containing the title of the product and an array of review data.

Example of a return:

{
	title: "Product Name",
	reviews: [
		{
			id: "R16DIYH5RRPEWK",
			title: "Review Title",
			rating: 5,
			text: "The product review body text.",
			author: "Reviewer Name"
			date: "October 26, 2016",
			link: "https://www.amazon.com/gp/customer-reviews/R16DIYH5RRPEWK/ref=cm_cr_arp_d_rvw_ttl?ie=UTF8&ASIN=0062472100"
		}
	]
}

Options

Options can also be provided to change the user agent string, review page, or elements being crawled.

Example:

reviewsCrawler('0062472100', {
		page: 'https://www.amazon.com/product-reviews/{{asin}}',
		userAgent: 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0',
		elements: {
			// Searches whole page
			productTitle: '.product-title',
			reviewBlock: '.review',

			// Searches within elements.reviewBlock
			link: 'a',
			title: '.review-title',
			rating: '.review-rating',
			ratingPattern: 'a-star-',
			text: '.review-text',
			author: '.review-byline a',
			date: '.review-date'
		},

		// Stops crawling when it hits a particular review ID
		// Useful for only crawling new reviews
		stopAtReviewId: false
	})
	.then(console.log)
	.catch(console.error)

amazon-reviews-crawler's People

Contributors

escaladesports-svc avatar kennedyrose avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

amazon-reviews-crawler's Issues

npm install fails

NPM install fails with:
npm ERR! code 1
npm ERR! Command failed: /usr/local/bin/git checkout 0.1.3
npm ERR! error: pathspec '0.1.3' did not match any file(s) known to git.
npm ERR!

npm ERR! A complete log of this run can be found in:
npm ERR! /Users/310189953/.npm/_logs/2017-08-25T12_32_32_881Z-debug.log

Getting

Hi. I'd like to understand which URL is being used to pull the reviews from. When I run the example, I get a seemingly random assortment of reviews, many of which are not displayed either in the homepage or reviews list.

Also, is there a way to crawl through all different review pages? Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.