node-scrapebp
the npm package
node-scrapebp
is DEPRECATED, source code of package can be found indeprecated
branch
with modern stack, there's not need for any wrapper library
this repo now serves as boilerplate for a Node.js scraper
Usage
git clone https://github.com/leesei/node-scrapebp/
npm install
# OR
yarn install
./index.js $URL
index.js
is the boilerplate code for setting up fetch()
and cheerio
.
You should implement a new Scraper to scrape your site and modify this line:
const scraper = require('./DemoScraper');
Note on Character Encoding
node-fetch@2
's Body.text()
API no longer do character conversion.
You have to :
npm install --save encoding
- use
Body.textConverted()
to detect encoding and convert to UTF8