Configuration version: <code class="notranslate"

Hey <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url=

Thanks <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-u

The bug was fixed and released in <a href="https://github.com/website-scraper/node-web

Hi <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="

Running `website-scraper` Without a `package.json` in the Working Directory Triggers Error about node-website-scraper HOT 5 CLOSED

manuth commented on June 11, 2024

Running `website-scraper` Without a `package.json` in the Working Directory Triggers Error

from node-website-scraper.

Comments (5)

manuth commented on June 11, 2024 1

Hey @s0ph1e
Thanks for the rapid answer 😄

Sorry for the inconvenience, I'll describe the steps for the reproduction real quick.

Create an empty directory
```
mkdir scrape-test && cd scrape-test
```
Create package.json as described below
Run npm install
Create index.js file as described below
Run the script using node ./index.js
Take note that everything works as expected
cd into a directory which does not contain a package.json (for example cd ..)
Run the script again node ./scrape-test/index.js
Take note that an running the script triggers an error now

Assets

package.json:

{
  "type": "module",
  "dependencies": {
    "website-scraper": "^5.3.0"
  },
  "devDependencies": {
    "@types/node": "^18.7.23",
    "@types/website-scraper": "^1.2.6"
  }
}

index.js

import { join } from "path";
import { fileURLToPath } from "url";
import websiteScraper from "website-scraper";

(
    async () =>
    {
        await websiteScraper(
            {
                directory: join(fileURLToPath(new URL(".", import.meta.url)), "output"),
                urls: [
                    "https://nodejs.org"
                ]
            });
    })();