Giter VIP home page Giter VIP logo

seo-audit-tool's Introduction

Features

This SEO audit tool can crawl all the web pages on your website and alert you to errors or issues that could affect SEO performance.

Input

Web URL is the most important input that you should set, as this is address is where the tool will start crawling.

Results

All results are stored in Apify datasets. Each result has the following attributes:

  • url - URL of page
  • title - title of page
  • isLoaded - true if page was loaded properly
  • isGoogleAnalyticsObject - check if Google Analytics object is included in the page
  • isGoogleAnalyticsFunc - check if Google Analytics function is included in the page
  • isCharacterEncode - check if page has meta tag with charset attribute
  • isMetaDescription - check if page has meta description
  • metaDescription - value from meta description
  • isMetaDescriptionEnoughLong - check if length of meta description is long enough
  • isDoctype - check if doctype is included in page
  • isTitle - check if title element is included in page
  • isTitleEnoughLong - check if title is long enough
  • isH1 - check if H1 element is present on page
  • h1 - value from H1 element
  • isH1OnlyOne - check that there is only one H1 on page
  • isH2 - check if H2 element is present on page
  • linksCount - number of links on page
  • isTooEnoughLinks - check if there are enough links on page
  • internalNoFollowLinks - list of no-follow links on page
  • internalNoFollowLinksCount - number of no-follow links on page
  • notOptimizedImages - list of unoptimized images (does not include alt tag)
  • notOptimizedImagesCount - number of unoptimized images
  • wordsCount - number of words on page
  • isContentEnoughLong - check if content is long enough
  • isViewport - check if meta tag viewport is set on page
  • isAmp - check if amp tag is set on page
  • isNotIframe - check that there is no iframe
  • brokenLinks - list of broken links on page
  • brokenLinksCount - number of broken links on page
  • brokenImages - list of broken images on page
  • brokenImagesCount - number of broken image links on page
  • jsonLd - data from JSON-LD specifications
  • microdata - data from microdata specifications

Example result

{
  "url": "https://www.apify.com/",
  "title": "Web Scraping, Data Extraction and Automation - Apify",
  "isLoaded": true,
  "isGoogleAnalyticsObject": true,
  "isGoogleAnalyticsFunc": false,
  "isCharacterEncode": true,
  "isMetaDescription": true,
  "metaDescription": "Apify extracts data from websites, crawls lists of URLs and automates workflows on the web. Turn any website into an API in a few minutes!",
  "isMetaDescriptionEnoughLong": true,
  "isDoctype": true,
  "isTitle": true,
  "isTitleEnoughLong": true,
  "isH1": true,
  "h1": "The web scraping and automation platform",
  "isH1OnlyOne": true,
  "isH2": true,
  "linksCount": 91,
  "isTooEnoughlinks": true,
  "internalNoFollowLinks": [],
  "internalNoFollowLinksCount": 0,
  "notOptimizedImgs": [],
  "notOptimizedImagesCount": 0,
  "wordsCount": 1373,
  "isContentEnoughLong": false,
  "isViewport": true,
  "isAmp": true,
  "isNotIframe": false,
  "brokenLinks": [],
  "brokenLinksCount": 0,
  "brokenImages": [],
  "brokenImagesCount": 0,
  "jsonLd": {
    "isJsonLd": false,
    "jsonLdData": {}
  },
  "microdata": {
    "isMicrodata": false,
    "microdata": []
  }
}

seo-audit-tool's People

Contributors

drobnikj avatar metalwarrior665 avatar pocesar avatar zpelechova avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar

Forkers

kamalsharma9392 smilefounder camilleroux redoper1 faizkhan18 brainedcode vundavallibalakrishna fairhopeweb zpelechova kekewind hxlxmjxbbxs hishamazad wlleroux84 ryntab quicklifesolutions

seo-audit-tool's Issues

Various improvements

There should be a limit on the number of pages crawled.

isTooEnoughLinks is not a very good English.

Also, the actor could have proxy configuration input option.

Build is failing

Version : 0.1 (latest)

2019-05-23T14:23:02.684Z ACTOR: Cloning https://github.com/drobnikj/seo-audit-tool
2019-05-23T14:23:02.700Z Cloning into '/tmp/apify-job-YzXH3szHdLrLWge7C-aSWNOo'...
2019-05-23T14:23:04.449Z ACTOR: Extracting actor documentation from README.md
2019-05-23T14:23:04.453Z ACTOR: Extracting input schema from INPUT_SCHEMA.json
2019-05-23T14:23:04.534Z ACTOR: Building Docker image
2019-05-23T14:23:04.687Z Step 1/4 : FROM apify/actor-node-puppeteer
2019-05-23T14:23:04.893Z  ---> 6f1b75505b12
2019-05-23T14:23:04.895Z Step 2/4 : COPY . ./
2019-05-23T14:23:07.029Z  ---> 68bbd02ad453
2019-05-23T14:23:07.031Z Step 3/4 : RUN npm --quiet set progress=false  && npm install --only=prod --no-optional  && echo "Installed NPM packages:"  && npm list  && echo "Node.js version:"  && node --version  && echo "NPM version:"  && npm --version
2019-05-23T14:23:07.847Z  ---> Running in 2cef078fd85a
2019-05-23T14:23:12.946Z �[91mnpm WARN tar�[0m
2019-05-23T14:23:12.947Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/package.json'
2019-05-23T14:23:12.949Z �[0m
2019-05-23T14:23:12.999Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/all.js'
2019-05-23T14:23:13.000Z �[0m
2019-05-23T14:23:13.018Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/allLimit.js'
2019-05-23T14:23:13.019Z �[0m
2019-05-23T14:23:13.029Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/allSeries.js'
2019-05-23T14:23:13.030Z �[0m
2019-05-23T14:23:13.042Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/any.js'
2019-05-23T14:23:13.044Z �[0m
2019-05-23T14:23:13.046Z �[91mnpm WARN tar�[0m
2019-05-23T14:23:13.047Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/apify-shared-7e5a51dd/linked_list.js'
2019-05-23T14:23:13.049Z �[0m
2019-05-23T14:23:13.050Z �[91mnpm WARN �[0m
2019-05-23T14:23:13.052Z �[91mtar�[0m
2019-05-23T14:23:13.053Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/graceful-fs-02c695a6/legacy-streams.js'
2019-05-23T14:23:13.055Z �[0m
2019-05-23T14:23:13.057Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/anyLimit.js'
2019-05-23T14:23:13.058Z �[0m
2019-05-23T14:23:13.076Z �[91mnpm WARN tar�[0m
2019-05-23T14:23:13.078Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/request-promise-a27ef81a/LICENSE'
2019-05-23T14:23:13.080Z �[0m
2019-05-23T14:23:13.082Z �[91mnpm WARN �[0m
2019-05-23T14:23:13.083Z �[91mtar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/graceful-fs-02c695a6/graceful-fs.js'
2019-05-23T14:23:13.085Z �[0m
2019-05-23T14:23:13.087Z �[91mnpm WARN�[0m
2019-05-23T14:23:13.091Z �[91m tar�[0m
2019-05-23T14:23:13.093Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/glob-c81cce02/LICENSE'
2019-05-23T14:23:13.094Z �[0m
2019-05-23T14:23:13.096Z �[91mnpm�[0m
2019-05-23T14:23:13.098Z �[91m WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/anySeries.js'
2019-05-23T14:23:13.099Z �[0m
2019-05-23T14:23:13.101Z �[91mnpm WARN�[0m
2019-05-23T14:23:13.102Z �[91m �[0m
2019-05-23T14:23:13.104Z �[91mtar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/commander-9971a428/LICENSE'
2019-05-23T14:23:13.105Z �[0m
2019-05-23T14:23:13.107Z �[91mnpm WARN�[0m
2019-05-23T14:23:13.108Z �[91m tar�[0m
2019-05-23T14:23:13.110Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/apify-shared-7e5a51dd/list_dictionary.js'
2019-05-23T14:23:13.111Z �[0m
2019-05-23T14:23:13.113Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/request-promise-a27ef81a/errors.js'
2019-05-23T14:23:13.114Z �[0m
2019-05-23T14:23:13.116Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/glob-c81cce02/README.md'
2019-05-23T14:23:13.117Z �[0m
2019-05-23T14:23:13.118Z �[91mnpm WARN tar�[0m
2019-05-23T14:23:13.120Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/commander-9971a428/Readme.md'
2019-05-23T14:23:13.121Z �[0m
2019-05-23T14:23:13.123Z �[91mnpm WARN�[0m
2019-05-23T14:23:13.124Z �[91m tar�[0m
2019-05-23T14:23:13.128Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/apify-shared-7e5a51dd/log.js'
2019-05-23T14:23:13.129Z �[0m
2019-05-23T14:23:13.131Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/graceful-fs-02c695a6/fs.js'
2019-05-23T14:23:13.133Z �[0m
2019-05-23T14:23:13.135Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/apply.js'
2019-05-23T14:23:13.136Z �[0m
2019-05-23T14:23:13.138Z �[91mnpm WARN tar�[0m
2019-05-23T14:23:13.140Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/applyEach.js'
2019-05-23T14:23:13.141Z �[0m
2019-05-23T14:23:13.143Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/es5-ext-9273cc13/global.js'
2019-05-23T14:23:13.146Z �[0m
2019-05-23T14:23:13.147Z �[91mnpm WARN�[0m
2019-05-23T14:23:13.149Z �[91m tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/fs-extra-1b8d2fed/LICENSE'
2019-05-23T14:23:13.151Z �[0m
2019-05-23T14:23:13.153Z �[91mnpm WARN tar�[0m
2019-05-23T14:23:13.155Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/es5-ext-9273cc13/index.js'
2019-05-23T14:23:13.157Z �[0m
2019-05-23T14:23:13.160Z �[91mnpm WARN �[0m
2019-05-23T14:23:13.162Z �[91mtar�[0m
2019-05-23T14:23:13.164Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/applyEachSeries.js'
2019-05-23T14:23:13.166Z �[0m
2019-05-23T14:23:13.167Z �[91mnpm WARN tar�[0m
2019-05-23T14:23:13.170Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/es5-ext-9273cc13/LICENSE'
2019-05-23T14:23:13.172Z �[0m
2019-05-23T14:23:13.174Z �[91mnpm �[0m
2019-05-23T14:23:13.176Z �[91mWARN �[0m
2019-05-23T14:23:13.177Z �[91mtar�[0m
2019-05-23T14:23:13.179Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/fs-extra-1b8d2fed/README.md'
2019-05-23T14:23:13.181Z �[0m
2019-05-23T14:23:13.183Z �[91mnpm WARN tar�[0m
2019-05-23T14:23:13.184Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/asyncify.js'
2019-05-23T14:23:13.186Z �[0m
2019-05-23T14:23:13.188Z �[91mnpm WARN tar�[0m
2019-05-23T14:23:13.190Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/async-4a06eb44/auto.js'
2019-05-23T14:23:13.191Z �[0m
2019-05-23T14:23:13.193Z �[91mnpm WARN�[0m
2019-05-23T14:23:13.195Z �[91m tar�[0m
2019-05-23T14:23:13.197Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/es5-ext-9273cc13/optional-chaining.js'
2019-05-23T14:23:13.199Z �[0m
2019-05-23T14:23:13.201Z �[91mnpm WARN tar�[0m
2019-05-23T14:23:13.202Z �[91m ENOENT: no such file or directory, open '/home/node/node_modules/.staging/es5-ext-9273cc13/README.md'
2019-05-23T14:23:13.204Z �[0m
2019-05-23T14:23:13.207Z �[91mnpm WARN �[0m
2019-05-23T14:23:13.209Z �[91mtar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/mime-db-449f1e49/HISTORY.md'
2019-05-23T14:23:13.210Z �[0m
2019-05-23T14:23:13.215Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/bluebird-35b173ed/js/browser/bluebird.core.min.js'
2019-05-23T14:23:13.217Z �[0m
2019-05-23T14:23:13.231Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/source-map-f29f7214/lib/source-map-generator.js'
2019-05-23T14:23:13.233Z �[0m
2019-05-23T14:23:13.252Z �[91mnpm WARN tar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/@types/node-fc3116ea/inspector.d.ts'
2019-05-23T14:23:13.253Z �[0m
2019-05-23T14:23:13.256Z �[91mnpm WARN �[0m
2019-05-23T14:23:13.258Z �[91mtar ENOENT: no such file or directory, open '/home/node/node_modules/.staging/jquery-379f0ec4/dist/jquery.min.js'
2019-05-23T14:23:13.260Z �[0m
2019-05-23T14:23:13.740Z �[91mnpm WARN [email protected] No repository field.
2019-05-23T14:23:13.741Z �[0m
2019-05-23T14:23:13.743Z �[91m
2019-05-23T14:23:13.745Z �[0m
2019-05-23T14:23:13.747Z �[91mnpm ERR! �[0m
2019-05-23T14:23:13.748Z �[91mcode E404
2019-05-23T14:23:13.749Z �[0m
2019-05-23T14:23:13.751Z �[91mnpm ERR!�[0m
2019-05-23T14:23:13.752Z �[91m 404�[0m
2019-05-23T14:23:13.754Z �[91m Not Found: [email protected]
2019-05-23T14:23:13.756Z �[0m
2019-05-23T14:23:22.182Z �[91m
2019-05-23T14:23:22.184Z npm ERR!�[0m
2019-05-23T14:23:22.185Z �[91m A complete log of this run can be found in:
2019-05-23T14:23:22.187Z npm �[0m
2019-05-23T14:23:22.188Z �[91mERR!�[0m
2019-05-23T14:23:22.190Z �[91m     /home/node/.npm/_logs/2019-05-23T14_23_13_747Z-debug.log
2019-05-23T14:23:22.192Z �[0m
2019-05-23T14:23:24.344Z Removing intermediate container 2cef078fd85a
2019-05-23T14:23:24.349Z ACTOR: Error: Docker image build failed: The command '/bin/sh -c npm --quiet set progress=false  && npm install --only=prod --no-optional  && echo "Installed NPM packages:"  && npm list  && echo "Node.js version:"  && node --version  && echo "NPM version:"  && npm --version' returned a non-zero code: 1

Issue when trying to run locally

I've been trying to run this locally like the google maps scraper however I came across this problem while trying to do it:
`PS F:\Scraping\seo-audit-tool-master> apify run --purge
Info: All default local stores were purged.
Warning: You are not logged in with your Apify Account. Some features like Apify Proxy will not work. Call "apify login" to fix that.
Run: npm.cmd start

[email protected] start
node ./src/main.js

INFO System info {"apifyVersion":"2.0.6","apifyClientVersion":"1.4.2","osType":"Windows_NT","nodeVersion":"v16.9.0"}
INFO SEO audit for https://apify.com started
INFO Web host name: apify.com
ERROR
ArgumentError: Did not expect property launchPuppeteerOptions to exist, got [object Object] in object PuppeteerCrawlerOptions
at Object.ow [as default] (F:\Scraping\node_modules\ow\dist\index.js:32:19)
at new PuppeteerCrawler (F:\Scraping\node_modules\apify\build\crawlers\puppeteer_crawler.js:241:21)
at F:\Scraping\seo-audit-tool-master\src\main.js:39:21
at async run (F:\Scraping\node_modules\apify\build\actor.js:182:13)
Error: npm.cmd exited with code 91`

This is what my INPUT.json looks like:

{
    "startUrl": "https://apify.com",
    "proxy": {
      "useApifyProxy": false
    },
    "maxRequestsPerCrawl": 100,
    "maxDepth": 2,
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36",
    "viewPortWidth": 1440,
    "viewPortHeight": 900,
    "pageTimeout": 30000,
    "maxRequestRetries": 2,
    "handlePageTimeoutSecs": 2,
    "seoParams": {
      "maxTitleLength": 70,
      "minTitleLength": 10,
      "maxMetaDescriptionLength": 140,
      "maxLinksCount": 3000,
      "maxWordsCount": 350,
      "outputLinks": false,
      "workingStatusCodes": [
        200,
        301,
        302,
        304
      ]
    }
  }

Any ideas?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.