jeffkala / pyurlcheck Goto Github PK

pyurlcheck can be used to scan through all of a projects documents and validate any `public` facing URLs are still reachable.

Python 98.65% Dockerfile 1.35%

pyurlcheck's Introduction

Hello all.

I’m currently working on Network Automation.
📫 How to reach me: Slack -> Join at http://slack.networktocode.com/

pyurlcheck's People

Stargazers

Watchers

Forkers

jvanderaa

pyurlcheck's Issues

Sys.Exit() Should Return > 0 On Fail

To execute the check and report back to a CI tool, when there is an URL that doesn't respond sys.exit() should return an integer greater than 0, likely 1, or the number of sites that failed.

Support a configuration file

supporting a configuration file like most other static analysis tools will provide an easy to use interface.

Feature Request: Display Warning on Redirects

I went looking to check on the code about what happens with a redirect. I believe it would be worth while to display a warning when there is a 3xx redirect that happens. This will allow for the proper link to get updated, but the content is still being displayed.

Nested directory lookups fail

▶ pyurlcheck ../nautobot/
Traceback (most recent call last):
  File "/Users/jeffkala/Library/Caches/pypoetry/virtualenvs/pyurlcheck-jbPNAqXn-py3.9/bin/pyurlcheck", line 5, in <module>
    main()
  File "/Users/jeffkala/Library/Caches/pypoetry/virtualenvs/pyurlcheck-jbPNAqXn-py3.9/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/jeffkala/Library/Caches/pypoetry/virtualenvs/pyurlcheck-jbPNAqXn-py3.9/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/jeffkala/Library/Caches/pypoetry/virtualenvs/pyurlcheck-jbPNAqXn-py3.9/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/jeffkala/Library/Caches/pypoetry/virtualenvs/pyurlcheck-jbPNAqXn-py3.9/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/jeffkala/Documents/github-clones/pyurlcheck/pyurlcheck/cli.py", line 20, in main
    files_urls = FindUrls(input_data).find_urls()
  File "/Users/jeffkala/Documents/github-clones/pyurlcheck/pyurlcheck/find.py", line 54, in find_urls
    results.update(_parse_file(f"{self.search}{file}"))
  File "/Users/jeffkala/Documents/github-clones/pyurlcheck/pyurlcheck/find.py", line 29, in _parse_file
    file_data = _read_in_file(filepath)
  File "/Users/jeffkala/Documents/github-clones/pyurlcheck/pyurlcheck/find.py", line 9, in _read_in_file
    return Path(file_to_read).read_text().splitlines()
  File "/usr/local/Cellar/[email protected]/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1255, in read_text
    with self.open(mode='r', encoding=encoding, errors=errors) as f:
  File "/usr/local/Cellar/[email protected]/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1241, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
IsADirectoryError: [Errno 21] Is a directory: '../nautobot/docker'

Find a better way to automatically ignore binary files

Currently a binary file causes a failure. Implement a check for this. Validation should only run on a file that is text based.

Message needs to be fixed.

pyurlcheck/pyurlcheck/find.py

Line 56 in 352db4b

raise ValueError("No File or Directory Provided.")

[ASA for Vagrant](https://techbloc.net/archives/2360)

This link is showing up consistently, yet works when I go to it manually.

[GNS3 DMVPN Intro](http://resources.intenseschool.com/gns3-lab-introduction-to-dmvpn/)

Feature to toggle whether rfc1918 addresses should be tested.

This maybe used in a closed in-house environment and testing private network space should be a toggle-able feature.

Error On Link Name With "." in it

### Zigbits Network Design

Main Page: [Zigbits.tech](https://zigbits.tech/)

Fails with the message:

requests.exceptions.MissingSchema: Invalid URL 'Zigbits.tech': No schema supplied. Perhaps you meant http://Zigbits.tech?