A GitHub Action for checking broken links
License: Apache License 2.0
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "github-actions[bot]")
![my-renovate[bot] avatar](https://avatars.githubusercontent.com/in/199026?v=4 "my-renovate[bot]")
Not sure if it's a bug or not, but I've tried to use an environment variable in the exclude parameter but it doesn't see to work, and I'm not sure how to diagnose this further.
To Reproduce
Try this:
on:
workflow_dispatch:
pull_request:
branches: [main]
name: test
jobs:
link-checker-test:
name: Check broken links
runs-on: ubuntu-latest
env:
LIST: |
mycompany.*
.*?github.com
steps:
- name: Check
uses: ruzickap/action-my-broken-link-checker@v2
with:
url: https://example.com
cmd_params: |
--follow-sitemap-xml
--skip-tls-verification
--max-connections=4
--color=always
--exclude="$(echo ${LIST}|tr ' ' '|')"I've tried various approaches but somehow
Expected behaviour
Some ability to resolve $() in the cmd_params field.
Thanks for any input!
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Location: .github/renovate.json5
Error type: The renovate configuration file contains some invalid settings
Message: Invalid packageRules[2].schedule: 'Invalid schedule: "0 0,2,4 1-7 * 0" has cron syntax, but doesn't have * as minutes'
Is your feature request related to a problem? Please describe
Not sure if it is in your hands of that of muffet, but it will be really useful to report not just what links are broken but also where to find them, so they can be fixed easily.
Describe the solution you'd like
For each broken link found, provide a list of places (within the target URL) where that link is found, if found in more than one place.
Describe alternatives you've considered
Nothing comes to my mind, just manually search my documentation for the broken links.
Additional context
None
Is your feature request related to a problem? Please describe
I use Hugo as open-source static site generator for my website. I would like to check the links of the generated HTML files before publishing my website anywhere. Hugo has a command to start a local web server at localhost:1313. However, I don't know how to provide the local web server to action-my-broken-link-checker.
I have tried:
- name: Start Web Server
run: |
hugo server &
while ! nc -z localhost 1313 ; do sleep 1s ; done
- name: Check Links
uses: ruzickap/action-my-broken-link-checker@v2
with:
url: http://localhost:1313/v2/Error:
*** INFO: [2022-09-11 09:34:50] Start checking: "http://localhost:1313/v2/"
*** ERROR: [2022-09-11 09:34:50] Something went wrong - see the errors above...
failed to fetch root page: dial tcp4 12[7](https://github.com/pmwmedia/tinylog-website/runs/8291244346?check_suite_focus=true#step:10:8).0.0.1:1313: connect: connection refused
I have also tried:
- name: Check Links
uses: ruzickap/action-my-broken-link-checker@v2
with:
url: https://tinylog.org/v2/
pages_path: ./public/Error:
*** INFO: Using path "./public/" as domain "tinylog.org" with URI "https://tinylog.org/"
*** INFO: [2022-09-11 10:04:48] Start checking: "https://tinylog.org/v2/"
failed to fetch root page: 404
{"level":"info","ts":166289068[7](https://github.com/pmwmedia/tinylog-website/runs/8291423464?check_suite_focus=true#step:9:8).[8](https://github.com/pmwmedia/tinylog-website/runs/8291423464?check_suite_focus=true#step:9:9)01[9](https://github.com/pmwmedia/tinylog-website/runs/8291423464?check_suite_focus=true#step:9:10)361,"msg":"using provided configuration","config_file":"/tmp/Caddyfile.AleHgB","config_adapter":""}
{"level":"warn","ts":1662890687.80263[11](https://github.com/pmwmedia/tinylog-website/runs/8291423464?check_suite_focus=true#step:9:12),"msg":"Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies","adapter":"caddyfile","file":"/tmp/Caddyfile.AleHgB","line":2}
{"level":"info","ts":1662890687.8031988,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["//[12](https://github.com/pmwmedia/tinylog-website/runs/8291423464?check_suite_focus=true#step:9:13)7.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
{"level":"info","ts":1662890687.8042064,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1662890687.8042216,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1662890687.8044977,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["tinylog.org"]}
{"level":"info","ts":1662890687.8068898,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc00063f730"}
{"level":"info","ts":1662890687.8072672,"logger":"tls.obtain","msg":"acquiring lock","identifier":"tinylog.org"}
{"level":"info","ts":1662890687.8124647,"logger":"tls.obtain","msg":"lock acquired","identifier":"tinylog.org"}
{"level":"info","ts":1662890687.8144195,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"tinylog.org"}
{"level":"info","ts":1662890687.8144295,"logger":"tls.obtain","msg":"releasing lock","identifier":"tinylog.org"}
{"level":"warn","ts":1662890687.8146987,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [tinylog.org]: no OCSP server specified in certificate","identifiers":["tinylog.org"]}
{"level":"warn","ts":1662890687.8197901,"logger":"pki.ca.local","msg":"installing root certificate (you might be prompted for password)","path":"storage:pki/authorities/local/root.crt"}
{"level":"info","ts":1662890687.8199[13](https://github.com/pmwmedia/tinylog-website/runs/8291423464?check_suite_focus=true#step:9:14)4,"msg":"Warning: \"certutil\" is not available, install \"certutil\" with \"apt install libnss3-tools\" or \"yum install nss-tools\" and try again"}
{"level":"info","ts":[16](https://github.com/pmwmedia/tinylog-website/runs/8291423464?check_suite_focus=true#step:9:17)62890687.8[19](https://github.com/pmwmedia/tinylog-website/runs/8291423464?check_suite_focus=true#step:9:20)9[20](https://github.com/pmwmedia/tinylog-website/runs/8291423464?check_suite_focus=true#step:9:21)8,"msg":"define JAVA_HOME environment variable to use the Java trust"}
{"level":"info","ts":1662890687.8548355,"msg":"certificate installed properly in linux trusts"}
{"level":"info","ts":1662890687.855012,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/github/home/.local/share/caddy"}
{"level":"info","ts":1662890687.8550277,"msg":"autosaved config (load with --resume flag)","file":"/github/home/.config/caddy/autosave.json"}
{"level":"info","ts":1662890687.8550408,"msg":"serving initial configuration"}
{"level":"info","ts":1662890687.8552792,"logger":"tls","msg":"finished cleaning storage units"}
Successfully started Caddy (pid=[23](https://github.com/pmwmedia/tinylog-website/runs/8291423464?check_suite_focus=true#step:9:24)) - Caddy is running in the background
sed: can't move '/etc/hostskhdlKE' to '/etc/hosts': Resource busy
I assume that the root page cannot be fetched because "./public/" should be mapped to "https://tinylog.org/v2/" instead of "https://tinylog.org/". The last output sed: can't move '/etc/hostskhdlKE' to '/etc/hosts': Resource busy looks also a bit odd.
Describe the solution you'd like
My favorite solution would be providing a shell command for starting the web server and action-my-broken-link-checker would wait until the web server will be up (no need for while ! nc -z localhost 1313 ; do sleep 1s ; done) and then runs muffet
Example:
- name: Check Links
uses: ruzickap/action-my-broken-link-checker@v2
with:
command: hugo server
url: http://localhost:1313/v2/Describe alternatives you've considered
An alternative would be passing a folder instead of an URL to action-my-broken-link-checker and the action starts a web server for static HTML files by itself.
Workaround
- name: Check Links
run: |
hugo server &
while ! nc -z localhost 1313 ; do sleep 1s ; done
wget -q https://github.com/raviqqe/muffet/releases/download/v2.6.1/muffet_2.6.1_Linux_x86_64.tar.gz
tar -zxvf muffet_2.6.1_Linux_x86_64.tar.gz muffet
chmod +x muffet
./muffet http://localhost:1313/v2/Describe the bug
The action stopped working.
To Reproduce
I have an Action config containing the job step:
- name: check for broken links
uses: ruzickap/[email protected]
with:
url: 'https://www.brimsecurity.com'
cmd_params: '--verbose --exclude=linkedin.com'
Expected behaviour
It worked in the past.
Screenshots
The messages around the failure in the Actions log:
2020-07-31T15:42:33.5034275Z �[0m⚠️ This installer only supports v1, which is obsoleted now that Caddy 2 is released. This script may change or go away soon. Please upgrade: https://caddyserver.com/docs/v2-upgrade
2020-07-31T15:42:33.5066870Z Downloading Caddy for linux/amd64 (personal license)...
2020-07-31T15:42:33.8694941Z Aborted, error 8 in command: wget --quiet --header "Authorization: Basic $(echo -ne "$CADDY_ACCOUNT_ID:$CADDY_API_KEY" | base64)" "$caddy_url" -O "$dl"
2020-07-31T15:42:34.1952082Z The command '/bin/ash -eo pipefail -c set -eux && apk add --no-cache bash ca-certificates sudo wget && if [ "${MUFFET_VERSION}" = "latest" ]; then MUFFET_URL=$(wget -qO- https://api.github.com/repos/raviqqe/muffet/releases/latest | grep "browser_download_url.*muffet_.*_Linux_x86_64.tar.gz" | cut -d \" -f 4) ; else MUFFET_URL="https://github.com/raviqqe/muffet/releases/download/${MUFFET_VERSION}/muffet_${MUFFET_VERSION}_Linux_x86_64.tar.gz" ; fi && wget -qO- "${MUFFET_URL}" | tar xzf - -C /usr/local/bin/ muffet && wget -qO- https://getcaddy.com | bash -s personal' returned a non-zero code: 1
2020-07-31T15:42:34.1960248Z ##[warning]Docker build failed with exit code 1, back off 5.277 seconds before retry.
Your YAML file
Alas, the full YAML is in a private repo, but hopefully the above excerpt will suffice. Based on its behavior I assume this is failing fore everybody who uses the Action. Perhaps the credentials in there have expired?
Describe the bug
I am using the action as follows
jobs:
my-broken-link-checker:
name: Broken Links Check
runs-on: ubuntu-latest
steps:
- name: Check
uses: ruzickap/action-my-broken-link-checker@v2
with:
url: https://petrhurtak.com
cmd_params: >
--timeout=30
--buffer-size=10000
--header="user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
--color=always
--verbose
And it fails with this error
invalid argument for flag `--header' (expected []string): invalid syntax
*** ERROR: [2022-07-30 16:56:24] Something went wrong - see the errors above...
So I am wondering if there is issue with how I specified the parameter or with how the action passes the params to muffet.
Unfortunatelly there is not much docs on the Muffer side about how pass this parameter correctly, the only docs I found is this https://github.com/raviqqe/muffet/blob/main/.snapshots/TestHelp#L19
Is your feature request related to a problem? Please describe
I would like to use the feature of configurable HTTP response status codes (raviqqe/muffet#364) of muffet 2.10.
Describe the solution you'd like
Please release the branch renovate/raviqqe-muffet-2.x as new version. It contains already muffet 2.10.1.
Is your feature request related to a problem? Please describe
When I use this link checker, I get very noisy results for issues I'm already aware of.
Describe the solution you'd like
ignorePatterns, as done here.Describe alternatives you've considered
We're currently using the action linked above because of this feature.
Describe the bug
The action fails with the following:
*** ERROR: [2024-02-09 03:38:43] Something went wrong - see the errors above...
But no errors are displayed.
Do you have any ideas what might be making it fail, or how I can get more details to fix it?
To Reproduce
Run the action with the following settings:
with:
url: https://owncast.online/docs
cmd_params: --buffer-size=8192 --max-connections=10 --color=always --rate-limit=1 --max-connections-per-host 3 --exclude="opencollective.com|github.com|example.com|mydomain.com|sitemap" --verbose
Describe the bug
I cannot use the action based on the quick-start. It references "v1" but that is unavailable.
I managed to use docker setup with:ruzickap/action-my-broken-link-checker@v2
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.