lycheeverse / lychee-action Goto Github PK

Github action to check for broken links in Markdown, HTML, and text files using lychee, a fast link checker written in Rust.

Home Page: https://lychee.cli.rs

License: Apache License 2.0

Shell 100.00%

action continuous-integration github-actions link-checker

lychee-action's People

Contributors

Stargazers

Watchers

lychee-action's Issues

Why Docker?

Using this action, I come to see that GitHub Docker actions are generally inefficient.
For e.g., if I understand correctly, it seems lake an intermediate container is created by GitHub for ever LABEL in the dockerfile 😬

Is there a specific reason(s) that the action was created as a Docker action rather than a composite action?

Would be interested to try to help to convert to composite action if there are no specific reasons for being a Docker action and there is no objection.

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total...........21
✅ Successful......19
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
🚫 Errors...........1

Errors in fixtures/TEST.md
✗ https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst (HTTP status server error (503 Service Unavailable) for url (https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst))

Full Github Actions output

Feature: Slack notifications

As a feature, we could support Slack notifications when link checking fails.

This could be done as a webhook or alternatively, using ready-made GitHub actions or other integrations. In this case the README could include example of how to enable this integration.

Exclude specific files from link checker

Hi, I have a use case where I want to exclude specific files from being checked in the link checker. Currently my args are as below:

args: --accept=200,403,429 "**/*.html" "**/*.md" "**/*.txt" "**/*.json"

I want to exclude some type of text files like files with names containing a specific word example anything with word file in the name like sample-file.txt. Is there a way to achieve this?

Thank you!

Issue not created

What am I missing here?

Checking only links from modified files in a Pull Request?

This is a feature that github-action-markdown-link-check offers.

Could this action support that as well? (or document advice on how to provide only relevant file paths to check)

Almost 18 months later, I revisited this request, comments of interest are:

Advice for Github request/API rate limits, which for PR workflows from third-party contributors lack secrets access (60 requests per hour limit). Affects link checking on Github services (eg: Github repo file/issue/pr links, or github pages).
- Related production branch scheduled runs with persisted cache.
Initial advice for getting line specific content, or relevant files from the PR changes.
- Related, individual line content changes feed to lychee via STDIN input is ill-advised depending on content/context.
Final advice for using two different approaches to collect relevant files to check, and then pre-process for compatibility to action input args in lychee-action.
Detailing xargs as viable for lychee, but not lychee-action as a user. Requested a separate lychee-action input, or for lychee an --inputs-from option.

Examples - Workflows that can be used currently

The first two support file paths containing spaces.

Without considering file paths containing spaces; if a mistake is made or a third-party contributor provides a file path containing a space, lychee will treat that path as separate files/inputs which would be invalid (or the wrong files to check).

Manually generating the file paths to check

I looked into this extensively, and I'm rather confident of this approach.

Click to view YAML

Remove or adjust the --diff-filter option as appropriate to your needs. View the git diff-tree docs linked for other options that may be of interest such as ignoring white-space only changes.
The sed command is specifically for lychee-action needs. It folds the multi-line output of git diff-tree (\n / line delimited list of file paths_) into a single line, separating paths with single quotes and later in the lychee-action.args input adding single quotes to the start/end, so that file paths can support spaces when provided as a single line to eval (which lychee-action uses internally). This handling is not required for lychee, which can instead replace sed with xargs -d '\n' lychee --.

jobs:
  build:
    runs-on: ubuntu-latest
    name: Test changed-files
    steps:
      # Compares the default generated "test merge commit" by Github (HEAD) to the base branch (HEAD^1)
      - uses: actions/checkout@v3
        with:
          fetch-depth: 2

      # Gets a list of files from the PR with a git status of Added (A) or Modified (M)
      - name: 'Example - Get a list of changed files to process'
        id: changed-files
        run: |
          CHANGED_FILES=$(git diff-tree --name-only --diff-filter 'AM' -r HEAD^1 HEAD | sed -z "s/\n$//;s/\n/' '/g")
          echo "::set-output name=all_changed_files::${CHANGED_FILES}"

      - name: 'Check Links'
        id: lychee
        uses: lycheeverse/[email protected]
        with:
          args: --verbose --no-progress -- '${{ steps.changed-files.outputs.all_changed_files }}'

Delegate to `tj-actions/changed-files` action to get the file paths

After this PR is resolved, the pre-processing step for lychee-action can likely be dropped and the output of tj-actions/changed-files provided to a new input for lychee-action that handles the delimited list of paths internally.

Click to view YAML

jobs:
  build:
    runs-on: ubuntu-latest
    name: Test changed-files
    steps:
      # Compares the default generated "test merge commit" by Github of the PR branch into the base branch
      - uses: actions/checkout@v3
        with:
          fetch-depth: 2

      # Gets a list of files from the PR with a git status of Added (A) or Modified (M) | (via outputs.all_changed_files)
      - name: 'Example - Get a list of changed files to process'
        id: get-changed-files
        uses: tj-actions/changed-files@v23
        with:
          separator: ','

      # lychee-action requires pre-processing for file paths with spaces (boundary quotes added in lychee-action input):
      - name: 'Compatibility with eval: Quote wrap paths to support spaces'
        id: changed-files
        run: |
          QUOTE_WRAPPED="$( sed "s/,/' '/g" <<< '${{ steps.get-changed-files.outputs.all_changed_files }}' )"
          echo "::set-output name=all_changed_files::${QUOTE_WRAPPED}"

      - name: 'Check Links'
        id: lychee
        uses: lycheeverse/[email protected]
        with:
          args: --verbose --no-progress -- '${{ steps.changed-files.outputs.all_changed_files }}'

Simpler version when file paths of inputs would never contain spaces

If you are confident that this won't be a concern to support, the extra pre-processing step can be avoided.

NOTE: If adapting to the "manual" example, you must still convert \n to space, unless lychee gets a --inputs-from option, then you can provide a file with git diff-tree output directly (or without writing an actual file).

Click to view YAML

jobs:
  build:
    runs-on: ubuntu-latest
    name: Test changed-files
    steps:
      # Compares the default generated "test merge commit" by Github of the PR branch into the base branch
      - uses: actions/checkout@v3
        with:
          fetch-depth: 2

      # Gets a list of files from the PR with a git status of Added (A) or Modified (M) | (via outputs.all_changed_files)
      - name: 'Example - Get a list of changed files to process'
        id: get-changed-files
        uses: tj-actions/changed-files@v23

      - name: 'Check Links'
        id: lychee
        uses: lycheeverse/[email protected]
        with:
          args: --verbose --no-progress -- ${{ steps.changed-files.outputs.all_changed_files }}

Security: Make SHA-pinning effective (e.g. allow pinning SHA of Docker image)

GitHub recommends pinning third-party Actions to a commit SHA, but since lycheeverse-action pulls a Docker image, that image should also be able to be pinned to a specific SHA.

To prevent churn, one approach could be to simply allow passing a Docker image SHA into the script from the end-user's .yml, which, if present, would then be used to pin the Docker image.

See, for example: https://github.com/returntocorp/bento/issues/318

Another approach: An option to have the action build lychee from scratch, which would be slower, but would also further reduce the attack surface.

Error with base dir `<empty>` : Found absolute local link

I'm checking html files passing a folder arg to Lychee :

      - name: Link checker
        uses: lycheeverse/[email protected]
        id: lychee
        with:
          args: --verbose --no-progress --exclude=w3.org --exclude=schema.org --exclude-mail build/client/**/*.html
      - name: Fail if there were link errors
        run: exit ${{ steps.lychee.outputs.exit_code }}

Since today, i've the following error :

Error: Error with base dir `<empty>` : Found absolute local link "/favicons/apple-touch-icon.png" but no base directory was set. Set with `--base`.

I never used the base arg and it's worked fine until today.
Anything related to this issue ?

How to test GH wikis?

E.g. https://github.com/cheeaun/life/wiki/Lives has a lot of bad links... any suggestions how to check pages like that?

Many thanks!

How to use exclude args

Hi,

I saw Lychee support exclude args, I tried using this action, but it seems like it doesn't work. So is it support exclude?

Thanks

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total............9
✅ Successful.......7
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
🚫 Errors...........1

Errors in README.md
↯ https://github.com/marketplace/actions/lychee-link-checker-action (GitHub token not specified. To check GitHub links reliably, use `--github-token` flag / `GITHUB_TOKEN` env var.)

Full Github Actions output

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total...........21
✅ Successful......19
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
🚫 Errors...........1

Errors in fixtures/TEST.md
✗ https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst (HTTP status server error (503 Service Unavailable) for url (https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst))

Full Github Actions output

Given example usage does not work

I'm using the same general format as the workflow, but it does not work:

name: Check Sources

on:
  #repository_dispatch:
  workflow_dispatch:
  schedule:
    - cron: "0 * * * *"

jobs:
  check_sources:
    runs-on: ubuntu-20.04

    steps:
      - name: Clone Repo
        uses: actions/checkout@v2

      - name: Check Links
        uses: lycheeverse/lychee-action@v1
        with:
          args: -vin -T 4 -m 5 sources/sources.txt
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Create Issue From File
        if: ${{ failure() }}
        uses: peter-evans/create-issue-from-file@v3
        with:
          title: Dead Source Report
          content-filepath: ./lychee/out.md
          labels: bug

The failure check is there b/c otherwise the reporting will not run. Even with it there, the file reporter cannot seem to find the lychee report.

GitHub links are also giving a 403 response, even w/ the GitHub token provided.

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total...........21
✅ Successful......19
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
🚫 Errors...........1

Errors in fixtures/TEST.md
✗ https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst (HTTP status server error (503 Service Unavailable) for url (https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst))

Full Github Actions output

Latest builds failing with RegEx parsing issue

See example job on OpenSearch:

Pattern syntax error near position 0: recursive wildcards must form a single path component

I've noticed the same problem on my own blog (private repository). Any thoughts on what might be causing it?

The job I am running is mapped to the following YAML file:

name: Link Checker

on:
  repository_dispatch:
  workflow_dispatch:
  schedule:
    - cron: "00 18 * * *"

jobs:
  linkChecker:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Load Excludes
        run: |
          LYCHEE_EXCLUDE=$(sed -e :a  -e 'N;s/\n/ /;ta' .lycheeexclude)
          echo "LYCHEE_EXCLUDE=$LYCHEE_EXCLUDE" >> $GITHUB_ENV

      - name: Link Checker
        uses: lycheeverse/lychee-action@master
        with:
          args: --accept=200,429 --exclude ${{ env.LYCHEE_EXCLUDE }} --exclude-mail "**/*.md"
        env:
          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
        
      - name: Create Issue From File
        uses: peter-evans/create-issue-from-file@v3
        with:
          title: Link Checker Report
          content-filepath: ./lychee/out.md
          labels: report, automated issue

Seems to be misparsing some links

I am reporting this on the action rather than the main utility repo because I am only seeing this behavior in the action.

I am getting the following failure:

🚫 https://raw.githubusercontent.com/informalsystems/apalache/99e58d6f5eebcc41f432a126a13a5f8d2ae7afe6/logo-apalache.svg/ [400 Bad Request]

while trying to run on a markdown file that contains this block of inline HTML:

<div align="center">
<img
src="https://raw.githubusercontent.com/informalsystems/apalache/99e58d6f5eebcc41f432a126a13a5f8d2ae7afe6/logo-apalache.svg"
alt="Apalache Logo">

lychee seems to be appending a / to the end of the link? You can see our failing CI step while trying to put this to use here: https://github.com/informalsystems/apalache/runs/1670269221?check_suite_focus=true#step:4:53

Exit Code 2 is not causing the Action to fail

We are in the process of setting up an Action to check our links, but the link checker catches an error due to a bad link, but the Action is still passing even though the link checker returns an exit code of 2.

How are we suppose to get the Action to fail? We don't want any issues to be created since any bad links should be caught in PRs before they are merged.

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total...........21
✅ Successful......19
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
🚫 Errors...........1

Errors in fixtures/TEST.md
✗ https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst (HTTP status server error (503 Service Unavailable) for url (https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst))

Full Github Actions output

Check Markdown Files Recursively

How do I pass a command to the argument with which I can make a recursive check? For example., my markdown files for my Github pages ate located in a docs folder which in turn contains many sub folders. In these sub folders I have several markdown files and I want to be able to test in all these markdown files.

Additionally I also have in my markdown files some links that use the a href tags, the reason being that I want those links to open in a new tab. Could this action check for a hrefs as well or will it only look for markdown style links with the ?

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total...........21
✅ Successful......19
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
🚫 Errors...........1

Errors in fixtures/TEST.md
✗ https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst (HTTP status server error (503 Service Unavailable) for url (https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst))

Full Github Actions output

Use with options that can take multiple arguments

Some lychee options can take multiple arguments, e.g. the --exclude options.
When using those options, one then needs to separate the list of files to check by a double dash --, see lycheeverse/lychee#113

For example:

lychee --verbose --timeout 40 --insecure  --exclude "https://www.msg.chem.iastate.edu/"  "https://openmopac.net/"  -- codes.md

The problem is: lychee-action appends --output /tmp/lychee/out.md to the command line, which is then interpreted as another input file:

lychee --verbose --timeout 40 --insecure --exclude https://www.msg.chem.iastate.edu/ https://openmopac.net/ -- codes.md --output /tmp/lychee/out.md
Error: Failed to read file: `--output`, reason: No such file or directory (os error 2)

Instead of appending the --output option, what about prepending it before the args?

Check gist and user profile links using GitHub tokens

We have a lot of gist links so CI constantly hits 429 Too Many Requests on them.

Doing the same for GH profile links would also help.

Thanks for the action. It's much better than what we used before 😉

Failed to read file

Hey first of, thank you for this awesome tool. It is definitely quicker than the alternatives with comparable feature sets.

I'm really not sure, if the bug / issue I'm describing is part of lychee-action or the lychee repo, so feel free to move the issue, if this is not the right place.

I'm currently trying to get lychee-actions working here: stm32-rs/stm32f3xx-hal#188

Now I've tried a bit and stumbled upon #10

You can see my attempts here:

I've tried around, because I wasn't sure, if lychee is getting the arguments right. I've run these commands locally with the newest lychee version from master, and they work as expected.
Now stm32-rs/stm32f3xx-hal@36936e7 is particularly interesting, because it fails like that:

+ lychee --verbose --output /tmp/lychee/out.md
Error: Failed to read file: `README.md`

Caused by:
    No such file or directory (os error 2)

(https://github.com/stm32-rs/stm32f3xx-hal/runs/2010954174)

which does hint towards a right direction: This invocation and the ones before do nothing, because no matching file was found in the repo. In this case an explicit error is return, because lychee defaults to README.md if no file-name is given.

I've no clue, why github-action does not find any valid files, so I assume this is a github-actions issue and more so a configuration issue of lychee-action?

Maybe I'm wrong though, and it is an issue on my side, but I can't see it.

Also, even though the exit code is 1 the check is marked as successfully passed.

Issues are created regardless of result

The readme states:

issues will be created when the action finds link problems.

That's not currently correct.

I ran the suggested workflow and got this issue:

refined-github/refined-github#5114

I fixed the links and then a new issue was opened, empty:

refined-github/refined-github#5116

For this to happen, lychee-action should probably export a variable that I can then pass to the next action. For example:

       - uses: lycheeverse/[email protected]
         id: lychee 
         with:
           args: --verbose --no-progress **/*.md **/*.html
         env:
           GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
       - name: Create Issue From File
         if: steps.lychee.outputs.errors
         uses: peter-evans/create-issue-from-file@v3
         with:
           title: Link Checker Report
           content-filepath: ./lychee/out.md

Feature Request: Add json output option

I would like to have an option to get the json report, and then further modify this output to generate a styled markdown comment. An example of such comment could be a markdown with links to the markdown files with errors.

Link Checker Report

Errors were reported while checking the availability of links.

📝 Summary

🔍 Total............8
✅ Successful.......7
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........0
🚫 Errors...........1

Input: README.md
⚡ lycheeverse/[email protected]
Runtime error (Invalid mail address: lycheeverse/[email protected])

Full Github Actions output

lychee-action produces no output and doesn't seem to do anything

I've tried using lychee-action v1.0.7 or v1.0.8 and see runs like this happening. It looks like nothing is actually running as part of the action?

The workflow file is here:

name: GitHub Actions CI
on:
  push:
    branches: master
  pull_request:

jobs:
  check_links:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2

      - name: Check links
        uses: lycheeverse/lychee-action@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - run: cat ./lychee/out.md
        if: ${{ always() }}

Not sure if I'm just doing something wrong or if there has been some kind of a regression.

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total...........21
✅ Successful......19
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
⚠ Unknown..........0
🚫 Errors...........1

Errors in fixtures/TEST.md
✗ file:///github/workspace/fixtures/awesome.png (Invalid file URI: file:///github/workspace/fixtures/awesome.png)

Full Github Actions output

Lychee not detecting markdown files

I am running Lychee as a GitHub action on https://github.com/balena-io/docs/, Lchee's GitHub action got updated v1.2.0 which lead to the pattern **/*.md failing.

For some reason, this in the file https://github.com/balena-io/docs/blob/master/.github/workflows/link-checker.yml

with:
           args: >
             --config ./lychee.toml
             --github-token ${{ secrets.GITHUB_TOKEN }}
             **/*.md

translates to just finding 2 markdown files in the repository when the action is running. Refer: https://github.com/balena-io/docs/runs/4626369590?check_suite_focus=true#step:5:19

lychee --format markdown --output /tmp/lychee/out.md --config ./lychee.toml --github-token *** pages/404.md shared/icon.md

I am trying to debug if there is something on our end that changed or is it Lychee's action update that caused this. I ran Lychee locally to confirm the pattern lychee --no-progress --config lychee.toml **/*.md and that worked just fine. Do let me know if you need any more information to help debug this further.

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total...........20
✅ Successful......18
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
🚫 Errors...........1

Errors in README.md
✗ http://opensource.org/licenses/MIT (HTTP status server error (502 Bad Gateway) for url (https://opensource.org/licenses/MIT))

Full Github Actions output

Error: Is a directory (os error 21)

I'm attempting to switch from peter-evans/link-checker to lycheeverse/lychee-action but I'm getting an error whenever my workflow runs:

Run lycheeverse/lychee-action@v1
/usr/bin/docker run --name cc49563e5ee5548076440b9eeed26b8e62143a_aa2ec5 --label cc4956 --workdir /github/workspace --rm -e INPUT_ARGS -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/EliteALD/EliteALD":"/github/workspace" cc4956:3e5ee5548076440b9eeed26b8e62143a -E -i -v *
+ LYCHEE_OUT=lychee/out.md
+ LYCHEE_TMP=/tmp/lychee/out.md
++ dirname /tmp/lychee/out.md
+ mkdir -p /tmp/lychee
+ lychee -E -i -v '*' --output /tmp/lychee/out.md
Error: Is a directory (os error 21)

Here is my workflow:

name: Link Checker v2 (Lychee)
on:
  schedule:
  - cron: 0 1 * * *
jobs:
  linkChecker:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Link Checker
      uses: lycheeverse/lychee-action@v1
      with:
        args: -E -i -v *
    - name: Create Issue From File
      uses: peter-evans/create-issue-from-file@v2
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        ISSUE_TITLE: Link Checker Report
        ISSUE_CONTENT_FILEPATH: ./lychee/out.md
        ISSUE_LABELS: report, automated issue

Can you suggest what I've done wrong?

Failing check when I want an informative check

I use this action simply:

name: Links

on:
  workflow_dispatch:
  push:

jobs:
  linkChecker:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Link Checker
        uses: lycheeverse/lychee-action@v1
        with:
          args: --verbose --no-progress **/*.mdwn

upon https://github.com/kaihendry/geekout.org.uk/blob/07328724ab98e8fc91105255654b399baa27fe58/.github/workflows/links.yaml

However it has two surprisingly issues:

it causes a failure, I assumed reading the documentation I needed run: exit ${{ steps.lychee.outputs.exit_code }} to actually cause a failure
doesn't appear to summarize the failing links in a sane way

What am I doing wrong please?

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total...........20
✅ Successful......18
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
🚫 Errors...........1

Errors in README.md
✗ http://opensource.org/licenses/MIT (HTTP status server error (502 Bad Gateway) for url (https://opensource.org/licenses/MIT))

Full Github Actions output

Link Checker Report

Errors were reported while checking the availability of links:
📝 Summary

🔍 Total: 8
✅ Successful: 7
⏳ Timeouts: 0
🔀 Redirected: 0
👻 Excluded: 0
🚫 Errors: 1

Create issue from file usage?

I wonder how to use the issues from file in the cronjob. It looks like the example in the README is broken.

name: Links

on:
  repository_dispatch:
  workflow_dispatch:
  schedule:
    - cron: "00 18 * * *"

jobs:
  linkChecker:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      - name: Link Checker
        uses: lycheeverse/[email protected]
        with:
          args: --verbose --no-progress **/*.md **/*.html
        env:
          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
        
      - name: Create Issue From File
        uses: peter-evans/create-issue-from-file@v2
        with:
          title: Link Checker Report
          content-filepath: ./lychee/out.md
          labels: report, automated issue

--output out.md isn't defined and thus we will never create any issue.

My understanding of https://github.com/peter-evans/create-issue-from-file is that it will create a issue with the data from the file no matter what it says in it.

lychee don't have any flag where you only create a file if you get an error, or have I missed something?
So we need to add some small script to first check if the output file generate only have 0 Errors in it.
If errors is 0 remove the file.

I will write a small script for this and create a PR for the README.

If i missed something please point me in the correct direction.

Action exit code is not set

Th Lychee action exit code is not reflected in the job

Example run:
https://github.com/lenisha/linkchecker-test/runs/2117110939?check_suite_focus=true

    steps:
      - uses: actions/checkout@v2
      - name: lychee Link Checker
        id: lychee
        uses: lycheeverse/[email protected]
        with:
          args: --exclude="^(javascript|chrome):.*"   --accept=200,403,429 "**/*.md" "**/*.html"
        env:
          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
      - name: Fail if there were link errors
        run: exit ${{ steps.lychee.outputs.exit_code }}

Although Lychee step exits with status code 2, next steps does not get the exit code properly, it gets 0 instead of 2:

+ lychee '--exclude=^(javascript|chrome):.*' --accept=200,403,429 '**/*.md' '**/*.html' --output /tmp/lychee/out.md
+ exit_code=2
+ '[' 2 -ne 0 ']'
++ dirname lychee/out.md
+ mkdir -p lychee
+ echo 'Errors were reported while checking the availability of links:'
+ cat /tmp/lychee/out.md
+ cat /tmp/lychee/out.md
+ echo ::set-output name=exit_code::2
📝 Summary
---------------------
🔍 Total..........519
✅ Successful.....462
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded........53
🚫 Errors...........4

Errors in python/auto_examples/plot_train_convert_predict.html
✗ https://scikit-learn.org/stable/modules/preprocessing.html/nPlease [404 Not Found]

Errors in docs/reference/execution-providers/Nuphar-ExecutionProvider.md
✗ https://support.microsoft.com/en-us/help/841290 [404 Not Found]

Errors in docs/how-to/build.md
✗ https://cmake.org/files/v3.13/cmake-3.16.1.tar.gz [404 Not Found]
✗ https://www.raspberrypi.org/downloads/raspbian/ [404 Not Found]::set-output name=exit_code::2
1s
Run exit 
  exit 
  shell: /usr/bin/bash -e {0}

URL exclusion lists

I'm successfully leveraging the action for my blog, however I've noticed that some URLs are being caught accidentally (e.g., XML schema pointers) and I'd like to exclude them. Is there an out-of-the-box way to configure a URL exclusion list without fiddling with custom arguments?

Update README example to use `peter-evans/create-issue-from-file@v3`

The README example uses peter-evans/create-issue-from-file@v2, but a new version has been released. It would be awesome if the example was updated to work with the new Action version

create-issue-from-file can't find the lychee output

I'm trying this action, but can't make it work.

Workflow file I'm using: https://github.com/GenericMappingTools/pygmt/pull/634/files
One of the latest run: https://github.com/GenericMappingTools/pygmt/actions/runs/576905543

The action can correctly find broken links, and the results are saved in /tmp/lychee/output.md:

However, the create-issue-from-file action can't find the /tmp/lychee/output.md:

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total...........21
✅ Successful......19
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
🚫 Errors...........1

Errors in fixtures/TEST.md
✗ https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst (HTTP status server error (503 Service Unavailable) for url (https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst))

Full Github Actions output

Link Checker Report

Errors were reported while checking the availability of links.

Full Github Actions output

Link Checker Report

Errors were reported while checking the availability of links.


📝 Summary
---------------------
🔍 Total...........21
✅ Successful......19
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........1
🚫 Errors...........1

Errors in fixtures/TEST.md
✗ https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst (HTTP status server error (503 Service Unavailable) for url (https://www.peerlyst.com/posts/a-list-of-static-analysis-tools-for-c-c-peerlyst))

Full Github Actions output

Link Checker Report

Errors were reported while checking the availability of links.

📝 Summary

🔍 Total............8
✅ Successful.......7
⏳ Timeouts.........0
🔀 Redirected.......0
👻 Excluded.........0
🚫 Errors...........1

Input: README.md
⚡ lycheeverse/[email protected]
Runtime error (Invalid mail address: lycheeverse/[email protected])

Full Github Actions output

Comment on existing PR instead

Can the lychee-action comment on existing PR if link check fails instead of creating another issue?

Print Markdown File Path in the Console

Is it possible to also print to the build console which file it being referenced?

Here is a snippet:

✅ https://ubuntu.com/tutorials/how-to-install-ubuntu-on-your-raspberry-pi#1-overview [200 OK]
✅ https://ubuntu.com/tutorials/install-the-arduino-ide#1-overview [200 OK]
🚫 https://navo-org.github.io//navo-docs/docs/tutorials/dc-motor-rpm.html#dc-motor-speed-control-arduino-sketch [404 Not Found]
🚫 https://en.wikipedia.org/wiki/PID_controller_test_ [404 Not Found]

Could the source Markdown file be printed along with this?

Error report not begin generated

In entrypoint.sh

lychee-action/entrypoint.sh

Line 2 in b9d270d

set -euxo pipefail

I think the -e option causes the script to end as soon as an error is encountered. This will mean that the if statement below is never reached.

lychee-action/entrypoint.sh

Lines 15 to 19 in b9d270d

 if [ $exit_code -ne 0 ]; then 

 mkdir -p "$(dirname $LYCHEE_OUT)" 

 echo 'Errors were reported while checking the availability of links:' > $LYCHEE_OUT 

 cat "$LYCHEE_TMP" >> $LYCHEE_OUT 

 fi

And unfortunately that means the $LYCHEE_OUT report is never created.

Is this correct?

One alternative would be to change set -euxo pipefail to set -uxo pipefail

Alternatively, lychee could be executed as lychee "$@" --output "$LYCHEE_OUT" (and then the $LYCHEE_OUT file be removed if not needed.

"unable to get local issuer certificate"

I'm encountering some URLs where I get the following error:

https://www.mysuezwater.com/ (error sending request for url (https://www.mysuezwater.com/): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate))

A quick check seems to show a valid cert when visiting that site in Chrome. Is this caused by the root certs that are available on the ubuntu-latest image?

Toikio runtime error

Very promising utility! Thank you :)

We are trying to set this up in our CI, but hitting a tokio runtime error. Here is a trace produced with RUST_BAKCTRACKE=full:

thread 'tokio-runtime-worker' panicked at 'not currently running on the Tokio runtime.', /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.22/src/runtime/handle.rs:118:28
stack backtrace:
   0:     0x5596babccc70 - std::backtrace_rs::backtrace::libunwind::trace::h04d12fdcddff82aa
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/../../backtrace/src/backtrace/libunwind.rs:100:5
   1:     0x5596babccc70 - std::backtrace_rs::backtrace::trace_unsynchronized::h1459b974b6fbe5e1
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x5596babccc70 - std::sys_common::backtrace::_print_fmt::h9b8396a669123d95
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x5596babccc70 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::he009dcaaa75eed60
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:46:22
   4:     0x5596babf032c - core::fmt::write::h77b4746b0dea1dd3
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/fmt/mod.rs:1078:17
   5:     0x5596babc5af2 - std::io::Write::write_fmt::heb7e50902e98831c
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/io/mod.rs:1518:15
   6:     0x5596babcf295 - std::sys_common::backtrace::_print::h2d880c9e69a21be9
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:49:5
   7:     0x5596babcf295 - std::sys_common::backtrace::print::h5f02b1bb49f36879
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:36:9
   8:     0x5596babcf295 - std::panicking::default_hook::{{closure}}::h658e288a7a809b29
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:208:50
   9:     0x5596babcef38 - std::panicking::default_hook::hb52d73f0da9a4bb8
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:227:9
  10:     0x5596babcfa31 - std::panicking::rust_panic_with_hook::hfe7e1c684e3e6462
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:593:17
  11:     0x5596babcf577 - std::panicking::begin_panic_handler::{{closure}}::h42939e004b32765c
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:499:13
  12:     0x5596babcd12c - std::sys_common::backtrace::__rust_end_short_backtrace::h9d2070f7bf9fd56c
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys_common/backtrace.rs:141:18
  13:     0x5596babcf4d9 - rust_begin_unwind
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/panicking.rs:495:5
  14:     0x5596babedfb1 - core::panicking::panic_fmt::ha0bb065d9a260792
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/panicking.rs:92:14
  15:     0x5596babedd43 - core::option::expect_failed::h4c033402fb75f2bf
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/core/src/option.rs:1260:5
  16:     0x5596baa6eff1 - tokio::runtime::handle::Handle::current::ha96f9b87de88945a
  17:     0x5596ba9513c5 - tokio::runtime::blocking::pool::spawn_blocking::h48d8eb66ca56c37c
  18:     0x5596ba960efc - <hyper::client::connect::dns::GaiResolver as tower_service::Service<hyper::client::connect::dns::Name>>::call::ha6caa77d3238ba36
  19:     0x5596ba8785c3 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h6d2fdba67ca2a4d9
  20:     0x5596ba87b2d6 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::hb369d20dc3c4460c
  21:     0x5596ba883272 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::he5270fa0d1b0e32a
  22:     0x5596ba883954 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::hec84679a73862f34
  23:     0x5596ba891700 - <hyper::service::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll::h8248a346f62ae3cf
  24:     0x5596ba8f9f43 - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h188bbbc53295e174
  25:     0x5596ba8fbf7c - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::hb744b892a0754836
  26:     0x5596ba8c6db0 - <futures_util::future::try_future::try_flatten::TryFlatten<Fut,<Fut as futures_core::future::TryFuture>::Ok> as core::future::future::Future>::poll::h1e1a2f078ac740d9
  27:     0x5596ba9065a3 - <hyper::common::lazy::Lazy<F,R> as core::future::future::Future>::poll::h8ef353031301abfb
  28:     0x5596ba8fadfe - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h5449dc8c5939a14b
  29:     0x5596ba8ec733 - <futures_util::future::future::flatten::Flatten<Fut,<Fut as core::future::future::Future>::Output> as core::future::future::Future>::poll::hdbaa70c7872ee2d5
  30:     0x5596ba8fb2be - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h6f3f6b04dbb91bd4
  31:     0x5596ba8c7607 - <futures_util::future::try_future::try_flatten::TryFlatten<Fut,<Fut as futures_core::future::TryFuture>::Ok> as core::future::future::Future>::poll::hff06c07390f9fe4c
  32:     0x5596ba8db33e - <futures_util::future::poll_fn::PollFn<F> as core::future::future::Future>::poll::h190cce32350404e8
  33:     0x5596ba9540b1 - <hyper::client::ResponseFuture as core::future::future::Future>::poll::hd5d3e67d2e5a386d
  34:     0x5596ba8df6c8 - <reqwest::async_impl::client::PendingRequest as core::future::future::Future>::poll::hecb61596f5e67e74
  35:     0x5596ba8df5cd - <reqwest::async_impl::client::Pending as core::future::future::Future>::poll::heefe428c8b026eba
  36:     0x5596ba81a74b - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::hf4c2b3d55226607f
  37:     0x5596ba8275af - <futures_util::future::try_future::try_flatten::TryFlatten<Fut,<Fut as futures_core::future::TryFuture>::Ok> as core::future::future::Future>::poll::h7b0c5395a8d9cc10
  38:     0x5596ba819306 - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::ha9c46b590bb1036b
  39:     0x5596ba82684d - <futures_util::future::try_future::try_flatten::TryFlatten<Fut,<Fut as futures_core::future::TryFuture>::Ok> as core::future::future::Future>::poll::h153d5f01992c0c10
  40:     0x5596ba7d1669 - <futures_util::future::try_future::AndThen<Fut1,Fut2,F> as core::future::future::Future>::poll::hcfea5f076abfc10d
  41:     0x5596ba81905c - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h86c54c84c6395c1f
  42:     0x5596ba7d1639 - <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll::h32bd97ae42308282
  43:     0x5596ba53a76e - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h8e0397bc56e02a18
  44:     0x5596ba59cc21 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hcf4a58d87c105805
  45:     0x5596ba5f14f8 - tokio::runtime::task::harness::Harness<T,S>::poll::hdfec8d71d6ccb250
  46:     0x5596baa74721 - std::thread::local::LocalKey<T>::with::h848442eb0d5943c5
  47:     0x5596baa9190c - tokio::runtime::thread_pool::worker::Context::run_task::habb45a59caf9f439
  48:     0x5596baa90d85 - tokio::runtime::thread_pool::worker::Context::run::hb0566a7d8888162f
  49:     0x5596baa76b35 - tokio::macros::scoped_tls::ScopedKey<T>::set::h77527d491a9f0090
  50:     0x5596baa90446 - tokio::runtime::thread_pool::worker::run::h85c2f07fb5f23aca
  51:     0x5596baa8bf40 - tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut::heea0107c5a745ef1
  52:     0x5596baa95bff - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h623daca22f10b316
  53:     0x5596baa9d263 - tokio::runtime::task::harness::Harness<T,S>::poll::hd92b4305da53de1b
  54:     0x5596baa87b09 - tokio::runtime::blocking::pool::Inner::run::he43dd590140b6e88
  55:     0x5596baa7420e - std::sys_common::backtrace::__rust_begin_short_backtrace::h5929b2ece450dc2e
  56:     0x5596baa9a694 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h7b3347f8d58bd52e
  57:     0x5596babd330a - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h09ff301006f1aeca
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/alloc/src/boxed.rs:1307:9
  58:     0x5596babd330a - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::he79488c8f00b5f31
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/alloc/src/boxed.rs:1307:9
  59:     0x5596babd330a - std::sys::unix::thread::Thread::new::thread_start::h587efff279c68ba7
                               at /rustc/e1884a8e3c3e813aada8254edfa120e85bf5ffca/library/std/src/sys/unix/thread.rs:71:17
  60:     0x7fd4ac75ffa3 - start_thread
  61:     0x7fd4ac8914cf - clone
  62:                0x0 - <unknown>
🚫 https://lamport.azurewebsites.net/tla/current-tools.pdf/n// [404 Not Found]

In case it's to do with the source being parsed, the relevent section of HTML looks like this:

<a href="https://lamport.azurewebsites.net/tla/current-tools.pdf">Current Versions of the TLA+ Tools</a>.

(Unless it is trying to parse out of a code block, which I don't think it is).

I can report that running lychee 0.4.1 locally on the same content does not produce this problem.

For reference, here is a failing pipeline: https://github.com/informalsystems/apalache/runs/1670165829?check_suite_focus=true#logs

Link Checker Report

Summary

Status	Count
🔍 Total	23
✅ Successful	21
⏳ Timeouts	0
🔀 Redirected	0
👻 Excluded	1
❓ Unknown	0
🚫 Errors	1

Errors per input

Errors in README.md

https://opensource.org/licenses/MIT: Failed: HTTP status server error (503 Service Unavailable) for url (https://opensource.org/licenses/MIT)

Full Github Actions output

Workflow not failing

Hi there!

I have seen #20 and #37, which seem to indicate that depending on version, I should see a failure either with or without exit ${{ steps.lychee.outputs.exit_code }} in the following step.

I am not seeing a failure in either case. That is to say, the workflow does not fail, even though it should.
See the Check links step on https://github.com/exercism/c/pull/706/checks?check_run_id=3888138029

Between the two mentioned issues, I a now confused as to what is the correct way to achieve this.
Could you clarify?

Thanks!

	if [ $exit_code -ne 0 ]; then
	mkdir -p "$(dirname $LYCHEE_OUT)"
	echo 'Errors were reported while checking the availability of links:' > $LYCHEE_OUT
	cat "$LYCHEE_TMP" >> $LYCHEE_OUT
	fi

lycheeverse / lychee-action Goto Github PK

lychee-action's People

Contributors

Stargazers

Watchers

Forkers

lychee-action's Issues

Examples - Workflows that can be used currently

Manually generating the file paths to check

Delegate to tj-actions/changed-files action to get the file paths

Simpler version when file paths of inputs would never contain spaces

Thanks for the action. It's much better than what we used before 😉

📝 Summary

Errors were reported while checking the availability of links: 📝 Summary

📝 Summary

Summary

Errors per input

Errors in README.md

Recommend Projects

Recommend Topics

Recommend Org

Delegate to `tj-actions/changed-files` action to get the file paths

Errors were reported while checking the availability of links:
📝 Summary