Hi,

Does this report supports GitHub Enterprise server?

We have tried using this, but the api call failed.
{
"message": "Not Found",
"documentation_url": "https://docs.github.com/rest/commits/comments#list-commit-comments"
}

As per the readme doc, it is possible to show the report to summary. In that case, how should we disable sha and pull_request_id?

set-env is deprecated

Hello!

I'm trying to implement a workflow that uses this action, and it works wonderfully when I open a PR from a branch created directly on my main repo.

However, when a PR is coming from a fork, the action lacks the required permissions to leave the comment. I try to specifically grant the token in my .yaml write-all permissions, and all settings I could find in the main repo should allow actions to have write permissions as well.

Has anyone faced a similar issue? Am I doing something wrong?
This is the yaml for reference:

name: dryrun

on: [push, pull_request]

permissions: write-all

jobs:
  dryrun:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Execute dryrun
        run: |
          <dryrun commands>
      - name: Parse output
        uses: joonvena/[email protected]
        with:
          report_path: test-output
          gh_access_token: ${{ secrets.GITHUB_TOKEN }}
          show_passed_tests: false

And this is the error I get:

Run joonvena/[email protected]
Run export OWNER="$(echo "red-hat-data-services/ods-ci" | awk -F / '{print $1}' | sed -e "s/:refs//")"
Run /home/runner/work/_actions/joonvena/robotframework-reporter-action/v2.1/report.sh
Unable to find image 'joonvena/robot-reporter:v2.1' locally
v2.1: Pulling from joonvena/robot-reporter
df9b9388f04a: Already exists
f265c46fcbe2: Pulling fs layer
8aee577374: Pulling fs layer
6fad9521d008: Pulling fs layer
8aee57710374: Verifying Checksum
8aee57710374: Download complete
6fad9521d008: Verifying Checksum
6fad9521d008: Download complete
f265c46fcbe2: Verifying Checksum
f265c46fcbe2: Download complete
f265c46fcbe2: Pull complete
8aee57710374: Pull complete
6fad9521d008: Pull complete
Digest: sha256:9910a6014905fb7053ef8486fac7346db0009e96511f14c7606184ffb9f5f60c
Status: Downloaded newer image for joonvena/robot-reporter:v2.1
2022/10/26 16:00:16 POST https://api.github.com/repos/red-hat-data-services/ods-ci/issues/604/comments: 403 Resource not accessible by integration []
Error: Process completed with exit code 1.

Hiya!

I'm attempting to use this action to comment robot logs to a commit. The repository is a fork, so due to permission issues I have to upload the commit's SHA number as an artifact from the testing workflow, and pass it to the separate robot log reporting workflow.

I'm currently getting this error:

2023/08/23 11:11:06 POST https://api.github.com/repos/{organisation and repository}/commits/%223b9fc46f01591b84c4950f6ec53f00b3614591c%22/comments: 422 No commit found for SHA: "3b9fc46f01591b84c4950f6ec53f00b3614591c" []
Error: Process completed with exit code 1.

The %22 in the url tells that there are extra quotes around the sha number when passed to your action, but I have not come to a solution where I could remove them on my end. Workflow file parses the SHA number as follows:

jobs:
  download:
    runs-on: ubuntu-latest
    steps:
      - name: 'Download artifact'
        uses: actions/[email protected]
        with:
          script: |
            var artifacts = await github.actions.listWorkflowRunArtifacts({
              owner: context.repo.owner,
              repo: context.repo.repo,
              run_id: ${{github.event.workflow_run.id }},
            });
            var matchArtifact = artifacts.data.artifacts.filter((artifact) => {
              return artifact.name == "sha"
            })[0];
            var download = await github.actions.downloadArtifact({
              owner: context.repo.owner,
              repo: context.repo.repo,
              artifact_id: matchArtifact.id,
              archive_format: 'zip',
            });
            var fs = require('fs');
            fs.writeFileSync('${{github.workspace}}/sha.zip', Buffer.from(download.data));
      - run: unzip sha.zip
      - name: 'Get sha Number'
        id: fetch_sha
        uses: actions/github-script@v6
        with:
          script: |
            var fs = require('fs');
            var issue_number = String(fs.readFileSync('./NR'));
            return issue_number.slice(0, -2).toString();
          # parses commit sha from artifact, removes \n from the end of the string
      - name: 'Send report to commit'
        uses: joonvena/[email protected]
        with:
          report_path: reports
          gh_access_token: ${{ secrets.GITHUB_TOKEN }}
          show_passed_tests: true
          sha: ${{steps.fetch_sha.outputs.result}}

The ${{steps.fetch_sha.outputs.result}} variable apparently uses JSON.stringify which adds quotes around a string variable.
In your action.yml the SHA number is used like this...

COMMIT_SHA: "${{ inputs.sha }}"

... which I am guessing might be the reason it is getting double-quoted..?

So, questions:

• Can anyone suggest any solution I could do on my end to get around this problem? And if not,
• Could you remove the quotes around ${{ inputs.sha }} variable on action.yml, or add some conditional formatting in it to prevent double-quotes around the variable?

Thank you in advance! 💯

Hi,

I have two different directories under reports like
reports/testsuite/testsuite1/output.xml
reports/testsuite/testsuite2/output.xml

When I try to read these xml files using below code
- name: Send reports summery
uses: joonvena/[email protected]
with:
gh_access_token: ${{ secrets.GITHUB_TOKEN }}
report_path: reports/* / *

it is searching for .../reposts/ * / */ output.xml, it is not considering * as sub directory.
I tried with reports/ ** / ** and 'reports/ ** / **' and 'reports/ * / *' all the combinations it is not reading the file under sub directories.
Can you please help me on how to read all output.xlm files from sub directories.

NOTE: I gave spaces after * and / in this query for readability, in real project I didn't give spaces

When there is a scenario with execution failure, the result of approved tests is being shown with ***.

I have this piece of code running on Github actions:

generate_report:
        if: always()
        needs: [Run-Api-Tests] 
        runs-on: ubuntu-22.04
        steps:
        - uses: actions/checkout@v2
        - name: Download reports
          uses: actions/download-artifact@v1
          with:
            name: reports
        - name: Send report to commit
          uses: joonvena/robotframework-reporter-action@v2
          with:
            gh_access_token: ${{ secrets.TOKEN }}
            summary: true

According to the documentation, adding summary: true should output the test summary but I get this error inside the Action output: Unexpected input(s) 'summary', valid inputs are ['gh_access_token', 'report_path', 'sha', 'pull_request_id']

We noticed a problem with commenting test results to PR and to commit messages when our tests were failing badly and there was lot of error messages in the failed tests table.

The error message we got was: PATCH https://api.github.com/repos/***ltd/xxx/issues/comments/2009605523: 422 Validation Failed [{Resource:IssueComment Field:body Code:custom Message:body is too long (maximum is 65536 characters)}]

This is most probably related to Github limitation mshick/add-pr-comment#93 (comment).

Would it be possible to cut the error messages shorter in case that the body would be longer than the maximum length allowed?

How to reproduce:

• Only summary = false
• Run tests so that really many tests are failing and producing error logs
  -> check the outcome

PR #31 broke the bash shell script used to set the repository owner and name by accidentally using single quotes instead of double quotes resulting in the a failing of expansion of variables.

The problem would manifest when using joonvena/[email protected], an example of the error is shown below:

Run joonvena/[email protected]
Run export OWNER='$(echo 'thin-edge/tedge-monit-setup' | awk -F / '{print $1}' | sed -e 's/:refs//')'
/home/runner/work/_temp/8a3a4c80-e9df-4cf9-aa36-e7e6386078e8.sh: line 1: export: `} | sed -e s/:refs//)': not a valid identifier
Error: Process completed with exit code 1.

Discovered on the following build

Hello,

In my current organisation, we've got a specific repository (say robot-tests) containing robot framework tests and corresponding secrets. In order to run those tests after deployment of any service stored in other repositories (serviceA, serviceB), I need to dispatch a workflow. I insist on the workflow_dispatch event because the workflow_call event from another repo cannot access secrets stored for robot-tests repository (and it's a bad practice to store everything at organisation level).

Therefore, I would like to enable this action to support passing another repository as input.

Following what I could see in the action code, the simplest way would be to add

repository:
    type: string
    description: Repository ([Owner/]Repository) of pull request to decorate with report
    default: "${{github.repository}}"

and modify the bash step to parse it instead of direct parsing of ${{github.repository}}

I will make a PR accordingly if you agree with it.

Could the outputs of the report be shown also as annotation in the relevant workflow run? This would help to find the test result summary in case that the testing workflow is started manually or by scheduler.

The place for the info like in the image below, but with the same content as you post to comments at the moment.

Hi there 👋

I'm using your action on self-hosted runners.
In my configuration the host were the robot-reporter container runs is reused.
Consequently, after completion these containers are never removed.

I suggest to add the --rm option when running the container here:

https://github.com/joonvena/robotframework-reporter-action/blob/413a90b26bb8a49eebc51af51f1631500ef0339d/report.sh#L3C1-L3C1

WDYT?

Thanks & have a nice day!

invalid character 'm' after object key:value pair Error: Process completed with exit code 1.

Test setting:
pabot runs robot tests parallel , generating output.xml
pabot re-runs failed test cases creating rerun.xml

rebot merges output.xml and rerun.xml creating a new output.xml file with merged results

Looks like the tool fails to read new output.xml file

Duration of test is displayed as 0s. Not sure if it is caused by only one test in the suite, but based on xml file there is no duration property. Attaching screenshot and xml file in zip for your reference.

output.zip

Hi
It would be really useful if it worked on windows as well. Now if I run it on windows-latest which has bash https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md I get this error as Windows bash does not understand Windows path separators: D:\a\_actions\joonvena\robotframework-reporter-action\v2.5/report.sh transforms to D:a_actionsjoonvenarobotframework-reporter-actionv2.5/report.sh

Here is the log I get when running on windows-latest (I wiped out owner and project-specific information):

▼ Run joonvena/[email protected]
  with:
    gh_access_token: ***
    report_path: <Path>
    sha: <sha>
    pull_request_id: 13
    summary: true
    only_summary: false
    show_passed_tests: true
    failed_tests_on_top: false
▼ Run export OWNER="$(echo '<Owner>/<Repo>' | awk -F / '{print $1}' | sed -e 's/:refs//')"
  export OWNER="$(echo '<Owner>/<Repo>' | awk -F / '{print $1}' | sed -e 's/:refs//')"
  export REPO="$(echo '<Owner>/<Repo>' | awk -F / '{print $2}' | sed -e 's/:refs//')"
  echo "REPOSITORY_OWNER=$OWNER" >> $GITHUB_ENV
  echo "REPOSITORY_NAME=$REPO" >> $GITHUB_ENV
  shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
▼ Run D:\a\_actions\joonvena\robotframework-reporter-action\v2.5/report.sh
  D:\a\_actions\joonvena\robotframework-reporter-action\v2.5/report.sh
  shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
  env:
    REPOSITORY_OWNER: <Owner>
    REPOSITORY_NAME: <Repo>
    GH_ACCESS_TOKEN: ***
    REPORT_PATH: <Path>
    REPOSITORY: <Repo>
    COMMIT_SHA: <sha>
    PR_ID: 13
    SUMMARY: true
    GITHUB_STEP_SUMMARY: $GITHUB_STEP_SUMMARY
    ONLY_SUMMARY: false
    SHOW_PASSED_TESTS: true
    GITHUB_API_URL: $GITHUB_API_URL
    FAILED_TESTS_ON_TOP: false
D:\a\_temp\707787c6-9cca-4ffb-9d42-73afba985f01.sh: line 1: D:a_actionsjoonvenarobotframework-reporter-actionv2.5/report.sh: No such file or directory
Error: Process completed with exit code 127.

When dispatching jobs to different runners using a strategy matrix:

    strategy:
      matrix:
        runner_type: [A, B]
    runs-on: ["self-hosted", "${{ matrix.runner_type }}"]

The comments of the github-bot are overwritten by the two runners (Can be recovered by looking at the history of the comment).

Is there a way to have a separate report per runner (but also not overwrite comments for repeated action-runs)?

There are tests run, but the total amount shows 0:

output.xml attached as a reference
output.zip

Hi guys,

During the "Send report to commit" step, we are getting a "permission denied" error when trying to access a summary file. As far as I can see on the server, the file has permission 644, so theoretically, it should be accessible.

Has anyone encountered this issue before?

Run /opt/actions-runner/_work/_actions/joonvena/robotframework-reporter-action/v2.1/report.sh

2023/06/08 13:25:47 open /opt/actions-runner/_work/_temp/_runner_file_commands/step_summary_7e756204-7af3-4e92-9c2a-ccaa7f359f37: permission denied

2023/06/08 13:25:47 invalid argument

workflow:

  generate_report:
    if: always()
    needs: [ tests ]
    runs-on: [ "self-hosted","linux","x64","test-runners" ]
    steps:
      - uses: actions/checkout@v2
      - name: Download reports
        uses: actions/download-artifact@v1
        with:
          name: reports
      - name: Send report to commit
        uses: joonvena/[email protected]
        with:
          gh_access_token: ${{ secrets.GITHUB_TOKEN }}

Could the pushed comment also include a list of successful robot tests? This would help demonstrate that user stories have been completed.

D:\a_temp\2e76fc8a-08bb-4c1d-8707-628ed09676a0.sh: line 1: D:a_actionsjoonvenarobotframework-reporter-actionv2.1/report.sh: No such file or directory
Error: Process completed with exit code 127.

below is my ymal code for it

• name: Robot Reporter
  # You may pin to the exact commit or the version.
  # uses: 1f1268f
  uses: joonvena/[email protected]
  with:
  # Token to access the API
  gh_access_token: ${{ secrets.GITHUB_TOKEN }}

     # Path to report
     report_path: downloads
     
     # SHA of the commit tha triggered the tests
     sha: ${{ github.sha }}
     
     # ID of the Pull Request
     pull_request_id: ${{ github.event.number }}
     
     # Add report to job summary
     summary: true
     
     # Only output report to job summary
     only_summary: false
     
     # If true only failed tests are shown
     show_passed_tests: true
  

This error started to appear yesterday and after some research on the internet that the info that I found could help to fix this issue: https://www.joeshaw.org/understanding-go-panic-output/
The error below is what I can see when running your reporter

open template.txt: no such file or directory
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x574cdb]

goroutine 1 [running]:
html/template.(*Template).escape(0x0, 0x0, 0x0)
	/usr/local/go/src/html/template/template.go:97 +0x3b
html/template.(*Template).Execute(0x0, 0x810340, 0xc000380600, 0x743c40, 0xc0003805d0, 0x2d, 0x0)
	/usr/local/go/src/html/template/template.go:121 +0x2f
main.main()
	/build/main.go:251 +0xd1c

In the report, the duration which is always showing in secs.

Please change that to HH::MM::SS format