Hi! Just caught this weird behavior today using this action where Merge Gatekeeper is considering previously failed runs and it keeps failing. Look at the log outputs:

Start processing validator: merge-gatekeeper....
Finish validator: merge-gatekeeper processing.
Error: validation failed, err: [10](https://github.com/my-organization/my-repo/runs/5407659157?check_suite_focus=true#step:3:10) out of [12](https://github.com/my-organization/my-repo/runs/5407659157?check_suite_focus=true#step:3:12)

  Total job count:     12
    jobs: ["code-quality" "label" "Validate PR title" "Validate PR title" "label" "label" "Validate PR title" "label" "Validate PR title" "code-quality" "label" "Validate PR title"]
  Completed job count: 10
    jobs: ["label" "Validate PR title" "Validate PR title" "label" "label" "Validate PR title" "label" "Validate PR title" "code-quality" "label"]
  Failed job count:    1
    jobs: ["Validate PR title"]

As you can see, the job log displays all previous runs of the "Validate PR title" job, including the failed ones, which prevents the Merge Gatekeeper job to succeed after we already fixed the problem (I'm using this action to validate PR titles: https://github.com/amannn/action-semantic-pull-request).

Steps to reproduce:

• Open a pull request with an "invalid name"
• Validation jobs will run and fail first
• Merge Gatekeeper will finish running and fail
• Change PR to a "valid name"
• Validation jobs will run and succeed
• Merge Gatekeeper does not re-run nor updates its failed state to reflect the
• Succeding runs does not exclude previously failed attempts

This behavior continues even after closing/ reopening or opening a completely new PR unless I change the commit hash

PS.: One of the things that solved it for me was changing the commit history so the "job cache" wouldn't take into consideration previous runs of failed jobs.

It would be cool if we could support regex matching on ignored jobs. This way we can specify something like deploy-* and have it ignore all the deploy jobs

I'm not 100% sure, but I think the current logic only checks for the successful job setup, and goes into loop until they all succeed, regardless of the actual target job status.

We should be probably failing early if there is any other job that has failed quickly. Having Merge Gatekeeper running for 10 min is only wasteful of GitHub Action minutes.

We are happily using Merge-Gatekeeper in our github projects.

We are also looking into Github Merge Queues, which are GA since July.
I played a little bit and triggered merge-gatekeeper for github.event_group and handing down the github ref. It kinda looked promising but It had some side affects. It stopped working for PRs. ;-).

But since you do not officially support merge queues I will not report a bug. Instead I would like to ask for that support? Or guidence how to configure merge-gatekeeper for merge queues if it support it by default?

Hi,

When executing the merge-gatekeeper on self-hosted runner I'm phasing the following error:
Build container for action use: '/runner/_work/_actions/upsidr/merge-gatekeeper/v1.2.1/Dockerfile'. Error: Docker build failed with exit code 1

In order to avoid this error, I need to implement one of the following:

1. find a way to authenticate the docker commands
2. Execute the merge-gatekeeper workflow on GitHub managed runners instead of my org self-hosted runners.

There's a simple solution or guidelines for the first option?

This project currently assumes all of the jobs need to pass. It's probably useful to have a whitelist / blacklist feature to suppress some errors that are irrelevant.

My self-hosted runners are running inside of Docker itself. I have not made them allowed to use docker, therefore I sadly cannot use this project as-is.

Previously I've made several other run by just pulling the actual pre-built binary and caching it, and running it directly. Since this is a Go project, it should be possible to create a self-contained binary. If added as a release artifact to every release, the action could in fact just download the binary from that release thus not needing Docker at all. It would also get much faster cold-start time as the binary would only need to be downloaded to start.

That way you wouldn't need to rewrite the project in JavaScript either as I've seen mentioned around the issues ;)

It'd be helpful to be able to retry in the event of a merge-gatekeeper failure

This is essentially a mirror of #17. We have got the "ignored" list where some jobs can be ignored from Merge Gatekeeper checks.
It will clearly be a great addition to have an extra control to list jobs that should be checked rather than checking all jobs.

Some discussion points for this support:

• Name for the flag - the ones ignored are handled as "ignored"; words such as "included" may be clear enough?
• When the job is specified for both lists (ignored and included), it is arguable how this should be handled. It can be 1.) a configuration error and fail with such message, 2.) ignored gets precedence, 3.) included gets precedence, 4.) determined by another flag to handle which one gets precedence.

Thank you for making this Action, super helpful! When deploying this action, I noticed that restricting the permissions to the following seemed sufficient:

name: merge-gatekeeper
on:
  pull_request:
    branches:
    - main

jobs:
  merge-gatekeeper:
    runs-on: ubuntu-latest
    permissions: # Restrict permissions. See https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
      checks: read
      statuses: read
    steps:
    - name: Run Merge Gatekeeper
      uses: upsidr/[email protected]
      with:
        token: ${{ secrets.GITHUB_TOKEN }}

I think it would be helpful to update the docs with these permissions if they look correct. Happy to make a PR with these docs changes!

I have a matrix job that tests a component in two configurations. When one fails, Merge Gatekeeper passes - the logs don't show my matrix jobs in the list reported for each status.

My matrix job is a Python library and is configured something like this:

jobs:
  test:
    runs-on: ubuntu-18.04
    strategy:
      matrix:
        tox-env: [env1, env2]
      fail-fast: false
    steps:
      - uses: actions/checkout@v2
      - run: tox -e ${{ matrix.tox-env }}

Hi,
I faced with an issue
The gatekeeper is checking webhook job correct only for status Successful

Total job count:     2
    jobs: ["atlantis/plan" "yaml: lint"]
  Completed job count: 2
    jobs: ["atlantis/plan" "yaml: lint"]
  Failed job count:    0
    jobs: []

The gatekeeper doesn't verify if webhook job is fail and that isn't stopped after timeout.

As you see only one GHA displayed

Hello Guys, I have configured merge-gatekeeper as a workflow by following the doc, but when I do the PR to my repository merge-gatekeeper is getting triggered but it says 0 out of 0 jobs found. I need like if any of the other workflows in the repo fails that should stop the merge(as the merge-gatekeeper acts as a check). Please help.

As said in the GitHub doc:

When using GITHUB_TOKEN, the rate limit is 1,000 requests per hour per repository.

src

This isn't a lot, if you check every 30 secs and have 9 PRs continuously checking you will cross the limit (and the GITHUB_TOKEN can be used for other queries).

Do you know a workaround or this is just the way it is?

Hi,

I'm using merge gatekeeper as a required workflow for all repos

sometimes when i open a pr
the merge gatekeeper already failed, when no job actually failed or even started to run

it happens in 60% of the time
my workaround is to commit an empty commit and then everything works as expected

what the issue could be?
maybe some kind of race?

Hi there,
I am using reusable workflows and most of the workflows trigger the same set of jobs like

  - build-and-test / build
  - build-and-test / terraform-plan (prd, xwrd)
  - build-and-test / terraform-plan (dev, xwrd)
  - check-prstats / stats

It turns out that merge-gatekeeper v1.2.0 treats jobs with a common name as one job, and if one workflow succeeds while others are still running, it returns success. If any workflow fail before the first success then it returns failure

Problem

Currently, the logging for execution looks something like this:

Total job count:     3
    jobs: ["my_job" "my_job2" "my_job3"]
  Completed job count: 2
    jobs: ["my_job" "my_job2"]
  Failed job count:    0
    jobs: []

This is fine for small amounts of jobs but once the list expands it becomes difficult to read the logs. Example

Total job count:     X
    jobs: ["my_job" "my_job2" "my_job3" "my_job" "my_job2" "my_job3" "my_job" "my_job2" "my_job3" "my_job" "my_job2" "my_job3" "my_job" "my_job2" "my_job3" "my_job" "my_job2" "my_job3" "my_job" "my_job2" "my_job3"]
  Completed job count: Y
    jobs: ["my_job" "my_job2" "my_job" "my_job2" "my_job3" "my_job" "my_job2" "my_job3""my_job" "my_job2" "my_job3""my_job" "my_job2" "my_job3" "my_job" "my_job2" "my_job3" "my_job" "my_job2" "my_job3"]
  Failed job count:    0
    jobs: []

(formatting in issues is slightly different than what is output in GitHub actions so YMMV on this)

Request

Part 1

print each job on a new line (yaml - esk), for example

Total job count:     3
    jobs: 
      - my_job
      - my_job2 
      - my_job3
  Completed job count: 2
    jobs:
      - my_job
      - my_job2 
  Failed job count:    0
    jobs: []

Part 2

Add a new section of the log that prints incomplete jobs (waiting for)

Total job count:     3
    jobs: 
      - my_job
      - my_job2 
      - my_job3
  Incomplete job count: 1
    jobs:
      - my_job3 
  Completed job count: 2
    jobs:
      - my_job
      - my_job2 
  Failed job count:    0
    jobs: []

I can split this into separate issues if wanted, lemme know. I can get started on a PR for this if the idea is liked

Currently this action does not supply a API URL to the GitHub client:

According to the documentation this causes the client to default to the GitHub.com API. Within Actions the correct API URL for the instance is available as GITHUB_API_URL.

I'd be happy to provide a PR to fix this!

Hi everyone,

Sometimes when merge-gatekeeper tries to call GitHub API, GitHub returns 5xx errors or EOF and they tend to be all transient network errors.

Would you be open to adding retries to GitHub API calls? I'm happy to discuss more and add the functionality.

Initial thoughts were adding retry flag to the cli and adding a retry roundTripper to the GitHub client.

Hello, thank you for all your work in building this amazing tool!

I wanted to run an idea by you all before coding it up, to see if it would make it into the main branch, or if it would be better off landing in a fork. Basically, in a repo that uses merge-gatekeeper, I want to be able to add something in my PR's title (such as [skip-mg], or whatever), which immediately causes merge-gatekeeper's check to succeed (e.g. I have a PR that's just updating documentation, or something).

This functionality already exists in GHA: you can skip an entire workflow run by adding a magic value somewhere in either the PR title or commit message: https://docs.github.com/en/actions/managing-workflow-runs/skipping-workflow-runs. Those values are:

• [skip ci]
• [ci skip]
• [no ci]
• [skip actions]
• [actions skip]

The problem is that merge-gatekeeper's design conflicts with those magic keywords. Because merge-gatekeeper is a required status check, if you mark a PR as [skip ci], then merge-gatekeeper itself never runs, and the PR becomes unmergeable, since the required status check was skipped.

The idea would be to have merge-gatekeeper look for a string like [skip-mg], and immediately exit as a success if it detects this string in the commit message / PR title — I believe we should have access to at least one of those values in the pull_request context object, so no additional network requests required or anything too complicated. A new special value like [skip-mg] would avoid the aforementioned conflict of e.g. [skip ci].

Please let me know your thoughts, and if this has already been considered. I am happy to spend some time on this if you believe it has the potential to be merged. Appreciate your consideration.

Hi,
I faced an issue
When PR has more than 30 jobs the merge-gatekeeper replace some job in the array list and if some job fails the MG can skip because that job is not in the list

  Total job count:     30
    jobs: ["unit-tests (test1)" "unit-tests (test2)" "unit-tests (test3)" "unit-tests (test4)" "unit-tests (test5)" "unit-tests (test6)" "unit-tests (test7)" "unit-tests (test8)" "unit-tests (test9)" "unit-tests (test10)" "unit-tests (test11)" "unit-tests (test12)" "unit-tests (test13)" "unit-tests (test14)" "unit-tests (test15)" "unit-tests (test16)" "unit-tests (test17)" "unit-tests (test18)" "unit-tests (test19)" "unit-tests (test20)" "unit-tests (test21)" "unit-tests (test22)" "unit-tests (test23)" "unit-tests (test23)" "unit-tests (test24)" "unit-tests (test25)" "unit-tests (test25)" "unit-tests (test26)" "unit-tests (test27)" "unit-tests (test28)"]
  Completed job count: 29
    jobs: ["unit-tests (test1)" "unit-tests (test2)" "unit-tests (test4)" "unit-tests (test5)" "unit-tests (test6)" "unit-tests (test7)" "unit-tests (test8)" "unit-tests (test9)" "unit-tests (test10)" "unit-tests (test11)" "unit-tests (test12)" "unit-tests (test13)" "unit-tests (test14)" "unit-tests (test15)" "unit-tests (test16)" "unit-tests (test17)" "unit-tests (test18)" "unit-tests (test19)" "unit-tests (test20)" "unit-tests (test21)" "unit-tests (test22)" "unit-tests (test23)" "unit-tests (test23)" "unit-tests (test24)" "unit-tests (test25)" "unit-tests (test25)" "unit-tests (test26)" "unit-tests (test27)" "unit-tests (test28)"]
  Failed job count:    0
    jobs: []

Finish validator: merge-gatekeeper processing.

  WARNING: Validation is yet to be completed. This is most likely due to some other jobs still running.
           Waiting for 15 seconds before retrying.

Start processing validator: merge-gatekeeper....
30 out of 30

  Total job count:     30
    jobs: ["Services Unit Tests" "unit-tests (test1)" "unit-tests (test2)" "unit-tests (test3)" "unit-tests (test4)" "unit-tests (test5)" "unit-tests (test6)" "unit-tests (test7)" "unit-tests (test8)" "unit-tests (test9)" "unit-tests (test10)" "unit-tests (test11)" "unit-tests (test12)" "unit-tests (test13)" "unit-tests (test14)" "unit-tests (test15)" "unit-tests (test16)" "unit-tests (test17)" "unit-tests (test18)" "unit-tests (test19)" "unit-tests (test20)" "unit-tests (test21)" "unit-tests (test22)" "unit-tests (test23)" "unit-tests (test23)" "unit-tests (test24)" "unit-tests (test25)" "unit-tests (test25)" "unit-tests (test26)" "unit-tests (test27)"]
  Completed job count: 30
    jobs: ["Services Unit Tests" "unit-tests (test1)" "unit-tests (test2)" "unit-tests (test3)" "unit-tests (test4)" "unit-tests (test5)" "unit-tests (test6)" "unit-tests (test7)" "unit-tests (test8)" "unit-tests (test9)" "unit-tests (test10)" "unit-tests (test11)" "unit-tests (test12)" "unit-tests (test13)" "unit-tests (test14)" "unit-tests (test15)" "unit-tests (test16)" "unit-tests (test17)" "unit-tests (test18)" "unit-tests (test19)" "unit-tests (test20)" "unit-tests (test21)" "unit-tests (test22)" "unit-tests (test23)" "unit-tests (test23)" "unit-tests (test24)" "unit-tests (test25)" "unit-tests (test25)" "unit-tests (test26)" "unit-tests (test27)"]
  Failed job count:    0
    jobs: []

As you see logs, the unit-tests (test28) was replaced Services Unit Tests.

It would be nice if the MK has an additional flag to increase the limit of jobs, like job-limit: 50

Docs and readme say that the default interval is 30 seconds, but in the code it's 5. Would be cool if the docs could be updated. :)

Also, I just want to say how big of a help this action is for us. We are using it together with the GH merge queue to finally set up the workflow how we need it. Thank you for creating this! We were almost ready to leave Github Actions behind because of this feature missing.

The current default is 30 sec. It works in most cases, but I think we should be able to query a bit more frequently, perhaps 5 sec or 10 sec.

Hello !

First thanks for this project :).

I'm working with merge-gatekeeper as a github action like this:

      - name: Run Merge Gatekeeper
        uses: upsidr/merge-gatekeeper@v1
        with:
          token: whatever

It seems that for our projects every Pull Requests triggers this action which always build a docker image using this Dockerfile.
The FROM uses a default docker repository which is prone to rate limits.

To my knowledge there is currently no way to avoid the issue, here are random thoughts to tackle that in the future:

• Allow to override the base repository
• Use amazon ECR repository
• Something else ?

What do you think ?

Original request from #43

Similar to #39, this PR removes trailling and leading whitespace from ignored job. It also adds support for using newlines in the ignored jobs. This should allow users to use merge gatekeeper in the following way:

jobs:
  merge-gatekeeper:
    runs-on: ubuntu-latest
    steps:
      - name: Run Merge Gatekeeper
        uses: upsidr/merge-gatekeeper@v1
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          ignored: |
            my_job1,
            my_job2,
            my_job3

--

We have merged a partial fix with #39, which adds this support to some extent. But there should be a better handling including providing more details as to which jobs are ignored.