Giter VIP home page Giter VIP logo

github / evergreen Goto Github PK

View Code? Open in Web Editor NEW
168.0 103.0 12.0 416 KB

GitHub Action to enable automated security updates and open a issue/PR in repos in an org that have dependency files but no dependabot.yaml file

Home Page: https://github.blog/2024-01-25-do-you-know-if-all-your-repositories-have-up-to-date-dependencies/

License: MIT License

Dockerfile 0.52% Makefile 0.67% Python 98.81%
dependabot dependabot-security dependencies org-admin ospo

evergreen's Introduction

Evergreen action

CodeQL Lint Code Base Python package OpenSSF Scorecard

This is a GitHub Action that given an organization or specified repositories, opens an issue/PR if dependabot is not enabled, or there are more package ecosystems that could be. It also enables automated security updates for the repository.

This action was developed by the GitHub OSPO for our own use and developed in a way that we could open source it that it might be useful to you as well! If you want to know more about how we use it, reach out in an issue in this repository.

Example use cases

  • As a part of the security team for my company, I want to make sure that all of the repositories in the company organizations are regularly updating their dependencies to ensure they are using the most secure version of the dependency available.
  • As an OSPO or maintainer, I want to automate everything I can to keep maintaining my project(s) easy and efficient.

Support

If you need support using this project or have questions about it, please open up an issue in this repository. Requests made directly to GitHub staff or support team will be redirected here to open an issue. GitHub SLA's and support/services contracts do not apply to this repository.

OSPO GitHub Actions as a Whole

All feedback regarding our GitHub Actions, as a whole, should be communicated through issues on our github-ospo repository.

Use as a GitHub Action

  1. Create a repository to host this GitHub Action or select an existing repository.
  2. Select a best fit workflow file from the examples below.
  3. Copy that example into your repository (from step 1) and into the proper directory for GitHub Actions: .github/workflows/ directory with the file extension .yml (ie. .github/workflows/evergreen.yml)
  4. Edit the values (ORGANIZATION, REPOSITORY, EXEMPT_REPOS, TYPE, TITLE, BODY) from the sample workflow with your information. If running on a whole organization then no repository is needed. If running the action on just one repository or a list of repositories, then no organization is needed. The type should be either issue or pull representing the action that you want taken after discovering a repository that should enable dependabot.
  5. Optionally, edit the value (CREATED_AFTER_DATE) if you are setting up this action to run regularly and only want newly created repositories to be considered. Otherwise, if you want all specified repositories regardless of when they were created to be considered, then leave blank.
  6. Optionally edit the value (UPDATE_EXISTING) if you want to update existing dependabot configuration files. If set to true, the action will update the existing dependabot configuration file with any package ecosystems that are detected but not configured yet. If set to false, the action will only create a new dependabot configuration file if there is not an existing one. The default value is false.
  7. Also edit the value for GH_ENTERPRISE_URL if you are using a GitHub Server and not using github.com. For github.com users, don't put anything in here.
  8. Update the value of GH_TOKEN. Do this by creating a GitHub API token with permissions to read the repository/organization and write issues or pull requests depending on what you put in for the TYPE. Additionally, you should set the administration:write permission on every repository in scope to successfully enable security updates. Then take the value of the API token you just created, and create a repository secret where the name of the secret is GH_TOKEN and the value of the secret the API token. Then finally update the workflow file to use that repository secret by changing GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} to GH_TOKEN: ${{ secrets.GH_TOKEN }}. The name of the secret can really be anything. It just needs to match between when you create the secret name and when you refer to it in the workflow file.
  9. If you want the resulting issue with the output to appear in a different repository other than the one the workflow file runs in, update the line token: ${{ secrets.GITHUB_TOKEN }} with your own GitHub API token stored as a repository secret. This process is the same as described in the step above. More info on creating secrets can be found here.
  10. Commit the workflow file to the default branch (often master or main)
  11. Wait for the action to trigger based on the schedule entry or manually trigger the workflow as shown in the documentation.

Configuration

Below are the allowed configuration options:

Authentication

This action can be configured to authenticate with GitHub App Installation or Personal Access Token (PAT). If all configuration options are provided, the GitHub App Installation configuration has precedence. You can choose one of the following methods to authenticate:

GitHub App Installation
field required default description
GH_APP_ID True "" GitHub Application ID. See documentation for more details.
GH_APP_INSTALLATION_ID True "" GitHub Application Installation ID. See documentation for more details.
GH_APP_PRIVATE_KEY True "" GitHub Application Private Key. See documentation for more details.
Personal Access Token (PAT)
field required default description
GH_TOKEN True "" The GitHub Token used to scan the repository. Must have read access to all repository you are interested in scanning, repo:write, and workflow privileges to create a pull request.

Other Configuration Options

field required default description
GH_ENTERPRISE_URL False "" The GH_ENTERPRISE_URL is used to connect to an enterprise server instance of GitHub. github.com users should not enter anything here.
ORGANIZATION Required to have ORGANIZATION or REPOSITORY The name of the GitHub organization which you want this action to work from. ie. github.com/github would be github
REPOSITORY Required to have ORGANIZATION or REPOSITORY The name of the repository and organization which you want this action to work from. ie. github/evergreen or a comma separated list of multiple repositories github/evergreen,super-linter/super-linter
EXEMPT_REPOS False "" These repositories will be exempt from this action considering them for dependabot enablement. ex: If my org is set to github then I might want to exempt a few of the repos but get the rest by setting EXEMPT_REPOS to github/evergreen,github/contributors
TYPE False pull Type refers to the type of action you want taken if this workflow determines that dependabot could be enabled. Valid values are pull or issue.
TITLE False "Enable Dependabot" The title of the issue or pull request that will be created if dependabot could be enabled.
BODY False Pull Request: "Dependabot could be enabled for this repository. Please enable it by merging this pull request so that we can keep our dependencies up to date and secure." Issue: "Please update the repository to include a Dependabot configuration file. This will ensure our dependencies remain updated and secure.Follow the guidelines in creating Dependabot configuration files to set it up properly.Here's an example of the code:" The body of the issue or pull request that will be created if dependabot could be enabled.
COMMIT_MESSAGE False "Create dependabot.yaml" The commit message for the pull request that will be created if dependabot could be enabled.
CREATED_AFTER_DATE False none If a value is set, this action will only consider repositories created on or after this date for dependabot enablement. This is useful if you want to only consider newly created repositories. If I set up this action to run weekly and I only want to scan for repos created in the last week that need dependabot enabled, then I would set CREATED_AFTER_DATE to 7 days ago. That way only repositories created after 7 days ago will be considered for dependabot enablement. If not set or set to nothing, all repositories will be scanned and a duplicate issue/pull request may occur. Ex: 2023-12-31 for Dec. 31st 2023
UPDATE_EXISTING False False If set to true, this action will update the existing dependabot configuration file with any package ecosystems that are detected but not configured yet. If set to false, the action will only create a new dependabot configuration file if there is not an existing one.
PROJECT_ID False "" If set, this will assign the issue or pull request to the project with the given ID. ( The project ID on GitHub can be located by navigating to the respective project and observing the URL's end.) The ORGANIZATION variable is required
DRY_RUN False False If set to true, this action will not create any issues or pull requests. It will only log the repositories that could have dependabot enabled. This is useful for testing.
GROUP_DEPENDENCIES False false If set to true, dependabot configuration will group dependencies updates based on dependency type (production or development, where supported)
FILTER_VISIBILITY False "public,private,internal" Use this flag to filter repositories in scope by their visibility (public, private, internal). By default all repository are targeted. ex: to ignore public repositories set this value to private,internal.
BATCH_SIZE False None Set this to define the maximum amount of eligible repositories for every run. This is useful if you are targeting large organizations and you don't want to flood repositories with pull requests / issues. ex: if you want to target 20 repositories per time, set this to 20.
ENABLE_SECURITY_UPDATES False true If set to true, Evergreen will enable Dependabot security updates on target repositories. Note that the GitHub token needs to have the administration:write permission on every repository in scope to successfully enable security updates.
EXEMPT_ECOSYSTEMS False "" A list of package ecosystems to exempt from the generated dependabot configuration. To ignore ecosystems set this to one or more of bundler,cargo, composer, pip, docker, npm, gomod, mix, nuget, github-actions and terraform. ex: if you don't want Dependabot to update Dockerfiles and Github Actions you can set this to docker,github-actions.
REPO_SPECIFIC_EXEMPTIONS False "" A list of repositories that should be exempt from specific package ecosystems similar to EXEMPT_ECOSYSTEMS but those apply to all repositories. ex: org1/repo1:docker,github-actions;org1/repo2:pip would set exempt_ecosystems for org1/repo1 to be ['docker', 'github-actions'], and for org1/repo2 it would be ['pip'], while for every other repository evaluated, it would be set by the env variable EXEMPT_ECOSYSTEMS. NOTE: If you want specific exemptions to be added on top of the already specified global exemptions, you need to add the global exemptions to each repo specific exemption.

Example workflows

Basic

---
name: Weekly dependabot checks
on:
  workflow_dispatch:
  schedule:
    - cron: '3 2 * * 6'

permissions:
  contents: read

jobs:
  evergreen:
    name: evergreen
    runs-on: ubuntu-latest
    permissions:
      issues: write

    steps:
      - name: Run evergreen action
        uses: github/evergreen@v1
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          ORGANIZATION: <YOUR_ORGANIZATION_GOES_HERE>

Advanced

---
name: Weekly dependabot checks
on:
  workflow_dispatch:
  schedule:
    - cron: '3 2 * * 6'

permissions:
  contents: read

jobs:
  evergreen:
    name: evergreen
    runs-on: ubuntu-latest
    permissions:
      issues: write

    steps:
      - shell: bash
        run: |
          # Get the current date
          current_date=$(date +'%Y-%m-%d')

          # Calculate the previous month
          previous_date=$(date -d "$current_date -7 day" +'%Y-%m-%d')

          echo "$previous_date..$current_date"
          echo "one_week_ago=$previous_date" >> "$GITHUB_ENV"

      - name: Run evergreen action
        uses: github/evergreen@v1
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          ORGANIZATION: <YOUR_ORGANIZATION_GOES_HERE>
          EXEMPT_REPOS: "org_name/repo_name_1, org_name/repo_name_2"
          TITLE: "Add dependabot configuration"
          BODY: "Please add this dependabot configuration so that we can keep the dependencies in this repo up to date and secure. for help, contact XXX"
          CREATED_AFTER_DATE: ${{ env.one_week_ago }}
          

Local usage without Docker

  1. Make sure you have at least Python3.11 installed
  2. Copy .env-example to .env
  3. Fill out the .env file with a token from a user that has access to the organization to scan (listed below). Tokens should have at least write:org access for organization scanning and write:repository for repository scanning.
  4. Fill out the .env file with the configuration parameters you want to use
  5. pip3 install -r requirements.txt
  6. Run python3 ./evergreen.py, which will output everything in the terminal

License

MIT

More OSPO Tools

Looking for more resources for your open source program office (OSPO)? Check out the github-ospo repo for a variety of tools designed to support your needs.

evergreen's People

Contributors

ashleywolf avatar chrheg avatar dependabot[bot] avatar echarrod avatar gbrindisi avatar jmeridth avatar lelia avatar martincostello avatar morganpattersonsignet avatar zkoppert avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

evergreen's Issues

TypeError: fromisoformat: argument must be str

Describe the bug

I'm testing this out for use. I'm getting the below error running on ubuntu-latest GitHub runner.

Traceback (most recent call last):
File "/action/workspace/evergreen.py", line 361, in
main() # pragma: no cover
^^^^^^
File "/action/workspace/evergreen.py", line 83, in main
if is_repo_created_date_before(repo.created_at, created_after_date):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/action/workspace/evergreen.py", line 164, in is_repo_created_date_before
repo_created_at_date = datetime.fromisoformat(repo_created_at).replace(tzinfo=None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: fromisoformat: argument must be str

To Reproduce

  1. Copied the basic example workflow from readme
  2. Updated 'Run evergreen action' step to the below
      - name: Run evergreen action
        uses: github/evergreen@v1
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          REPOSITORY: "myorg/myrepo"
          FILTER_VISIBILITY: "private,internal"
  1. Run workflow manually through GitHub Actions page

Expected behavior

As state in readme, it open issue/PR in the specified repo.

Screenshots

No response

Additional context

I did adding env CREATED_AFTER_DATE and hardcoded "2020-01-01" as value. I got the same error.

I did see the closed issue #82, and the error appears to committed code from the pr #83 that closed that issue.

Add Issue Templates

To simplify user interaction with this repository we want to add GitHub Issue Templates.

  • bug report
  • feature request
  • config
    • link to discussions
    • link to github-ospo repo for overall OSPO GitHub Action issues

Rate Limiting

Would be great to implement rate limiting so users don't get failed workflow runs because it was zooming through repos too fast.

Automatically configure the GitHub server URL

GitHub Actions provides information about the URLs associated with the GitHub instance running the action, whether that is GitHub.com or GitHub Enterprise Server via the following properties of the github context:

  • api_url
  • graphql_url
  • server_url

Accordingly, when run as a GitHub Action (not as a standalone container), the action could self-configure itself for the current environment, only requiring the URL to be specified if it does not match the execution environment. This would make the action "just workโ„ข๏ธ" in GitHub Enterprise Server without having to explicitly configure the action.

For example:

---
name: 'Evergreen action'
author: 'github'
description: 'A GitHub Action to request dependabot enablement on eligible repositories in an organization.'
runs:
  using: 'docker'
  image: 'docker://ghcr.io/github/evergreen:v1'
+ env:
+   GH_ENTERPRISE_URL: ${{ github.server_url }}
branding:
  icon: 'file-plus'
  color: 'green'

In this scenario, renaming the variable to GH_URL would probably make the most sense, but it could be still be optional when using the container directly as it is today.

Test coverage

Improve test coverage and set failure threshold to 80 percent

Current coverage

Name Stmts Miss Cover Missing
auth.py 10 1 90% 26
dependabot_file.py 175 87 50% 32-40, 50-51, 55-63, 73-74, 77-85, 96-97, 100-108, 111-119, 122-130, 133-141, 152-153, 156-164, 175-176, 187-188, 191-199, 210-211, 214-222, 226-236, 247-248, 252-260, 270-271, 277-279, 281-288, 292
env.py 49 6 88% 46, 75, 83, 91, 100, 106
evergreen.py 47 23 51% 130-140, 145-152, 157-173, 177

TOTAL Stmnts: 281 Miss: 117 Cover: 58%

DRY_RUN is truthy

The logic around the DRY_RUN option behaves truthily. Instead of validating the value is true or false and then setting the value, it always acts as a dry run regardless of the value.

evergreen/env.py

Lines 105 to 107 in 7bd605e

if dry_run not in ("true", "false"):
raise ValueError("DRY_RUN environment variable not 'true' or 'false'")
dry_run_bool = True

Monthly contributor report

Contributors

  • Date range for contributor list: 2024-01-01 to 2024-01-31
  • Repository: ['github/evergreen']
Total Contributors Total Contributions % New Contributors
3 50 66.67%
Username Contribution Count New Contributor Sponsor URL Commits
zkoppert 41 False Sponsor Link https://github.com/github/evergreen/commits?author=zkoppert&since=2024-01-01&until=2024-01-31
chrheg 6 True not sponsorable https://github.com/github/evergreen/commits?author=chrheg&since=2024-01-01&until=2024-01-31
martincostello 3 True Sponsor Link https://github.com/github/evergreen/commits?author=martincostello&since=2024-01-01&until=2024-01-31

this file was generated by the Contributors GitHub Action

Check for duplicate issues

Pull requests currently check for a pending pull request that has already been opened, but for issues, there is no existing issue checking. This would be a nice improvement.

v1.11.1 422 Error

Describe the bug

It seems that the v1.11.1 version introduced a validation error being returned from the GitHub API at the very end of the process.

The error message:

Checking org/**** for compatible package managers
Traceback (most recent call last):
  File "/action/workspace/evergreen.py", line 427, in <module>
    main()  # pragma: no cover
    ^^^^^^
  File "/action/workspace/evergreen.py", line 162, in main
    pull = commit_changes(
           ^^^^^^^^^^^^^^^
  File "/action/workspace/evergreen.py", line 321, in commit_changes
    pull = repo.create_pull(
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/github3/decorators.py", line 24, in auth_wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/github3/repos/repo.py", line 1185, in create_pull
    return self._create_pull(data)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/github3/repos/repo.py", line 117, in _create_pull
    json = self._json(self._post(url, data=data), 201)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/github3/models.py", line 161, in _json
    raise exceptions.error_for(response)
github3.exceptions.UnprocessableEntity: 422 Validation Failed

To Reproduce

  1. Pull latest
  2. Run as normal with the config below

Expected behavior

Expect a successful processing of repos created in the last week.

Screenshots

No response

Additional context

Configuration

---
name: Weekly dependabot checks
on:
  workflow_dispatch:
    inputs:
      batch_size:
        description: "Number of repositories to process in a single run"
        required: false
        default: 5
        type: number
  schedule:
    # Runs twice per week, Tuesdays and Saturdays at 2:00 AM UTC
    - cron: "0 2 * * 2,6"

jobs:
  evergreen:
    name: evergreen
    runs-on: ubuntu-latest

    steps:
      - shell: bash
        run: |
          # Get the current date
          current_date=$(date +'%Y-%m-%d')

          # Calculate the previous month
          previous_date=$(date -d "$current_date -7 day" +'%Y-%m-%d')

          echo "$previous_date..$current_date"
          echo "one_week_ago=$previous_date" >> "$GITHUB_ENV"

      - name: Run evergreen action
        uses: github/evergreen@v1
        env:
          GH_APP_ID: "<id>"
          GH_APP_INSTALLATION_ID: "<id>"
          GH_APP_PRIVATE_KEY: ${{ secrets.GH_APP_PRIVATE_KEY }}
          ORGANIZATION: <org>
          GROUP_DEPENDENCIES: true
          PROJECT_ID: 9
          BATCH_SIZE: ${{ inputs.batch_size || 50 }}
          CREATED_AFTER_DATE: ${{ env.one_week_ago }}

Monthly contributor report

Contributors

  • Date range for contributor list: 2024-05-01 to 2024-05-31
  • Repository: ['github/evergreen']
Total Contributors Total Contributions % New Contributors
3 174 33.33%
Username Contribution Count New Contributor Commits
@zkoppert 126 False https://github.com/github/evergreen/commits?author=zkoppert&since=2024-05-01&until=2024-05-31
@jmeridth 47 False https://github.com/github/evergreen/commits?author=jmeridth&since=2024-05-01&until=2024-05-31
@echarrod 1 True https://github.com/github/evergreen/commits?author=echarrod&since=2024-05-01&until=2024-05-31

this file was generated by the Contributors GitHub Action

Optimize performance and reduce API calls by failing fast when building a dependabot.yml file

Describe the bug

Not exactly a bug but more of a performance improvement.

In the logic of the build_dependabot_file() function, it should check all other conditions before hitting the API for a file_contents check.

BEFORE:

try:
        if (
            repo.file_contents("Gemfile")
            and not bundler_found
            and "bundler" not in exempt_ecosystems
        ):
            compatible_package_manager_found = True
            bundler_found = True
            dependabot_file += make_dependabot_config("bundler", group_dependencies)
    except github3.exceptions.NotFoundError:
        pass

AFTER:

try:
        if (
            not bundler_found
            and "bundler" not in exempt_ecosystems
            and repo.file_contents("Gemfile")
        ):
            compatible_package_manager_found = True
            bundler_found = True
            dependabot_file += make_dependabot_config("bundler", group_dependencies)
    except github3.exceptions.NotFoundError:
        pass

To Reproduce

Use the action

Expected behavior

The most common logical false to be first and the API hit to be last.

Screenshots

No response

Additional context

No response

Support grouping of dependencies updates

Hi,
it would be useful to have the option to automatically enable some basic grouping for dependency updates from the start, since dependabot can become quite noisy.

A barebone configuration with grouping enabled could be the one using the default production/development groups:

---
version: 2
updates:
  - package-ecosystem: 'npm'
    directory: '/'
    schedule:
        interval: 'weekly'
    groups:
        production-dependencies:
            dependency-type: 'production'
        development-dependencies:
            dependency-type: 'development'

With basic workflow I get 403 Resource not accessible by integration

Hi all ๐Ÿ‘‹๐Ÿป

I tried to execute this workflow for my org:

name: Weekly dependabot checks
on:
  workflow_dispatch:
  schedule:
    - cron: '3 2 1 * *'

permissions: write-all  

jobs:
  evergreen:
    name: evergreen
    runs-on: ubuntu-latest

    steps:
      - name: Run evergreen action
        uses: github/evergreen@v1
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          ORGANIZATION: returngis
          # DRY_RUN: true

But I get the following error:

Checking returngis/tour-of-heroes-nuget
	Failed to enable Dependabot security updates.
Traceback (most recent call last):
  File "/action/workspace/evergreen.py", line 32[8](https://github.com/returngis/.github/actions/runs/7773680765/job/21197625687#step:3:9), in <module>
    main()  # pragma: no cover
    ^^^^^^
  File "/action/workspace/evergreen.py", line [12](https://github.com/returngis/.github/actions/runs/7773680765/job/21197625687#step:3:13)1, in main
    pull = commit_changes(
           ^^^^^^^^^^^^^^^
  File "/action/workspace/evergreen.py", line 2[14](https://github.com/returngis/.github/actions/runs/7773680765/job/21197625687#step:3:15), in commit_changes
    repo.create_ref(front_matter + branch_name, default_branch_commit)
  File "/usr/local/lib/python3.12/site-packages/github3/decorators.py", line 24, in auth_wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/github3/repos/repo.py", line 1228, in create_ref
    json = self._json(self._post(url, data=data), 201)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/github3/models.py", line [16](https://github.com/returngis/.github/actions/runs/7773680765/job/21197625687#step:3:17)1, in _json
    raise exceptions.error_for(response)
github3.exceptions.ForbiddenError: 403 Resource not accessible by integration

If I enable DRY_RUNI can see all dependabot configurations for my repos but If I leave to false I get this error. I tried to put as permission contents: write and also permissions: write-all but I get the same error.

Any ideas?

Thank you so much!

Support filtering by team

Enable filtering repositories to run on by team, e.g. run on my organisation just for repos owned by the engineering team

Wrong package-ecosystem value in dependabot config for Hex package manager

According to the docs the correct config value for the Hex ecosystem is mix and not hex as used right now in dependabot-file.py here:

    try:
        if repo.file_contents("mix.exs") and not hex_found:
            compatible_package_manager_found = True
            hex_found = True
            dependabot_file += make_dependabot_config("hex", group_dependencies)
    except github3.exceptions.NotFoundError:
        pass
    try:
        if repo.file_contents("mix.lock") and not hex_found:
            compatible_package_manager_found = True
            hex_found = True
            dependabot_file += make_dependabot_config("hex", group_dependencies)
    except github3.exceptions.NotFoundError:
        pass

Monthly contributor report

Contributors

  • Date range for contributor list: 2024-04-01 to 2024-04-30
  • Repository: ['github/evergreen']
Total Contributors Total Contributions % New Contributors
2 120 0.0%
Username Contribution Count New Contributor Commits
@zkoppert 90 False https://github.com/github/evergreen/commits?author=zkoppert&since=2024-04-01&until=2024-04-30
@jmeridth 30 False https://github.com/github/evergreen/commits?author=jmeridth&since=2024-04-01&until=2024-04-30

this file was generated by the Contributors GitHub Action

Monthly contributor report

Contributors

  • Date range for contributor list: 2024-03-01 to 2024-03-31
  • Repository: ['github/evergreen']
Total Contributors Total Contributions % New Contributors
2 100 0.0%
Username Contribution Count New Contributor Commits
@zkoppert 75 False https://github.com/github/evergreen/commits?author=zkoppert&since=2024-03-01&until=2024-03-31
@gbrindisi 25 False https://github.com/github/evergreen/commits?author=gbrindisi&since=2024-03-01&until=2024-03-31

this file was generated by the Contributors GitHub Action

feature request: Add ability for per project ecosystem exemption

          Thank you for adding this feature.

I briefly looked through the changes, and it's not apparent to me how a project can mark that a ecosystem shouldn't be added to the dependabot-file. You can exclude ecosystems globally through the exempt_repos config, but can it be done per project?

Originally posted by @vidwah-nte in #140 (comment)

In order to add a per repository ecosystem exemption, we would need to modify the env handling to accept a new variable repo_specific_exemptions that contained a string of repository names and ecosystem exemptions where multiple exemptions per repository are allowed. ie. repo_specific_exemptions = "org1/repo1:docker,github-actions;org1/repo2:pip would set exempt_ecosystems for org1/repo1 to be ['docker', 'github-actions'], and for org1/repo2 it would be ['pip'], while for every other repo evaluated, it would be set by the env variable EXEMPT_ECOSYSTEMS.

Monthly contributor report

Contributors

  • Date range for contributor list: 2024-07-01 to 2024-07-31
  • Repository: ['github/evergreen']
Total Contributors Total Contributions % New Contributors
2 219 0.0%
Username Contribution Count New Contributor Commits
@zkoppert 156 False https://github.com/github/evergreen/commits?author=zkoppert&since=2024-07-01&until=2024-07-31
@jmeridth 63 False https://github.com/github/evergreen/commits?author=jmeridth&since=2024-07-01&until=2024-07-31

this file was generated by the Contributors GitHub Action

Project Support Permissions

Describe the bug

When using the PROJECT_ID environment variable, getting the following error:

Request failed: 401 Client Error: Unauthorized for url: https://api.github.com/graphql

I am authenticating using a GitHub app with the following permissions:

  • Read access to metadata
  • Read and write access to administration, code, issues, pull requests, repository custom properties, repository projects, and security events
  • Admin access to organization projects

To Reproduce

  1. Setup github action
  2. Use GitHub app to authenticate
  3. Use the permissions described above
  4. Create a project
  5. Run with batch size of 1

Expected behavior

The PR that is created should be associated to the project of the id that is passed in.

Screenshots

No response

Additional context

Things that I have tried:

  • Every permission that could make sense
  • I have triple-checked that the installation has been installed properly to the organization
  • Debug mode (didn't get any extra output, unfortunately)

Environment Variables Used

GH_APP_ID: "<redacted>"
GH_APP_INSTALLATION_ID: "<redacted>"
GH_APP_PRIVATE_KEY: ${{ secrets.GH_APP_PRIVATE_KEY }}
ORGANIZATION: <redacted>
COMMIT_MESSAGE: "chore: add dependabot.yml"
TITLE: "Add dependabot configuration"
BODY: "Please add this dependabot configuration so that we can keep the dependencies in this repo up to date and secure. For more information on dependabot configuration, check out the [official docs](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file)."
GROUP_DEPENDENCIES: true
PROJECT_ID: 9
BATCH_SIZE: 50

Terraform dependabot files need a test written

Is your feature request related to a problem?

Terraform dependabot files need a test written

Describe the solution you'd like

write a test please

Describe alternatives you've considered

No response

Additional context

No response

Monthly contributor report

Contributors

  • Date range for contributor list: 2024-06-01 to 2024-06-30
  • Repository: ['github/evergreen']
Total Contributors Total Contributions % New Contributors
2 205 0.0%
Username Contribution Count New Contributor Commits
@zkoppert 151 False https://github.com/github/evergreen/commits?author=zkoppert&since=2024-06-01&until=2024-06-30
@jmeridth 54 False https://github.com/github/evergreen/commits?author=jmeridth&since=2024-06-01&until=2024-06-30

this file was generated by the Contributors GitHub Action

Detect new ecosystems

Is your feature request related to a problem?

If you have a dependabot.yml with config for e.g. ecosystem maven, then add a new ecosystem in the same project (i.e. npm), then evergreen won't try to update dependabot.yml to include npm.

Describe the solution you'd like

A new PR/Issue should be created when a project contains a dependabot.yml-file, but with missing ecosystems.

Describe alternatives you've considered

No response

Additional context

No response

Add github actions test

Is your feature request related to a problem?

github-actions code dependabot configuration is currently untested

Describe the solution you'd like

we should add a test for it similar to what is in #155

Describe alternatives you've considered

not testing and living everyday with the knowledge that my test coverage could have been higher and any changes to the github-actions related code will have lower confidence of its quality. ๐Ÿ˜ฆ

Additional context

No response

Create Dry run mode

It would be great if I could run this and get a report of repos that would be changed and dependabot files contents for that repo so I could feel more comfortable with running this across the organization.

Can't pull 7-zip

Describe the bug

Get-EvergreenApp does not return any results for 7zip. It seems that "Resolve-SystemNetWebRequest" cannot resolve sourceforge address properly.

To Reproduce

get-evergreenapp -name "7zip" -verbose

Expected behavior

7zip query returns expected values such as URI, version, etc.

Screenshots

Screenshot 2024-05-21 at 3 45 28 PM

Additional context

The get endpoint cmd returns additional URIs for 7zip. Is is possible to ignore or specify certain URIs?

Get-EvergreenEndpoint -Name "7zip"

Application Endpoints                                                                                  Ports

----------- ---------                                                                                  -----

7zip        {[nchc.dl.sourceforge.net](http://nchc.dl.sourceforge.net/), [sourceforge.net](http://sourceforge.net/), www.7-zip.org, [cytranet-dal.dl.sourceforge.net](http://cytranet-dal.dl.sourceforge.net/)} {443}

Document which permissions are required to automatically enable security updates

#14 allows Evergreen to automatically enable security updates on all repos in an org. However, there's no documentation of which permissions are required for this to work.

The docs in the REST API only state that "The authenticated user must have admin access to the repository", but I don't know how that translates to fine-grained tokens. I assume "Administration", but the REST API uses "admin access" broadly, so clarity in the docs is useful.

Improve existing configurations

It may occur that a repo is elgible to enable both github-actions and nuget but only has nuget current enabled. This action would skip over this repository because it already has a dependabot configuration and assumes that configuration has everything needed.

There is an opportunity to detect existing dependabot configurations and to ensure each of the package ecosystems that could be enabled are actually enabled. Too far? Open to hearing others thoughts here.

Duplicate PRs filed against same eligible repository

If you run evergreen multiple times against the same eligible repository, duplicate PRs may be generated, as it is checking for the presence / lack of a dependabot.y*ml file each time:

Screenshot 2024-02-21 at 7 19 38โ€ฏPM

A possible solution could be to do something similar to what the Scorecard Action installer does, which simply checks if a matching branch already exists, and skips the PR creation step whenever there's a match.

However, this may require reworking the branch commit logic, which currently uses the pattern of "dependabot-[uuid-from-commit-SHA]" to create PR branches. We don't necessarily have to match the branch name 100%, but I'm not sure if partially matching on dependabot-* could introduce other issues - or maybe not, if Dependabot does not, in fact, exist on said repository.

Dependency ecosystems not correctly detected

The logic to detect whether to configure package updates for most ecosystems, except for GitHub Actions and Terraform only appears to consider files in the root of the repository.

I found this after testing the action out on a .NET repository where the .csproj files are in subdirectories within the repository. The action generated a dependabot.yml file that only specified GitHub Actions updates.

Each file tested for should recurse through the repository to try and files (maybe within a configurable depth limit to reduce impact on rate limits) so that the generated file is a more accurate reflection of what should be configured.

Monthly contributor report

Contributors

  • Date range for contributor list: 2024-02-01 to 2024-02-29
  • Repository: ['github/evergreen']
Total Contributors Total Contributions % New Contributors
5 72 80.0%
Username Contribution Count New Contributor Sponsor URL Commits
zkoppert 56 False Sponsor Link https://github.com/github/evergreen/commits?author=zkoppert&since=2024-02-01&until=2024-02-29
gbrindisi 6 True not sponsorable https://github.com/github/evergreen/commits?author=gbrindisi&since=2024-02-01&until=2024-02-29
MorganPattersonSignet 6 True not sponsorable https://github.com/github/evergreen/commits?author=MorganPattersonSignet&since=2024-02-01&until=2024-02-29
lelia 3 True not sponsorable https://github.com/github/evergreen/commits?author=lelia&since=2024-02-01&until=2024-02-29
ashleywolf 1 True not sponsorable https://github.com/github/evergreen/commits?author=ashleywolf&since=2024-02-01&until=2024-02-29

this file was generated by the Contributors GitHub Action

BUG: TypeError when replacing TZ info

Getting an TypeError when trying to run evergreen:

Traceback (most recent call last):
File "/action/workspace/evergreen.py", line 349, in
main() # pragma: no cover
^^^^^^
File "/action/workspace/evergreen.py", line 79, in main
if created_after_date and repo.created_at.replace(
^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: str.replace() takes no keyword arguments

cc/ @MorganPattersonSignet since this is a code change you authored previously.

Add option to create issue if dependabot security updates did not turn on.

Currently when the action runs, if it determines that security updates aren't enabled, it tries to enable them, but if permissions are limited, then it fails, prints to the actions log, and continues. I think it would be an improvement to also open an issue in the target repo to ask the repo maintainers to turn on Dependabot security updates.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.