Giter VIP home page Giter VIP logo

scorecard's Introduction

OpenSSF Scorecard

OpenSSF Scorecard OpenSSF Best Practices build CodeQL Go Reference Go Report Card codecov SLSA 3 Slack

Overview

Using Scorecard

Important

OpenSSF Scorecard has opened a survey to better understand user expectations and needs from the project. The survey will remain open through OSS NA and the results will help steer the roadmap. Survey link

Please let us know what is working, what is not, and what you would like to see from the project. Thank you for participating!

Checks

Other Important Recommendations

Scoring

Contribute

FAQ

Overview

What is Scorecard?

We created Scorecard to help open source maintainers improve their security best practices and to help open source consumers judge whether their dependencies are safe.

Scorecard is an automated tool that assesses a number of important heuristics ("checks") associated with software security and assigns each check a score of 0-10. You can use these scores to understand specific areas to improve in order to strengthen the security posture of your project. You can also assess the risks that dependencies introduce, and make informed decisions about accepting these risks, evaluating alternative solutions, or working with the maintainers to make improvements.

The inspiration for Scorecard’s logo: "You passed! All D's ... and an A!"

Project Goals

  1. Automate analysis and trust decisions on the security posture of open source projects.

  2. Use this data to proactively improve the security posture of the critical projects the world depends on.

Prominent Scorecard Users

Scorecard has been run on thousands of projects to monitor and track security metrics. Prominent projects that use Scorecard include:

View a Project's Score

To see scores for projects regularly scanned by Scorecard, navigate to the webviewer, replacing the placeholder text with the platform, user/org, and repository name: https://scorecard.dev/viewer/?uri=<github_or_gitlab>.com/<user_name_or_org>/<repository_name>.

For example:

To view scores for projects not included in the webviewer, use the Scorecard CLI.

Public Data

We run a weekly Scorecard scan of the 1 million most critical open source projects judged by their direct dependencies and publish the results in a BigQuery public dataset.

This data is available in the public BigQuery dataset openssf:scorecardcron.scorecard-v2. The latest results are available in the BigQuery view openssf:scorecardcron.scorecard-v2_latest.

You can query the data using BigQuery Explorer by navigating to Add Data > Star a project by name > 'openssf'. For example, you may be interested in how a project's score has changed over time:

SELECT date, score FROM `openssf.scorecardcron.scorecard-v2` WHERE repo.name="github.com/ossf/scorecard" ORDER BY date ASC

You can extract the latest results to Google Cloud storage in JSON format using the bq tool:

# Get the latest PARTITION_ID
bq query --nouse_legacy_sql 'SELECT partition_id FROM
openssf.scorecardcron.INFORMATION_SCHEMA.PARTITIONS WHERE table_name="scorecard-v2"
AND partition_id!="__NULL__" ORDER BY partition_id DESC
LIMIT 1'

# Extract to GCS
bq extract --destination_format=NEWLINE_DELIMITED_JSON
'openssf:scorecardcron.scorecard-v2$<partition_id>' gs://bucket-name/filename-*.json

The list of projects that are checked is available in the cron/internal/data/projects.csv file in this repository. If you would like us to track more, please feel free to send a Pull Request with others. Currently, this list is derived from projects hosted on GitHub ONLY. We do plan to expand them in near future to account for projects hosted on other source control systems.

Using Scorecard

Scorecard GitHub Action

The easiest way to use Scorecard on GitHub projects you own is with the Scorecard GitHub Action. The Action runs on any repository change and issues alerts that maintainers can view in the repository’s Security tab. For more information, see the Scorecard GitHub Action installation instructions.

Scorecard REST API

To query pre-calculated scores of OSS projects, use the REST API.

To enable your project to be available on the REST API, set publish_results: true in the Scorecard GitHub Action setting.

Data provided by the REST API is licensed under the CDLA Permissive 2.0.

Scorecard Badges

Enabling publish_results: true in Scorecard GitHub Actions also allows maintainers to display a Scorecard badge on their repository to show off their hard work. This badge also auto-updates for every change made to the repository. See more details on this OSSF blogpost.

To include a badge on your project's repository, simply add the following markdown to your README:

[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/{owner}/{repo}/badge)](https://scorecard.dev/viewer/?uri=github.com/{owner}/{repo})

Scorecard Command Line Interface

To run a Scorecard scan on projects you do not own, use the command line interface installation option.

Prerequisites

Platforms: Currently, Scorecard supports OSX and Linux platforms. If you are using a Windows OS you may experience issues. Contributions towards supporting Windows are welcome.

Language: You must have GoLang installed to run Scorecard (https://golang.org/doc/install)

Installation

Docker

scorecard is available as a Docker container:

docker pull gcr.io/openssf/scorecard:stable

To use a specific scorecard version (e.g., v3.2.1), run:

docker pull gcr.io/openssf/scorecard:v3.2.1
Standalone

To install Scorecard as a standalone:

Visit our latest release page and download the correct zip file for your operating system.

Add the binary to your GOPATH/bin directory (use go env GOPATH to identify your directory if necessary).

Verifying SLSA provenance for downloaded releases

We generate SLSA3 signatures using the OpenSSF's slsa-framework/slsa-github-generator during the release process. To verify a release binary:

  1. Install the verification tool from slsa-framework/slsa-verifier#installation.
  2. Download the signature file attestation.intoto.jsonl from the GitHub releases page.
  3. Run the verifier:
slsa-verifier -artifact-path <the-zip> -provenance attestation.intoto.jsonl -source github.com/ossf/scorecard -tag <the-tag>
Using package managers
Package Manager Supported Distribution Command
Nix NixOS nix-shell -p nixpkgs.scorecard
AUR helper Arch Linux Use your AUR helper to install scorecard
Homebrew macOS or Linux brew install scorecard

Authentication

GitHub imposes api rate limits on unauthenticated requests. To avoid these limits, you must authenticate your requests before running Scorecard. There are two ways to authenticate your requests: either create a GitHub personal access token, or create a GitHub App Installation.

  • Create a classic GitHub personal access token. When creating the personal access token, we suggest you choose the public_repo scope. Set the token in an environment variable called GITHUB_AUTH_TOKEN, GITHUB_TOKEN, GH_AUTH_TOKEN or GH_TOKEN using the commands below according to your platform.
# For posix platforms, e.g. linux, mac:
export GITHUB_AUTH_TOKEN=<your access token>
# Multiple tokens can be provided separated by comma to be utilized
# in a round robin fashion.
export GITHUB_AUTH_TOKEN=<your access token1>,<your access token2>

# For windows:
set GITHUB_AUTH_TOKEN=<your access token>
set GITHUB_AUTH_TOKEN=<your access token1>,<your access token2>

OR

  • Create a GitHub App Installation for higher rate-limit quotas. If you have an installed GitHub App and key file, you can use the three environment variables below, following the commands (set or export) shown above for your platform.
GITHUB_APP_KEY_PATH=<path to the key file on disk>
GITHUB_APP_INSTALLATION_ID=<installation id>
GITHUB_APP_ID=<app id>

These variables can be obtained from the GitHub developer settings page.

Basic Usage

Using repository URL

Scorecard can run using just one argument, the URL of the target repo:

$ scorecard --repo=github.com/ossf-tests/scorecard-check-branch-protection-e2e
Starting [CII-Best-Practices]
Starting [Fuzzing]
Starting [Pinned-Dependencies]
Starting [CI-Tests]
Starting [Maintained]
Starting [Packaging]
Starting [SAST]
Starting [Dependency-Update-Tool]
Starting [Token-Permissions]
Starting [Security-Policy]
Starting [Signed-Releases]
Starting [Binary-Artifacts]
Starting [Branch-Protection]
Starting [Code-Review]
Starting [Contributors]
Starting [Vulnerabilities]
Finished [CI-Tests]
Finished [Maintained]
Finished [Packaging]
Finished [SAST]
Finished [Signed-Releases]
Finished [Binary-Artifacts]
Finished [Branch-Protection]
Finished [Code-Review]
Finished [Contributors]
Finished [Dependency-Update-Tool]
Finished [Token-Permissions]
Finished [Security-Policy]
Finished [Vulnerabilities]
Finished [CII-Best-Practices]
Finished [Fuzzing]
Finished [Pinned-Dependencies]

RESULTS
-------
Aggregate score: 7.9 / 10

Check scores:
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
|  SCORE  |          NAME          |             REASON             |                         DOCUMENTATION/REMEDIATION                         |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 10 / 10 | Binary-Artifacts       | no binaries found in the repo  | github.com/ossf/scorecard/blob/main/docs/checks.md#binary-artifacts       |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 9 / 10  | Branch-Protection      | branch protection is not       | github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection      |
|         |                        | maximal on development and all |                                                                           |
|         |                        | release branches               |                                                                           |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| ?       | CI-Tests               | no pull request found          | github.com/ossf/scorecard/blob/main/docs/checks.md#ci-tests               |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | CII-Best-Practices     | no badge found                 | github.com/ossf/scorecard/blob/main/docs/checks.md#cii-best-practices     |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 10 / 10 | Code-Review            | branch protection for default  | github.com/ossf/scorecard/blob/main/docs/checks.md#code-review            |
|         |                        | branch is enabled              |                                                                           |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | Contributors           | 0 different companies found -- | github.com/ossf/scorecard/blob/main/docs/checks.md#contributors           |
|         |                        | score normalized to 0          |                                                                           |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | Dependency-Update-Tool | no update tool detected        | github.com/ossf/scorecard/blob/main/docs/checks.md#dependency-update-tool |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | Fuzzing                | project is not fuzzed in       | github.com/ossf/scorecard/blob/main/docs/checks.md#fuzzing                |
|         |                        | OSS-Fuzz                       |                                                                           |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 1 / 10  | Maintained             | 2 commit(s) found in the last  | github.com/ossf/scorecard/blob/main/docs/checks.md#maintained             |
|         |                        | 90 days -- score normalized to |                                                                           |
|         |                        | 1                              |                                                                           |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| ?       | Packaging              | no published package detected  | github.com/ossf/scorecard/blob/main/docs/checks.md#packaging              |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 8 / 10  | Pinned-Dependencies    | unpinned dependencies detected | github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies    |
|         |                        | -- score normalized to 8       |                                                                           |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | SAST                   | no SAST tool detected          | github.com/ossf/scorecard/blob/main/docs/checks.md#sast                   |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | Security-Policy        | security policy file not       | github.com/ossf/scorecard/blob/main/docs/checks.md#security-policy        |
|         |                        | detected                       |                                                                           |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| ?       | Signed-Releases        | no releases found              | github.com/ossf/scorecard/blob/main/docs/checks.md#signed-releases        |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 10 / 10 | Token-Permissions      | tokens are read-only in GitHub | github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions      |
|         |                        | workflows                      |                                                                           |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
| 10 / 10 | Vulnerabilities        | no vulnerabilities detected    | github.com/ossf/scorecard/blob/main/docs/checks.md#vulnerabilities        |
|---------|------------------------|--------------------------------|---------------------------------------------------------------------------|
Docker

The GITHUB_AUTH_TOKEN has to be set to a valid token

docker run -e GITHUB_AUTH_TOKEN=token gcr.io/openssf/scorecard:stable --show-details --repo=https://github.com/ossf/scorecard

To use a specific scorecard version (e.g., v3.2.1), run:

docker run -e GITHUB_AUTH_TOKEN=token gcr.io/openssf/scorecard:v3.2.1 --show-details --repo=https://github.com/ossf/scorecard
Showing Detailed Results

For more details about why a check fails, use the --show-details option:

./scorecard --repo=github.com/ossf-tests/scorecard-check-branch-protection-e2e --checks Branch-Protection --show-details
Starting [Pinned-Dependencies]
Finished [Pinned-Dependencies]

RESULTS
-------
|---------|------------------------|--------------------------------|--------------------------------|---------------------------------------------------------------------------|
|  SCORE  |          NAME          |             REASON             |            DETAILS             |                         DOCUMENTATION/REMEDIATION                         |
|---------|------------------------|--------------------------------|--------------------------------|---------------------------------------------------------------------------|
| 9 / 10  | Branch-Protection      | branch protection is not       | Info: 'force pushes' disabled  | github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection      |
|         |                        | maximal on development and all | on branch 'main' Info: 'allow  |                                                                           |
|         |                        | release branches               | deletion' disabled on branch   |                                                                           |
|         |                        |                                | 'main' Info: linear history    |                                                                           |
|         |                        |                                | enabled on branch 'main' Info: |                                                                           |
|         |                        |                                | strict status check enabled    |                                                                           |
|         |                        |                                | on branch 'main' Warn: status  |                                                                           |
|         |                        |                                | checks for merging have no     |                                                                           |
|         |                        |                                | specific status to check on    |                                                                           |
|         |                        |                                | branch 'main' Info: number     |                                                                           |
|         |                        |                                | of required reviewers is 2     |                                                                           |
|         |                        |                                | on branch 'main' Info: Stale   |                                                                           |
|         |                        |                                | review dismissal enabled on    |                                                                           |
|         |                        |                                | branch 'main' Info: Owner      |                                                                           |
|         |                        |                                | review required on branch      |                                                                           |
|         |                        |                                | 'main' Info: 'administrator'   |                                                                           |
|         |                        |                                | PRs need reviews before being  |                                                                           |
|         |                        |                                | merged on branch 'main'        |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------|---------------------------------------------------------------------------|
Showing Maintainers Annotations (Experimental)

To see the maintainers annotations for each check, use the --show-annotations option.

For more information on how to configure annotations or what are the available annotations, see the configuration doc.

Using a GitLab Repository

To run Scorecard on a GitLab repository, you must create a GitLab Access Token with the following permissions:

  • read_api
  • read_user
  • read_repository

You can run Scorecard on a GitLab repository by setting the GITLAB_AUTH_TOKEN environment variable:

export GITLAB_AUTH_TOKEN=glpat-xxxx

scorecard --repo gitlab.com/<org>/<project>/<subproject>

For an example of using Scorecard in GitLab CI/CD, see here.

Self Hosted Editions

While we focus on GitLab.com support, Scorecard also works with self-hosted GitLab installations. If your platform is hosted at a subdomain (e.g. gitlab.foo.com), Scorecard should work out of the box. If your platform is hosted at some slug (e.g. foo.com/bar/), you will need to set the GL_HOST environment variable.

export GITLAB_AUTH_TOKEN=glpat-xxxx
export GL_HOST=foo.com/bar
scorecard --repo foo.com/bar/<org>/<project>
Using GitHub Enterprise Server (GHES) based Repository

To use a GitHub Enterprise host github.corp.com, use the GH_HOST environment variable.

# Set the GitHub Enterprise host without https prefix or slash with relevant authentication token
export GH_HOST=github.corp.com
export GITHUB_AUTH_TOKEN=token

scorecard --repo=github.corp.com/org/repo
# OR without github host url
scorecard --repo=org/repo
Using a Package manager

For projects in the --npm, --pypi, --rubygems, or --nuget ecosystems, you have the option to run Scorecard using a package manager. Provide the package name to run the checks on the corresponding GitHub source code.

For example, --npm=angular.

Running specific checks

To run only specific check(s), add the --checks argument with a list of check names.

For example, --checks=CI-Tests,Code-Review.

Formatting Results

The currently supported formats are default (text) and json.

These may be specified with the --format flag. For example, --format=json.

Checks

Scorecard Checks

The following checks are all run against the target project by default:

Name Description Risk Level Token Required GitLab Support Note
Binary-Artifacts Is the project free of checked-in binaries? High PAT, GITHUB_TOKEN Supported
Branch-Protection Does the project use Branch Protection ? High PAT (repo or repo> public_repo), GITHUB_TOKEN Supported (see notes) certain settings are only supported with a maintainer PAT
CI-Tests Does the project run tests in CI, e.g. GitHub Actions, Prow? Low PAT, GITHUB_TOKEN Supported
CII-Best-Practices Has the project earned an OpenSSF (formerly CII) Best Practices Badge at the passing, silver, or gold level? Low PAT, GITHUB_TOKEN Validating
Code-Review Does the project practice code review before code is merged? High PAT, GITHUB_TOKEN Validating
Contributors Does the project have contributors from at least two different organizations? Low PAT, GITHUB_TOKEN Validating
Dangerous-Workflow Does the project avoid dangerous coding patterns in GitHub Action workflows? Critical PAT, GITHUB_TOKEN Unsupported
Dependency-Update-Tool Does the project use tools to help update its dependencies? High PAT, GITHUB_TOKEN Unsupported
Fuzzing Does the project use fuzzing tools, e.g. OSS-Fuzz, QuickCheck or fast-check? Medium PAT, GITHUB_TOKEN Validating
License Does the project declare a license? Low PAT, GITHUB_TOKEN Validating
Maintained Is the project at least 90 days old, and maintained? High PAT, GITHUB_TOKEN Validating
Pinned-Dependencies Does the project declare and pin dependencies? Medium PAT, GITHUB_TOKEN Validating
Packaging Does the project build and publish official packages from CI/CD, e.g. GitHub Publishing ? Medium PAT, GITHUB_TOKEN Validating
SAST Does the project use static code analysis tools, e.g. CodeQL, LGTM (deprecated), SonarCloud? Medium PAT, GITHUB_TOKEN Unsupported
Security-Policy Does the project contain a security policy? Medium PAT, GITHUB_TOKEN Validating
Signed-Releases Does the project cryptographically sign releases? High PAT, GITHUB_TOKEN Validating
Token-Permissions Does the project declare GitHub workflow tokens as read only? High PAT, GITHUB_TOKEN Unsupported
Vulnerabilities Does the project have unfixed vulnerabilities? Uses the OSV service. High PAT, GITHUB_TOKEN Validating
Webhooks Does the webhook defined in the repository have a token configured to authenticate the origins of requests? Critical maintainer PAT (admin: repo_hook or admin> read:repo_hook doc EXPERIMENTAL

Detailed Checks Documentation

To see detailed information about each check, its scoring criteria, and remediation steps, check out the checks documentation page.

Beginner's Guide to Scorecard Checks

For a guide to the checks you should use when getting started, see the beginner's guide to scorecard checks.

Other Important Recommendations

Two-factor Authentication (2FA)

Two-factor Authentication (2FA) adds an extra layer of security when logging into websites or apps. 2FA protects your account if your password is compromised by requiring a second form of authentication, such as codes sent via SMS or authentication app, or touching a physical security key.

We strongly recommend that you enable 2FA on any important accounts where it is available. 2FA is not a Scorecard check because GitHub and GitLab do not make that data about user accounts public. Arguably, this data should always remain private, since accounts without 2FA are so vulnerable to attack.

Though it is not an official check, we urge all project maintainers to enable 2FA to protect their projects from compromise.

Enabling 2FA

For users

Follow the steps described at Configuring two-factor authentication

If possible, use either:

  • physical security key (preferred), such as Titan or Yubikey
  • recovery codes, stored in an access protected and encrypted vault

As a last option, use SMS. Beware: 2FA using SMS is vulnerable to SIM swap attack.

For an organization
  1. Prepare to require 2FA in your organization
  2. Require 2FA in your organization

Scoring

Aggregate Score

Each individual check returns a score of 0 to 10, with 10 representing the best possible score. Scorecard also produces an aggregate score, which is a weight-based average of the individual checks weighted by risk.

  • “Critical” risk checks are weighted at 10
  • “High” risk checks are weighted at 7.5
  • “Medium” risk checks are weighted at 5
  • “Low” risk checks are weighted at 2.5

See the list of current Scorecard checks for each check's risk level.

Contribute

Report Problems

If you have what looks like a bug, please use the GitHub issue tracking system. Before you file an issue, please search existing issues to see if your issue is already covered.

Contribute to Scorecard

Before contributing, please follow our Code of Conduct.

See the Contributing documentation for guidance on how to contribute to the project.

Adding a Scorecard Check

If you'd like to add a check, please see guidance here.

Connect with the Scorecard Community

If you want to get involved in the Scorecard community or have ideas you'd like to chat about, we discuss this project in the OSSF Best Practices Working Group meetings.

Artifact Link
Scorecard Dev Forum ossf-scorecard-dev@
Scorecard Announcements Forum ossf-scorecard-announce@
Community Meeting VC Link to z o o m meeting
Community Meeting Calendar APAC-friendly Biweekly on Thursdays at 1:00-2:00 PM Pacific (OSSF Public Calendar)
Video Call: LFX Zoom
EMEA-friendly Every 4 Mondays at 7:00-8:00 AM Pacific (OSSF Public Calendar)
Video Call: LFX Zoom
Meeting Notes Notes
Slack Channel #scorecard

Maintainers are listed in the CODEOWNERS file.

Report a Security Issue

To report a security issue, please follow instructions here.

Join the Scorecard Project Meeting

Zoom

APAC-friendly Biweekly on Thursdays at 1:00-2:00 PM Pacific (OSSF Public Calendar)

Video Call: LFX z o o m

EMEA-friendly Every 4 Mondays at 7:00-8:00 AM Pacific (OSSF Public Calendar)

Video Call: LFX z o o m

Agenda

You can see the agenda and meeting notes here.

Stargazers over time

Stargazers over time

FAQ

FAQ

See the FAQ for answers to Frequently Asked Questions about Scorecard.

scorecard's People

Contributors

adamkorcz avatar afmarcum avatar aidenwang9867 avatar asraa avatar azeemsgoogle avatar azeemshaikh38 avatar calebbrown avatar cpanato avatar david-a-wheeler avatar davidkorczynski avatar dependabot[bot] avatar dlorenc avatar evverx avatar gabibguti avatar inferno-chromium avatar joycebrum avatar justaugustus avatar katzj avatar kimsterv avatar laurentsimon avatar lehors avatar moorereason avatar nathannaveen avatar naveensrinivasan avatar olivekl avatar oliverchang avatar pnacht avatar r0mdau avatar raghavkaul avatar spencerschrock avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

kimsterv dlorenc rajkrishnamurthy preetam finnigja dut3062796s b-xiang shuaizend magnologan tools-env zerolugithub yeohoonyun asked637 quantumpacket rheehot a2l007 xkhahuss trendingtechnology duzhanyuan altanateser karimelmel boost-entropy-repos-org tom-vanbraband-sonarsource diegopacheco be-secure moorereason naveensrinivasan chrisjsimpson goryudyuma scovetta jorgevrgs highkay willpowerforever nathannaveen jhutchings1 vin01 isgasho nirmalay12 asraa lichnak bondzio wha28 ristomcgehee trmiller dhruv-sachdev1313 henryzz0 oliverchang bugdisclose laurentsimon zamhal cowens alexcontini pacospace sgoggins rizwanali-sv doomhammerhell weiplanet techris45 sarulon uionto lajpatshah sumodgeorge toancanh y0d4a purplesparkle dm407 mlynchcogent 3ur0c3nt5y masteramuk fraggler0ck cmeninwa duanshuaimin mfrank2016 jkulrativid vpnj012k sarvex lyrl org-mars sonyeric schulzdl bluseking willie-lin ben-moss gdemarcsek vikram-yerneni ekmixon iainsmith-snippets gdraperi johann-smith xmillies nudnov1 pithomlabs sec-js notanton minkione rbrt-weiler gh0st0ne brahman81 5l1v3r1 aevaonline

scorecard's Issues

Show negative results in details

Howdy! New here. 👋 🤠

While using scorecard to evaluate projects I contribute to, I found myself having to read (and patch) the checks code to understand why I was seeing failing or marginal results. It would be very helpful to see negative results in the --show-details output instead of only positive results. It makes it easier for newcomers like me to learn what is considered bad practice, and I assume it would be help everyone to investigate any negative findings.

For example, I've patched scorecard locally to give negative results (starting with !!) for some checks:

$ ./scorecard --repo=github.com/gohugoio/hugo --show-details --checks=Code-Review,CI-Tests,Pull-Requests,Signed-Releases,Signed-Tags
RESULTS
-------
CI-Tests: Fail 4
    !! found committed PR without CI test: 8075
    !! found committed PR without CI test: 8070
    CI test found: context: continuous-integration/travis-ci/pr, url: https://api.github.com/repos/gohugoio/hugo/statuses/1056701da088d5e87e6e31cdc6e0c455862697cc
    !! found committed PR without CI test: 8059
    CI test found: context: continuous-integration/travis-ci/pr, url: https://api.github.com/repos/gohugoio/hugo/statuses/34ecd28779d4836c74d4e71a5227b196d5d2cbec
    CI test found: context: continuous-integration/travis-ci/pr, url: https://api.github.com/repos/gohugoio/hugo/statuses/713792077b06504e9fdd0c8abdc2aebace5dcf2a
    CI test found: context: continuous-integration/travis-ci/pr, url: https://api.github.com/repos/gohugoio/hugo/statuses/3a38df2c4f96e560ddedc5fd5266fb54972a7fb6
    !! found committed PR without CI test: 8020
    CI test found: context: continuous-integration/travis-ci/pr, url: https://api.github.com/repos/gohugoio/hugo/statuses/7475bb324f208c8acd5fb3b6aa0f1b07c8d26d94
    CI test found: context: continuous-integration/travis-ci/push, url: https://api.github.com/repos/gohugoio/hugo/statuses/cd5a53bb240ef23977c03d2a47cd76f8ee915899
    !! found committed PR without CI test: 7999
    CI test found: context: continuous-integration/travis-ci/pr, url: https://api.github.com/repos/gohugoio/hugo/statuses/15ee0346fc60265c36530dcc3b70be3955f006ef
    CI test found: context: continuous-integration/travis-ci/pr, url: https://api.github.com/repos/gohugoio/hugo/statuses/bb228c9faf6a44fe2bb60d1d3d6f5ec538a7c786
    CI test found: context: continuous-integration/travis-ci/pr, url: https://api.github.com/repos/gohugoio/hugo/statuses/f2b60ecc084b1142c9cf73f7e3736a754067943f
    CI test found: context: continuous-integration/travis-ci/pr, url: https://api.github.com/repos/gohugoio/hugo/statuses/3a89789e40cc093f36cb9c03814f8dc30a9b1921
Code-Review: Pass 8
    found PR with committer different than author: 8075
    found PR with committer different than author: 8070
    !! found unreviewed PR committed by author: 8065
    !! found unreviewed PR committed by author: 8059
    found review approved PR: 8042
    found PR with committer different than author: 8035
    found PR with committer different than author: 8034
    found PR with committer different than author: 8020
    found review approved PR: 8008
    found PR with committer different than author: 8004
    found PR with committer different than author: 7999
    found PR with committer different than author: 7998
    !! found unreviewed PR committed by author: 7991
    found PR with committer different than author: 7989
    found PR with committer different than author: 7988
    github code reviews found for 12 of 15 merged PRs
Pull-Requests: Fail 5
    !! found commit without PR: 6c294182788f7da358243c4a0ef0a98772491067
    !! found commit without PR: 10ae7c3210cd1add14d3750aa9512a87df0e1146
    !! found commit without PR: a2d146ec32a26ccca9ffa68d3c840ec5b08cca96
    !! found commit without PR: 21fa1e86f2aa929fb0983a0cc3dc4e271ea1cc54
    !! found commit without PR: c84ad8db821c10225c0e603c6ec920c67b6ce36f
    !! found commit without PR: 718e09ed4bc538f4fccc4337f99e9eb86aea31f3
    !! found commit without PR: 32d4bf68da7d16302f138dde343c70f9667933c4
    !! found commit without PR: 1415efdcd838cf482072ef08e765a8ce960bfdde
    !! found commit without PR: 4e6bf7907dc5dfde697ace251b91a9399e0e3c39
    !! found commit without PR: 50be4370b0c46c6c34430eb45bdc53d1926dd800
    !! found commit without PR: 3d2e6a30d43079d48eb241505d5e0d9628dedf15
    !! found commit without PR: 4f1e4bb3fe8241d7a900f57e156f9679768aff24
    !! found commit without PR: 9f1265fde4b9ef186148337c99f08601633b6056
    !! found commit without PR: d162bbd7990b6a523bdadcd10bf60fcb43ecf270
    found PRs for 16 out of 30 commits
Signed-Releases: Fail 10
    release found: v0.79.1
    !! release v0.79.1 has no signed artifacts
    release found: v0.79.0
    !! release v0.79.0 has no signed artifacts
    release found: v0.78.2
    !! release v0.78.2 has no signed artifacts
    release found: v0.78.1
    !! release v0.78.1 has no signed artifacts
    release found: v0.78.0
    !! release v0.78.0 has no signed artifacts
    release found: v0.77.0
    !! release v0.77.0 has no signed artifacts
Signed-Tags: Fail 10
    !! unsigned tag found: v0.78.0, commit: 2f1a31211c08f7fd52738ba2f817055a7fab9373
    !! unsigned tag found: v0.78.1, commit: 0cb2fd5cc8eca7cd6af59e246492f1587b69819b
    !! unsigned tag found: v0.78.2, commit: 21103fa0df2b9d23e5792ac4f27def0453118f7e
    !! unsigned tag found: v0.79.0, commit: 626facbfa32823bc2d1152f97e6db67ed051a307
    !! unsigned tag found: v0.79.1, commit: ea1d515f9750581769311d95064f52165c89edd0

I can submit a PR for the above if this idea is acceptable.

How to measure maintaince

Current check measures commits and releases.

The question is - does something with a lot of commits imply a secure project? How does commits correlate with security. Mature projects release far less often.

Less of an issue until this results in some form of automated checks/badging. Some of these checks should be higher weight than others. I would give this a low weight if kept measuring in this fashion....

Publish Docker image

Most of us would like to run a docker run , instead of pulling down the repo.

There is an existing dockerfile. The docker container has to be built and pushed into a registry.

Scorecard scalability limitation: Reduce GitHub API calls

I've been running scorecard over ~40 Envoy dependencies a few times in the past hour and even using a personal access token for GitHub, I've hit the rate limit. I'm wondering what can be done to make this work better in scorecard. Suggestions:

  • Profile GitHub API calls, determine the most costly, optimize to API alternatives. Share details on which are most expensive in public documentation.
  • Add support for local caching of results for some time bound period.
  • Provide a scorecard network service that collects results for projects periodically.
  • Have scorecard JSON examples generalized to some contributor editable file that will collect projects nightly.

RFE: Tool / Check should provide suggested remediation

Currently, a check indicates a simple pass/fail with confidence. For a developer, this does not actually help them know how they should improve the situation. Checks should provide a suggested remediation on how to fix or improve.

Signed Tags Check: Github "Verified" is a poor metric for source authenticity

Github blanket signs releases with their own internal key. There are listed as "Verified" in the UI, and I suspect the library in use does the same. However this is poor UX and should not be relied upon to provide any form of "Signed-Tags" or "Signed-Releases".

An attacker can get access to some account that doesn't have 2FA enabled and have Github generate a "Verified" tag to satisfy the requirement.

There should be a mechanism to ensure the key is only a valid key from the given user, and not the Github signing key.

Enable gosec for golangci-lint

The present golangci-lint does not have https://github.com/securego/gosec enabled.

Enable the gosec for the golangci-lint with config file.

Fix the gosec warning in the code by ignoring it, which is not an issue.

golangci-lint run --enable gosec
roundtripper/roundtripper.go:37:2: G101: Potential hardcoded credentials (gosec)
	GITHUB_AUTH_TOKEN          = "GITHUB_AUTH_TOKEN"
	^

Official downloadable releases?

Hey all,

This projects looks really great. I was naively expecting to download an official release but I see none is being built/proposed. Any reason why? Feels a shame not benefit from on of the great golang asset ;)

security policy check

You check only for "security.md" and "SECURITY.md" but Some project has "SECURITY.MD".

Unexpected fail of Signed-Tags check

The scorecard on bkimminich/juice-shop gives this result:

RESULTS
-------
Active: Pass 10
CI-Tests: Pass 10
CII-Best-Practices: Pass 10
Code-Review: Fail 10
Contributors: Pass 10
Frozen-Deps: Fail 5
Fuzzing: Fail 3
Pull-Requests: Pass 7
SAST: Pass 8
Security-Policy: Pass 10
Signed-Releases: Fail 10
Signed-Tags: Fail 9

I get the Code-Review and Fuzzing fails, but not the Signed-Tags fail:

image

All releases (of the last 2+ years at least) are based on signed tags. And since 1 year+ all releases have MD5 signatures of all artifacts as well as artifacts themselves, so the Signed-Releases fail seems also unexpected:

image

Release schedule

Need to define a release schedule. Between v1.0.0 and the master there are quite a few commits.

7ab314d (HEAD -> main, tag: v1.1, tag: ls, upstream/main) Fix - dependabot githubactions location
bcf8d0d Fix - dependabot yaml error
4ad4a42 Feature - enabled dependabot for githubactions
f385b0d Feature - run scans from npm pacakge name
0d77d89 Fix - tarball URL trailing slash
038e3b6 Bump github.com/onsi/gomega from 1.10.4 to 1.10.5
717701b Bump github.com/onsi/ginkgo from 1.14.2 to 1.15.0
8493b0b Add remediation steps for various checks.
93373f7 Fixes - Incorrect result for branch protection
2a1463b Feature - Report codecoverage to codecov.io
09b83b9 Fixes
33e9189 fix - panic on nil
c00aa4b (origin/main, origin/HEAD) Add e2e tests for remaining checks.
bcaa2e7 Lint fix.
b5096bf Fix backslash.
b278475 Fix CodeQL failure.
5b7ddc5 Add e2e test.
dc8d1fe Add packaging check.
c4c99cd feature - Included the e2e into the PR workflows
91bfea5 feat - Close stale issues
1d26654 Document - Included instruction for GITHUB_AUTH_TOKEN
1700c3a feature - Pull request template (#127)
b11fad8 feature - Included the status badge in README (#125)
7b740ce fix - Handle nil structs in branch protection (#124)
9d4e5c0 feature - CODEOWNERS for github branch protection feature (#123)
fcf0ac4 Merge pull request #119 from naveensrinivasan/feature/protected-branches
3191c55 Update README.md
938b9f2 Merge branch 'main' into feature/protected-branches
b506c6f Merge pull request #122 from ossf/b5
650fe0a Update README.md
3c94ffa Remove releases from active check.
5d84b86 (origin/feature/protected-branches, feature/protected-branches) Merge branch 'main' into feature/protected-branches
b86fae0 Fix https://github.com/ossf/scorecard/issues/121
9ce57c0 feature - Checks for branch protections
15a1ba0 feat - nonroot docker container (#114)
9e0388f Merge pull request #118 from naveensrinivasan/feature/update-readme
c5c51b9 (origin/feature/update-readme, feature/update-readme) feature - Update the CONTRIBUTING guidelines
b216a1e Feat - implemented goreleaser for releases (#117)
f77da77 feat-e2e tests for signed tags and signed releases (#115)
3df1191 Create Dependabot config file (#116)
ddc82c6 (feat/dependabot) Add --show-details to the cron job. (#113)
329a4cf Merge pull request #109 from moorereason/release-tagname
88d5218 Use release tag name instead of name in log messages
a239820 Merge pull request #108 from moorereason/iss95-ci-tests
39464a5 Refactor CI-Tests to show negative results
7937da4 Merge pull request #103 from naveensrinivasan/fix/golangrun-ci-issue
9b1e28e Merge pull request #106 from ossf/b3
2d348a7 Merge pull request #105 from naveensrinivasan/feat/makefile
91780fd Allow skipping scheme, fix regression.
a56f707 (origin/feat/makefile, feat/makefile) Feat - Implemented Makefile and actions for PR
06f2616 (origin/fix/golangrun-ci-issue, fix/golangrun-ci-issue) fix - golangci-lint issues
c308663 Merge pull request #102 from naveensrinivasan/fix/shellcheck
3de6a1b (origin/fix/shellcheck, fix/shellcheck) fix - shellcheck violations for cron.sh
6549ecc Create codeql-analysis.yml (#101)
f7cb4d7 Merge pull request #100 from naveensrinivasan/fix/http-path
4362368 (origin/fix/http-path, fix/http-path) Tests updated to include validation for parsing
fd3a2a8 fix - URL with trailing slash
6b80b78 Merge pull request #98 from moorereason/iss95
ac55575 Adjust details logging on a few checks
348bedb Show negative results in Signed-Releases details
eb0d488 Show negative results in Signed-Tags details
4ec34e9 Show negative results to Pull-Requests details
1991617 Merge pull request #94 from ossf/b3
7a10bed (origin/b3) Improve SAST check.
c5abb92 Merge pull request #91 from ossf/a12
87d6954 Merge pull request #92 from ossf/b1
0bcd8ea (origin/b1) Improve fuzzing check.
ab2c9d4 (origin/a12) Add support for yarn, composer in frozen deps check.
983e406 Merge pull request #90 from dlorenc/moreprojects
cd16def Add 50 Google projects.

OAuth token seems to be required (not recommended) to avoid rate limits

The first time I ran scorecard (on Windows 10 Home edition, with go version: go version go1.15.3 windows/amd64) I got a warning message after [Fuzzing] finished that the rate limit was exceeded in roundtripper.go (further investigation showed the warning was logged after attempting a request to https://api.github.com/graphql). Only after setting GITHUB_AUTH_TOKEN to a personal access token of mine everything worked fine. Hence, perhaps the wording in the README should be updated from "It is recommended" to "It is required"?

Two related suggestions:

  • Specify what the scope of the token should be. It seems I didn't have to specify any scope (which makes sense) but I think it could be useful to have this explicit.
  • Specify how to set the access token as an environment variable on Windows. If I'm not mistaken one should use set GITHUB_AUTH_TOKEN=<your access token> for that.

Frozen-Deps: Handle npm-shrinkwrap.json like package-lock.json

According to npm, both files have the same effect of "freezing" dependencies: https://docs.npmjs.com/cli/v6/configuring-npm/shrinkwrap-json

We have a CLI project which uses npm-shrinkwrap.json instead of package-lock.json: https://github.com/SAP/ui5-cli

I would expect the "Frozen-Deps" check to pass. But it fails since it does not check for npm-shrinkwrap.json:

scorecard/checks/frozen_deps.go

Lines 80 to 82 in 9d18604

case "package-lock.json":
c.Logf("nodejs packages found: %s", name)
return passResult

Is there any reason for that?

By the way, GitHub's handling is very odd in that regard. It seems to detect the content of npm-shrinkwrap.json but declares it as package-lock.json: https://github.com/SAP/ui5-cli/network/dependencies.
Also npm-shrinkwrap.json seems to be missing in the corresponding documentation. I'll try and reach out to GitHub to clarify that 👍

Panic in SASTToolInCheckRuns

I ran into an odd issue today:

$ ./scorecard --repo=github.com/adnanh/webhook --show-details
Starting [Active]
Starting [Branch-Protection]
Starting [CI-Tests]
Starting [CII-Best-Practices]
Starting [Code-Review]
Starting [Contributors]
Starting [Frozen-Deps]
Starting [Fuzzing]
Starting [Packaging]
Starting [Pull-Requests]
Starting [SAST]
Starting [Security-Policy]
Starting [Signed-Releases]
Starting [Signed-Tags]
Finished [Fuzzing]
Finished [CII-Best-Practices]
Finished [Branch-Protection]
Finished [Packaging]
Finished [Frozen-Deps]
Finished [Signed-Tags]
Finished [Signed-Releases]
Finished [Contributors]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x7696f1]

goroutine 29 [running]:
github.com/ossf/scorecard/checks.SASTToolInCheckRuns(0x9a53c0, 0xc00009c000, 0xc0000e0840, 0xc0000b00d0, 0xc000097620, 0xc00009a103, 0x6, 0xc00009a10a, 0x7, 0xc000032320, ...)
        /src/github.com/ossf/scorecard/checks/sast.go:54 +0x231
github.com/ossf/scorecard/checker.MultiCheck.func1(0x9a53c0, 0xc00009c000, 0xc0000e0840, 0xc0000b00d0, 0xc000097620, 0xc00009a103, 0x6, 0xc00009a10a, 0x7, 0xc000032320, ...)
        /src/github.com/ossf/scorecard/checker/check.go:57 +0xf8
github.com/ossf/scorecard/checks.SAST(0x9a53c0, 0xc00009c000, 0xc0000e0840, 0xc0000b00d0, 0xc000097620, 0xc00009a103, 0x6, 0xc00009a10a, 0x7, 0xc000032320, ...)
        /src/github.com/ossf/scorecard/checks/sast.go:32 +0xd5
github.com/ossf/scorecard/checker.(*Runner).Run(0xc0001fbea8, 0x933ba0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /src/github.com/ossf/scorecard/checker/checker.go:54 +0x1b5
github.com/ossf/scorecard/pkg.RunScorecards.func1(0xc00009c414, 0x9a53c0, 0xc00009c000, 0xc0000e0840, 0xc0000b00d0, 0xc000097620, 0xc00009a103, 0x6, 0xc00009a10a, 0x7, ...)
        /src/github.com/ossf/scorecard/pkg/scorecard.go:107 +0xd4
created by github.com/ossf/scorecard/pkg.RunScorecards
        /src/github.com/ossf/scorecard/pkg/scorecard.go:104 +0x353

If I limit the checks to just SAST, I haven't seen it panic yet. I added some debugging print statements before the range loop to see what the github package is returning. It looks something like this:

pr     = https://github.com/adnanh/webhook/pull/463
error  = <nil>
resp   = &{0xc000778d80 0 0 0 0  github.Rate{Limit:5000, Remaining:3034, Reset:github.Timestamp{2021-01-18 10:01:53 -0600 CST}}}
status = 200 OK
crs    = (*github.ListCheckRunsResults)(nil)

It would be easy to add a nil-check before accessing csr, but it seems like the github package should be returning an error or a non-nil response.

$ go version
go version go1.16beta1 linux/amd64

$ git log -1 --oneline
c00aa4b (HEAD -> main, origin/main, origin/HEAD) Add e2e tests for remaining checks.

I tried updating to github/v33, but the problem remains.

Question RE value of Contributors Check

"Does the project have contributors from at least two different organizations?"

While I fully endorse this from a governance perspective, I fail to see what it has to do with the security of a project specifically.

Eat your own dog food

Running the scorecard on the scorecard repo should succeed.

./scorecard --repo=https://github.com/ossf/scorecard Starting [Active] Starting [CI-Tests] Starting [CII-Best-Practices] Starting [Code-Review] Starting [Contributors] Starting [Frozen-Deps] Starting [Fuzzing] Starting [Pull-Requests] Starting [SAST] Starting [Security-Policy] Starting [Signed-Releases] Starting [Signed-Tags] Finished [Fuzzing] Finished [CII-Best-Practices] Finished [Signed-Releases] Finished [Frozen-Deps] Finished [Contributors] Finished [Signed-Tags] Finished [Security-Policy] Finished [SAST] Finished [Pull-Requests] Finished [Active] Finished [Code-Review] Finished [CI-Tests]

RESULTS

Active: Pass 10
CI-Tests: Fail 0
CII-Best-Practices: Fail 10
Code-Review: Pass 10
Contributors: Fail 10
Frozen-Deps: Pass 10
Fuzzing: Fail 10
Pull-Requests: Pass 9
SAST: Fail 10
Security-Policy: Fail 10
Signed-Releases: Fail 0
Signed-Tags: Fail 0

Active != 90-day releases

The current definition of "active" is:

  • Did the project get any commits and releases in last 90 days?

I think this is a terrible definition. Many projects intentionally do not release every 90 days, nor do their users want that kind of churn.

Can we please change "and" to "or"? Or alternatively, remove "and releases" entirely?

Change Fuzzing measurement

The current definition of "Fuzzing" is:

  • Does the project use OSS-Fuzz?

I think the definition should be:

  • Does the project use fuzzing to detect security vulnerabilities (e.g., by using OSS-Fuzz)?

If they do fuzzing some other way it should be fine. It may be harder to detect, but that's okay, that can be improved over time as long as the definition allows for it.

Fails when trailing slash is passed in the URL for the GitHub repository

The code fails to fetch the repository information when there is a trailing slash in the URL.

./scorecard --repo=https://github.com/volcano-sh/volcano/ --show-details
Starting [Active]
Starting [CI-Tests]
Starting [CII-Best-Practices]
Starting [Code-Review]
Starting [Contributors]
Starting [Frozen-Deps]
Starting [Fuzzing]
Starting [Pull-Requests]
Starting [SAST]
Starting [Security-Policy]
Starting [Signed-Releases]
Starting [Signed-Tags]
Finished [Fuzzing]
Finished [CII-Best-Practices]
Finished [SAST]
Finished [Pull-Requests]
Finished [Frozen-Deps]
Finished [Contributors]
Finished [Signed-Releases]
Finished [Signed-Tags]
Finished [CI-Tests]
Finished [Active]
Finished [Code-Review]
Finished [Security-Policy]

RESULTS
-------
Active: Fail 0
    error, retrying: GET https://api.github.com/repos/volcano-sh/volcano//releases: 404 Not Found []
CI-Tests: Fail 0
    error, retrying: GET https://api.github.com/repos/volcano-sh/volcano//pulls?state=closed: 404 Not Found []
CII-Best-Practices: Pass 10
    badge level: passing
Code-Review: Fail 0
    error, retrying: GET https://api.github.com/repos/volcano-sh/volcano//commits: 404 Not Found []
Contributors: Fail 0
    error, retrying: GET https://api.github.com/repos/volcano-sh/volcano//contributors: 404 Not Found []
Frozen-Deps: Fail 0
    error, retrying: GET https://api.github.com/repos/volcano-sh/volcano/: 404 Not Found []
Fuzzing: Fail 10
Pull-Requests: Fail 0
    error, retrying: GET https://api.github.com/repos/volcano-sh/volcano//commits: 404 Not Found []
SAST: Fail 10
Security-Policy: Fail 10
Signed-Releases: Fail 0
    error, retrying: GET https://api.github.com/repos/volcano-sh/volcano//releases: 404 Not Found []
Signed-Tags: Fail 0
    error, retrying: Could not resolve to a Repository with the name 'volcano-sh/volcano/'.
    ```

Searching for vuln reporting info outside of GH's security.md

For the Security Policy field, some projects that use GitHub do have a documented security policy, but are not using the GitHub Security Policy feature, so scraping just for GH's Security Policy via security.md would inaccurately show a false in this field.

I think this is most commonly seen with projects that were established before the Security Policy feature, or for projects mirror to GitHub.

The score card should indicate when checks don't support a particular language/ecosystem

The scorecard can be populated for any open source project without any work or interaction from maintainers.

Maintainers must be provided with a mechanism to correct any automated scorecard findings they feel were made in error, provide "hints" for anything we can't detect automatically, and even dispute the applicability of a given scorecard finding for that repository.

There are some checks, like fuzzing, that are very specific to particular languages.

I think any score card should make clear when a particular check doesn't natively support a language or ecosystem.

As someone who is a consumer, and a maintainer, it would be good to know that a project didn't really score zero on something. It's just that the check doesn't support that language/ecosystem vs the maintainer has gone to the effort of somehow flagging that they do something the automated checks can't pick up.

Is the project abandoned?

I can't think of a good way to check this.

See twitter.com/alexellisuk/status/1315711970729955328 for an example.

Specifically, I want to make sure we differentiate between "done" projects that simply don't receive many PRs and projects that are abandoned, with changes/PRs piling up.

Add details to the public data

It would be great if the public data included the details (--show-details) as well as the pass/fail and confidence indicator. This would give consumers of those data files a bit more information when parsing them.

Branch Protection are failing for some repositories

The branch protection checks are working for this

./scorecard --repo=https://github.com/ossf/scorecard --show-details --checks Branch-Protection
Starting [Branch-Protection]
Finished [Branch-Protection]

It is failing for envoy repository.

curl https://api.github.com/repos/envoyproxy/envoy/branches/main

{
  "name": "main",
  "commit": {
    "sha": "88140146696fe09cebad628321020528a7e9ecd0",
    "node_id": "MDY6Q29tbWl0NjUyMTQxOTE6ODgxNDAxNDY2OTZmZTA5Y2ViYWQ2MjgzMjEwMjA1MjhhN2U5ZWNkMA==",
    "commit": {
      "author": {
        "name": "Alex Konradi",
        "email": "[email protected]",
        "date": "2021-01-25T22:39:00Z"
      },
      "committer": {
        "name": "GitHub",
        "email": "[email protected]",
        "date": "2021-01-25T22:39:00Z"
      },
      "message": "overload: scale transport socket connect timeout (#13800)\n\nAdd support for scaling the transport socket connect timeout with load.\r\n\r\nRisk Level: low\r\nTesting: added tests and ran affected tests\r\nDocs Changes: none\r\nRelease Notes: none\r\nPlatform Specific Features: none\r\nFixes: #11426\r\n\r\nSigned-off-by: Alex Konradi <[email protected]>",
      "tree": {
        "sha": "598f129ae2765942adcff11cff6a714d5fc4c04c",
        "url": "https://api.github.com/repos/envoyproxy/envoy/git/trees/598f129ae2765942adcff11cff6a714d5fc4c04c"
      },
      "url": "https://api.github.com/repos/envoyproxy/envoy/git/commits/88140146696fe09cebad628321020528a7e9ecd0",
      "comment_count": 0,
      "verification": {
        "verified": true,
        "reason": "valid",
        "signature": "-----BEGIN PGP SIGNATURE-----\n\nwsBcBAABCAAQBQJgD0iECRBK7hj4Ov3rIwAAdHIIALC1oGf32w8tyz3GU9JF0eY+\nkU5KSQsuU7csZl+f9l+hjs+FnnYHpx8xwirbf7vTqSfhkY8jwXAxT5AV5uLFVNIR\nC6StmfvWxZAI1mdjbiLPhLOxrKPL6oxqMf6wjq1CVhHbu0FYDrOW8kjM9nV9iqqA\nDAQtkFlhcNzTyemgcn+AAYWokqdGhzoQ/HaD+1yTAv8OyRWA34V3RuH0G6KUkyw9\nT/UJ9bxAyZFDSBrGM++VQJQEhpaPykO3UQrEzWLGs41M6VhNUBzqVdxF0gGfHdPf\nK6Q7kQUnVxmueiyESCCmbED3BwtAKOzfuCvgn9LL+/5ee9jvbDfIr980mstSZD8=\n=PFhs\n-----END PGP SIGNATURE-----\n",
        "payload": "tree 598f129ae2765942adcff11cff6a714d5fc4c04c\nparent 90534ebdaeb05306b3c4ebcfd3b20d8c5c364529\nauthor Alex Konradi <[email protected]> 1611614340 -0500\ncommitter GitHub <[email protected]> 1611614340 -0800\n\noverload: scale transport socket connect timeout (#13800)\n\nAdd support for scaling the transport socket connect timeout with load.\r\n\r\nRisk Level: low\r\nTesting: added tests and ran affected tests\r\nDocs Changes: none\r\nRelease Notes: none\r\nPlatform Specific Features: none\r\nFixes: #11426\r\n\r\nSigned-off-by: Alex Konradi <[email protected]>"
      }
    },
    "url": "https://api.github.com/repos/envoyproxy/envoy/commits/88140146696fe09cebad628321020528a7e9ecd0",
    "html_url": "https://github.com/envoyproxy/envoy/commit/88140146696fe09cebad628321020528a7e9ecd0",
    "comments_url": "https://api.github.com/repos/envoyproxy/envoy/commits/88140146696fe09cebad628321020528a7e9ecd0/comments",
    "author": {
      "login": "akonradi",
      "id": 2974888,
      "node_id": "MDQ6VXNlcjI5NzQ4ODg=",
      "avatar_url": "https://avatars.githubusercontent.com/u/2974888?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/akonradi",
      "html_url": "https://github.com/akonradi",
      "followers_url": "https://api.github.com/users/akonradi/followers",
      "following_url": "https://api.github.com/users/akonradi/following{/other_user}",
      "gists_url": "https://api.github.com/users/akonradi/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/akonradi/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/akonradi/subscriptions",
      "organizations_url": "https://api.github.com/users/akonradi/orgs",
      "repos_url": "https://api.github.com/users/akonradi/repos",
      "events_url": "https://api.github.com/users/akonradi/events{/privacy}",
      "received_events_url": "https://api.github.com/users/akonradi/received_events",
      "type": "User",
      "site_admin": false
    },
    "committer": {
      "login": "web-flow",
      "id": 19864447,
      "node_id": "MDQ6VXNlcjE5ODY0NDQ3",
      "avatar_url": "https://avatars.githubusercontent.com/u/19864447?v=4",
      "gravatar_id": "",
      "url": "https://api.github.com/users/web-flow",
      "html_url": "https://github.com/web-flow",
      "followers_url": "https://api.github.com/users/web-flow/followers",
      "following_url": "https://api.github.com/users/web-flow/following{/other_user}",
      "gists_url": "https://api.github.com/users/web-flow/gists{/gist_id}",
      "starred_url": "https://api.github.com/users/web-flow/starred{/owner}{/repo}",
      "subscriptions_url": "https://api.github.com/users/web-flow/subscriptions",
      "organizations_url": "https://api.github.com/users/web-flow/orgs",
      "repos_url": "https://api.github.com/users/web-flow/repos",
      "events_url": "https://api.github.com/users/web-flow/events{/privacy}",
      "received_events_url": "https://api.github.com/users/web-flow/received_events",
      "type": "User",
      "site_admin": false
    },
    "parents": [
      {
        "sha": "90534ebdaeb05306b3c4ebcfd3b20d8c5c364529",
        "url": "https://api.github.com/repos/envoyproxy/envoy/commits/90534ebdaeb05306b3c4ebcfd3b20d8c5c364529",
        "html_url": "https://github.com/envoyproxy/envoy/commit/90534ebdaeb05306b3c4ebcfd3b20d8c5c364529"
      }
    ]
  },
  "_links": {
    "self": "https://api.github.com/repos/envoyproxy/envoy/branches/main",
    "html": "https://github.com/envoyproxy/envoy/tree/main"
  },
  "protected": true,
  "protection": {
    "enabled": true,
    "required_status_checks": {
      "enforcement_level": "non_admins",
      "contexts": [
        "DCO",
        "envoy-presubmit (check linux_x64 api)",
        "envoy-presubmit (check linux_x64 asan)",
        "envoy-presubmit (check linux_x64 clang_tidy)",
        "envoy-presubmit (check linux_x64 compile_time_options)",
        "envoy-presubmit (check linux_x64 coverage)",
        "envoy-presubmit (check linux_x64 fuzz_coverage)",
        "envoy-presubmit (check linux_x64 gcc)",
        "envoy-presubmit (check linux_x64 tsan)",
        "envoy-presubmit (docker linux multiarch)",
        "envoy-presubmit (linux_arm64 release)",
        "envoy-presubmit (linux_x64 release)",
        "envoy-presubmit (precheck docs)",
        "envoy-presubmit (precheck format)",
        "envoy-presubmit (verify examples)",
        "envoy-presubmit (windows docker)",
        "envoy-presubmit (windows release)",
        "envoy-presubmit (precheck dependencies)"
      ]
    }
  },
  "protection_url": "https://api.github.com/repos/envoyproxy/envoy/branches/main/protection"
}

But where was this API is failing?

curl https://api.github.com/repos/envoyproxy/envoy/branches/main/protection
{
 "message": "Not Found",
 "documentation_url": "https://docs.github.com/rest/reference/repos#get-branch-protection"
}

Fix golang-lint ci issues

Enable golang-lint ci issues in the code based on this config.

---
run:
  concurrency: 6
  deadline: 5m
linters:
  disable-all: true
  enable:
    - asciicheck
    - bodyclose
    - deadcode
    - depguard
    - dogsled
    - dupl
    - errcheck
    - errorlint
    - exhaustive
    - exportloopref
    - gci
    - gochecknoinits
    - gocognit
    - goconst
    - gocritic
    - gocyclo
    - godot
    - godox
    - goerr113
    - gofmt
    - gofumpt
    - goheader
    - goimports
    - golint
    - gomnd
    - gomodguard
    - goprintffuncname
    - gosec
    - gosimple
    - govet
    - ineffassign
    - interfacer
    - lll
    - makezero
    - maligned
    - misspell
    - nakedret
    - nestif
    - noctx
    - nolintlint
    - paralleltest
    - prealloc
    - predeclared
    - rowserrcheck
    - scopelint
    - sqlclosecheck
    - staticcheck
    - structcheck
    - stylecheck
    - thelper
    - tparallel
    - typecheck
    - unconvert
    - unparam
    - unused
    - varcheck
    - whitespace
    - wrapcheck
    # - exhaustivestruct
    # - forbidigo
    # - funlen
    # - gochecknoglobals
    # - nlreturn
    # - testpackage
    # - wsl
linters-settings:
  gci:
    local-prefixes: sigs.k8s.io/security-profiles-operator
  errcheck:
    check-type-assertions: true
    check-blank: true
  godox:
    keywords:
      - BUG
      - FIXME
      - HACK
  gocritic:
    enabled-checks:
      # Diagnostic
      - appendAssign
      - argOrder
      - badCond
      - caseOrder
      - codegenComment
      - commentedOutCode
      - deprecatedComment
      - dupArg
      - dupBranchBody
      - dupCase
      - dupSubExpr
      - exitAfterDefer
      - flagDeref
      - flagName
      - nilValReturn
      - offBy1
      - sloppyReassign
      - weakCond
      - octalLiteral

      # Performance
      - appendCombine
      - equalFold
      - hugeParam
      - indexAlloc
      - rangeExprCopy
      - rangeValCopy

      # Style
      - assignOp
      - boolExprSimplify
      - captLocal
      - commentFormatting
      - commentedOutImport
      - defaultCaseOrder
      - docStub
      - elseif
      - emptyFallthrough
      - emptyStringTest
      - hexLiteral
      - ifElseChain
      - methodExprCall
      - regexpMust
      - singleCaseSwitch
      - sloppyLen
      - stringXbytes
      - switchTrue
      - typeAssertChain
      - typeSwitchVar
      - underef
      - unlabelStmt
      - unlambda
      - unslice
      - valSwap
      - wrapperFunc
      - yodaStyleExpr

      # Opinionated
      - builtinShadow
      - importShadow
      - initClause
      - nestingReduce
      - paramTypeCombine
      - ptrToRefParam
      - typeUnparen
      - unnamedResult
      - unnecessaryBlock

Fix the direct http call to get user org

The code in the contributors.go uses the direct HTTP call url := fmt.Sprintf("https://api.github.com/users/%s/orgs", contrib.GetLogin()) instead of the go API for github.

This call has to be replaced with a standard go GitHub api.

Signed-Tags checks fail on certain repositories

Signed-Tags checks fail

./scorecard --repo=https://github.com/volcano-sh/volcano --show-details --checks Signed-Tags
Starting [Signed-Tags]
Finished [Signed-Tags]

RESULTS
-------
Signed-Tags: Fail 0
    error, retrying: GET https://api.github.com/repos/volcano-sh/volcano/git/tags/89c2fa6d85de17ee7c140c080a7ecce17fa32453: 404 Not Found []

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.