We turned on GH secrets detection. There are now 14 open issues in https://github.com/cloud-gov/caulking/security/secret-scanning. I should make them go away.

Acceptance Criteria

• Zero unaddressed findings in secrets detection

Currently, the Makefile system is responsible for creating the ~/.git-support directory, copying the relevant files into it, and configuring core.hooksPath to point at it. This means we have to remember to run make install whenever we pull new changes to this repo, and we have to ensure make install is idempotent.

I think it might simplify things if we instead had make install create ~/.git-support as a symlink into the locally cloned copy of this repository, and we embedded the required files directly in here. That would simplify the installation process, and would make upgrades as easy as git pull.

This repo's structure would then look something like:

~/workspace/caulking/
├── README.md
├── ...
├── gitleaks.toml
└── hooks
    └── pre-commit

....git-support would look like:

$HOME/.git-support -> ~/workspace/caulking

...and make install would run something like:

ln -sf "$(dirname "$0")" ~/.git-support

Whadayathink?

In order to more fully ensure AWS keys can't be inadvertently leaked, add a rule to detect any string that might be an AWS key.

This is a finding from our 3PAO review of caulking

This will generate lots of false positives, so it would be considered a mitigation until we better ensure that only short-lived authenticators are used.

Security considerations

To update rules, no implications as such

Implementation sketch

In order to prevent FPs, we want not to alert on "secrets" in PNG files.

Security considerations

Fewer FPs is good.

In order to [reason/outcome/goal], [someone or "we"] want [a specific change in product implementation/behavior]

Acceptance Criteria

• GIVEN [a precondition]
  AND [another precondition]
  WHEN [test step]
  AND [test step]
  THEN [verification step]
  AND [verification step]

Security considerations

[note any potential changes to security boundaries, practices, documentation, risk that arise directly from this story]

Implementation sketch

[links to background notes, sketches, and/or relevant documentation

• [first thing to do]
• [another thing to do]

A new version of gitleaks has been released to bring it to version 8.0.0 (now at 8.1.1 at the time of this writing), which brings some breaking changes with it to our current setup.

Specifically, when I go to commit a series of changes now, gitleaks will error out due to an unknown flag, --unstaged. This flag was removed in the newest versions of the tool and appears to have been replaced/incorporated into a new protect subcommand.

It looks like the line we need to change is line 17 in our pre-commit shell script, but I suspect we'll have to do a bit more than just swap a flag with a subcommand. It looks like some of the flags/options may have changed as well according to the Usage section of the README in the latest release.

It also looks like the new version (8.1.1) ignores gitleaks.toml by default; I'm not sure if this will have any impact, either, since I see it referenced in our script as well.

The alternative is to pin our version again to a known working version for us (I believe that would be 7.6.1 - though it looks like we try and pin to 7.2.1) until we have time to come back to this.

See screenshot below. Two issues:

1. False positive of CF_PASSWORD: ((production-cf-password)) (which is fine - not the point of this issue)
2. Caught this secret when it was being removed from the repo (this is the point of this issue). Can we configure gitleaks to only stop commits that add leaks?

Running make audit, I get

 ✗ audit fails if AWS keys are in ~/.aws
   (in test file caulked.bats, line 69)
     `[ ${status} -eq 1 ]' failed with status 2
   Initialized empty Git repository in /private/var/folders/8x/vrpz2njj5vv0ls0xtr32_w240000gn/T/gittest.ChNDn1/.git/

I don't have a .aws directory. At least, I don't think I do. (I can't find one in $HOME.)

I'm wondering if the grep is returning a non-zero code if the directory doesn't exist? Not sure... that's just based on a quick look on my part.

I don't know that there is anything unique about my setup. Happy to provide more info if it helps.

In order to make users and developers lives simpler, the Makefile should be self-documenting a la https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html

Security considerations

None. Refactor in functionality

Currently, caulking checks all modified files for suspected leaks, not just those to be committed. It would be a much better experience to only check those files that are being committed.

Example:

$ git diff --cached
diff --git a/.gitignore b/.gitignore
index 8f8109e..177b00d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -3,3 +3,4 @@ venv
 .vscode
 **/__pycache__/*
 
+flask_session

$ git commit -m 'not gonna work'
{
        "line": "    monkeypatch.setenv(\"UAA_AUTH_URL\", \"https://uaa.example.com/authorize\")",
        "lineNumber": 24,
        "offender": "keypatch.setenv(\"UAA_AUTH_URL\"",
        "commit": "0000000000000000000000000000000000000000",
        "repo": "kibana-cf-auth-proxy",
        "rule": "Generic Credential",
        "commitMessage": "***STAGED CHANGES***",
        "author": "",
        "email": "",
        "file": "tests/unit/test_config.py",
        "date": "1970-01-01T00:00:00Z",
        "tags": "key, API, generic",
        "operation": "equal"
}
WARN[2021-01-21T15:53:35-08:00] 3 leaks detected in staged changes           
Error: gitleaks has detected sensitive information in your changes.
For examples use: CHANGEME|changeme|feedabee|EXAMPLE|23.22.13.113|1234567890
If you know what you are doing you can disable this check using:
    git config --local hooks.gitleaks false; 
    git commit ....; 
    git config --local hooks.gitleaks true; 

In order to not get flagged for patterns in upstream code bases, we should whitelist Go vendor stuff in vendor/gopkg.in and vendor/github.com

See for example Van's branch https://github.com/cloud-gov/aws-broker/tree/redis-service

Security considerations

Port numbers aren't considered CUI.

In order to reduce the amount of operational intelligence our Terraform repos provide, we should probably fail on .tf files with:

module "foo_provision_user" {
  source        = "../../modules/iam_user/iam_cert_provision"
  username      = "foo-provision"
  aws_partition = data.aws_partition.current.partition
  account_id    = data.aws_caller_identity.current.account_id
}

but allow:

module "foo_provision_user" {
  source        = "../../modules/iam_user/iam_cert_provision"
  username      = var.provision_user
  aws_partition = data.aws_partition.current.partition
  account_id    = data.aws_caller_identity.current.account_id
}

While we may get flags where this unobfuscated behavior is OK, it's better to flag it and OK it, then to let too much info leak out.

Security considerations

Improves our stance.

Implementation sketch

• Change the success/fail status of the current tests in devlopment.bats.
• Change the pattern .toml file

related to #5

tl;dr: If I have global hooks enabled, and those global hooks check gitleaks, I think the check for gitleaks in local hooks is unnecessary

One of the tests run by make audit looks like this:

• check to see if .git/hooks/pre-commit exists in the repo. If not, return success.
• check to see if .git/hooks/pre-commit has the string gitleaks in a way that doesn't look like a comment. If not, fail.

It seems, on my setup at least, that my global hooks run even when my local hooks don't have gitleaks in them (relevant info below)

$ git commit --allow-empty -m "test-commit"
black................................................(no files to check)Skipped
INFO[2020-10-15T10:48:07-07:00] No leaks detected in staged changes
[test-branch 0e419fb] test-commit
$  cat .git/hooks/pre-commit
#!/usr/local/opt/pre-commit/libexec/bin/python3
# File generated by pre-commit: https://pre-commit.com
# ID: 138fd403232d2ddd5efb44317e38bf03
import os
import sys

# we try our best, but the shebang of this script is difficult to determine:
# - macos doesn't ship with python3
# - windows executables are almost always `python.exe`
# therefore we continue to support python2 for this small script
if sys.version_info < (3, 3):
    from distutils.spawn import find_executable as which
else:
    from shutil import which

# work around https://github.com/Homebrew/homebrew-core/issues/30445
os.environ.pop('__PYVENV_LAUNCHER__', None)

# start templated
INSTALL_PYTHON = '/usr/local/opt/pre-commit/libexec/bin/python3'
ARGS = ['hook-impl', '--config=.pre-commit-config.yaml', '--hook-type=pre-commit']
# end templated
ARGS.extend(('--hook-dir', os.path.realpath(os.path.dirname(__file__))))
ARGS.append('--')
ARGS.extend(sys.argv[1:])

DNE = '`pre-commit` not found.  Did you forget to activate your virtualenv?'
if os.access(INSTALL_PYTHON, os.X_OK):
    CMD = [INSTALL_PYTHON, '-mpre_commit']
elif which('pre-commit'):
    CMD = ['pre-commit']
else:
    raise SystemExit(DNE)

CMD.extend(ARGS)
if sys.platform == 'win32':  # https://bugs.python.org/issue19124
    import subprocess

    if sys.version_info < (3, 7):  # https://bugs.python.org/issue25942
        raise SystemExit(subprocess.Popen(CMD).wait())
    else:
        raise SystemExit(subprocess.call(CMD))
else:
    os.execvp(CMD[0], CMD)
$ git config -l
credential.helper=osxkeychain
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
filter.lfs.clean=git-lfs clean -- %f
filter.lfs.smudge=git-lfs smudge -- %f
filter.lfs.clean=git-lfs clean -- %f
filter.lfs.smudge=git-lfs smudge -- %f
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
gitseekret.version=1
gitseekret.rulespath=/Users/my_user/.git-support/seekret-rules
gitseekret.rulesenabled=aws.secret_access_key,aws.access_key_id,aws.account_id,mandrill.username,mandrill.password,mandrill.api_key,newrelic.license_key,slack.api_token
gitseekret.exceptionsfile=
hooks.gitleaks=true
core.hookspath=/Users/my_user/.git-support/hooks
pull.ff=only
init.defaultbranch=main
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true
core.ignorecase=true
core.precomposeunicode=true
[email protected]:cloud-gov/external-domain-broker-migrator.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.main.remote=origin
branch.main.merge=refs/heads/main
hooks.gitleaks=true

In order to better ensure alerting and response to any secrets that might yet get committed to GitHub, assess leakd.io and gitguardian

Acceptance Criteria

Security considerations

None - would only use publicly available APIs

Implementation sketch

The issue with the core.hooksPath setting is that it uses the global directory instead of the per-repository $GIT_DIR/hooks directory:

$ man git-config
...
core.hooksPath
  By default Git will look for your hooks in the $GIT_DIR/hooks directory. 
  Set this to different path, e.g.  /etc/git/hooks, and Git will try to 
  find your hooks in that directory, e.g.  /etc/git/hooks/pre-receive 
  instead of in $GIT_DIR/hooks/pre-receive.

This breaks a number of plugins like git-crypt (and maybe git-LFS) and doesn't allow for pre-commit linters, etc. Even if we don't use any of those capabilities now, it feels like a recipe for confusion in the future. I can think of two solutions:

1. Use a much more complicated structure such as pivotal-cf/git-hooks-core. This would be very flexible and would allow us to add all kinds of nifty global git-hook behavior. It would also be a huge change, and somewhat complicated to introduce. Or...

2. Add this snippet to the existing ~/.git-support/hooks/pre-commit script:

   git_dir=$( git rev-parse --git-dir )
   if [ -f "${git_dir}/hooks/pre-commit" ]; then
     "${git_dir}/hooks/pre-commit" $@
   fi

   This is a simple (and only slightly dirty) solution.

Thoughts?

concourse_password: blahBLARGblah123blah was not caught

Notes

Acceptance Criteria

• anything_password should be looked at

In order to fix the false positive on these file types, we may want to ignore image files and generated lock files used in dependency management.

Acceptance Criteria

• GIVEN images and generate files when trying to commit them they should be ignored

Security considerations

Looks to be a false positive

Implementation sketch

Error Examples

## package-lock.json example
{
	"line": "      \"resolved\": \"https://registry.npmjs.org/source-map-support/-/source-map-support-0.5.16.tgz\",",
	"offender": "port-0.5",
	"commit": "0000000000000000000000000000000000000000",
	"repo": "cg-site",
	"rule": "Port",
	"commitMessage": "***STAGED CHANGES***",
	"author": "",
	"email": "",
	"file": "package-lock.json",
	"date": "1970-01-01T00:00:00Z",
	"tags": ""
}

## svg file example
{
	"line": "\u003csvg xmlns=\"http://www.w3.org/2000/svg\" viewBox=\"0 0 233 47\" id=\"logo\"\u003e\u003ctitle\u003elogo\u003c/title\u003e\u003cg fill=\"none\" fill-rule=\"evenodd\"\u003e\u003cpath d=\"M57.547 18.534a3.71 3.71 0 0 0-1.679-1.276 5.563 5.563 0 0 0-2.02...",
	"offender": " 2.403.268.729.",
	"commit": "0000000000000000000000000000000000000000",
	"repo": "cg-site",
	"rule": "IPv4 addresses",
	"commitMessage": "***STAGED CHANGES***",
	"author": "",
	"email": "",
	"file": "assets/img/cloud-gov-logo.svg",
	"date": "1970-01-01T00:00:00Z",
	"tags": "IPv4, IP, addresses"
}

• Ignore images
• Ignore generated files

I performed a new install of caulking and proceeded to run make audit, which had one failure:

 ✗ leak prevention catches IPv4 address in test repo
   (in test file caulked.bats, line 56)
     `[ ${status} -eq 1 ]' failed
   Initialized empty Git repository in /private/var/folders/j7/qqyv0gtj4rj5nl0_jr2sgh5c0000gn/T/gittest.JBJs85/.git/

test_helper.sh uses a private IP address range in the test, line 86:

Host: 10.20.30.40

If private ranges are supposed to be allowed in commits, then the test should be changed to not use a private range (e.g., 11.22.33.44 instead of 10.20.30.40). On the other hand, if private ranges should be considered secrets, then probably the regexes in local.toml need to be updated to catch them.

In order to not trivially leak personal email address, we want to ensure our git configuration is set to use GSA as our email.

Security considerations

[note any potential changes to security boundaries, practices, documentation, risk that arise directly from this story]

Implementation sketch

[links to background notes, sketches, and/or relevant documentation

• [first thing to do]
• [another thing to do]

In order to avoid false positives, do not flag on with things like "offender": "password\": \"${mailer_password}\""

Security considerations

Fewer false positives are better.

make audit should confirm the local version of caulking is up-to-date with the latest release.

Notes

• Implementation idea is to create a VERSION file and compare local copy to remote copy. Fail if they're different.

Acceptance Criteria

• make audit fails when on an older clone

✗ repos have hooks.gitleaks set to true
   (in test file caulked.bats, line 60)
     `./check_repos.sh $HOME check_hooks_gitleaks >&3' failed
   Initialized empty Git repository in /private/var/folders/8x/vrpz2njj5vv0ls0xtr32_w240000gn/T/gittest.aVPb6T/.git/

This does not give me, as a user, a lot to go on. What repository did I do this awful thing to?

(After 10m of digging around... perhaps this could be a PR, but...)

My solution was to add the following to the else case of check_hooks_gitleaks() in check_repos.sh:

        >&2 echo "hooks.gitleaks set to false: $gitrepo"

right before return 1. This then told me exactly which repositories were at fault.

$ echo 127.0.0.1 >> README.md 

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")

Note that README.md has a "secret", but isn't staged for commit, so this shouldn't happen:

$ git commit -m "wat"
caulking: {
caulking: 	"line": "127.0.0.1\n",
caulking: 	"offender": "127.0.0.1\n",
caulking: 	"commit": "0000000000000000000000000000000000000000",
caulking: 	"repo": "REDACTED",
caulking: 	"rule": "IPv4 addresses",
caulking: 	"commitMessage": "***STAGED CHANGES***",
caulking: 	"author": "",
caulking: 	"email": "",
caulking: 	"file": "README.md",
caulking: 	"date": "1970-01-01T00:00:00Z",
caulking: 	"tags": "IPv4, IP, addresses"
caulking: }
caulking: WARN[2020-04-10T08:25:56-07:00] 1 leaks detected in staged changes           
caulking:
caulking: If you know what you are doing you can disable this check using:
caulking:   git config --local hooks.gitleaks false
caulking:   git commit ....
caulking:   git config --local hooks.gitleaks true

Caulking found an email in my code which was intended to be there as it was contained in a header block about the maintainer:
__email__ = "[email protected]"

Caulking should allow this type of occurrence of an email address in code.

In order to avoid false positives, commits like adding cg-scripts/log4j*files should not trigger the Generic Credentials rules:

Here's sample output:

INFO[0000] opening .
{
	"line": "csvwriter.writerow([\"Path\",\"Node_0\",\"Instance_GUID\",\"Customer_Path\",\"Plugin_URLS\",\"App_GUID\",\"App_Name\",\"Space_Name\",\"Org_Name\",\"Org_Managers\",\"Space_Devs\"])",
	"lineNumber": 2,
	"offender": "GUID\",\"Customer_Path\",\"Plugin_URLS\",\"App_GUID\",\"App_Name\",\"Space_Name\",\"Org_Name\",\"Org_Managers\",\"Space_Devs\"",
	"offenderEntropy": -1,
	"commit": "0000000000000000000000000000000000000000",
	"repo": "cg-scripts",
	"repoURL": "",
	"leakURL": "",
	"rule": "Generic Credential",
	"commitMessage": "",
	"author": "",
	"email": "",
	"file": "audit/log4j-nessus-parser.py",
	"date": "1970-01-01T00:00:00Z",
	"tags": "key, API, generic"
}
{
	"line": "app_guid_file = open(\"app_guids\", \"r\")",
	"lineNumber": 2,
	"offender": "guid_file = open(\"app_guids\"",
	"offenderEntropy": -1,
	"commit": "0000000000000000000000000000000000000000",
	"repo": "cg-scripts",
	"repoURL": "",
	"leakURL": "",
	"rule": "Generic Credential",
	"commitMessage": "",
	"author": "",
	"email": "",
	"file": "audit/log4j-report-users.py",
	"date": "1970-01-01T00:00:00Z",
	"tags": "key, API, generic"
}

Acceptance Criteria

• GIVEN a commit with the cg-scripts/audit/log4j* files,
  THEN no findings

Security considerations

Reducing false positives supports use of this tool.

The repo https://github.com/cloud-gov/shibboleth-boshrelease contains a submodule in https://github.com/cloud-gov/shibboleth-boshrelease/tree/master/src. When attempting to update this submodule and commit, caulking throws the following error:

ERRO[0000] read /Users/***/workspace/cloud-gov/shibboleth-boshrelease/src/shibboleth-idp3-totp-auth: is a directory

In order to [reason/outcome/goal], [someone or "we"] want [a specific change in product implementation/behavior]

This may be a fairly restricted set of patterns to minimize false positives

Acceptance Criteria

• GIVEN [a precondition]
  AND [another precondition]
  WHEN [test step]
  AND [test step]
  THEN [verification step]
  AND [verification step]

Security considerations

[note any potential changes to security boundaries, practices, documentation, risk that arise directly from this story]

Implementation sketch

[links to background notes, sketches, and/or relevant documentation

• [first thing to do]
• [another thing to do]

Security considerations

[note any potential changes to security boundaries, practices, documentation, risk that arise directly from this story]

In order to prevent mysterious make audit results, caulking should detect if a git setup is not ready-to-use.

A user hadn't set setup git config --global user.email so git was always failing, so make audit was failing too.

Acceptance Criteria

• GIVEN a fresh workstation WITH a fresh make install THEN make audit SHOULD fail with a better error message

Security considerations

None

Implementation sketch

Any reason why the work-around when you encounter a false positive is to disable hooks, instead of just use the git commit --no-verify option?

Hook output currently recommends:

  git config --local hooks.gitleaks false
  git commit -m "message" 
  git config --local hooks.gitleaks true

In order to avoid false positives, we want lock files, such as package-lock.json, yarn.lock, pnpm-lock.yaml, etc... to be excluded from checks.

Acceptance Criteria

• GIVEN a lockfile
  WHEN it contains a bad word like key
  THEN it should not be flagged by caulking

Security considerations

None - lockfiles are meant to be checked into version control and will not contain secrets.

Implementation sketch

[links to background notes, sketches, and/or relevant documentation

• [first thing to do]
• [another thing to do]

Back with gitleaks 7.x we had this test:

@test "it fails a flask secret key" {
  cat > $REPO_PATH/webapp.py <<END
    app.secret_key = (
        '39a45464-cb1d-4b8d-aa1f-83c7c04fa673'
    )
END
    run testCommit $REPO_PATH
    should_fail
}

But that stopped working in 8.8.4.

For now I've changed the test to single line, but it would be good to detect across the newline.

I don't know if this is a bug/feature in 8.8.4 or perhaps in Go regexes.

Security considerations

1. This issue is safe, 2. Fixing this is a good thing

In order to successfully engage the 3PAO, we need to have our work complete enough to assess

Security considerations

No direct security considerations

Implementation sketch

Ensure the following caulking related outcomes are ready to test:

Commit no more secrets to Git

• ensure test suite in laptop covers our use cases for potential secrets
• update the laptop caulking repository with more patterns to match cloud.gov use cases
• use a bats script in cg-script to ensure operators have controls to prevent committing secrets
• Add scan for AWS secrets in ~/.aws
• Add a regular and periodic audit to verify the secret pre-commit hook is working on all dev laptops
  • Need to update SSP for this
• update documentation - see cg-site

Ensure we have no secrets now , or later, in GitHub

• Choose a tool to scan all our repos for the patterns identified above and report back
  • Will need to be gitleaks instead of further investigation
• Use such a tool at least monthly, or regularly as a service, or continually as part of periodic process, like an existing pipeline, or a regular manual process
  • Needs to be a part of conmon and added to SSP
• Update documentation on how we scan and how to conduct such a scan to minimize false positives
• NEW: ponder a github action to use gitleaks on every push/update