GoCD plugin to introduce a material that watches on a sub-directory of a Git repository.
This is the missing GoCD support for mono-repos.
...that you can't do with the built-in Git material:
- Support mono-repos without excessive triggering from a material that watches the entire repository
- Implement side-channels into deployment pipelines for source-controlled environment configuration
- Version control deployment configuration in the same repository as your code without rebuilding software every time unrelated configuration is changed; supporting twelve-factor apps
- Fix fan-in - materials that monitor different paths in a repo are considered separate when evaluating fan-in conditions
- Provide clean, filtered change logs for pipeline/VSM comparisons showing only changes on the monitored paths within a repository
This plugin is intended as
- a drop-in replacement for the built-in GoCD Git material (see known issues for some caveats)
- replaces the problematic use of whitelists and blacklists
- whitelists and blacklists in GoCD tend to prevent/block fan-in because materials with different whitelists are still considered equivalent by the fan-in algorithm. This makes them unsuitable for use as environment side-channels or dealing with mono-repos.
Wildcard whitelists - When the Git command line is available on your agents; you can also use wildcards like config/*/prod.yaml. Anything that works with git log will work here.
Shallow clones - supported in the same way as the GoCD Git Material supports them
Manually
Dockerized GoCD automatic plugin install
- Find release URL
- Set environment variable like the below; replacing
${VERSION}GOCD_PLUGIN_INSTALL_gocd-git-path-material-plugin=https://github.com/TWChennai/gocd-git-path-material-plugin/releases/download/v${VERSION}/gocd-git-path-material-plugin-${VERSION}.jar
| Plugin Version | Requirements | Recommended |
|---|---|---|
2.2+ |
GoCD >= 19.9 Server and agent running Java 11+ |
GoCD 20.9 for full functionality git binary available on path |
2.1 |
GoCD >= 18.10 Server and agent running Java 9+ |
At least GoCD 19.2, but >= 20.9 for full functionality git binary available on path |
1.x |
GoCD >= 15.1 Server and agent running Java 7+ |
From GoCD 19.2.0 onwards you can use via pluggable material configuration with gocd-yaml-config-plugin
materials:
path-filtered-material:
plugin_configuration:
id: git-path
options:
url: https://github.com/TWChennai/gocd-git-path-sample.git
username: username # optional
path: path1, path2/subpath
shallow_clone: false # optional
secure_options: # optional
password: 'encrypted_value'
destination: destDirAvailable from GoCD 19.8.0 onwards.
Available on all compatible versions of GoCD.
You can see a sample here.
VSM view
View changes grouped by material
This plugin relies on the underlying Git CLI to interpret path expressions. For that reason, you can construct almost any expression you could do with a git pathspec including
- including certain paths
- including only certain files deep inside a path structure
- excluding certain paths or files (using
:(exclude)or:!or:^) - using glob syntax (using
:(glob))
Restrictions
- cannot contain commas (used for splitting)
- whitespace before and after splitting commas is ignored
You can normally test your expressions using something like the below to see the last revision that would have triggered a build
EXPR='path1, :(exclude)*README.md'
git log -1 $(echo "${EXPR}" | tr -d ',' )For example, these are all valid expressions
# Monitor for changes in path1 and path2/subpath
path1, path2/subpath
# Monitor for changes in path1, but ignore README changes
path1, :(exclude)*README.md
# Monitor for changes to all config files for the qa environment, in any folder
config/*/qa.yamlv2 is a major overhaul of the plugin, and prefers the use of the git command line to the previously preferred jgit.
This makes the plugin consistent with more recent GoCD server & agent Docker releases, and improves ability to support
standard git features such as shallow and no-checkout clones and wildcard matches in paths.
As a result the plugin ID has changed between the two versions.
| v1 plugin id | v2+ plugin id |
|---|---|
GitPathMaterial |
git-path |
- If you use the gocd-yaml-config-plugin to configure your pipelines as code; you can update the ID per the above example.
- If you configure your pipelines via the UI, you will need to add new materials and remove the old ones one-by-one.
Custom source control material plugins like the gocd-git-path-material-plugin only have support for GoCD secrets management
and secret variable interpolation (syntax like {{SECRET:[file-secrets][test-password]}}) from GoCD 20.8.0 onwards
(changelog).
On earlier versions if you use pipelines-as-code to source control your material definitions, you can however
- use
secure_optionsto source control an encrypted password (see gocd-yaml-config-plugin#pluggable) - use SSH keys within your GoCD server and agent to authenticate with a private repository
- If running GoCD on Kubernetes, there is support for doing so based on Kubernetes
Secrets within the GoCD Helm chart or any other solution that allows mounting of a file into a container.
- If running GoCD on Kubernetes, there is support for doing so based on Kubernetes
Custom source control material plugins like the gocd-git-path-material-plugin have support for
triggering from webhooks from GoCD 20.9.0 onwards (changelog). This contrasts with the convention material polling approach (see #27 and gocd/gocd#8170)
This plugin uses Git commands to filter the history and determine the most recent revision that matches the
monitored paths. This means that changes that are not monitored in your paths may be "stale". The plugin does
not rm un-monitored paths from a clone; meaning your build task could accidentally depend on files in the
repository that are out-of-date.
Be careful with your repository structure to keep your monitored path expressions simple, so you can easily reason about whether a given pipeline should have been triggered for a given commit.
You will not be able to see the Git Path material as a material type when creating a new pipeline. Add a dummy git material, then edit to add a new Git Path material and remove the old one. This problem has been resolved in newer GoCD versions via the pipeline creation wizard.
execute the following command to build the plugin
./gradlew clean buildYou can quickly test the plugin using Docker Compose, via Docker Desktop, Colima or equivalent.
Execute the following gradle task to start the go-server
./gradlew clean startGoCdYou can now access the go-server via port 8153
If you like to reload the go-server with new build run,
./gradlew clean restartGoCdYou can stop the running docker instance with the following gradle task
./gradlew clean stopGoCdReleased via the gocd-plugin-gradle-task-helpers onto GitHub.
- Check the
gocdPlugin.pluginVersionversion inbuild.gradleand bump if necessary- The release helpers add a
-${COMMIT_NUM}suffix to the version, so this is not strictly necessary for minor changes.
- The release helpers add a
- Tag and publish
export GITHUB_TOKEN=blah PRERELEASE=no ./gradlew clean build githubRelease - Edit the release description on
GitHubif necessary.
This plugin contains Git sub-process handling code originally written and released under Apache 2 licenses by
- the GoCD core team
- Srinivas Upadhya via https://github.com/srinivasupadhya/git-cmd/
- Ashwanth Kumar and Chad Wilson via https://github.com/ashwanthkumar/git-cmd
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent