Support for secrets plugin interpolation of git passwords about gocd-git-path-material-plugin HOT 10 CLOSED

Hello @rra08,
You can use the encrypted password as a part of secure_options.
Taking your example:

materials:
  git-myrepo:
        plugin_configuration:
          id: git-path
        options:
          url: https://github.com/myrepo.git
          branch: master
          username: user1
          path: src/**
          #password: '{{SECRET:[github-credentials][password]}}'
          shallow_clone: true
        secure_options:
          password: 'encrypted_value'

from gocd-git-path-material-plugin.

Hi @rra08 - I'm not sure where you got the {{...}} syntax from above, but if that's what you have in your pipeline config, I don't believe it will work - to my knowledge there is no such interpolation logic applied by the gocd-yaml-config-plugin.

If you want to source control material credentials, the normal approach in GoCD 19.4+ is to use encrypted_password, however I am not personally sure if there is some magic which allows this to work with custom materials like this one. Personally I've always used SSH keys for this, which we ensure can be mounted by the GoCD Server and relevant agents and available to the ssh binary.

Unfortunately, I'm not sure if there is a secure way to have a password source controlled via pipelines-as-code with custom plugins like this.

from gocd-git-path-material-plugin.

Hi @chadlwilson, thanks for replying to my issue.

'{{SECRET:[github-credentials][password]}}' is supported by inbuilt git material in GoCD and this works fine (in yaml). Example in step4 in the GoCd Secret documentation.

All sorts of secrets are sourced via kubernetes[EKS] secrets, that is where my GoCD is provisioned.

Reason to explore gocd-git-path-material-plugin is to fix a workflow issue with fan-in when using alongside a mono repo.

I'm still exploring options..

from gocd-git-path-material-plugin.

Thanks for the link - if I get some time at the weekend I might look at what is actually happening with the pluggable SCM material. My guess is that GoCD is not interpolating that secret value and passing the resolved value to the plugin - generally this is probably because in the GoCD SCM plugin API model, the main GoCD Server doesn't actually know which field is which, so the plugin config model would need to have some way to indicate to GoCD that "the config string called password is a secure password/secret".

if you are using Kubernetes/EKS for your server and agent, it's relatively simple to mount an SSH key and use ssh:// Git URLs via server.security.ssh and then put your SSH key into a K8S Secret called gocd-server-ssh (default name). You can do the same on static agents with agent.security.ssh, and even use with elastic agent with something like the below

apiVersion: v1
kind: Pod
metadata:
  name: gocd-agent-{{ POD_POSTFIX }}
  labels:
    app: gocd-agent
spec:
  serviceAccountName: default
  containers:
    - name: gocd-agent-{{ CONTAINER_POSTFIX }}
      image: gocd/gocd-agent-alpine
      imagePullPolicy: Always
      env:
        - name: GIT_SSH_COMMAND
          value: "ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"
      volumeMounts:
        - name: ssh-secrets
          readOnly: true
          mountPath: /home/go/.ssh
  volumes:
    - name: ssh-secrets
      secret:
        secretName: gocd-agent-ssh

This is the approach I/we have been using with Git SSH.

from gocd-git-path-material-plugin.

Appreciate @chadlwilson for taking time to reply.

Thanks for your suggestions too. It might involve some additional work to re-provision and verify all products working fine etc.. I was really looking for a drop-in replacement for the built-in GoCD Git material as this plugin intended.

Please let me know if you have further findings.

from gocd-git-path-material-plugin.

Fair enough, thanks for the feedback. I should note these things in the caveats/known issues as they are discovered. Once it's confirmed I will update it.

It's not too difficult to get SSH keys mounted if you are deploying into EKS via the Helm chart and OK with creating Secret resources in your namespace to contain these. I find SSH clones to be faster in the general sense too, but YMMV.

Unfortunately GoCD team have done a lot of things on server side in past few years that are not supported by the custom SCM material plugin API. That API seems to have become stagnant (probably because basically everyone uses git so there was a lot less need for such plugins to handle different types of source control).

from gocd-git-path-material-plugin.

I've had a look at the GoCD Server code. It does indeed seem that it doesn't support this. I've raised an issue at gocd/gocd#8234 so let's see if anything can be done. I believe if it were possible to implement it on GoCD Server side, there might not be anything required to be done on the plugin, since there is already an existing way for a plugin to indicate that a value is secure (i.e should be displayed as **** and redacted from logs) which may also be able to indicate that a variable is eligible for secrets interpolation.

from gocd-git-path-material-plugin.

Thanks @chadlwilson for taking a deep look on this issue.

On the other hand, does your plugin supports encrypted_password. It doesn't seem to work for me.
I encrypted password to AES encrypted (encrypted using gocd api).

from gocd-git-path-material-plugin.

Unfortunately encrypted_password suffers from the same problem of requiring specific GoCD Server support for plugins in the API, which it doesn't currently have.

from gocd-git-path-material-plugin.

I'm closing this now since there appears a workaround with secure_options in the short term (I'm not sure why are there so many mechanisms to do something similar..), and the interpolation will be supported in GoCD 20.8.0 according to gocd/gocd#8234

from gocd-git-path-material-plugin.