debops / ansible-pki Goto Github PK

In inventories with multiple domains, the defaults result in debops-pki generating certificates that are invalid (do not fulfil the Name Constraints) whenever a certificate for a non-preferred domain is issued.

I hereby recommend adding name_constraints to the Useful global parameters section, where debops-pki users are pointed to the full variable documentation and discussion.

With no configuration changes, tiny-acme now fails to request the certificate. I’ve run debops --tags role::nginx:servers and debops --tags role::pki to make sure my configurations were okay, they run with no errors. I’ve also deleted the error.log files and rerun the commands, but no luck.

Here’s a copy of one of the error.log files:

Parsing account key...
Parsing CSR...
Registering account...
Already registered!
Verifying lucascherkewski.com...
lucascherkewski.com verified!
Verifying www.lucascherkewski.com...
Traceback (most recent call last):
  File "/usr/local/lib/pki/acme-tiny", line 198, in <module>
    main(sys.argv[1:])
  File "/usr/local/lib/pki/acme-tiny", line 194, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca)
  File "/usr/local/lib/pki/acme-tiny", line 123, in get_crt
    wellknown_path, wellknown_url))
ValueError: Wrote file to /srv/www/sites/acme/public/.well-known/acme-challenge/Pf-KGKW79jBEdqdUS3UC15JwoL9qGRX9fIb1jFRL6no, but couldn't download http://www.lucascherkewski.com/.well-known/acme-challenge/Pf-KGKW79jBEdqdUS3UC15JwoL9qGRX9fIb1jFRL6no

Happy to share any details that would help.

This is a bit related to #41. I'm really not that familiar with pki infrastructure and how it's used through out DebOps. I might be asking for something that can't be done.

I'm wondering if there's a way we can make debops.pki do less through configuration. If I look at the WordPress project, we don't seem to need internal certificate authorities, root certificates and all the stuff generated by default. We only need to support two scenarios:

• Upload certificates that someone bought
• Let's Encrypt certificates (not done yet)

99% of the users of the WordPress project will not need a lot of the features of this role. Yet it puts a lot of burden on them that I'd like to remove. I don't want to have to ask them to update bash or openssl if they won't use the internal certificate authorities at all.

Ideally maybe an option like pki_enabled, but just for the internal certificate authority generation stuff. Maybe something like pki_generate_certificates or pki_generate.

I'm also not sure how much of the role or other roles depend on those certificates being present. That said, I'd rather work through those issues in those roles.

Does this make sense? Or am I crazy to ask this lol

Looking at the domain certificates generated by this role

        Subject: CN = host1.internal.example.jp
...
            X509v3 Subject Alternative Name: 
                DNS:internal.example.jp, DNS:*.host1.internal.example.jp, 
                DNS:host1.internal.example.jp, DNS:*.internal.example.jp

I have already found that setting pki_default_realms to the below removes the *.host1.internal.example.jp and *.internal.example.jp, from SAN but I currently can't figure if there is a way to remove just the internal.example.jp

pki_default_realms:
  - name: 'domain'
    acme: False
    default_subdomains:
      - '{{ ansible_hostname }}'

So what I am looking for is this:

        Subject: CN = host1.internal.example.jp
...
            X509v3 Subject Alternative Name: 
                DNS:host1.internal.example.jp

Unless of course this is there for a reason, and if so just let me know.

Is this role appropriate to handle lets encrypt certificates?
I mean, to add support to it for lets encrypt.

I love debops and I even saw ACME work once, but I have to say it is super hard to get it working. I've never had this kind of difficulty with other letsencrypt tools or ansible roles.
I've tried to piece it together from the official docs, the debops-wordpress project and a few issues documented here, but I'm really not having much joy.

I'm currently stuck on this error, in /etc/pki/realms/mydomain.com/acme/error.log

Parsing account key...
Parsing CSR...
Found domains: www.mydomain.com, mydomain.com
Getting directory...
Directory found!
Registering account...
Traceback (most recent call last):
  File "/usr/local/lib/pki/acme-tiny", line 197, in <module>
    main(sys.argv[1:])
  File "/usr/local/lib/pki/acme-tiny", line 193, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact)
  File "/usr/local/lib/pki/acme-tiny", line 111, in get_crt
    account, code, acct_headers = _send_signed_request(directory['newAccount'], reg_payload, "Error registering")
KeyError: 'newAccount'

So apparently it is failing when it attempts to send the signed CSR request to letsencrypt. I can't tell much more than that by looking at the code.

Is there any way to get more verbose output to actually see what the problem is? I assume there is something wrong with my CSR request, but really not sure where to look next.

This is in my host inventory (though I've tried a bunch of other permutations, with and without www, with and without subdomain settings):

pki_host_realms:
  - name: www.mydomain.com
    enabled: True
    acme: True

In group_vars/all I have

pki_ca_domain: mycompany.com
pki_ca_organization: My Company
pki_acme: True
pki_acme_install: True
pki_acme_default_subdomains: []

(Note this is a different domain than the website domain I'm trying to get working.)

Then I have an nginx site (which is working great otherwise)

nginx__servers:
  - name: ['www.mydomain.com', 'mydomain.com']
    redirect_from: True
    enabled: True
    ssl: True
    redirect_to_ssl: False
    ...etc

The DNS for the root and www domain resolve correctly to the IP address of the host. The fqdn of the host (gamma.mycompany.com) is not the same as the website (and is on a different domain, if that makes a difference).

The internal certificate is working correctly, and is used when viewing the site in https. Obviously this generates warnings due to the untrusted CA.

Basically my approach is to try something, then delete /etc/pki/realms/mydomain.com on the managed host (and sometimes need to delete /ansible/secret/pki/realms/by-host/gamma.mycompany.com/mydomain.com on the controller host) then try something else and run again - is there a more graceful way (or other things I need to delete before a new run)?

The only time I've had ACME work was when I finally got the configuration right, and then started from scratch on a new server. So I suspect maybe I'm just not cleaning up enough debris from my trial and error.

At this point I'm thinking it may be best to disable debops.pki and try a different tool for managing letsencrypt certificates. But I would love to get this working. Apparently it is possible.

What am I doing wrong / or where should I be looking to get more insight?

As those forms of certificate status checking are currently not supported by the role, it would be nice to allow to disable them. Normal browsers don’t yet require OCSP but I have my browsers configured to do that. I think for internal (staging) CAs, OCSP could be disabled.

TASK [debops.pki : Initialize PKI authorities] *********************************
failed: [vbox -> localhost] => (item={u'subdomain': u'root-ca', u'name': u'root', u'subject': [u'o=Vagrant-ubuntu-trusty-64 Certificate Authority']}) => {"changed": true, "cmd": ["./lib/pki-authority", "init", "--name", "root", "--default-sign-base", "365", "--root-sign-multiplier", "12", "--ca-sign-multiplier", "10", "--cert-sign-multiplier", "3"], "delta": "0:00:00.030746", "end": "2016-04-21 13:50:51.097910", "failed": true, "item": {"name": "root", "subdomain": "root-ca", "subject": ["o=Vagrant-ubuntu-trusty-64 Certificate Authority"]}, "rc": 2, "start": "2016-04-21 13:50:51.067164", "stderr": "./lib/pki-authority: line 77: declare: -g: invalid option\ndeclare: usage: declare [-afFirtx] [-p] [name[=value] ...]", "stdout": "", "stdout_lines": [], "warnings": []}
failed: [vbox -> localhost] => (item={u'subdomain': u'domain-ca', u'name': u'domain', u'issuer_name': u'root', u'subject': [u'o=Vagrant-ubuntu-trusty-64', u'ou=Domain CA']}) => {"changed": true, "cmd": ["./lib/pki-authority", "init", "--name", "domain", "--default-sign-base", "365", "--root-sign-multiplier", "12", "--ca-sign-multiplier", "10", "--cert-sign-multiplier", "3"], "delta": "0:00:00.029932", "end": "2016-04-21 13:50:51.378080", "failed": true, "item": {"issuer_name": "root", "name": "domain", "subdomain": "domain-ca", "subject": ["o=Vagrant-ubuntu-trusty-64", "ou=Domain CA"]}, "rc": 2, "start": "2016-04-21 13:50:51.348148", "stderr": "./lib/pki-authority: line 77: declare: -g: invalid option\ndeclare: usage: declare [-afFirtx] [-p] [name[=value] ...]", "stdout": "", "stdout_lines": [], "warnings": []}

TASK [debops.pki : Sign certificate requests for current hosts] ************************************************************************************
fatal: [node1.domain]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'pki_fact_fqdn'\n\nThe error appears to have been in virt-ansible/roles/external/debops.pki/tasks/main.yml': line 350, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n# Sign certificate requests [[[\n- name: Sign certificate requests for current hosts\n ^ here\n\nexception type: <class 'ansible.errors.AnsibleUndefinedVariable'>\nexception: 'dict object' has no attribute 'pki_fact_fqdn'"}

line 349 of tasks/main.yml:

command: ./lib/pki-authority sign-by-host
{% for host in play_hosts %}{{ hostvars[host].pki_fact_fqdn }} {% endfor %}

It would be useful to be able to copy a cert/key from an external CA from a variable instead of having to save it to the secret directory first.

That way I could do something like this:
In vault file:

my_crt: |
  -----BEGIN CERTIFICATE-----
  ALKSICERTIFICATESKJOLKJLJ
  ALKSIDKLEVALUEDLKJOLKJLJ
  -----END CERTIFICATE-----

In nginx definition:

nginx_server_default:
  default: True
  enabled: True
  name: ['my_domain']
  pki_crt: '{{ my_crt }}'
  pki_key: '{{ my_key }}'

This doesn't seem possible currently.

I think since yesterday after a debops-update the pki role failed with every second run:
TASK [debops.pki : Sign certificate requests for current hosts] **************************************************************************************************************************************
fatal: [teamcity -> localhost]: FAILED! => {"changed": false, "cmd": ["./lib/pki-authority", "sign-by-host", "teamcity.domain.com"], "delta": "0:00:00.305034", "end": "2018-03-26 09:41:34.501734", "msg": "non-zero return code", "rc": 2, "start": "2018-03-26 09:41:34.196700", "stderr": "pki-authority: Error: failed to run verify -CAfile issuer/subject/cert.pem -untrusted subject/cert.pem /home/ansible/debops/ansible/secret/pki/realms/by-host/teamcity.domain.com/domain/internal/cert.pem (Exitcode: 2)\n\nDetails:\n/home/ansible/debops/ansible/secret/pki/realms/by-host/teamcity.domain.com/domain/internal/cert.pem: CN = teamcity.domain.com\nerror 47 at 0 depth lookup:permitted subtree violation", "stderr_lines": ["pki-authority: Error: failed to run verify -CAfile issuer/subject/cert.pem -untrusted subject/cert.pem /home/ansible/debops/ansible/secret/pki/realms/by-host/teamcity.domain.com/domain/internal/cert.pem (Exitcode: 2)", "", "Details:", "/home/ansible/debops/ansible/secret/pki/realms/by-host/teamcity.domain.com/domain/internal/cert.pem: CN = teamcity.domain.com", "error 47 at 0 depth lookup:permitted subtree violation"], "stdout": "", "stdout_lines": []}

Based on IRC chat with drybjed I add this request here, and not to ansible-nginx.

HPKP is a trust on first use security mechanism which protects HTTPS websites from impersonation using fraudulent certificates issued by compromised certificate authorities.

Read more about in links below.

Links:
https://tools.ietf.org/html/rfc7469
https://developer.mozilla.org/en-US/docs/Web/Security/HTTP_strict_transport_security
https://raymii.org/s/articles/HTTP_Public_Key_Pinning_Extension_HPKP.html
https://github.com/debops/ansible-nginx
https://en.wikipedia.org/wiki/HTTP_Public_Key_Pinning

Is there a step by actual step guide available somewhere? It doesn't even need to be in the official docs (although it should be).

Guess I'm looking for actual examples after spending too many hours trying to get this working.

My config is pretty simple:

pki_default_realms: []
pki_inventory_groups: [ 'test' ]
pki_realms:

• name: test

realms
│   ├── by-group
│   │   └── test
│   │       ├── test
│   │       │   ├── external
│   │       │   │   ├── cert.pem
│   │       │   │   ├── intermediate.pem
│   │       │   │   └── root.pem
│   │       │   └── private
│   │       │       └── key.pem

And yet it treats it as internal and creates requests etc. Any pointers?

Getting this error from pki-realm:

/usr/local/lib/pki/pki-realm: line 573: selfsigned/gnutls.conf: No such file or directory

I think it's because it's running on an already created PKI domain before the selfsigned changes. The directory doesn't exist so it can't find the file. I tried looking at the script to see how it was doing, but couldn't find where you call create_gnutls_config.

This is the actual error:

$ cat /etc/pki/realms/REALM/acme/error.log             
Parsing account key...                                                         
Parsing CSR...                                                                 
Found domains: domain.ex                                            
Getting directory...                                                           
Directory found!                                                               
Registering account...                                                         
Traceback (most recent call last):                                             
  File "/usr/local/lib/pki/acme-tiny", line 197, in                    
    main(sys.argv[1:])                                                         
  File "/usr/local/lib/pki/acme-tiny", line 193, in main                       
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact)
  File "/usr/local/lib/pki/acme-tiny", line 111, in get_crt                    
    account, code, acct_headers = _send_signed_request(directory['newAccount'], reg_payload, "Error registering")
KeyError: 'newAccount

@egon0 found that this is an issue with the actual tiny-acme code. Telling the debops.pki role to use the tiny-acme version "4ed13950c0" fixes this issue.

pki_acme_tiny_version: '4ed13950c0'

It seems the acme_challenge_dir is hard coded. The /usr/local/lib/pki/pki-realm script allows for overwriting args["acme_challenge_dir"], but that's not done when the "run" sub command is called:

# Execute PKI realm commands [[[
- name: Execute PKI realm commands
  environment:
    PKI_SESSION_TOKEN: '{{ pki_fact_session_token }}'
  command: '"{{ pki_fact_lib_path }}/pki-realm" run -n "{{ item.name }}"'

And "run" is the only place where "request_acme_tiny_certificate" is called.

It would also be nice to have some logging output of the pki-realm script to determine why it does not create ACME certificates. It tests for, amongst other things, [ "${config['pki_acme']}" = "true" ] && [ -x "${config['acme_client_script']}" ] && [ -d "${config['acme_challenge_dir']}" ] and it would be helpful to see why it skipped creation of a certificate. If only with --verbose flag or so.

The problem with these files is that there is no warning.
In my case the error was that
Execute source Makefiles
failed but no error was generated at all.
The Makefile did not fail. There was just nothing to do, because of that the Makefile returned a code != 0.
What I expect is that make.sh checks for these lock files and exits with a warning that there are these lockfiles.
I did not look into the lockfiles, but if possible it would be cool to write user and hostname into this file. We are using them via a shared network mount.

The vulnerability goes in the same direction as other vulns on https://www.ansible.com/security which are compromised remote hosts exploiting the Ansible controller.

Steps to reproduce:

pki_ca_domain: 'example.org'
pki_ca_organization: 'example'
pki_default_authority: 'example-issuing-ca'

pki_authorities:

  - name: 'example-root-ca'
    domain: '{{ network__public_dns_fqdn }}'
    subdomain: 'root-ca'
    subject: [ 'o=example Internal Root CA' ]
    key_size: '{{ pki_ca_root_key_size }}'

  - name: 'example-issuing-ca'
    domain: '{{ network__public_dns_fqdn }}'
    subdomain: 'issuing-ca'
    subject: [ 'o=example', 'ou=example Internal Issuing CA' ]
    issuer_name: 'example-root-ca'
    key_size: '{{ pki_ca_domain_key_size }}'

# What the Ansible controller expects:
pki_host_realms:

  - name: 'good.{{ pki_ca_domain }}'

# That is what the remote host will give us (the Ansible controller):
pki_evil_host_realms:

  - name: 'good.{{ pki_ca_domain }}'
    domains:
      - 'evil.{{ pki_ca_domain }}'

Then simply replace all:

  with_flattened:
    - '{{ pki_realms }}'
    - '{{ pki_group_realms }}'
    - '{{ pki_host_realms }}'
    - '{{ pki_default_realms }}'
    - '{{ pki_dependent_realms }}'

with:

  with_flattened:
    - '{{ pki_evil_host_realms }}'

in tasks where pki-realm is called on the remote host (which can be spoofed by the remote host to their liking).

Then run the role. The remote host gets this cert issued:

Certificate:
    Data:
        Version: 3 (0x2)
    Signature Algorithm: sha256WithRSAEncryption
        Subject: CN=good.example.org
        X509v3 extensions:
            X509v3 Subject Alternative Name:
                DNS:evil.example.org

The above is just my way of demonstrating/reproducing it.

Basically, the pki-realms script is always executed by the remote host, so the input and output of the script could be controlled by an attacker if the remote host got compromised. That is basically what I simulated here. I defined a second variable in the inventory on the controller pki_evil_host_realms representing the input by which pki-realms gets called despite the fact that the controller sends pki_host_realms. I could have done it manually on the remote host but, oh well, automation.

pki-authority must validate pki_*realms and check if the admin actually authorized the CSR generated by that particular (untrusted) remote host.

Unfortunately, DebOps does not have a patch for this and it is currently not clear when one will be available as it might end in a rewrite of pki-authority. For transparency reasons, we therefore decided to make this public to not leave users in the dark. Technically, DebOps is not yet recommended for production.

@drybjed Already wrote how this can be solved. It is probably best when he pastes that in himself.

My idea was:
pki-authority could just get all pki_*realms of the current Ansible run, encoded as JSON, build Subject and SANs for each realm. Then subject and SANs (and possibly other interesting parts of the CSR) get extracted from the CSR the remote host provided, sorted and compared. Python would make that a bit easier.

Timeline:

• 2017-04-17: Internal discussion with @drybjed
• 2017-04-23: Decision that the vulnerability can/should be made public.

Then installing a fresh system, I get this error: (line-breaks and ellipsis by me)

TASK: [debops.pki/env | Prepare debops.pki environment] *********************** 
fatal: [server] => Failed to template
{{ lookup("template", "lookup/pki_env_secret_directories.j2") | from_yaml }}:
unable to read [....]/debops.pki/env/tasks/lookup/pki_env_secret_directories.j2

The requested file exists in debops.pki/env/templates/lookup/ (templates, not tasks)

Using current HEAD (279905a).

When changing the pki_acme_ca from testing to live,

1. certificates are not re-generated.
   Workaround: rm default.key config/realm.conf acme/{error.log,*.pem}
2. the nginx server is not reloaded (even after applying the workaround for 1.

debops: 0.5.0
debops.pki: v0.2.14-50-gb98db7f

Running debops for a new host, using all defaults for pki

TASK [debops.pki : Sign certificate requests for current hosts] ****************
fatal: [my-host -> localhost]: FAILED! => 
{
  "changed": false,
  "cmd": [
    "./lib/pki-authority",
    "sign-by-host",
    "my-host.fqdn"
  ],
  "delta": "0:00:00.079191",
  "end": "2017-11-02 07:50:07.169723",
  "failed": true,
  "msg": "non-zero return code",
  "rc": 2,
  "start": "2017-11-02 07:50:07.090532",
  "stderr": "pki-authority: Error: failed to run verify -CAfile issuer/subject/cert.pem -untrusted subject/cert.pem /home/myuser/fivepointoh/ansible/secret/pki/realms/by-host/my-host.fqdn/domain/internal/cert.pem (Exitcode: 2)\n\nDetails:\n/home/myuser/fivepointoh/ansible/secret/pki/realms/by-host/my-host.fqdn/domain/internal/cert.pem: CN = my-host.fqdn\nerror 47 at 0 depth lookup:permitted subtree violation",
  "stderr_lines": [
    "pki-authority: Error: failed to run verify -CAfile issuer/subject/cert.pem -untrusted subject/cert.pem /home/myuser/fivepointoh/ansible/secret/pki/realms/by-host/my-host.fqdn/domain/internal/cert.pem (Exitcode: 2)",
    "",
    "Details:",
    "/home/myuser/fivepointoh/ansible/secret/pki/realms/by-host/my-host.fqdn/domain/internal/cert.pem: CN = my-host.fqdn",
    "error 47 at 0 depth lookup:permitted subtree violation"
  ],
  "stdout": "",
  "stdout_lines": []
}

I don't get the error the first time but it happens on every subsequent run. If I remove the internal/gnutls.conf and internal/request.pem files as suggested at the bottom of this page in the docs: https://docs.debops.org/en/latest/ansible/roles/ansible-pki/docs/internal-ca.html then it runs fine.

Is there minimal configuration recommended or required to get this task to succeed so I don't have to delete these files before each run? Still trying to wrap my head around other roles before I deal with pki.

pki-realm may fail to generate a request.pem when the gnutls.conf file is invalid. In my case it complains about an unknown uri option.

$ certtool --generate-request --template gnutls.conf --load-privkey ../private/key.pem --outfile request.pem.tmp
error parsing command line: gnutls.conf: unknown option 'uri' at position 480 in config file
$ echo $?
11

The failure is not visible in the ansible output and one wonders why the realm is not correctly setup.

pki_realms:
  - name: 'myrealm'
    authority: 'service'
    internal: True
    subject_alt_names:
      - 'ip:{{ ansible_default_ipv4.address }}'
      - 'uri:https://{{ ansible_domain }}/'
      - 'dns:*.{{ ansible_domain }}'
      - 'dns:{{ ansible_domain }}'

Removing the uri from the above realm configuration makes things work.

This happened with an older Debian wheezy system.

$ certtool --version
certtool (GnuTLS) 2.12.20
Packaged by Debian (2.12.20-8+deb7u5)

$ apt-cache policy gnutls-bin 
gnutls-bin:
  Installed: 3.0.22-3+really2.12.20-8+deb7u5
  Candidate: 3.0.22-3+really2.12.20-8+deb7u5
  Version table:
 *** 3.0.22-3+really2.12.20-8+deb7u5 0
        500 http://cdn.debian.net/debian/ wheezy/main amd64 Packages
        500 http://security.debian.org/ wheezy/updates/main amd64 Packages
        100 /var/lib/dpkg/status

So the error from the certtool output should probably shown to the user and the task aborted.

When I declare a new realm in a playbook, the task Sign certificate requests for current hosts fails with the following error:

fatal: [noah.host -> localhost]: FAILED! =>
{
  "changed": false,
  "cmd": [
    "./lib/pki-authority",
    "sign-by-host",
    "noah.host"
  ],
  "delta": "0:00:00.760960",
  "end": "2016-08-09 12:58:03.344391",
  "failed": true,
  "rc": 1,
  "start": "2016-08-09 12:58:02.583431",
  "stderr": "pki-authority: Error: failed to run ca -batch -notext -in /ansible/ansible/secret/pki/requests/domain/noah.host/k8s/request.pem -out /ansible/ansible/secret/pki/realms/by-host/noah.host/k8s/internal/cert.pem.tmp -config config/openssl-sign.conf (Exitcode: 1)\n\nDetails:\nUsing configuration from config/openssl-sign.conf\n/ansible/ansible/secret/pki/realms/by-host/noah.host/k8s/internal/cert.pem.tmp: No such file or directory\n140100935210648:error:02001002:system library:fopen:No such file or directory:bss_file.c:398:fopen('/ansible/ansible/secret/pki/realms/by-host/noah.host/k8s/internal/cert.pem.tmp','w')\n140100935210648:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:400:",
  "stdout": "",
  "stdout_lines": [],
  "warnings": []
}

It looks like some directory creations are missing, since the playbooks runs perfectly when I create the following empty directory structure locally:

ansible/secret/pki/realms/by-host/noah.host/
`-- k8s
    |-- external
    `-- internal

Extract from the playbook:

  vars:

    kubernetes_pki_dependent_realms:
      - name: 'k8s'
        subject_alt_names:
          - 'ip:{{ ansible_default_ipv4.address }}'
          - 'ip:10.200.0.1'
          - 'dns:kubernetes.default.svc.gini.cluster'
          - 'dns:kubernetes.default.svc'
          - 'dns:kubernetes.default'
          - 'dns:kubernetes'

  roles:

    - role: debops.pki
      tags: [ 'role::pki' ]
      pki_dependent_realms:
        - '{{ kubernetes_pki_dependent_realms }}'

Hi,

Running debops on a new host host1 results in failure of debops.pki.

TASK [debops.pki : DebOps pre_tasks hook] **************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ImportError: No module named debops
fatal: [aditi1]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""}

PLAY RECAP *********************************************************************
host1                     : ok=45   changed=0    unreachable=0    failed=1

Any idea what causes error of out of box run?

$ ansible --version
ansible 2.2.2.0

$ echo $PYTHONPATH
/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages:/usr/local/Cellar/ansible/2.2.2.0/libexec/vendor/lib/python2.7/site-packages:

Ansible has been installed using homebrew on an osx box. The PYTHONPATH had to be added. Due No module named ansible error. This error sounds similar, however, I don't see any python modules named debops in /usr/local/Cellar or other locations.

$ debops -vvv
TASK [debops.pki : DebOps pre_tasks hook] **************************************
task path: /Users/thomas/Library/Application Support/debops/debops-playbooks/roles/debops.pki/tasks/main.yml:4
An exception occurred during task execution. The full traceback is:
Traceback (most recent call last):
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 126, in run
    res = self._execute()
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 443, in _execute
    include_file = templar.template(include_file)
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/template/__init__.py", line 383, in template
    disable_lookups=disable_lookups,
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/template/__init__.py", line 584, in do_template
    res = j2_concat(rf)
  File "<template>", line 8, in root
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/jinja2/runtime.py", line 196, in call
    return __obj(*args, **kwargs)
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/template/__init__.py", line 493, in _lookup
    instance = self._lookup_loader.get(name.lower(), loader=self._loader, templar=self)
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/plugins/__init__.py", line 339, in get
    self._module_cache[path] = self._load_module_source('.'.join([self.package, name]), path)
  File "/usr/local/Cellar/ansible/2.2.2.0/libexec/lib/python2.7/site-packages/ansible/plugins/__init__.py", line 324, in _load_module_source
    module = imp.load_source(name, path, module_file)
  File "/Users/thomas/Library/Application Support/debops/debops-playbooks/playbooks/lookup_plugins/task_src.py", line 33, in <module>
    from debops import *
ImportError: No module named debops

Currently, the realm key used for all certificates of this realm (internal, external, ACME) is created on realm creation. In the case of ACME and LE, the certificate is renewed at least all 3 months. Other certs are rotated less frequently. It should be possible to specify the wanted private key renewal period which can happen at cert renewal.

Ref: https://letsencrypt.org/2015/11/09/why-90-days.html

Just got an issue based on #61. I already had a workaround in place to disable certificate authority generation and thus bypass the bash requirement for macOS. I don't want to force anyone to update bash when we don't need it for simpler setups like mine.

Are we ok with disabling this if we don't need to run the local script?

The 'Create PKI realms' task runs only on creation of realm., this is due to the "creates: '/etc/pki/realms/{{ item.name }}/default.key'" line. Is it necessary? I mean running the "pki-realm new-realm -n" command doesn't really change anything, only updates configs which is handy if it comes to reconfiguration, otherwise you have to delete the default.key link in each realm.
Also I think it would be helpful to include in the docs that the 'Create PKI realms' task needs "internal: True" on the realm in order to run. Otherwise variables crucial to ACME certs wont be included. Am I missing something?

I'm trying to generate a Let's Encrypt certificate for a domain but I'm not having any luck.

I consulted the numerous other issues related to the pki role but I couldn't find a clear answer. I should note that the certificate is for a domain different to that of the host's domain.

I have the following pki config in ansible/inventory/host_vars/flowww/pki.yml:

---
pki_realms:
    - name: 'staging.mycity.co.za'
      acme: True
      acme_domains: []
      acme_default_subdomains: []
      acme_ca: 'le-staging-v2'

I get the following error:

fatal: [flowww -> localhost]: FAILED! => changed=false
  cmd:
  - ./lib/pki-authority
  - sign-by-host
  - flowww.upfronthosting.co.za
  delta: '0:00:00.175256'
  end: '2020-05-26 14:33:43.809087'
  msg: non-zero return code
  rc: 2
  start: '2020-05-26 14:33:43.633831'
  stderr: |-
    pki-authority: Error: failed to run verify -CAfile issuer/subject/cert.pem -untrusted subject/cert.pem /home/roche/debops/servers/ansible/secret/pki/realms/by-host/flowww.upfronthosting.co.za/staging.mycity.co.za/internal/cert.pem (Exitcode: 2)

    Details:
    CN = staging.mycity.co.za
    error 47 at 0 depth lookup: permitted subtree violation
    error /home/roche/debops/servers/ansible/secret/pki/realms/by-host/flowww.upfronthosting.co.za/staging.mycity.co.za/internal/cert.pem: verification failed
  stderr_lines: <omitted>
  stdout: ''

This is the output of openssl x509 -in /home ansible/secret/pki/realms/by-host/flowww.upfronthosting.co.za/staging.mycity.co.za/internal/cert.pem -text -noout:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            19:e5:ce:27:e5:0a:22:61:bb:38:07:0d:fa:78:21:64
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: O = Upfronthosting, OU = Domain CA
        Validity
            Not Before: May 26 12:26:43 2020 GMT
            Not After : May 26 12:26:43 2023 GMT
        Subject: CN = staging.mycity.co.za
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:c6:60:82:86:bb:10:7f:90:f7:1e:94:9a:6e:c8:
                    4f:90:76:d1:35:84:ae:e4:ba:d3:b4:51:b1:c0:51:
                    cf:50:ca:16:95:51:69:79:1f:dd:aa:6e:8d:96:0f:
                    ac:47:9b:25:cd:9d:d7:a8:85:e3:cb:ae:87:16:19:
                    54:a8:ff:73:5f:d9:a4:a5:6d:09:4f:3e:4e:2c:b4:
                    d3:15:51:c3:61:99:a9:b1:fc:43:17:d9:bc:de:b7:
                    b7:51:ab:56:a6:6a:e6:1c:bc:b7:a0:89:6f:e8:55:
                    bc:64:c5:af:21:8f:53:49:c3:90:a1:63:37:51:ed:
                    e1:1b:ab:b4:ae:af:9b:a4:50:af:c0:cb:9d:8c:e3:
                    4e:43:c5:ea:02:8f:ea:ae:61:70:fe:c7:6a:62:ac:
                    8f:f7:c6:20:c1:2c:a9:af:00:e2:8f:2e:c2:aa:2b:
                    82:70:82:d7:f9:8e:71:68:a5:eb:65:1e:74:e5:3a:
                    60:7c:39:b7:6e:af:11:fc:1e:fa:86:9a:d1:fd:b0:
                    35:78:24:a7:53:07:38:fc:d2:d9:62:35:5f:f5:8c:
                    ad:6a:8f:c3:61:02:79:4b:70:53:66:50:2d:0a:57:
                    85:6a:b8:72:3d:6a:a7:c6:39:f4:58:94:6a:95:00:
                    90:d6:72:ba:19:ba:4c:8d:f5:64:c0:aa:a1:c8:6c:
                    36:e3
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            Authority Information Access:
                CA Issuers - URI:http://domain-ca.upfronthosting.co.za/crt/
                OCSP - URI:http://domain-ca.upfronthosting.co.za/ocsp/

            X509v3 Authority Key Identifier:
                keyid:36:22:C6:14:D7:FD:BF:7A:D9:07:81:0B:BF:1E:2F:22:92:1C:E7:CB
                DirName:/O=Upfronthosting Certificate Authority
                serial:76:33:37:78:69:9E:1D:18:A8:50:89:C7:5B:E0:2D:2D

            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 CRL Distribution Points:

                Full Name:
                  URI:http://domain-ca.upfronthosting.co.za/crl/

            X509v3 Extended Key Usage:
                TLS Web Client Authentication, TLS Web Server Authentication
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Subject Key Identifier:
                C1:DF:47:9E:1F:C3:53:88:3D:CE:C5:14:9E:AC:36:9B:F4:48:95:9A
            X509v3 Subject Alternative Name:
                DNS:staging.mycity.co.za, DNS:*.staging.mycity.co.za
    Signature Algorithm: sha256WithRSAEncryption
         06:f9:d8:88:2d:54:c5:49:f1:93:61:ed:a3:64:07:a0:94:02:
         f4:72:a6:f0:90:e8:b9:74:9a:8d:ab:87:34:e0:9e:64:f7:57:
         fe:b0:71:48:62:37:f9:5c:e4:f3:bd:86:22:96:f8:01:33:21:
         fc:53:d0:b1:36:d0:39:2b:8d:48:20:22:59:64:bb:37:89:40:
         52:4a:df:91:2f:4b:fa:d3:f9:88:c3:a9:67:11:cd:d9:da:84:
         d9:ed:e0:7b:90:c2:70:6c:b2:e4:18:e2:bd:59:9b:23:0d:99:
         4a:c9:67:6d:f6:27:88:69:ee:53:42:f8:34:dc:fa:01:19:6c:
         38:e3:d6:25:6a:93:c3:1b:bf:a8:d5:13:a8:78:0c:6a:2a:76:
         c1:be:f7:34:be:7c:8c:49:53:38:36:9d:54:e7:05:ba:df:9f:
         a3:1c:5c:cc:62:8a:c3:ff:d8:94:a0:11:70:1f:93:e3:63:4d:
         09:56:d8:b0:7f:1c:21:90:a6:e1:3c:e1:a4:0e:37:86:66:1f:
         03:e0:92:2d:21:0c:07:9c:77:11:a3:9f:a1:36:bd:3f:be:f0:
         ac:dd:fb:91:ac:64:e0:c1:e7:fb:70:78:d6:c6:39:fb:c6:c7:
         d6:c7:9e:8c:91:f3:06:6f:5b:c2:54:ef:a1:9b:fc:34:95:27:
         db:fe:de:ea:cd:ea:b3:17:b5:48:0f:01:97:c1:92:bc:44:0e:
         65:6c:79:46:56:18:0f:8f:57:98:16:9d:ea:49:6f:5e:f2:13:
         aa:08:9f:f7:33:1c:23:63:bf:be:23:6a:39:36:3c:c0:fe:f4:
         72:d8:97:64:a5:57:69:90:46:97:8e:18:7f:d3:73:82:37:2b:
         00:80:dd:bb:39:8f:dc:97:4a:26:4f:dd:51:10:86:bb:de:e9:
         45:c3:eb:e5:9e:af:45:5d:03:d5:9c:df:17:be:03:c7:d9:26:
         77:d8:51:0b:9f:21:28:08:a3:59:45:60:d6:a8:b5:f3:30:83:
         1c:55:24:7c:5b:c1:5d:37:ac:e5:1b:e9:c9:2c:25:a6:2c:bb:
         bc:66:36:f5:89:0c:f1:83:4a:33:65:69:a2:05:13:19:44:f4:
         d4:f7:dc:63:c5:e1:0a:78:cc:bc:fa:86:a9:36:33:dd:5f:bd:
         69:77:15:b6:4a:41:2d:7f:85:a3:30:4f:bc:01:10:50:30:3c:
         6c:3f:64:d5:ea:ea:15:ca:3f:e0:01:66:85:fb:3c:7a:29:cc:
         f0:b5:1b:cc:3e:47:e4:f9:ac:0e:bb:5a:ac:2e:86:b7:7f:a6:
         17:47:35:d1:aa:99:57:20:b6:99:d8:af:f1:66:20:25:84:87:
         c5:71:5b:fc:52:d8:bc:b2

I would appreciate any help.

As recommended/explained on BetterCrypto.org in "Random Number Generators" -> "Recommendations" VMs and embedded systems might have a low entropy pool. It would be nice to allow to generate realm private keys on the Ansible controller, transfer them to remote systems and then destroy them on the Ansible controller. This should not necessarily be the default but could be as the Ansible controller is part of the TCB of remote systems.

Changeing parameters for ACME has no effect.

• Changing acme-subject, acme_domains, acme_subdomains or acme_alt_names has no effect, since these values is passed to the server only in Create new PKI realms - which is only run if /etc/pki/realms/{{ item.name }}/default.key not yet exists.

• Changing acme_subdomains has no effect at all, since this is never passed over to the script pki-realm.

• Changing acme_ca, acme_ca_api, acme_default_subdomains or acme_challenge_dir has no effect, since these values is passed to the server only in Initialize PKI realms – which is only run if /etc/pki/realms/{{ item.name }}/config/realm.conf not yet exists.

• Even after rm default.key config/realm.conf acme/error.log the file acme/request.pem is not re-created with new values.

I assume this effects other pki configuration parameters, too.

Workaround: rm default.key config/realm.conf acme/error.log acme/request.pem

Example playbook:

---
- hosts: myacmehost
  become: True
  vars:
      secret_directories:
      - '{{ pki_env_secret_directories }}'
  roles:
    - { role: debops.pki/env}
    - { role: debops.secret }
    - { role: debops.pki }
    - { role: debops.nginx }

Example config (ansible/inventory/host_vars/myacmehost.yml)

pki_acme: True
pki_acme_install: True
pki_acme_ca: 'le-staging' # for testing
_hostnames_:
  - share.example.com
  - cloud.example.com
pki_host_realms:
 - name: 'some-realm'
   acme: True
   domains: '{{ _hostnames_ }}'
   acme_domains: '{{ _hostnames_ }}'
   acme_default_subdomains: []

Hi. Like the title says. I was previously using the reference playbook to perform PKI certificate creation and exchange in a 4 node cluster. So I'm pretty sure my playbook used to work. I currently tried running it under both Ansible 2.4 and Ansible 2.6 on both MacOS and Debian 9 stretch. Both fail with the same error message:

bash-3.2$ ansible-playbook site-pki.yml --ask-vault-pass
Vault password:
ERROR! the role 'debops.pki/env' was not found in /Users/jjzazuet/code/gopher/devops/roles:/Users/jjzazuet/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/Users/jjzazuet/code/gopher/devops

The error appears to have been in '/Users/jjzazuet/code/gopher/devops/site-pki.yml': line 4, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
    - role: debops.pki/env
      ^ here

bash-3.2$ 

And under Debian:

root@ny-mightygopher:~/gopher/devops# ansible-playbook site-pki.yml --ask-vault-pass
Vault password:
ERROR! the role 'debops.pki/env' was not found in /root/gopher/devops/roles:/root/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/root/gopher/devops

The error appears to have been in '/root/gopher/devops/site-pki.yml': line 4, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  roles:
    - role: debops.pki/env
      ^ here

root@mightygopher:~/qnoa/devops#

Upon further inspection of the role's code, it appears as if the env subrole is using symlinks to point back to shared parent role file resources (but I'm not fully certain). I also see that the role's codebase is a ew years old in general, so it's also possible that a new Ansible release broke the role's resource resolution strategy.

Any help or feedback is appreciated. Thanks for the awesome framework! 👍

md5 is deprecated, and sha1 will soon not be supported by Chrome, so sha256 support is desirable.

Very impressive project btw (the whole thing, debops).

Is there an advantage with copying the program in this case?

This happens with ansible 2.4.0.1 and latest debops on a fresh Debian Stretch machine.

TASK [debops.pki : Initialize PKI authorities] ******************************************************************************************************************
failed: [server -> localhost] (item={u'subdomain': u'root-ca', u'name': u'root', u'key_size': u'4096', u'subject': [u'o=MyDomain Certificate Authority']}) => {"changed": true, "cmd": ["./lib/pki-authority", "init", "--name", "root", "--default-sign-base", "365", "--root-sign-multiplier", "12", "--ca-sign-multiplier", "10", "--cert-sign-multiplier", "3"], "delta": "0:00:00.021152", "end": "2017-11-17 19:00:21.720638", "failed": true, "item": {"key_size": "4096", "name": "root", "subdomain": "root-ca", "subject": ["o=MyDomain Certificate Authority"]}, "msg": "non-zero return code", "rc": 2, "start": "2017-11-17 19:00:21.699486", "stderr": "./lib/pki-authority: line 164: declare: -g: invalid option\ndeclare: usage: declare [-afFirtx] [-p] [name[=value] ...]", "stderr_lines": ["./lib/pki-authority: line 164: declare: -g: invalid option", "declare: usage: declare [-afFirtx] [-p] [name[=value] ...]"], "stdout": "", "stdout_lines": []}
failed: [server -> localhost] (item={u'key_size': u'4096', u'subject': [u'o=MyDomain', u'ou=Domain CA'], u'subdomain': u'domain-ca', u'issuer_name': u'root', u'name': u'domain'}) => {"changed": true, "cmd": ["./lib/pki-authority", "init", "--name", "domain", "--default-sign-base", "365", "--root-sign-multiplier", "12", "--ca-sign-multiplier", "10", "--cert-sign-multiplier", "3"], "delta": "0:00:00.014699", "end": "2017-11-17 19:00:21.955539", "failed": true, "item": {"issuer_name": "root", "key_size": "4096", "name": "domain", "subdomain": "domain-ca", "subject": ["o=MyDomain", "ou=Domain CA"]}, "msg": "non-zero return code", "rc": 2, "start": "2017-11-17 19:00:21.940840", "stderr": "./lib/pki-authority: line 164: declare: -g: invalid option\ndeclare: usage: declare [-afFirtx] [-p] [name[=value] ...]", "stderr_lines": ["./lib/pki-authority: line 164: declare: -g: invalid option", "declare: usage: declare [-afFirtx] [-p] [name[=value] ...]"], "stdout": "", "stdout_lines": []}

The execution of the following playbook...

---

- name: PKI realms
  hosts: all
  become: True

  vars:

    custom_pki_realms:

      - name: 'k8s'
        subject_alt_names:
          - 'ip:{{ ansible_default_ipv4.address }}'
          - 'ip:{{ kubernetes_services_net | ipaddr("1") | ipaddr("host") | ipaddr("address") }}'
          - 'dns:kubernetes.default.svc.{{ kubernetes_cluster_domain }}'
          - 'dns:kubernetes.default.svc'
          - 'dns:kubernetes.default'
          - 'dns:kubernetes'

      - name: 'etcd'
        subject_alt_names:
          - 'ip:{{ ansible_default_ipv4.address }}'
          - 'ip:{{ kubernetes_services_net | ipaddr("3") | ipaddr("host") | ipaddr("address") }}'
          - 'dns:etcd.kube-system.svc.{{ kubernetes_cluster_domain }}'
          - 'dns:etcd.kube-system.svc'
          - 'dns:etcd.kube-system'
          - 'dns:etcd'

  roles:

    - role: debops.pki/env
      tags: [ 'role::pki' ]
      pki_dependent_realms: '{{ custom_pki_realms }}'  # see issue #78

    - role: debops.secret
      tags: [ 'role::pki' ]
      secret_directories:
        - '{{ pki_env_secret_directories }}'

    - role: debops.pki
      tags: [ 'role::pki' ]
      pki_dependent_realms:
        - '{{ custom_pki_realms }}'

..throws the following error:

TASK [debops.pki : Download public realm contents by host] *********************
ok: [noah] => (item=[{u'name': u'domain', u'acme': False}, u'external'])
ok: [noah] => (item=[{u'name': u'domain', u'acme': False}, u'internal'])
fatal: [noah]: FAILED! => {"failed": true, "msg": "Unexpected templating type error occurred on ({{ secret + \"/pki/realms/by-host/\" + pki_fqdn + \"/\" + item.0.name + \"/\" + item.1 + \"/\" }}): coercing to Unicode: need string or buffer, dict found"}

The pki-realm script support renewal of scripts that are about to expire, but if I modify the configuration of a certificate it wont update the certificate. Right now only deletion of the whole realm directory works.

A check like the following could do the check and then remove the keys like the pki-realm script already does upon cert expiration.

cur_san=$(openssl x509 -in public/cert.pem -noout -text |awk '/X509v3 Subject Alternative Name:/{ P=1; next } /X509v3 Certificate Policies:/ {exit} P' | sed 's/ //g' |tr ',' '\n' |sed 's/DNS://' |sort) 
new_san=$(grep 'DNS.' acme/openssl.conf |cut -d= -f2 |sed 's/ //g' |sort)
diff $cur_san $new_san

I had a typo in a script of mine that accidentally passed a CIDR ip range instead of an IP as a subject_alt_name.

Everything seemed to run normally, except the generated certs never appeared on the host. The request file was left as request.pem.tmp. The issue is that the role appears to complete successfully, influencing later roles, and making debugging quite difficult.

Result after completing a broken run:

.
├── [4.0K]  acme
├── [4.0K]  config
│   └── [2.8K]  realm.conf  
├── [4.0K]  external
├── [4.0K]  internal
│   ├── [ 745]  gnutls.conf 
│   └── [   0]  request.pem.tmp
├── [4.0K]  private
│   ├── [1.6K]  key.pem
│   └── [1.6K]  realm_key.pem
└── [4.0K]  public

I would guess the issue is in one of these lines , given that that's where request.pem.tmp is created.

Maybe there is a better way to get the exit status from certtool, or perhaps first check the validity of the generated file?

The exact line in my gnutls.conf that caused the error was:

ip_address = "123.123.123.123/16" when it should have been
ip_address = "123.123.123.123"

Maybe it's just me, but the whole system seems really brittle and every time it breaks it makes me wish I could just run certbot and be done with it.

/usr/local/lib/pki/pki-realm is a ~2500 lines bash script and it's a pain to debug when something goes wrong.

This could be an umbrella bug to improve the whole experience. I think the main points to tackle are:

• Document how the system works
• Document pki-realm existence and the arguments it takes, also implement a --help switch
• Have a verbose and interactive mode, where pki-realm tells the user what it's doing, without running in background.
• Don't just create error.log, but send it to the sysadmin by mail as well
• Be clear when subsequent runs fail because of the error.log presence.
• Simplify the whole script, if possible.

The docs to ACME support contain an example for "I want a certificate for subdomains but domain".

Please include a simple example for the standard case: a host serving several vhosts.

Thanks.

When attempting to call lib/pki-authority sign manually (I want to manually sign some CSRs with the authority provided by debops), it fails with: lib/pki-authority: line 661: 4: unbound variable. The line in sub_sign only calls sign_${library}_certificate with 3 arguments;

sign_${library}_certificate "config/${library}-sign.conf" "${input}" "${output}"

Additionally, there is no such function sign_gnutls_certificate, so using that library wouldn't work. It looks like sign-by-host is the only variant used by Ansible tasks here so I doubt this bug would be encountered under normal operation.

If an external certificate expires and get's renewed and replaced on the controller, the new certificate does not get copied to the host. This is because the relevant copy task has force: False.

In the current role whether force: False is set on a copy task in the pki role is rather inconsistent. Or in other words it's not clear to me what the rules are.

IMHO all these tasks should either use force: True or at least have this configurable. Would you accept a pull request for this?

I created a clean project with debops-init and bootstraped the host. Then i let debops -l <host> run.

Expected that debops.pki generates certs and setups the secrests/pki dir. But nothing happens, here the log:

TASK: [debops.pki | debug var=pki] ******************************************** 
ok: [owncloudsta] => {
    "var": {
        "pki": "True"
    }
}

TASK: [debops.pki | Create private groups if requested] *********************** 
skipping: [owncloudsta]

TASK: [debops.pki | Check if snapshot file exists on the Ansible Controller] *** 
ok: [owncloudsta -> localhost]

TASK: [debops.pki | Restore PKI from snapshot] ******************************** 
skipping: [owncloudsta]

TASK: [debops.pki | Create realm destination directories] ********************* 
skipping: [owncloudsta] => (item={'source': 'domain', 'ca': ['root/RootCA'], 'destination': 'domain', 'makefile': False, 'name': 'domain'})
skipping: [owncloudsta] => (item={'source': 'service', 'ca': ['root/ServiceCA'], 'destination': 'service', 'makefile': False, 'name': 'service'})
skipping: [owncloudsta] => (item={'source': u'hosts/owncloudsta/service', 'default_ca': u'CA/ServiceCA-.crt', 'name': 'host-service', 'default': u'srv.owncloudsta', 'ou': u' Service', 'destination': 'service', 'authority': 'root/ServiceCA'})
skipping: [owncloudsta] => (item={'source': u'hosts/owncloudsta/domain', 'name': 'host-domain', 'default': u'wildcard.domain.owncloudsta', 'ou': u'Data Center Operations', 'destination': 'domain', 'authority': 'ca/domain'})
skipping: [owncloudsta] => (item={'source': u'hosts/owncloudsta/host', 'name': 'host-internal', 'default': u'owncloudsta', 'ou': u' Data Center', 'destination': 'host', 'authority': 'ca/internal'})

TASK: [debops.pki | Generate main Makefile] *********************************** 
skipping: [owncloudsta]

TASK: [debops.pki | Generate realm Makefile] ********************************** 
skipping: [owncloudsta] => (item={'source': 'domain', 'ca': ['root/RootCA'], 'destination': 'domain', 'makefile': False, 'name': 'domain'})
skipping: [owncloudsta] => (item={'source': 'service', 'ca': ['root/ServiceCA']

I stumbled upon this error:

TASK [debops.pki : Install local PKI scripts] **********************************
task path: /home/vagrant/catena/roles/debops.pki/tasks/main.yml:32
ESTABLISH LOCAL CONNECTION FOR USER: vagrant
localhost EXEC rc=flag; [ -r /home/vagrant/catena/inventory/../secret/pki/lib/pki-authority ] || rc=2; [ -f /home/vagrant/catena/inventory/../secret/pki/lib/pki-authority ] || rc=1; [ -d /home/vagrant/catena/inventory/../secret/pki/lib/pki-authority ] && rc=3; /usr/bin/python -V 2>/dev/null || rc=4; [ x"$rc" != "xflag" ] && echo "${rc}  "/home/vagrant/catena/inventory/../secret/pki/lib/pki-authority && exit 0; (/usr/bin/python -c 'import hashlib; BLOCKSIZE = 65536; hasher = hashlib.sha1();
afile = open("'/home/vagrant/catena/inventory/../secret/pki/lib/pki-authority'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
        hasher.update(buf)
        buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())' 2>/dev/null) || (/usr/bin/python -c 'import sha; BLOCKSIZE = 65536; hasher = sha.sha();
afile = open("'/home/vagrant/catena/inventory/../secret/pki/lib/pki-authority'", "rb")
buf = afile.read(BLOCKSIZE)
while len(buf) > 0:
        hasher.update(buf)
        buf = afile.read(BLOCKSIZE)
afile.close()
print(hasher.hexdigest())' 2>/dev/null) || (echo '0  '/home/vagrant/catena/inventory/../secret/pki/lib/pki-authority)
localhost EXEC ( umask 22 && mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455030274.0-84911499590959 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455030274.0-84911499590959 )" )
localhost PUT /home/vagrant/catena/roles/debops.pki/files/secret/pki/lib/pki-authority TO /home/vagrant/.ansible/tmp/ansible-tmp-1455030274.0-84911499590959/source
localhost EXEC ( umask 22 && mkdir -p "$( echo $HOME/.ansible/tmp/ansible-tmp-1455030274.01-251127249931061 )" && echo "$( echo $HOME/.ansible/tmp/ansible-tmp-1455030274.01-251127249931061 )" )
localhost PUT /tmp/tmpEWM0vg TO /home/vagrant/.ansible/tmp/ansible-tmp-1455030274.01-251127249931061/copy
localhost EXEC LANG=it_IT.UTF-8 LC_ALL=it_IT.UTF-8 LC_MESSAGES=it_IT.UTF-8 /usr/bin/python /home/vagrant/.ansible/tmp/ansible-tmp-1455030274.01-251127249931061/copy; rm -rf "/home/vagrant/.ansible/tmp/ansible-tmp-1455030274.01-251127249931061/" > /dev/null 2>&1
fatal: [catena -> localhost]: FAILED! => {"changed": false, "checksum": "9164ed19f42472f4c5dc197731c47f3240789134", "failed": true, "invocation": {"module_args": {"backup": false, "content": null, "delimiter": null, "dest": "/home/vagrant/catena/inventory/../secret/pki/lib/", "directory_mode": null, "follow": false, "force": true, "group": null, "mode": "0755", "original_basename": "pki-authority", "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/home/vagrant/.ansible/tmp/ansible-tmp-1455030274.0-84911499590959/source", "validate": null}}, "msg": "Could not replace file: /home/vagrant/.ansible/tmp/ansible-tmp-1455030274.0-84911499590959/source to /home/vagrant/catena/inventory/../secret/pki/lib/pki-authority: [Errno 26] Text file busy"}

Ansible installed and running into the guest machine (ubuntu 14.04, windows 10 host) where /home/vagrant/ is a mounted shared folder via Vagrant/Virtualbox.

Any suggestion?

I'm not sure if it's a configuration issue on my part or not. Let's say I have a webserver inventory group and I have this in my configuration file:

pki_inventory_groups: [ 'webserver' ]

If I put files in secret/pki/realms/by-group/webserver/domain/external, I'd expect them to be uploaded. They're not. I think it's because pki_group_realms is empty and doesn't have the pki_default_realms in it.

You can fix the issue if you do this:

pki_group_realms: '{{ pki_default_realms }}'

That said, I'm not sure that's the best solution. It should be handled by the task itself I think. What do you think?

I've running tests with the new role on OSX. I can't get the PKI role to work at all. At first, I thought it was a server issue, but pki-authority runs on the local machine. This creates a bunch of issues.

The first issue is with the bash 4.3.0 requirement. The requirement isn't a deal breaker in itself, but Apple has no intention of supporting GPLv3 versions of bash. This means that we can only install bash 4.3.0 using homebrew. We can't and shouldn't replace the bash executable found in /bin/bash.

Because of this, pki-authority doesn't work. It runs using /bin/bash which is always going to be on bash 3.2. You'll get this error:

line 62: declare: -g: invalid option

I fixed it by editing pki-authority to use /usr/local/bin/bash. This got rid of the error, but that's not a long-term option.

I also found another issue once the script could go a bit further. $(hostname --fqdn) throws this error on OSX:

hostname: illegal option -- - 

I haven't found a solution to this error. That said, I'm not sure the script is designed to run on OSX at all at this point. This is a pretty big problem for people that use DebOps on OSX. We can't run the debops command to run the default playbook without it failing.

Using this config:

pki_acme: True
pki_realms:
  - name: 'test'
    acme: True
    acme_default_subdomains: []
    acme_subject: [ 'cn=test.example.com' ]
    acme_domains: [ 'test.example.com' ]
    domains: [ 'test.example.com' ]
    #acme_ca: 'le-staging'  # uncomment for staging tests

After the first debops run, the certificates created are not signed by LE.

Workaround is to manually delete /etc/pki/realms/test and then run debops service/pki again.

Hi

Just to keep track of it, I was just required to do a manual (external CA) certificate switch to a new set of private key and certificate. I updated the following files on the controller:

• ansible/secret/pki/realms/by-group/$domain_name/domain/external/cert.pem
• ansible/secret/pki/realms/by-group/$domain_name/domain/private/key.pem

And rerun the role. This did not have the desired affect. I did not have time to look into this further and just deleted the realm on the remote host and rerun the role which fixed it.