saltstack-formulas / bind-formula Goto Github PK

As written, "pillar.example" has four instances of "bind:" formatted to look like the base of a pillar entry.

Each one has distinct types of data, so this n00b took that to mean that Salt would condense the file into distinct pillar items without any trouble. Instead, the pillar doesn't compile at all, and the errors it throws didn't help (me at least) know what the reason was.

Proposal: Either remove all but the first "bind:" heading, or insert comments to explain how they should be used.

In named.conf.local file is optional see here.

Due to the way the formula is written at present this means we need to define an entire site in available_zones just to add the file directive to named.conf.local rather than just skipping writing the file directive if not specified.

I've put a bodge into my local copy to get round my pressing issue, but need to look into a sane way of doing it in the macro in files/named.conf.local.jinja.

Hello,

I am trying kitchen-salt to play with this formula.
I am loding the pillar.example file to pillar data, running the bind.config state.

       Transfering files to <default-ubuntu-1404-i386>
[INFO    ] Using cached minion ID from /etc/salt/minion_id: default-ubuntu-1404-i386.vagrantup.com       
[INFO    ] Loading fresh modules for state activity       
[INFO    ] Creating module dir '/var/cache/salt/minion/extmods/modules'       
[INFO    ] Syncing modules for environment 'base'       
[INFO    ] Loading cache from salt://_modules, for base)       
[INFO    ] Caching directory '_modules/' for environment 'base'       
[INFO    ] Creating module dir '/var/cache/salt/minion/extmods/states'       
[INFO    ] Syncing states for environment 'base'       
[INFO    ] Loading cache from salt://_states, for base)       
       [INFO    ] Caching directory '_states/' for environment 'base'
[INFO    ] Creating module dir '/var/cache/salt/minion/extmods/grains'       
[INFO    ] Syncing grains for environment 'base'       
[INFO    ] Loading cache from salt://_grains, for base)       
[INFO    ] Caching directory '_grains/' for environment 'base'       
[INFO    ] Creating module dir '/var/cache/salt/minion/extmods/renderers'       
[INFO    ] Syncing renderers for environment 'base'       
[INFO    ] Loading cache from salt://_renderers, for base)       
[INFO    ] Caching directory '_renderers/' for environment 'base'       
[INFO    ] Creating module dir '/var/cache/salt/minion/extmods/returners'       
[INFO    ] Syncing returners for environment 'base'       
       [INFO    ] Loading cache from salt://_returners, for base)
[INFO    ] Caching directory '_returners/' for environment 'base'       
[INFO    ] Creating module dir '/var/cache/salt/minion/extmods/outputters'       
[INFO    ] Syncing outputters for environment 'base'       
[INFO    ] Loading cache from salt://_outputters, for base)       
[INFO    ] Caching directory '_outputters/' for environment 'base'       
[INFO    ] Loading fresh modules for state activity       
[CRITICAL] Rendering SLS "base:bind.config" failed: Conflicting ID "zones-"       
local:       
    Data failed to compile:       
       ----------
           Rendering SLS "base:bind.config" failed: Conflicting ID "zones-"
salt-call exit code: 0       
salt-call output grep exit code 1       
       Finished converging <default-ubuntu-1404-i386> (6m12.68s).

Isn't pillar example data suposed to work out of the box ?

🚨 The automated release from the master branch failed. 🚨

I recommend you give this issue a high priority, so other packages depending on you could benefit from your bug fixes and new features.

You can find below the list of errors reported by semantic-release. Each one of them has to be resolved in order to automatically publish your package. I’m sure you can resolve this 💪.

Errors are usually caused by a misconfiguration or an authentication problem. With each error reported below you will find explanation and guidance to help you to resolve it.

Once all the errors are resolved, semantic-release will release your package the next time you push a commit to the master branch. You can also manually restart the failed CI job that runs semantic-release.

If you are not sure how to resolve this, here is some links that can help you:

• Usage documentation
• Frequently Asked Questions
• Support channels

If those don’t help, or if this issue is reporting something you think isn’t right, you can always ask the humans behind semantic-release.

Missing package.json file.

A package.json file at the root of your project is required to release on npm.

Please follow the npm guideline to create a valid package.json file.

Good luck with your project ✨

Your semantic-release bot 📦🚀

I started out thinking that this was something I was doing wrong, and I'm still not sure it isn't, but after backing out all the relevant stuff I've done I still get the same error: the 'bind_local_config' state won't render, with a Jinja complaint,

SaltRenderError: Jinja variable 'dict object' has no attribute 'notify'

...and that still happens even if I take every reference to 'notify' out of my pillar.

Help.

Writeup I posted on gist:

https://gist.github.com/mjinks/cfefc0c50a5a6dd3f5d4

Full text of the error I'm hitting:

ID: bind_local_config

Function: file.managed

    Name: /etc/bind/named.conf.local
  Result: False
 Comment: An exception occurred in this state: Traceback (most recent call last):
            File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1560, in call
              **cdata['kwargs'])
            File "/usr/lib/python2.7/dist-packages/salt/states/file.py", line 1423, in managed
              **kwargs
            File "/usr/lib/python2.7/dist-packages/salt/modules/file.py", line 3142, in check_managed_changes
              **kwargs)
            File "/usr/lib/python2.7/dist-packages/salt/modules/file.py", line 2841, in get_managed
              **kwargs)
            File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 121, in render_tmpl
              output = render_str(tmplstr, context, tmplpath)
            File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 324, in render_jinja_tmpl
              buf=tmplstr)
          SaltRenderError: Jinja variable 'dict object' has no attribute 'notify'
 Started: 19:38:23.729014
Duration: 68.164 ms
 Changes:   

Due to an issue with nesting, you end up with two pillar IDs that are identical.

I believe this is due to the dnssec logic that this supposed to be inside the 'included' file, is actually outside that if block.

Bind doesn't reload zones with dynamic updates: http://jon.netdork.net/2008/08/21/bind-dynamic-zones-and-updates/

A workaround is:

rndc freeze example.com
rndc reload example.com
rndc thaw example.com

Looking here:

https://github.com/saltstack-formulas/bind-formula/blob/master/bind/config.sls#L133

You are generating a list of zones from configured_zones, but then require available_zones to contain a zone name with a 'file' key. Why are there two authoritative sources for creating a zone? Why couldn't configured_zones simply contain a 'file' key so you don't have to duplicate zone definitions in two places in pillar data? Am I missing something?

i know bind9 accepts both by default, but should there be a way to also choose one or the other?

ipv6 lookups on our ipv4 network are making a mess of the logs and appear to be breaking ipv4 lookups. adding -4 to /etc/default/bind9 OPTIONS resolved this.

i've fixed this by creating my own state to manage /etc/default/bind9, but it seems more appropriate that it be handled by this formula...

When I set configured_views in my pillar file the bind.config state by calling named.local.jinja template doesn't update properly named.conf.local. In fact, in addition to correctly put zone block in view block, bind.config copy a second time zone block at the top of the file. And you get the following error from bind9

/etc/bind/named.conf.local:11: when using 'view' statements, all zones must be in views
loading configuration: failure

To get able to use views I put the zone macro in if statement

https://github.com/saltstack-formulas/bind-formula/blob/master/bind/files/named.conf.local.jinja

line 92 to 94
{%- if salt['pillar.get']('bind:configured_views', {}) is not defined %}
{{ zone(key, args, file, masters) }}
{%- endif %}

Hi,

Is there anyway to not erase existing records in a zone file ?
We use dynamic records, so each time we run salt, the zone file is reseted

Best regards,

Hi,

I'm not able to use this module without modifying bind/files/named.conf and remove the ipv6 conditional section.

I don't understand how's that supposed to work, because it's not pillar or grains information...

Thanks

It seems impossible with this bind formula to set a TTL on individual records that differs from the TTL set in the soa data structure.

$TTL 1w ; default for the rest of the zone. 
foo         IN      A      192.168.254.3  ; uses default of 1w
     
$TTL 2d ; default for the rest of the zone. 
joe   3h   IN      A      192.168.254.3  ; overrides default     
www        IN      A      192.168.254.3  ; uses default = 2 days

Changing the default for the rest of the zone may nog be very practical with the current set up that sorts the entries on record type and alphabetically, but the override per record should be possible?

I believe this showed up in #45, but I now get this when running this state with test = True (though I don't think that matters):

----------
          ID: bind_default_config
    Function: file.managed
        Name: /etc/default/bind9
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1591, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/states/file.py", line 1446, in managed
                  **kwargs
                File "/usr/lib/python2.7/dist-packages/salt/modules/file.py", line 3694, in check_managed_changes
                  **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/modules/file.py", line 3393, in get_managed
                  **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 180, in render_tmpl
                  output = render_str(tmplstr, context, tmplpath)
                File "/usr/lib/python2.7/dist-packages/salt/utils/templates.py", line 376, in render_jinja_tmpl
                  tmplstr)
              SaltRenderError: Jinja syntax error: Encountered unknown tag 'param'. Jinja was looking for the following tags: 'elif' or 'else' or 'endif'. The innermost block that needs to be closed is 'if'.; line 4

              ---
              {% set protocol = salt['pillar.get']('bind:config:protocol', False) -%}
              {% set param = ['-u bind'] -%}
              {% if protocol -%}
                {%- param.append('-' + protocol|string) -%}    <======================
              {% endif -%}
              # run resolvconf?
              RESOLVCONF=no

              # startup options for the server
              [...]
              ---
     Started: 16:56:08.504971
    Duration: 39.773 ms
     Changes:   
----------

I just upgraded from 2015.5.2 to 2015.8.1:

$ sudo salt-master --version
salt-master 2015.8.1 (Beryllium)
$ sudo salt-minion --version
salt-minion 2015.8.1 (Beryllium)

The state bind.config doesn't check the contents of zone files with named-checkzone.

I would be nice to have this state fail on zones with errors in stead of exiting with "Succeeded", and to find with further investigation that bind didn't actually load the faulty zone because of errors.

Hi!

I get an error with this logrotate :

/var/log/bind9/query.log {
  rotate 7
  daily
  missingok
  notifempty
  sharedscripts
  copytruncate
  compress
  create 0664 bind root
  su
}

switching euid to -1 and egid to -1
error: error switching euid to -1 and egid to -1: Invalid argument

I think we should pass user and group to the su directive : su bind bind.

Thanks!

bind:
  keys:
    "core_dhcp":
    algorithm: HMAC-SHA512
    secret: {{ conf.bind.secret }}

creates /etc/bind/named.conf.key:

key "core_dhcp" {
  algorithm HMAC-SHA512;
  secret MY-SUP/ER-SECU/RE-K/EY;
};

... but bind refuses to restart unless the secret is "quoted". (probably slashes in the key causing issues)

this needs to be fixed in the named.conf.key template:

  secret "{{ args['secret'] }}";

happy to do a PR for this if its agreeable.

If bind:available_zones does not mactch bind:configured_zones or if match but bind:available_zones:[ZONE]:file does not exists, file.managed will be a directory.

Creating a PR soon.

Due to issues with rollerd, dnssec-tools was removed from testing and did not make it into Debian Jessie. map.jinja currently specs dnssect-tools for all Debian installs which causes failures on Jessie installations.

Convention over configuration:

If a zone is defined in a pillar, I want the salt states to ensure it is declared in the conf file and also the records are declared in the zone file.

@ppieprzycki thought it was a good idea for Debian in 2947dde

It's probably a good idea in general.

I'm following pillar.example and having issues enabling extensive logging. If I remove extensive logging all is ok with rendering.

I noticed line 117 of named.conf.local.jinja seems to expect use_extensive_logging: would be a boolean.

      ID: bind_local_config
Function: file.managed
    Name: /etc/named.conf.local
  Result: False
 Comment: Unable to manage file: Jinja variable 'salt.utils.odict.OrderedDict object' has no attribute 'logging_config'
 Started: 20:23:38.504949
Duration: 139.781 ms
 Changes:

      ID: bind
Function: service.running
    Name: named
  Result: False
 Comment: One or more requisite failed: bind.config.bind_local_config
 Started: 20:23:38.645581
Duration: 0.024 ms
 Changes:

Pillar File
...

config:
tmpl: salt://bind/files/redhat/named.conf
user: root
group: named
mode: 640
enable_logging: true
use_extensive_logging:
channel:
default_log:
file: default
size: '200m'
versions: '10'
print-time: yes
print-category: yes
print-severity: yes
severity: info
queries_log:
file: queries
print-time: yes
print-category: yes
print-severity: yes
severity: info
query-errors_log:
file: query-errors
print-time: yes
print-category: yes
print-severity: yes
severity: dynamic
default_syslog:
print-time: yes
print-category: yes
print-severity: yes
syslog: daemon
severity: info
default_debug:
file: named.run
print-time: yes
print-category: yes
print-severity: yes
severity: info
category:
default:
- default_syslog
- default_debug
- default_log
config:
- default_syslog
- default_debug
- default_log
network:
- default_syslog
- default_debug
- default_log
general:
- default_syslog
- default_debug
- default_log
queries:
- queries_log
query-errors:
- query-errors_log

...

Changed to following:

...

config:
tmpl: salt://bind/files/redhat/named.conf
user: root
group: named
mode: 640
enable_logging: true

...

      ID: bind_local_config
Function: file.managed
    Name: /etc/named.conf.local
  Result: True
 Comment: File /etc/named.conf.local updated
 Started: 20:28:37.335889
Duration: 136.169 ms
 Changes:
          ----------
          diff:
              ---
              +++
              @@ -0,0 +1,19 @@
              +# vim: sts=2 ts=2 sw=2 et ai
              +//
              +// Do any local configuration here
              +//
              +
              +// Consider adding the 1918 zones here, if they are not used in your
              +// organization
              +//include "/etc/bind/zones.rfc1918";
              +
              +
              +
              +
              +logging {
              +  channel "querylog" {
              +    file "/var/named/data/query.log";
              +    print-time yes;
              +  };
              +  category queries { querylog; };
              +};
          group:
              named
          mode:
              0640

      ID: bind
Function: service.running
    Name: named
  Result: True
 Comment: Service reloaded
 Started: 20:28:37.521079
Duration: 116.846 ms
 Changes:
          ----------
          named:
              True

I've been looking at this today, but I can't figure out how to define my local zone. The config.sls looks to

salt://bind/zones/{{ file }}

But if I am using this over gitfs, I don't think I can define this file locally while still using the rest of the definitions from gitfs.

So is the preferred usage of this (as it stands now) to fork the repository to add zone templates? Or am I missing something?

Currently:

bind:
lookup:
pkgs:
- bind
service: named

Shouldn't that be:

bind:
lookup:
pkgs: bind
service: named

...?

Formulas iterating over dictionaries should use the items() method instead of iteritems() because the latter method has been removed from Python 3. While the items() method in Python 2 uses more memory than iteritems(), these dictionaries are typically small (e.g., sets of settings from Pillar).

Why bind/zones a symbolic link?
Why it is hardcoded without possibility of custom path?

What about a root_dir pointing to ~/workspace/servers/ instead of /srv/salt?
What about file_roots other than default base setup?

file_roots:
  dev:
    - /srv/salt/dev/states
    - /srv/salt/prod/states
  prod:
    - /srv/salt/prod/states

Currently I need to manual change bind-formula in order to make things work.

Maybe add another file_root entry pointing to my custom bind directory can work but this seems a hack instead of a default expected approach:

file_roots:
  dev:
    - /srv/salt/prod/states/files/var/
    - /srv/salt/dev/states
    - /srv/salt/prod/states
  prod:
    - /srv/salt/prod/states/files/var/
    - /srv/salt/prod/states

var/
└── bind/
    └── hosts
       ├── zone-file1.local
       ├── zone-file2.local

EDIT: Hacky symlinks will be a issue because master stop responding when I've specifics symlinks

tks (:

notice returncode 0 even if errors.
I wonder if use of -xc flag will change this, although not convinced by docs:

-xc Display a message associated with a zonesigner exit value. This option is intended for use by those programs who wish for zonesigner to run silently, but need a description for why
zonesigner has exited with an error.

      ID: named_directory
Function: file.directory
    Name: zonesigner -zone example.com example.com
  Result: True
 Comment: Command "zonesigner -zone example.com example.com" run
 Started: 08:23:53.414959
Duration: 157.958 ms
 Changes:   
          ----------
          pid:
              17455
          retcode:
              0
          stderr:
              UNIVERSAL->import is deprecated and will be removed in a future perl at /usr/share/perl5/Net/DNS/SEC/Tools/tooloptions.pm line 19.
          stdout:
              dns_master_load: example.com:24: main.example.com: CNAME and other data
              zone example.com/IN: loading from master file example.com failed: CNAME and other data
              zone example.com/IN: not loaded due to errors.

See https://deepthought.isc.org/article/AA-00711/0/In-line-Signing-With-NSEC3-in-BIND-9.9-A-Walk-through.html for a walkthrough.

The directory declaration in named.conf.options appears to be statically defined; even though named_directory is specified in the map; for most archs, the value in the map and in named.conf.options matches - but not for Debian.

As a result, when using auto_serial, the include file root is based from the directory declaration, and the file is located in named_directory.

The named.conf.options template should reference the named_directory variable from the map.

Is there an option to configure slave servers?

The bind911 package is deprecated and the formula should probably default to the bind916 package.

I searched repo for "secret" since a key defined for dynamic updates using CentOS (RedHat) was not being rendered in named.conf or named.conf.local . I found that only Debian and BSD had the following code block.

{%- if 'keys' in salt['pillar.get']('bind') %}
{% for key,args in salt['pillar.get']('bind:keys', {})|dictsort -%}
key "{{ key }}" {
  algorithm {{ args['algorithm'] | default('HMAC-MD5.SIG-ALG.REG.INT') }};
  secret "{{ args['secret'] }}";
};
{% endfor %}
{% endif %}

I placed at top of bind/files/named.conf.local.jinja and key section was now rendered. Could you add this missing section for RedHat?

Hello,

Thank you for this formula. I'd like to use LDAP as a backend for bind, but I can't find any option here, did I miss something?

The generated configuration would looks like:

// A standard zone
zone "deimos.fr" {
        type master;
        database "ldap ldap://127.0.0.1/ou=deimos.fr,ou=dns,o=deimos,dc=fr???? \
        !bindname=cn=dnsadmin%2cou=dns%2co=deimos%2cdc=fr,!x-bindpw=DnsPassword 172800";
        notify yes;
};
 
// A reverse zone
zone "1.168.192.in-addr.arpa" {
        type master;
        database "ldap ldap://127.0.0.1/ou=1.168.192.in-addr.arpa,ou=dns,o=deimos,dc=fr???? \
        !bindname=cn=dnsadmin%2cou=dns%2co=deimos%2cdc=fr,!x-bindpw=DnsPassword 172800";
        notify yes;
};

Thank you!

Source: https://wiki.deimos.fr/Mise_en_place_d%27un_serveur_Bind_en_backend_LDAP

I currently cannot get this module to see the zones_source_dir on RedHat (CentOS) because it is configured with a leading slash. If I edit the module source and remove the leading slash I can get it to deploy a file from "/srv/salt/base/srv/salt/zones/", which is quite ugly.

I want to override this variable to be exactly as it is on Arch, "zones"

This is a saner default, this way I can present a file in /srv/salt/base/zones/ and it will find it.

I am running Ubuntu 12.04. The following section causes bind to fail on startup:

logging {
    channel "querylog" { file "/var/log/bind9/query.log"; print-time yes; };
    category queries { querylog; };
};

Since this is a logging issue, there are no log files for this. Syslog shows the following problems:

Nov 19 01:46:11 dns named[13324]: isc_stdio_open '/var/log/bind9/query.log' failed: permission denied
Nov 19 01:46:11 dns named[13324]: configuring logging: permission denied
Nov 19 01:46:11 dns named[13324]: loading configuration: permission denied
Nov 19 01:46:11 dns named[13324]: exiting (due to fatal error)

Commenting out the logging lines solved my problems.

I do not know bind well, so I do not know if we should just remove this section, or if it is important.

On RHEL7/CentOS7 selinux is enabled by default the bind-formula state enables logging by default. The default logging location is not suitable for the default selinux settings however. RHEL7/CentOS7 standard packaging expects named to log to /var/named/data instead of /var/log/named.

type=AVC msg=audit(1535031676.458:4738): avc: denied { open } for pid=33274 comm="named" path="/var/log/named/query.log" dev="dm-3" ino=262174 scontext=system_u:system_r:named_t:s0 tcontext=system_u:object_r:var_log_t:s0 tclass=file

So the default state settings on RHEL/CentOS7 is broken because the service cannot start due to not being able to write to the log file.

We need to ensure that the version of logrotate is being checked (for the usage of su) as it breaks for people on old versions of logrotate as noted here: 6c02efe#commitcomment-6085039

When running this formula under python3, multiple issues crop up:

• dictsort is used extensively and has changed functionality between py2 and py3. Python2 would happily sort a dictionary containing both strings and integers. Python3 however will return a TypeError saying that int and str are incompatible types.
  This is a known change in the sorted() function.

• {{ zone_records | tojson }} seems to break on mixed data as well. I did not dig into the reason yet.

There are two possible ways of resolving the issue:

1. Document that zone data must be all strings. Especially when configuring PTR RRs. Instead of using an int as key, this needs to be wrapped in quotes to ensure there are only strings. This ill work but has the drawback that e.g. '110' is now seen as smaller than e.g. '40' and the order of entries in the zone files appear incorrectly sorted.

2. Remove all use of dictsort and instead rely on the python3 behavior that dicts are now returned in deterministic (but not sorted) order.

What is the preferred solution?

For 2. an example patch might look like this:

diff --git a/bind/config.sls b/bind/config.sls
index 0d200e7..d9a62b8 100644
--- a/bind/config.sls
+++ b/bind/config.sls
@@ -223,9 +223,9 @@ bind_rndc_client_config:

 {%- set views = {False: salt['pillar.get']('bind', {})} %}{# process non-view zones in the same loop #}
 {%- do views.update(salt['pillar.get']('bind:configured_views', {})) %}
-{%- for view, view_data in views|dictsort %}
+{%- for view, view_data in views.items() %}
 {%-   set dash_view = '-' + view if view else '' %}
-{%-   for zone, zone_data in view_data.get('configured_zones', {})|dictsort -%}
+{%-   for zone, zone_data in view_data.get('configured_zones', {}).items() -%}
 {%-     if 'file' in zone_data %}
 {%-       set file = zone_data.file %}
 {%-       set zone = file|replace(".txt", "") %}
@@ -260,7 +260,7 @@ zones{{ dash_view }}-{{ zone }}{{ '.include' if serial_auto else '' }}:
         zone: zones{{ dash_view }}-{{ zone }}
         soa: {{ salt['pillar.get']("bind:available_zones:" + zone + ":soa") | json }}
         includes: {{ salt['pillar.get']("bind:available_zones:" + zone + ":includes") | json }}
-        records: {{ zone_records | json }}
+        records: {{ zone_records }}
         include: False
     {% endif %}
     - user: {{ salt['pillar.get']('bind:config:user', map.user) }}
diff --git a/bind/files/zone.jinja b/bind/files/zone.jinja
index 9583197..ac77c62 100644
--- a/bind/files/zone.jinja
+++ b/bind/files/zone.jinja
@@ -28,11 +28,11 @@ $TTL {{ soa['ttl'] }}
 {% if include %}
 $INCLUDE {{ include }}
 {% else %}
-{% for type, rrs in records|dictsort %}
+{% for type, rrs in records.items() %}
 ;
 ; {{ type }} RRs
 ;
-{%- for host, data in rrs|dictsort %}
+{%- for host, data in rrs.items() %}
 {%- if data is mapping %}
 {%- if 'ttl' in data %}
 {{ host }} {{ data['ttl'] }} {{ data['type'] }} {{ data['record'] }}

allow-recursion: '{ any; };' # Never include this on a public resolver

https://github.com/saltstack-formulas/bind-formula/blob/master/pillar.example#L14

The semicolon after the close brace seems to cause an error when bind starts as the file is compiled to have two semicolons in a row