leowmjw / reclass Goto Github PK
View Code? Open in Web Editor NEWThis project forked from madduck/reclass
A recursive external node classifier for automation tools like Ansible, Puppet, and Salt
License: Other
This project forked from madduck/reclass
A recursive external node classifier for automation tools like Ansible, Puppet, and Salt
License: Other
============================================================= reclass — recursive external node classification ============================================================= reclass is © 2007–2013 martin f. krafft <[email protected]> and available under the terms of the Artistic Licence 2.0 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' reclass is an "external node classifier" (ENC) as can be used with automation tools, such as Puppet, Salt, and Ansible. It is also a stand-alone tool for merging data sources recursively. The purpose of an ENC is to allow a system administrator to maintain an inventory of nodes to be managed, completely separately from the configuration of the automation tool. Usually, the external node classifier completely replaces the tool-specific inventory (such as site.pp for Puppet, ext_pillar/master_tops for Salt, or /etc/ansible/hosts). reclass allows you to define your nodes through class inheritance, while always able to override details further up the tree (i.e. in more specific nodes). Think of classes as feature sets, as commonalities between nodes, or as tags. Add to that the ability to nest classes (multiple inheritance is allowed, well-defined, and encouraged), and piece together your infrastructure from smaller bits, eliminating redundancy and exposing all important parameters to a single location, logically organised. In general, the ENC fulfills two jobs: - it provides information about groups of nodes and group memberships - it gives access to node-specific information, such as variables In this document, you will find an overview of the concepts of reclass and the way it works. Have a look at README.Salt and README.Ansible for information about integration of reclass with these tools. Installation ~~~~~~~~~~~~ Before you can use reclass, you need to install it into a place where Python can find it. Unless you installed a package from your distribution, the following step should install the package to /usr/local: $ python setup.py install If you want to install to a different location, use --prefix like so: $ python setup.py install --prefix=/opt/local Just make sure that the destination is in the Python module search path, which you can check like this: $ python -c 'import sys; print sys.path' More options can be found in the output of $ python setup.py install --help $ python setup.py --help $ python setup.py --help-commands $ python setup.py --help [cmd] If you just want to run reclass from source, e.g. because you are going to be making and testing changes, install it in "development mode": $ python setup.py develop To uninstall (the rm call is necessary due to http://bugs.debian.org/714960): $ python setup.py develop --uninstall $ rm /usr/local/bin/reclass* Uninstallation currently isn't possible for packages installed to /usr/local as per the above method, unfortunately: http://bugs.python.org/issue4673. The following should do: $ rm -r /usr/local/lib/python*/dist-packages/reclass* /usr/local/bin/reclass* reclass concepts ~~~~~~~~~~~~~~~~ reclass assumes a node-centric perspective into your inventory. This is obvious when you query reclass for node-specific information, but it might not be clear when you ask reclass to provide you with a list of groups. In that case, reclass loops over all nodes it can find in its database, reads all information it can find about the nodes, and finally reorders the result to provide a list of groups with the nodes they contain. Since the term 'groups' is somewhat ambiguous, it helps to start off with a short glossary of reclass-specific terminology: node: A node, usually a computer in your infrastructure class: A category, tag, feature, or role that applies to a node Classes may be nested, i.e. there can be a class hierarchy application: A specific set of behaviour to apply parameter: Node-specific variables, with inheritance throughout the class hierarchy. A class consists of zero or more parent classes, zero or more applications, and any number of parameters. A node is almost equivalent to a class, except that it usually does not (but can) specify applications. When reclass parses a node (or class) definition and encounters a parent class, it recurses to this parent class first before reading any data of the node (or class). When reclass returns from the recursive, depth first walk, it then merges all information of the current node (or class) into the information it obtained during the recursion. Furthermore, a node (or class) may define a list of classes it derives from, in which case classes defined further down the list will be able to override classes further up the list. Information in this context is essentially one of a list of applications or a list of parameters. The interaction between the depth-first walk and the delayed merging of data means that the node (and any class) may override any of the data defined by any of the parent classes (ancestors). This is in line with the assumption that more specific definitions ("this specific host") should have a higher precedence than more general definitions ("all webservers", which includes all webservers in Munich, which includes "this specific host", for example). Here's a quick example, showing how parameters accumulate and can get replaced. All unixnodes (i.e. nodes who have the 'unixnodes' class in their ancestry) have /etc/motd centrally-managed (through the 'motd' application), and the unixnodes class definition provides a generic message-of-the-day to be put into this file. All debiannodes, which are descendants of unixnodes, should include the Debian codename in this message, so the message-of-the-day is overwritten in the debiannodes class. The node 'quantum.example.org' will have a scheduled downtime this weekend, so until Monday, an appropriate message-of-the-day is added to the node definition. When the 'motd' application runs, it receives the appropriate message-of-the-day (from 'quantum.example.org' when run on that node) and writes it into /etc/motd. At this point it should be noted that parameters whose values are lists or key-value pairs don't get overwritten by children classes or node definitions, but the information gets merged (recursively) instead. Similarly to parameters, applications also accumulate during the recursive walk through the class ancestry. It is possible for a node or child class to _remove_ an application added by a parent class, by prefixing the application with '~'. Finally, reclass happily lets you use multiple inheritance, and ensures that the resolution of parameters is still well-defined. Here's another example building upon the one about /etc/motd above: 'quantum.example.org' (which is back up and therefore its node definition no longer contains a message-of-the-day) is at a site in Munich. Therefore, it is a child of the class 'hosted@munich'. This class is independent of the 'unixnode' hierarchy, 'quantum.example.org' derives from both. In this example infrastructure, 'hosted@munich' is more specific than 'debiannodes' because there are plenty of Debian nodes at other sites (and some non-Debian nodes in Munich). Therefore, 'quantum.example.org' derives from 'hosted@munich' _after_ 'debiannodes'. When an electricity outage is expected over the weekend in Munich, the admin can change the message-of-the-day in the 'hosted@munich' class, and it will apply to all hosts in Munich. However, not all hosts in Munich have /etc/motd, because some of them are 'windowsnodes'. Since the 'windowsnodes' ancestry does not specify the 'motd' application, those hosts have access to the message-of-the-day in the node variables, but the message won't get used… … unless, of course, 'windowsnodes' specified a Windows-specific application to bring such notices to the attention of the user. It's also trivial to ensure a certain order of class evaluation. Here's another example: The 'ssh.server' class defines the 'permit_root_login' parameter to 'no'. The 'backuppc.client' class defines the parameter to 'without-password', because the BackupPC server might need to log in to the host as root. Now, what happens if the admin accidentally provides the following two classes? - backuppc.client - ssh.server Theoretically, this would mean 'permit_root_login' gets set to 'no'. However, since all 'backuppc.client' nodes need 'ssh.server' (at least in most setups), the class 'backuppc.client' itself derives from 'ssh.server', ensuring that it gets parsed before 'backuppc.client'. When reclass returns to the node and encounters the 'ssh.server' class defined there, it simply skips it, as it's already been processed. reclass operations ~~~~~~~~~~~~~~~~~~ While reclass has been built to support different storage backends through plugins, currently only the 'yaml_fs' storage backend exists. This is a very simple, yet powerful, YAML-based backend, using flat files on the filesystem (as suggested by the _fs postfix). yaml_fs works with two directories, one for node definitions, and another for class definitions. It is possible to use a single directory for both, but that could get messy and is therefore not recommended. Files in those directories are YAML-files, specifying key-value pairs. The following three keys are read by reclass: classes: a list of parent classes appliations: a list of applications to append to the applications defined by ancestors. If an application name starts with '~', it would remove this application from the list, if it had already been added — but it does not prevent a future addition. E.g. '~firewalled' parameters: key-value pairs to set defaults in class definitions, override existing data, or provide node-specific information in node specifications. By convention, parameters corresponding to an application should be provided as subkey-value pairs, keyed by the name of the application, e.g. applications: - ssh.server parameters: ssh.server: permit_root_login: no reclass starts out reading a node definition file, obtains the list of classes, then reads the files corresponding to these classes, recursively reading parent classes, and finally merges the applications list and the parameters. Merging of parameters is done recursively, meaning that lists and dictionaries are extended (recursively), rather than replaced. However, a scalar value *does* overwrite a dictionary or list value. While the scalar could be appended to an existing list, there is sane default assumption in the context of a dictionary, so this behaviour seems the most logical. Finally, parameters may reference each other, including deep references, e.g.: parameters: location: Munich, Germany motd: header: This node sits in ${location} for_demonstration: ${motd:header} dict_reference: ${motd} After merging and interpolation, which happens automatically inside the storage modules, the 'for_demonstration' parameter will have a value of "This node sits in Munich, Germany". Types are preserved if the value contains nothing but a reference. Hence, the value of 'dict_reference' will actually be a dictionary. Version control ~~~~~~~~~~~~~~~ I recommend you maintain your reclass inventory database in Git, right from the start. Usage ~~~~~ For information on how to use reclass directly, invoke reclass with --help and study the output. The three options --inventory-base-uri, --nodes-uri, and --classes-uri together specify the location of the inventory. If the base URI is specified, then it is prepended to the other two URIs, unless they are absolute URIs. If these two URIs are not specified, they default to 'nodes' and 'classes'. Therefore, if your inventory is in '/etc/reclass/nodes' and '/etc/reclass/classes', all you need to specify is the base URI as '/etc/reclass' — which is actually the default (see reclass/defaults.py). If you've installed reclass as per the above instructions, try to run it from the source directory like this: $ reclass -b examples/ --inventory $ reclass -b examples/ --node localhost Those data come from examples/nodes and examples/classes, and you can surely make your own way from here. More commonly, however, use of reclass will happen indirectly, and through so-called adapters, e.g. /…/reclass/adapters/salt. The job of an adapter is to translate between different invocation paradigms, provide a sane set of default options, and massage the data from reclass into the format expected by the automation tool in use. Please have a look at the respective README files for these adapters. Configuration file ~~~~~~~~~~~~~~~~~~ reclass can read some of its configuration from a file. The file is a YAML-file and simply defines key-value pairs. The configuration file can be used to set defaults for all the options that are otherwise configurable via the command-line interface, so please use the --help output of reclass for reference. The command-line option '--nodes-uri' corresponds to the key 'nodes_uri' in the configuration file. For example: storage_type: yaml_fs pretty_print: True output: json inventory_base_uri: /etc/reclass nodes_uri: ../nodes reclass first looks in the current directory for the file called 'reclass-config.yml' (see reclass/defaults.py) and if no such file is found, it looks in $HOME, then in /etc/reclass, and then "next to" the reclass script itself, i.e. if the script is symlinked to /srv/provisioning/reclass, then the the script will try to access /srv/provisioning/reclass-config.yml. Note that yaml_fs is currently the only supported storage_type, and it's the default if you don't set it. Adapters may implement their own lookup logic, of course, so make sure to read their READMEs. -- martin f. krafft <[email protected]> Wed, 07 Aug 2013 16:21:04 +0200
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.