Ansible playbooks for installing the Confluent Platform.
Note: These playbooks are provided without support and are intended to be a guideline. Any issues encountered can be reported via GitHub issues and will be addressed on a best effort basis.
- Confluent Platform 4.1 or higher
- Ansible 2.5.x or higher (on control node)
- passwordless ssh between all hosts
- sudo access for ssh user for all hosts
Ansible provides a simple way to deploy, manage, and configure the Confluent Platform services. This repository provides playbooks and templates to easily spin up a Confluent Platform installation. Specifically this repository:
- Installs Confluent Platform packages
- Starts services using systemd scripts
- Provides configuration options for plaintext, SSL, and SASL_SSL communication amongst the services
The services that can be installed from this repository are:
- ZooKeeper
- Kafka
- Schema Registry
- REST Proxy
- Confluent Control Center
- Kafka Connect (distributed mode)
These Ansible playbooks are intended as a general template for setting up a production-ready proof of concept environment. There are three available templates.
- PLAINTEXT -- use these templates if you have no requirements for a secured environment
- SSL -- use these templates if you require only SSL encryption
- SASL_SSL -- use these templates if you require plaintext SASL authentication and SSL encryption
The playbooks and templates generate self-signed certificates and use SASL plaintext authentication for a simple proof of concept that demonstrates the use of security features. The following are not in scope:
- Generating production-ready SSL certificates
- Setting up a Kerberos KDC or integration with Active Directory
- Guidance on business usage of authentication and authorization
- Provisioning of machines
Future work and additional features should be tracked in the GitHub issues for the repository.
PLAINTEXT, SSL, SASL_SSL each have example playbooks and hosts files in their respective plaintext
, ssl
, sasl_ssl
directories.
The hosts.yml
and all.yml
files at the repository root match the sasl_ssl
and this can be considered the default path.
Each service has three template properties files. Which properties file template will be used depends on the value of security_mode
set. Valid options are plaintext
, ssl
, sasl_ssl
.
These templates hardcode some security parameters for ease of setup in a proof of concept environment. If you want to use this setup in a production environment, you will likely want to use
SSL certificates provided by your own Certificate Authority and use Kerberos keytabs for SASL authentication.
The service properties file templates in this repository expect certificates to be stored under /var/ssl/private
. In this directory, each host stores a keystore and trustore for clients
and a keystore and truststore for Brokers. No differentiation between services is made for simplicity. You can update the exact path to the certificate stores by updating
roles/<service>/templates/<service>_ssl.properties.j2
or roles/<service>/templates/<service>_sasl_ssl.properties.j2
depending on the security mode you have chosen.
The service properties file templates in this repository use the plaintext SASL mechanism that is shipped with Kafka. The JAAS configurations are specified in the properties files directly, so
if you choose to use Kerberos keytabs for authentication, you can modify roles/<service>/templates/<service>_sasl_ssl.properties.j2
to use the SASL mechanism GSSAPI and update the JAAS
configuration. Please consult the security documentation for specific examples on updating the configuration.
ansible-playbook -i hosts.yml all.yml
ansible-playbook --check -i hosts.yml all.yml
ansible-playbook -i hosts.yml all.yml
This repository makes use of the systemd scripts provided in Confluent Platform. As such, there is an expected default user/service mapping that follows the convention of using the prefix
cp-
followed by the service name. For example cp-kafka
or cp-schema-registry
. The one exception is that ZooKeeper is run as the cp-kafka
user. This matches the systemd scripts as well.