Comments (12)
Can you please retry with the 1.4.4
version of kafka-python
as stated in the requirements.txt
file?
from ansible-kafka-admin.
root@kafka:~# pip show kafka-python|grep -i version
Version: 1.4.4
- pip:
name: kafka-python
version: 1.4.4
tags: py
- name: create acl
kafka:
resource: 'acl'
api_version: "2.3.0"
acl_resource_type: "topic"
name: "*"
acl_principal: "User:Anal"
acl_operation: "all"
acl_permission: "allow"
state: "present"
zookeeper: kafka.local:2181
bootstrap_servers: kafka.local:9093
security_protocol: 'SSL'
ssl_check_hostname: true
ssl_cafile: "{{ kafka.ssl.usersdir }}/ansible/CARoot.pem"
ssl_certfile: "{{ kafka.users.ansible.cert }}"
ssl_keyfile: "{{ kafka.users.ansible.key }}"
tags: acl
thank you! it works )
root@kafka:~# /opt/kafka/kafka_2.12-2.3.0/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --list|grep -i anal
User:Anal has Allow permission for operations: All from hosts: *
from ansible-kafka-admin.
Hello,
I personally use this library on a daily basis with Kafka in version 2.1.1 so I donβt think it is related.
How did you configure your Kafka broker (listeners, advertised address, etc.)?
from ansible-kafka-admin.
all is default but i have try with
listeners=PLAINTEXT://:$myip9092
advertised.listeners=PLAINTEXT://$myip:9092
from ansible-kafka-admin.
default is localhost and protocol PLAINTEXT
Zookeper and kakfka in same node
from ansible-kafka-admin.
Could you try adding:
import logging
log = logging.getLogger('kafka')
log.addHandler(logging.StreamHandler(sys.stdout))
log.setLevel(logging.INFO)
to get some logs?
from ansible-kafka-admin.
hello. i have same problem
kafka_2.12-2.3.0
ansible:
- name: create acl
kafka:
resource: 'acl'
api_version: "2.3.0"
acl_resource_type: "topic"
name: "*"
acl_principal: "User:Anal"
acl_operation: "all"
acl_permission: "allow"
state: "present"
zookeeper: kafka.local:2181
bootstrap_servers: kafka.local:9093
security_protocol: 'SSL'
ssl_check_hostname: true
ssl_cafile: "{{ kafka.ssl.usersdir }}/ansible/CARoot.pem"
ssl_certfile: "{{ kafka.users.ansible.cert }}"
ssl_keyfile: "{{ kafka.users.ansible.key }}"
tags: acl
ansibe
TASK [create acl]
fatal: [kafka]: FAILED! => {"changed": false, "msg": "Cannot determine a controller for your current Kafka server. Is your Kafka server running and available on 'kafka.local:9093' with security protocol 'SSL'?"}
kafka:
listeners=SSL://kafka.local:9093
advertised.listeners=SSL://kafka.local:9093
advertised.host.name=kafka.local
ssl.keystore.location=/opt/kafka/ssl/kafka.server.keystore.jks
ssl.keystore.password=secret
ssl.key.password=secret
ssl.truststore.location=/opt/kafka/ssl/kafka.server.truststore.jks
ssl.truststore.password=secret
ssl.client.auth=required
listener.security.protocol.map=SSL:SSL
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:CN=kafka.local;User:kafka;User:CN=ansible
allow.everyone.if.no.acl.found=false
security.inter.broker.protocol=SSL
netstat:
root@kafka:~# netstat -tulpan | grep -i listen|grep -P '(2181|9093)'
tcp6 0 0 10.254.242.241:9093 :::* LISTEN 23950/java
tcp6 0 0 :::2181 :::* LISTEN 21589/java
root@kafka:~# grep -i endpoint /opt/kafka/kafka_2.12-2.3.0/logs/server.log|tail -1
[2019-10-01 14:03:30,093] INFO Registered broker 0 at path /brokers/ids/0 with addresses: ArrayBuffer(EndPoint(kafka.local,9093,ListenerName(SSL),SSL)), czxid (broker epoch): 337 (kafka.zk.KafkaZkClient)
but with python i can connect to kafka
from ansible-kafka-admin.
Hi,
Which version of kafka-python
are you using? Which version of the lib?
Can you please use the latest version of this lib (master
or tag 0.8.0
) and start your playbook in DEBUG
mode: ANSIBLE_DEBUG=true ansible-playbook my-playbook.yml
. Then, please copy and paste the line where kafka-python
logs are present. This should be something like:
13193 1569943505.41220: _low_level_execute_command() done: rc=0, stdout=Bootstrapping cluster metadata from [('localhost', 9092, <AddressFamily.AF_UNSPEC: 0>)]
<BrokerConnection node_id=bootstrap host=localhost:9092 <connecting> [IPv6 ('::1', 9092, 0, 0)]>: connecting to localhost:9092 [('::1', 9092, 0, 0) IPv6]
<BrokerConnection node_id=bootstrap host=localhost:9092 <connecting> [IPv6 ('::1', 9092, 0, 0)]>: Connection complete.
Bootstrap succeeded: found 1 brokers and 1 topics.
<BrokerConnection node_id=bootstrap host=localhost:9092 <connected> [IPv6 ('::1', 9092, 0, 0)]>: Closing connection.
{"changed": false, "msg": "topic 'test': nothing to do.", "invocation": {"module_args": {"resource": "topic", "api_version": "2.2.0", "name": "test", "partitions": 2, "replica_factor": 1, "options": {"retention.ms": 574930}, "state": "present", "zookeeper": "localhost:2181", "bootstrap_servers": "localhost:9092", "acl_resource_type": "topic", "acl_pattern_type": "literal", "acl_permission": "allow", "acl_host": "*", "zookeeper_auth_scheme": "digest", "zookeeper_auth_value": "", "zookeeper_ssl_check_hostname": true, "zookeeper_sleep_time": 5, "zookeeper_max_retries": 5, "security_protocol": "PLAINTEXT", "ssl_check_hostname": true, "sasl_mechanism": "PLAIN", "acl_principal": null, "acl_operation": null, "zookeeper_ssl_cafile": null, "zookeeper_ssl_certfile": null, "zookeeper_ssl_keyfile": null, "zookeeper_ssl_password": null, "ssl_cafile": null, "ssl_certfile": null, "ssl_keyfile": null, "ssl_password": null, "ssl_crlfile": null, "ssl_supported_protocols": null, "ssl_ciphers": null, "sasl_plain_username": null, "sasl_plain_password": null, "sasl_kerberos_service_name": null}}}
from ansible-kafka-admin.
I used latest version of this lib(0.8.0)
root@kafka:~# pip show kafka-python|grep -i version
Version: 1.4.7
TASK [create acl] ********************************************************************************************************************************************************************************************************************
fatal: [kafka]: FAILED! => {"changed": false, "msg": "Cannot determine a controller for your current Kafka server. Is your Kafka server running and available on 'kafka.local:9093' with security protocol 'SSL'?"}
[kafka] TASK: create acl (debug)> p result._result
{'_ansible_no_log': False,
'_ansible_parsed': True,
'changed': False,
u'failed': True,
u'invocation': {u'module_args': {u'acl_host': u'*',
u'acl_operation': u'all',
u'acl_pattern_type': u'literal',
u'acl_permission': u'allow',
u'acl_principal': u'User:Anal',
u'acl_resource_type': u'topic',
u'api_version': u'2.3.0',
u'bootstrap_servers': u'kafka.local:9093',
u'name': u'*',
u'options': None,
u'partitions': 0,
u'replica_factor': 0,
u'resource': u'acl',
u'sasl_kerberos_service_name': None,
u'sasl_mechanism': u'PLAIN',
u'sasl_plain_password': None,
u'sasl_plain_username': None,
u'security_protocol': u'SSL',
u'ssl_cafile': u'/opt/kafka/ssl/users/ansible/CARoot.pem',
u'ssl_certfile': u'/opt/kafka/ssl/users/ansible/ansible.kafka.cert',
u'ssl_check_hostname': True,
u'ssl_ciphers': None,
u'ssl_crlfile': None,
u'ssl_keyfile': u'VALUE_SPECIFIED_IN_NO_LOG_PARAMETER',
u'ssl_password': None,
u'ssl_supported_protocols': None,
u'state': u'present',
u'zookeeper': u'kafka.local:2181',
u'zookeeper_auth_scheme': u'digest',
u'zookeeper_auth_value': u'',
u'zookeeper_max_retries': 5,
u'zookeeper_sleep_time': 5,
u'zookeeper_ssl_cafile': None,
u'zookeeper_ssl_certfile': None,
u'zookeeper_ssl_check_hostname': True,
u'zookeeper_ssl_keyfile': None,
u'zookeeper_ssl_password': None}},
u'msg': u"Cannot determine a controller for your current Kafka server. Is your Kafka server running and available on 'kafka.local:9093' with security protocol 'SSL'?"}
[kafka] TASK: create acl (debug)>
from ansible-kafka-admin.
@imvld great, thanks for your feedback.
@chris93111 can you check your versions on your side?
from ansible-kafka-admin.
Downgrading kafka-python from 1.4.6 to 1.4.4 did not completely resolve the issue for me:
failed: [host -> 127.0.0.1] (item=topic) => {"ansible_loop_var": "item", "changed": false, "item": "topic", "msg": "Error while initializing Zookeeper client : init() got unexpected keyword arguments: ['certfile', 'ca', 'keyfile_password', 'use_ssl', 'keyfile', 'verify_certs']. Is your Zookeeper server available and running on 'host1:2181,host2:2181,host3:2181'?"}
But then I upgraded kazoo from 2.5.0 to 2.6.1 and it is working now. Guess I should have read requirements.txt as well
from ansible-kafka-admin.
@nkonopinski thanks for your feedback.
I opened #43 that would help in the future
from ansible-kafka-admin.
Related Issues (20)
- [Module] Add `kafka_users` module
- [Module] Unable to use ssl one way with zookeeper
- Output variable name difference for lib kafka_info is not optimal HOT 2
- UnboundLocalError: local variable 'manager' referenced before assignment while using SSL HOT 1
- Get unspecified Error Message on module kafka_info for topics HOT 3
- Unnecessary requirement for acl_operation in module kafka_acls HOT 1
- Unable to get Topic Information when connecting to a Cluster using SSL HOT 20
- spelling: Seomthing -> Something
- Question: How to add ACLs with transactional-ids? HOT 5
- kafka_topic: invalid check_mode behaviour when updating an existing topic HOT 2
- Enhancement: allow to define common configurations from env vars HOT 1
- Could not find imported module support code for kafka_users: Looked for ansible.module_utils.kafka_scram HOT 1
- kafka_user: Multiple mechanisms for password during user creation HOT 1
- Ansible-galaxy is not updated HOT 1
- Add publishing to Ansible Galaxy as collection HOT 4
- Missing asset in release 0.18.1 HOT 3
- Add quotas management for KRaft-based clusters HOT 2
- Consumer group ACL creation issue on 0.19.0 HOT 1
- outdated example HOT 1
- Add mTLS SSL support HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ansible-kafka-admin.