Giter VIP home page Giter VIP logo

Comments (12)

StephenSorriaux avatar StephenSorriaux commented on September 17, 2024 1

Can you please retry with the 1.4.4 version of kafka-python as stated in the requirements.txt file?

from ansible-kafka-admin.

imvld avatar imvld commented on September 17, 2024 1
root@kafka:~# pip show kafka-python|grep -i version
Version: 1.4.4
  - pip:
      name: kafka-python
      version: 1.4.4
    tags: py

  - name: create acl
    kafka:
      resource: 'acl'
      api_version: "2.3.0"
      acl_resource_type: "topic"
      name: "*"
      acl_principal: "User:Anal"
      acl_operation: "all"
      acl_permission: "allow"
      state: "present"
      zookeeper: kafka.local:2181
      bootstrap_servers: kafka.local:9093
      security_protocol: 'SSL'
      ssl_check_hostname: true
      ssl_cafile: "{{ kafka.ssl.usersdir }}/ansible/CARoot.pem"
      ssl_certfile: "{{ kafka.users.ansible.cert }}"
      ssl_keyfile: "{{ kafka.users.ansible.key }}"
    tags: acl

thank you! it works )

root@kafka:~# /opt/kafka/kafka_2.12-2.3.0/bin/kafka-acls.sh --authorizer-properties zookeeper.connect=localhost:2181 --list|grep -i anal
	User:Anal has Allow permission for operations: All from hosts: * 

from ansible-kafka-admin.

StephenSorriaux avatar StephenSorriaux commented on September 17, 2024

Hello,

I personally use this library on a daily basis with Kafka in version 2.1.1 so I don’t think it is related.
How did you configure your Kafka broker (listeners, advertised address, etc.)?

from ansible-kafka-admin.

chris93111 avatar chris93111 commented on September 17, 2024

all is default but i have try with

listeners=PLAINTEXT://:$myip9092
advertised.listeners=PLAINTEXT://$myip:9092

from ansible-kafka-admin.

chris93111 avatar chris93111 commented on September 17, 2024

default is localhost and protocol PLAINTEXT

Zookeper and kakfka in same node

from ansible-kafka-admin.

ryarnyah avatar ryarnyah commented on September 17, 2024

Could you try adding:

import logging

log = logging.getLogger('kafka')
log.addHandler(logging.StreamHandler(sys.stdout))
log.setLevel(logging.INFO)

to get some logs?

from ansible-kafka-admin.

imvld avatar imvld commented on September 17, 2024

hello. i have same problem
kafka_2.12-2.3.0

ansible:

  - name: create acl
    kafka:
      resource: 'acl'
      api_version: "2.3.0"
      acl_resource_type: "topic"
      name: "*"
      acl_principal: "User:Anal"
      acl_operation: "all"
      acl_permission: "allow"
      state: "present"
      zookeeper: kafka.local:2181
      bootstrap_servers: kafka.local:9093
      security_protocol: 'SSL'
      ssl_check_hostname: true
      ssl_cafile: "{{ kafka.ssl.usersdir }}/ansible/CARoot.pem"
      ssl_certfile: "{{ kafka.users.ansible.cert }}"
      ssl_keyfile: "{{ kafka.users.ansible.key }}"
    tags: acl

ansibe

TASK [create acl] 
fatal: [kafka]: FAILED! => {"changed": false, "msg": "Cannot determine a controller for your current Kafka server. Is your Kafka server running and available on 'kafka.local:9093' with security protocol 'SSL'?"}
 

kafka:

listeners=SSL://kafka.local:9093
advertised.listeners=SSL://kafka.local:9093
advertised.host.name=kafka.local
ssl.keystore.location=/opt/kafka/ssl/kafka.server.keystore.jks
ssl.keystore.password=secret
ssl.key.password=secret
ssl.truststore.location=/opt/kafka/ssl/kafka.server.truststore.jks
ssl.truststore.password=secret
ssl.client.auth=required
listener.security.protocol.map=SSL:SSL
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
super.users=User:CN=kafka.local;User:kafka;User:CN=ansible
allow.everyone.if.no.acl.found=false
security.inter.broker.protocol=SSL

netstat:

root@kafka:~# netstat -tulpan | grep -i listen|grep -P '(2181|9093)'
tcp6       0      0 10.254.242.241:9093     :::*                    LISTEN      23950/java      
tcp6       0      0 :::2181                 :::*                    LISTEN      21589/java  
root@kafka:~# grep -i endpoint /opt/kafka/kafka_2.12-2.3.0/logs/server.log|tail -1
[2019-10-01 14:03:30,093] INFO Registered broker 0 at path /brokers/ids/0 with addresses: ArrayBuffer(EndPoint(kafka.local,9093,ListenerName(SSL),SSL)), czxid (broker epoch): 337 (kafka.zk.KafkaZkClient)

but with python i can connect to kafka

from ansible-kafka-admin.

StephenSorriaux avatar StephenSorriaux commented on September 17, 2024

Hi,

Which version of kafka-python are you using? Which version of the lib?

Can you please use the latest version of this lib (master or tag 0.8.0) and start your playbook in DEBUG mode: ANSIBLE_DEBUG=true ansible-playbook my-playbook.yml. Then, please copy and paste the line where kafka-python logs are present. This should be something like:

 13193 1569943505.41220: _low_level_execute_command() done: rc=0, stdout=Bootstrapping cluster metadata from [('localhost', 9092, <AddressFamily.AF_UNSPEC: 0>)]
<BrokerConnection node_id=bootstrap host=localhost:9092 <connecting> [IPv6 ('::1', 9092, 0, 0)]>: connecting to localhost:9092 [('::1', 9092, 0, 0) IPv6]
<BrokerConnection node_id=bootstrap host=localhost:9092 <connecting> [IPv6 ('::1', 9092, 0, 0)]>: Connection complete.
Bootstrap succeeded: found 1 brokers and 1 topics.
<BrokerConnection node_id=bootstrap host=localhost:9092 <connected> [IPv6 ('::1', 9092, 0, 0)]>: Closing connection.

{"changed": false, "msg": "topic 'test': nothing to do.", "invocation": {"module_args": {"resource": "topic", "api_version": "2.2.0", "name": "test", "partitions": 2, "replica_factor": 1, "options": {"retention.ms": 574930}, "state": "present", "zookeeper": "localhost:2181", "bootstrap_servers": "localhost:9092", "acl_resource_type": "topic", "acl_pattern_type": "literal", "acl_permission": "allow", "acl_host": "*", "zookeeper_auth_scheme": "digest", "zookeeper_auth_value": "", "zookeeper_ssl_check_hostname": true, "zookeeper_sleep_time": 5, "zookeeper_max_retries": 5, "security_protocol": "PLAINTEXT", "ssl_check_hostname": true, "sasl_mechanism": "PLAIN", "acl_principal": null, "acl_operation": null, "zookeeper_ssl_cafile": null, "zookeeper_ssl_certfile": null, "zookeeper_ssl_keyfile": null, "zookeeper_ssl_password": null, "ssl_cafile": null, "ssl_certfile": null, "ssl_keyfile": null, "ssl_password": null, "ssl_crlfile": null, "ssl_supported_protocols": null, "ssl_ciphers": null, "sasl_plain_username": null, "sasl_plain_password": null, "sasl_kerberos_service_name": null}}}

from ansible-kafka-admin.

imvld avatar imvld commented on September 17, 2024

I used latest version of this lib(0.8.0)

root@kafka:~# pip show kafka-python|grep -i version
Version: 1.4.7
TASK [create acl] ********************************************************************************************************************************************************************************************************************
fatal: [kafka]: FAILED! => {"changed": false, "msg": "Cannot determine a controller for your current Kafka server. Is your Kafka server running and available on 'kafka.local:9093' with security protocol 'SSL'?"}
[kafka] TASK: create acl (debug)> p result._result
{'_ansible_no_log': False,
 '_ansible_parsed': True,
 'changed': False,
 u'failed': True,
 u'invocation': {u'module_args': {u'acl_host': u'*',
                                  u'acl_operation': u'all',
                                  u'acl_pattern_type': u'literal',
                                  u'acl_permission': u'allow',
                                  u'acl_principal': u'User:Anal',
                                  u'acl_resource_type': u'topic',
                                  u'api_version': u'2.3.0',
                                  u'bootstrap_servers': u'kafka.local:9093',
                                  u'name': u'*',
                                  u'options': None,
                                  u'partitions': 0,
                                  u'replica_factor': 0,
                                  u'resource': u'acl',
                                  u'sasl_kerberos_service_name': None,
                                  u'sasl_mechanism': u'PLAIN',
                                  u'sasl_plain_password': None,
                                  u'sasl_plain_username': None,
                                  u'security_protocol': u'SSL',
                                  u'ssl_cafile': u'/opt/kafka/ssl/users/ansible/CARoot.pem',
                                  u'ssl_certfile': u'/opt/kafka/ssl/users/ansible/ansible.kafka.cert',
                                  u'ssl_check_hostname': True,
                                  u'ssl_ciphers': None,
                                  u'ssl_crlfile': None,
                                  u'ssl_keyfile': u'VALUE_SPECIFIED_IN_NO_LOG_PARAMETER',
                                  u'ssl_password': None,
                                  u'ssl_supported_protocols': None,
                                  u'state': u'present',
                                  u'zookeeper': u'kafka.local:2181',
                                  u'zookeeper_auth_scheme': u'digest',
                                  u'zookeeper_auth_value': u'',
                                  u'zookeeper_max_retries': 5,
                                  u'zookeeper_sleep_time': 5,
                                  u'zookeeper_ssl_cafile': None,
                                  u'zookeeper_ssl_certfile': None,
                                  u'zookeeper_ssl_check_hostname': True,
                                  u'zookeeper_ssl_keyfile': None,
                                  u'zookeeper_ssl_password': None}},
 u'msg': u"Cannot determine a controller for your current Kafka server. Is your Kafka server running and available on 'kafka.local:9093' with security protocol 'SSL'?"}
[kafka] TASK: create acl (debug)> 

from ansible-kafka-admin.

StephenSorriaux avatar StephenSorriaux commented on September 17, 2024

@imvld great, thanks for your feedback.
@chris93111 can you check your versions on your side?

from ansible-kafka-admin.

nkonopinski avatar nkonopinski commented on September 17, 2024

Downgrading kafka-python from 1.4.6 to 1.4.4 did not completely resolve the issue for me:

failed: [host -> 127.0.0.1] (item=topic) => {"ansible_loop_var": "item", "changed": false, "item": "topic", "msg": "Error while initializing Zookeeper client : init() got unexpected keyword arguments: ['certfile', 'ca', 'keyfile_password', 'use_ssl', 'keyfile', 'verify_certs']. Is your Zookeeper server available and running on 'host1:2181,host2:2181,host3:2181'?"}

But then I upgraded kazoo from 2.5.0 to 2.6.1 and it is working now. Guess I should have read requirements.txt as well

from ansible-kafka-admin.

StephenSorriaux avatar StephenSorriaux commented on September 17, 2024

@nkonopinski thanks for your feedback.

I opened #43 that would help in the future

from ansible-kafka-admin.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.