Comments (14)
Fobhep
commented on July 16, 2024
Ok - I could solve that issue by using python3.
Using python2 I could not even get that working:
$ python
>> import kafka.errors
no such module kafka.errors
from ansible-kafka-admin.
Fobhep
commented on July 16, 2024
feel free to close this issue if that is ok for you
from ansible-kafka-admin.
StephenSorriaux
commented on July 16, 2024
Hello,
It seems like a dependency problem to me, are you using a virtualenv to start Ansible?
from ansible-kafka-admin.
Fobhep
commented on July 16, 2024
yes I do now.
To get it running I had to install ansible also inside of the virtualenv besides the reqs of this repo in the virtualenv.
Using my system-wide ansible-installation did not work.
from ansible-kafka-admin.
StephenSorriaux
commented on July 16, 2024
Seems ok to me then, thank you for your confirmation.
from ansible-kafka-admin.
pgacek
commented on July 16, 2024
@Fobhep could share how you managed to fix the issue? I think I got a similar problem. I'm trying to run the playbook inside of the docker container with the ansible and get the same error. I tried with a few Python versions, this is how my Dockerfile and ansible-playbook looks:
FROM alpine:3.7
RUN apk add --no-cache python3 git ansible \
&& python3 -m ensurepip \
&& pip3 install --upgrade pip setuptools \
&& rm -r /usr/lib/python*/ensurepip && \
if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && \
rm -r /root/.cache
- name: Install pip modules
become: yes
pip:
name: "{{ item }}"
with_items:
- kafka-python>=2.0.2
- kazoo==2.6.1
- pure-sasl==0.5.1
- name: create topic
kafka_topic:
resource: "topic"
api_version: "{{ kafka_api_version }}"
name: "{{ item.name }}"
partitions: "{{ item.partitions | default('8') }}"
replica_factor: "{{ item.replica_factor | default('3') }}"
state: "present"
zookeeper: "{{ kafka_zookeeper_connection_string }}"
bootstrap_servers: "{{ kafka_bootstrap_brokers }}"
options:
compression.type: "{{ item.options_compression_type | default('producer') }}"
retention.ms: "{{ item.options_retention_ms | default('604800000') }}"
with_items:
- "{{ kafka_topics }}"
modules install correctly, but just after that a bunch of errors. I also installed the kafka package via the pip - didn't help
failed: [localhost] (item={u'name': u'aggregator-incoming-orders-events'}) => {"changed": false, "failed": true, "item": {"name": "aggregator-incoming-orders-events"}, "module_stderr": "Traceback (most recent call last):\n File \"/tmp/ansible_e0r7Bv/ansible_module_kafka_topic.py\", line 15, in <module>\n from ansible.module_utils.kafka_lib_topic import process_module_topic\n File \"/tmp/ansible_e0r7Bv/ansible_modlib.zip/ansible/module_utils/kafka_lib_topic.py\", line 4, in <module>\nImportError: No module named kafka.errors\n", "module_stdout": "", "msg": "MODULE FAILURE", "rc": 0}
from ansible-kafka-admin.
Fobhep
commented on July 16, 2024
As I can see you are installing ansible via package manager.
I created a dedicated virtualenv and installed inside that ansible.
I guess in your case where you don't seem to use a venv you could simply try to install ansible via pip3
from ansible-kafka-admin.
pgacek
commented on July 16, 2024
I changed the image and installed ansible via pip, as you suggested, but still get the same error message :(
FROM python:3.7.6-stretch
RUN pip install pip --upgrade
RUN pip install ansible
RUN apt-get update -y && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
sshpass
from ansible-kafka-admin.
Fobhep
commented on July 16, 2024
mmh - sorry then I don't really have other suggestions.
from ansible-kafka-admin.
pgacek
commented on July 16, 2024
Thanks a lot.
@StephenSorriaux maybe you would suggest something? :D
from ansible-kafka-admin.
StephenSorriaux
commented on July 16, 2024
Hey @pgacek
I think the become: yes may be the issue, since you are installing the libs for the root user. Do you still have the issue if you are removing the become: yes line or adding one to the call to create topic task?
- name: Install pip modules
become: yes
pip:
name: "{{ item }}"
with_items:
- kafka-python>=2.0.2
- kazoo==2.6.1
- pure-sasl==0.5.1
- name: create topic
kafka_topic:
resource: "topic"
api_version: "{{ kafka_api_version }}"
name: "{{ item.name }}"
partitions: "{{ item.partitions | default('8') }}"
replica_factor: "{{ item.replica_factor | default('3') }}"
state: "present"
zookeeper: "{{ kafka_zookeeper_connection_string }}"
bootstrap_servers: "{{ kafka_bootstrap_brokers }}"
options:
compression.type: "{{ item.options_compression_type | default('producer') }}"
retention.ms: "{{ item.options_retention_ms | default('604800000') }}"
with_items:
- "{{ kafka_topics }}"
from ansible-kafka-admin.
pgacek
commented on July 16, 2024
actually, I'm getting another error - syntax
TASK [create topic] **************************************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: SyntaxError: invalid syntax
failed: [localhost] (item={'name': 'aggregator-incoming-orders-events'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aggregator-incoming-orders-events"}, "module_stderr": "Traceback (most recent call last):\n File \"/root/.ansible/tmp/ansible-tmp-1620764755.0318098-98-39190393483405/AnsiballZ_kafka_topic.py\", line 102, in <module>\n _ansiballz_main()\n File \"/root/.ansible/tmp/ansible-tmp-1620764755.0318098-98-39190393483405/AnsiballZ_kafka_topic.py\", line 94, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/root/.ansible/tmp/ansible-tmp-1620764755.0318098-98-39190393483405/AnsiballZ_kafka_topic.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible.modules.kafka_topic', init_globals=None, run_name='__main__', alter_sys=True)\n File \"/usr/local/lib/python3.7/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/local/lib/python3.7/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/local/lib/python3.7/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_kafka_topic_payload_pyvv4v_9/ansible_kafka_topic_payload.zip/ansible/modules/kafka_topic.py\", line 15, in <module>\n File \"<frozen importlib._bootstrap>\", line 983, in _find_and_load\n File \"<frozen importlib._bootstrap>\", line 967, in _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line 668, in _load_unlocked\n File \"<frozen importlib._bootstrap>\", line 638, in _load_backward_compatible\n File \"/tmp/ansible_kafka_topic_payload_pyvv4v_9/ansible_kafka_topic_payload.zip/ansible/module_utils/kafka_lib_topic.py\", line 4, in <module>\n File \"/usr/local/lib/python3.7/site-packages/kafka/__init__.py\", line 23, in <module>\n from kafka.producer import KafkaProducer\n File \"/usr/local/lib/python3.7/site-packages/kafka/producer/__init__.py\", line 4, in <module>\n from .simple import SimpleProducer\n File \"/usr/local/lib/python3.7/site-packages/kafka/producer/simple.py\", line 54\n return '<SimpleProducer batch=%s>' % self.async\n ^\nSyntaxError: invalid syntax\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: SyntaxError: invalid syntax
failed: [localhost] (item={'name': 'aggregator-incoming-orders-events.dlt', 'replica_factor': 2, 'partitions': 1}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aggregator-incoming-orders-events.dlt", "partitions": 1, "replica_factor": 2}, "module_stderr": "Traceback (most recent call last):\n File \"/root/.ansible/tmp/ansible-tmp-1620764755.3576074-98-148721332064075/AnsiballZ_kafka_topic.py\", line 102, in <module>\n _ansiballz_main()\n File \"/root/.ansible/tmp/ansible-tmp-1620764755.3576074-98-148721332064075/AnsiballZ_kafka_topic.py\", line 94, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/root/.ansible/tmp/ansible-tmp-1620764755.3576074-98-148721332064075/AnsiballZ_kafka_topic.py\", line 40, in invoke_module\n runpy.run_module(mod_name='ansible.modules.kafka_topic', init_globals=None, run_name='__main__', alter_sys=True)\n File \"/usr/local/lib/python3.7/runpy.py\", line 205, in run_module\n return _run_module_code(code, init_globals, run_name, mod_spec)\n File \"/usr/local/lib/python3.7/runpy.py\", line 96, in _run_module_code\n mod_name, mod_spec, pkg_name, script_name)\n File \"/usr/local/lib/python3.7/runpy.py\", line 85, in _run_code\n exec(code, run_globals)\n File \"/tmp/ansible_kafka_topic_payload_s6t5jzfd/ansible_kafka_topic_payload.zip/ansible/modules/kafka_topic.py\", line 15, in <module>\n File \"<frozen importlib._bootstrap>\", line 983, in _find_and_load\n File \"<frozen importlib._bootstrap>\", line 967, in _find_and_load_unlocked\n File \"<frozen importlib._bootstrap>\", line 668, in _load_unlocked\n File \"<frozen importlib._bootstrap>\", line 638, in _load_backward_compatible\n File \"/tmp/ansible_kafka_topic_payload_s6t5jzfd/ansible_kafka_topic_payload.zip/ansible/module_utils/kafka_lib_topic.py\", line 4, in <module>\n File \"/usr/local/lib/python3.7/site-packages/kafka/__init__.py\", line 23, in <module>\n from kafka.producer import KafkaProducer\n File \"/usr/local/lib/python3.7/site-packages/kafka/producer/__init__.py\", line 4, in <module>\n from .simple import SimpleProducer\n File \"/usr/local/lib/python3.7/site-packages/kafka/producer/simple.py\", line 54\n return '<SimpleProducer batch=%s>' % self.async\n ^\nSyntaxError: invalid syntax\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: SyntaxError: invalid syntax
from ansible-kafka-admin.
StephenSorriaux
commented on July 16, 2024
I see, a SyntaxError is raised because of self.async in kafka-python, as async is a reserved keyword from Python 3.7 I think. But it seems like the version of kafka-python installed on the machine is not >=2.0.2 as this was changed in 1.4.3 release. Can you share the complete the playbook?
from ansible-kafka-admin.
ryarnyah
commented on July 16, 2024
@pgacek when you use python image you need to specify where python is located like in our tests: https://github.com/StephenSorriaux/ansible-kafka-admin/blob/master/molecule/default/molecule.yml#L238
If not you will can have 2 python version inside your image with one with kafka-python and the other no.
To be sure of which version your using, use /usr/local/bin/python -m pip install xxx when your calling pip.
from ansible-kafka-admin.
Related Issues (20)
- [Module] Add `kafka_users` module
- [Module] Unable to use ssl one way with zookeeper
- Output variable name difference for lib kafka_info is not optimal HOT 2
- UnboundLocalError: local variable 'manager' referenced before assignment while using SSL HOT 1
- Get unspecified Error Message on module kafka_info for topics HOT 3
- Unnecessary requirement for acl_operation in module kafka_acls HOT 1
- Unable to get Topic Information when connecting to a Cluster using SSL HOT 20
- spelling: Seomthing -> Something
- Question: How to add ACLs with transactional-ids? HOT 5
- kafka_topic: invalid check_mode behaviour when updating an existing topic HOT 2
- Enhancement: allow to define common configurations from env vars HOT 1
- Could not find imported module support code for kafka_users: Looked for ansible.module_utils.kafka_scram HOT 1
- kafka_user: Multiple mechanisms for password during user creation HOT 1
- Ansible-galaxy is not updated HOT 1
- Add publishing to Ansible Galaxy as collection HOT 4
- Missing asset in release 0.18.1 HOT 3
- Add quotas management for KRaft-based clusters HOT 2
- Consumer group ACL creation issue on 0.19.0 HOT 1
- outdated example HOT 1
- Add mTLS SSL support HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ansible-kafka-admin.