nickjj / ansigenome Goto Github PK
View Code? Open in Web Editor NEWA tool to help you gather information and manage your Ansible roles.
License: GNU General Public License v3.0
A tool to help you gather information and manage your Ansible roles.
License: GNU General Public License v3.0
I just installed ansigenome with pip install. Don't know if use it correctly, but it crash when I try to export a graph.
xxxx@xxxx:~/ansible/roles$ ansigenome export -o graph.png
Traceback (most recent call last):
File "/usr/local/bin/ansigenome", line 326, in
main()
File "/usr/local/bin/ansigenome", line 322, in main
fn(args, options, reloaded_config, parser)
File "/usr/local/bin/ansigenome", line 177, in execute_export
Scan(args, options, config, export=True)
File "/usr/local/lib/python2.7/dist-packages/ansigenome/scan.py", line 83, in init
self.scan_roles()
File "/usr/local/lib/python2.7/dist-packages/ansigenome/scan.py", line 117, in scan_roles
self.report["roles"][key] = self.report_role(key)
File "/usr/local/lib/python2.7/dist-packages/ansigenome/scan.py", line 190, in report_role
"meta": self.gather_meta(),
File "/usr/local/lib/python2.7/dist-packages/ansigenome/scan.py", line 213, in gather_meta
dep_list.append(dependency["role"])
TypeError: string indices must be integers, not str
I noticed when running this it did not report any defaults for my role however there are a number of defaults outside of the standard tasks folder
Hi,
would it be possible to track variables visually in a similar way?
we have 3 places from where a variable can be changed: role, group_vars and host_vars
Thanks!
now that I have somehow got it working, thanks ppl!
can we have multiple authors?
Hi, we use the latest ansigenome 0.6.0 at this point and we are getting into the habit of using our own templates.
The template is there in place, but the ansigenome gendoc cannot find it.
Traceback (most recent call last):
File "/usr/bin/ansigenome", line 326, in
main()
File "/usr/bin/ansigenome", line 322, in main
fn(args, options, reloaded_config, parser)
File "/usr/bin/ansigenome", line 161, in execute_gendoc
Scan(args, options, config, gendoc=True)
File "/usr/lib/python2.7/site-packages/ansigenome/scan.py", line 83, in init
self.scan_roles()
File "/usr/lib/python2.7/site-packages/ansigenome/scan.py", line 125, in scan_roles
self.write_readme(key)
File "/usr/lib/python2.7/site-packages/ansigenome/scan.py", line 435, in write_readme
j2_out = self.readme_template.render(self.readme_template_vars)
File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
reraise(exc_type, exc_value, tb)
File "", line 1, in top-level template code
File "/usr/lib/python2.7/site-packages/jinja2/loaders.py", line 286, in get_source
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: redacted_path/README.md.j2
Thank you!
If I use the command line to export roles dependencies into a png file, the bellow error is displayed:
$ ansigenome export ./git_repos/devops/roles/ -o ./git_repos/devops/roles/role_schema.png
Warning: <stdin>: syntax error in line 1 near 'No'
But it works if I go to role directory and execute the command like as bellow:
$ cd ./git_repos/devops/roles/ && ansigenome export -o role_schema.png
I attempted to run this on my repository to get a feel for what our graph looks like and it failed with the error message:
Traceback (most recent call last):
File "./ansigenome", line 326, in <module>
main()
File "./ansigenome", line 322, in main
fn(args, options, reloaded_config, parser)
File "./ansigenome", line 153, in execute_scan
Scan(args, options, config)
File "/home/kwoodson/git/ansigenome/ansigenome/scan.py", line 83, in __init__
self.scan_roles()
File "/home/kwoodson/git/ansigenome/ansigenome/scan.py", line 117, in scan_roles
self.report["roles"][key] = self.report_role(key)
File "/home/kwoodson/git/ansigenome/ansigenome/scan.py", line 185, in report_role
"total_lines": self.gather_lines(),
File "/home/kwoodson/git/ansigenome/ansigenome/scan.py", line 315, in gather_lines
with open(full_path, "r") as f:
IOError: [Errno 40] Too many levels of symbolic links: '/home/kwoodson/git/openshift-ansible/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks/roles/openshift_certificate_expiry/examples/playbooks'
It looks like it started doing a recursive decent and was broken by the symlinks.
Possible fixes:
If I can find some time I'll look at submitting a patch.
Pick a license type by choosing a number:
1. MIT
2. GPLv2
3. GPLv3
5. LGPL
6. Apache-2.0
7. BSDv2
8. BSDv3
9. Other
[]: 9
Traceback (most recent call last):
File "/usr/local/bin/ansigenome", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/Users/kgarland/Code/ansigenome/bin/ansigenome", line 326, in <module>
main()
File "/Users/kgarland/Code/ansigenome/bin/ansigenome", line 309, in main
(config, config_path) = load_config()
File "/Users/kgarland/Code/ansigenome/bin/ansigenome", line 248, in load_config
Config([], {}, {})
File "/Users/kgarland/Code/ansigenome/ansigenome/config.py", line 28, in __init__
out_config = self.ask_questions()
File "/Users/kgarland/Code/ansigenome/ansigenome/config.py", line 77, in ask_questions
out_config["license_type"] = c.LICENSE_TYPES[answer][0]
IndexError: list index out of range
When trying to use Ansigenome with Ansible 2.0 i get this error:
ERROR! 'ansigenome_info' is not a valid attribute for a RoleMetadata
I checked the MetaData class in ansible 2.0 and it seems to be more strict than earlier.
https://github.com/ansible/ansible/blob/devel/lib/ansible/playbook/role/metadata.py
Do you have any idea how this could be solved? I volunteer to help fix it, but I'm unsure how to proceed.
Would be nice to also create the "files"-folder when using ´ansigenome init´.
ansible-galaxy init
also creates this directory.
Hi,
this very nice tool we are adopting for some time ad try to fully automate the generation based on commits (not only precommit) (gitlab runner, jenkins job) , now gets more attention, in this phase we were wondering if the templates can be included from an online repo?
Thanks!
I already mentioned this in #47 (comment)
86a62d4 broke role.galaxy_name
and with this also the links based on it.
I did not look more closely what the patch actually does.
Here is a mini working example using this template:
{{ role.galaxy_name | replace('-', '--')}}
{{ role.name | replace('-', '--')}}
And this role as an example: https://github.com/debops-contrib/ansible-apparmor
With 86a62d4, this generates the following:
debops--contrib.debops--contrib.apparmor
debops--contrib.apparmor
And with eb85483 (one commit before), it generates this:
debops--contrib.apparmor
apparmor
cc: @jimbocoder
PS: Might come in handy, I wrote a little wrapper script for ansigenome gendoc
to handle multiple environments: https://github.com/ypid/ypid-ansible-common/tree/master/template_READMEs
Is there support for Python2.6 planned? I want to use ansigenome on RHEL6 and Python 2.6 is the default there.
Currently, it crashes with the following exception, as Python 2.6 doesn't support dict comprehensions (I guess).
Traceback (most recent call last):
File "/usr/bin/ansigenome", line 5, in <module>
pkg_resources.run_script('ansigenome==0.5.1', 'ansigenome')
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 461, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 1194, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/lib/python2.6/site-packages/ansigenome-0.5.1-py2.6.egg/EGG-INFO/scripts/ansigenome", line 8, in <module>
import ansigenome.constants as c
File "/usr/lib/python2.6/site-packages/ansigenome-0.5.1-py2.6.egg/ansigenome/constants.py", line 4, in <module>
import utils
File "/usr/lib/python2.6/site-packages/ansigenome-0.5.1-py2.6.egg/ansigenome/utils.py", line 128
return {v[key]: v for v in items}.values()
^
SyntaxError: invalid syntax
Hi, very useful tool,
we finally managed to have a PoC of ansigenome working on my local mac, with ansigenome 0.5.6 from ypid. ANd it works ok with the meta/ansigenome.yml a custom template in md format.
The same role and the same ansigenome.conf used on a Linux host with ansigenome 0.6.0 installed by pip ignore anything in the ansigenome.yml.
I can provide logs if instructed how to.
Thanks!
roles without meta/main.yml files seem to produce spurious dependencies. I think this is due to a mistake in scan.py around line 202:
It would be nice to include all the required configuration and templates into the role itself so that everyone can regenerate the readme.
Related to: weareinteractive/gem-ansible-role#2
How hard would it be to bend this fantastic tool into a dynamic machine graph (via gather_facts)?
My usecase would be to have ansigenome assess the "ansible coverage" over a bunch of machines. That is, how many machines have been "ansiblized" using roles vs those that were installed manually (legacy manual installations).
Like regular developers do with test coverage.
It's mentioned in the readme that ansigenome
supports custom templates for ./meta/main.yml
. However I have failed to find a configuration option to plug-in a custom meta files path.
There used to be a pluggable template till v0.2.0
https://pypi.org/project/ansigenome/0.2.0/
Hi,
I'm currently using Molecule. It creates a molecule
sub-folder in each role, in which we can find some playbooks unrelated to the functioning of the role. Ansigenome scans this directory and finds some variables that are listed as "internal variables".
Would it be possible to add the ability to ignore some role's sub-folders to Ansigenome? It would allow me to ignore the molecule sub-folder and prevent useless variables to be listed in the generated README.
Regards
Hi
Great project. Is it possible to specify the path to the readme template as follows, where ~
would be expanded as $HOME
?
options_readme_template: ~/.ansigenome/ypid.README.rst.j2
Currently, ansigenome says:
The following template could not be found:
~/.ansigenome/README.rst.j2
or
The following template could not be found:
$HOME/.ansigenome/README.rst.j2
Example: https://github.com/debops/ansible-apt#role-dependencies
debops.etc_services
could link to https://github.com/debops/ansible-etc_services
Please fix/handle Python 3 incompatibility.
When I tried to install, I got this:
halis@eleanor ~ (master) $ pip install ansigenome
Collecting ansigenome
Using cached ansigenome-0.5.6.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/tmp/pip-build-xng2b7ao/ansigenome/setup.py", line 8
print "Ansigenome needs setuptools in order to build. " + \
^
SyntaxError: Missing parentheses in call to 'print'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-xng2b7ao/ansigenome
Some version info:
halis@eleanor ~ (master) $ pip -V
pip 7.1.2 from /usr/lib64/python3.4/site-packages (python 3.4)
halis@eleanor ~ (master) $ python
Python 3.4.3 (default, Nov 11 2015, 11:23:26)
[GCC 4.9.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
Picking licenses and providing URLs for them is hard.
Show an user a list of known popular licenses - MIT, BSD, Apache 2.0, GPLv2, GPLv3, other (to specify name and URL). If a user selects one of the known licenses, configure it with an URL to http://tldrlegal.com/.
Hi,
The picture will likely speak for itself:
The roles are not connected to one another and instead of having a graph where everything starts from the comb/
role and then connects to one another, ansigenome
seems to miss that the roles should indeed all be connected.
Could you point me into the direction to fix it, please?
Hi
I just noticed that Ansigenome missed packages__whitelist_by_vars_combined
which is used in the same set_fact
tasks as packages__blacklist_by_vars_combined
(is which is listed as expected). Without looking into the code I would expected that Ansigenome only finds the first occurrence of variables in set_fact
tasks?
Currently, running ansigenome scan still marks roles with markdown readmes (named README.md) as missing readmes, even if these readmes are generated by ansigenome itself.
Are there any requirements for what ansigenome expects when does a scan? I ask because I ran the following:
ansigenome scan directory-here/
ansigenome export -o /tmp/out.png
where directory-here contains my yaml playbooks and other include files that those yaml files refer to.
However, when I open /tmp/out.png, it just shows what seem to be a series of unconnected lines, laid out sequentially in a row. When I zoom in, it doesn't show me much; I can't make out anything for each line. I expected to see a graph of relationships, much like is shown in the README file on this github site.
I thought the reason I'm not seeing anything may have been because I have yaml files that are essentially just dictionary definitions or other files that are referred to, i.e. included, in the actual playbooks that have roles and hosts defined, i.e. that have a roles: and a hosts: section in the .yml file. So I copied to a subdirectory, and ran a scan on that subdirectory, but that doesn't even find any roles.
What do I need to do to see a graph of relationships like shown in the README?
Use case:
You have 80 roles and want to keep track of the versions. Versioning the requirements file by hand is tedious.
Solution:
Each role has a VERSION file which is a git tag. When you run the reqs
command with -v|--version
it will read in that version and write it out to the requirements file.
Roles that are unversioned by either an empty VERSION file or no VERSION file will be written out as master
for the version.
we have roles that start with xx-role-name
when we use gendoc the name of the role in the readme is only -role-name
When running ansigenome export
on my project I get:
contains 1 or more syntax errors:
could not determine a constructor for the tag '!vault'
in "<string>", line 15, column 22:
foo: !vault |
ansigenome
needs to tell PyYAML how to load them, like https://github.com/ansible/ansible/blob/e241e15899d864fdad67e198c0bce3457622a01c/lib/ansible/parsing/yaml/objects.py#L73
I've installed ansigenome
in virtual env running Python 2.7.3. After running ansigenome config
I got following traceback:
Traceback (most recent call last):
File "/home/scibi/.virtualenvs/ansible_xxxxx/bin/ansigenome", line 326, in <module>
main()
File "/home/scibi/.virtualenvs/ansible_xxxxx/bin/ansigenome", line 309, in main
(config, config_path) = load_config()
File "/home/scibi/.virtualenvs/ansible_xxxxx/bin/ansigenome", line 248, in load_config
Config([], {}, {})
File "/home/scibi/.virtualenvs/ansible_xxxxx/local/lib/python2.7/site-packages/ansigenome/config.py", line 29, in __init__
utils.write_config(self.config_path, out_config)
File "/home/scibi/.virtualenvs/ansible_xxxxx/local/lib/python2.7/site-packages/ansigenome/utils.py", line 398, in write_config
string_to_file(path, config_as_string)
File "/home/scibi/.virtualenvs/ansible_xxxxx/local/lib/python2.7/site-packages/ansigenome/utils.py", line 50, in string_to_file
file.write(input)
File "/home/scibi/.virtualenvs/ansible_xxxxx/lib/python2.7/codecs.py", line 691, in write
return self.writer.write(data)
File "/home/scibi/.virtualenvs/ansible_xxxxx/lib/python2.7/codecs.py", line 351, in write
data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 81: ordinal not in range(128)
The cause of that was letter 'Ś' in my name. Replacing it with 'S' "solved" the issue.
Hi, we would like ansignenome to automatically be called when we commit for example.
Are there any automation already done? Thank you!
@drybjed "Move[d] [all] ansigenome metadata to a separate file " some time ago. Reading thought the Readme, one gets the idea to put everything into meta/main.yml
. As I know @drybjed by now, there is usually a good reason to do something a certain way. What is the best-practice here and why?
Hi Nick,
Thanks for putting together ansigenome, which I found at https://github.com/nickjj/ansigenome. It seems to be what I’m looking for, i.e. a tool to graphically show me what’s in my playbooks. However, I’m having issues getting it to install, and am wondering if you have thoughts on what’s wrong. Note that setuptools is already installed, so not sure why it’s giving that message.
On MacOS 10.11.6, python 3.6.5:
$ pip3 install ansigenome
Collecting ansigenome
Using cached https://files.pythonhosted.org/packages/68/82/f41ce20b98bd91e64ea49aee3660516c4d40f0a09ffa2c2932d4552c44c8/ansigenome-0.6.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/private/var/folders/bh/7zd_g3hd4b7bkv2hf2_cnmwsf0zxn9/T/pip-install-9h5i5f1r/ansigenome/setup.py", line 8
print "Ansigenome needs setuptools in order to build. " +
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Ansigenome needs setuptools in order to build. " + )?
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/bh/7zd_g3hd4b7bkv2hf2_cnmwsf0zxn9/T/pip-install-9h5i5f1r/ansigenome/
On RHEL 6.7, python 3.4.2:
$ dzdo pip3.4 install ansigenome
Downloading/unpacking ansigenome
Downloading ansigenome-0.6.0.tar.gz (863kB): 863kB downloaded
Running setup.py (path:/tmp/pip_build_root/ansigenome/setup.py) egg_info for package ansigenome
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip_build_root/ansigenome/setup.py", line 8
print "Ansigenome needs setuptools in order to build. " +
^
SyntaxError: Missing parentheses in call to 'print'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 17, in
File "/tmp/pip_build_root/ansigenome/setup.py", line 8
print "Ansigenome needs setuptools in order to build. " + \
^
SyntaxError: Missing parentheses in call to 'print'
Cleaning up...
On MacOS, installing from source:
$ sudo python3 setup.py develop
Password:
File "setup.py", line 8
print "Ansigenome needs setuptools in order to build. " +
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Ansigenome needs setuptools in order to build. " + )?
$ pip3 install setuptools
Requirement already satisfied: setuptools in /usr/local/lib/python3.6/site-packages (39.0.1)
As this tool is tightly related to Ansible and folks using Ansigenome are probably very familiar with Ansible’s custom Jinja filters I think it would come in handy to support them. From what I saw when I last checked, Ansible has all the filters defined in one Python module. I played with that for a different project but Ansible has quite some dependencies. So not too easy unfortunately.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.