batfish / ansible Goto Github PK

Snapshot of a Batfish Ansible integration from 2020

License: Apache License 2.0

Python 95.98% Shell 4.02%

ansible's Introduction

Got questions, feedback, or feature requests? Join our community on Slack!

What is Batfish?

Batfish is a network validation tool that provides correctness guarantees for security, reliability, and compliance by analyzing the configuration of network devices. It builds complete models of network behavior from device configurations and finds violations of network policies (built-in, user-defined, and best-practices).

A primary use case for Batfish is to validate configuration changes before deployment (though it can be used to validate deployed configurations as well). Pre-deployment validation is a critical gap in existing network automation workflows. By including Batfish in automation workflows, network engineers can close this gap and ensure that only correct changes are deployed.

Batfish does NOT require direct access to network devices. The core analysis requires only the configuration of network devices. This analysis may be enhanced using additional information from the network such as:

BGP routes received from external peers
Topology information represented by LLDP/CDP

See www.batfish.org for technical information on how it works.

What kinds of correctness checks does Batfish support?

The Batfish YouTube channel (subscribe!) and Python notebooks illustrate many checks. Batfish checks span a range of network behaviors.

Configuration Compliance

Flag undefined-but-referenced or defined-but-unreferenced structures (e.g., ACLs, route maps)
Configuration settings for MTUs, AAA, NTP, logging, etc. match templates
Devices can only be accessed using SSHv2 and password is not null

Reliability

End-to-end reachability is not impacted for any flow after any single-link or single-device failure
Certain services (e.g., DNS) are globally reachable

Security

Sensitive services can be reached only from specific subnets or devices
Paths between endpoints are as expected (e.g., traverse a firewall, have at least 2 way ECMP, etc...)

Change Analysis

End-to-end reachability is identical across the current and a planned configuration
Planned ACL or firewall changes are provably correct and causes no collateral damage for other traffic
Two configurations, potentially from different vendors, are functionally equivalent

How do I get started?

1. Run the Batfish service

Getting started with Batfish is easy. Just pull and run the latest allinone Docker container that includes Batfish as well as example Jupyter notebooks.

docker pull batfish/allinone

docker run --name batfish -v batfish-data:/data -p 8888:8888 -p 9997:9997 -p 9996:9996 batfish/allinone

The second command starts the Batfish service and maps the necessary TCP ports.

Advanced Docker configuration:

The amount of memory available to Batfish is determined by the Docker configuration. You may wish to supply the --memory command-line argument to explicitly set this value.

On Linux systems that run the OOM Killer, you may also wish to supply the --oom-kill-disable argument, which runs in conjunction with the --memory argument to prevent Linux from killing Batfish when there is memory pressure on the system.

2. Browse example notebooks (optional)

If you are new to Batfish, consider walking through our notebooks which highlight different capabilities and use cases of Batfish. Point your browser to http://localhost:8888, and in the Password or token: prompt, enter the token that Jupyter showed when you ran the container (e.g. token=abcdef123456...).

Jupyter will show you the list of available notebooks. "Getting Started with Batfish" is a good one to start with. This README explains what each notebook does.

3. Install Pybatfish

To analyze your network configurations, you also need Pybatfish, a Python 3 SDK to interact with the Batfish service. Though not strictly necessary, we recommend that you install Pybatfish in a virtual environment.

To install Pybatfish run the following commands (in a virtual environment if applicable):

python3 -m pip install --upgrade pybatfish

4. Develop your analysis

After installing Pybatfish, use your Python environment of choice (e.g., PyCharm, interactive Python shell, Jupyter, ..) to interact with Batfish. The notebooks provide examples of such scripts.

See complete documentation of Pybatfish on readthedocs.

System Requirements for running Batfish

Batfish can be run on any operating system that supports Docker. The containers are actively tested on Mac OS X and Ubuntu 16.04 LTS.

To get started with the example Jupyter notebooks, all you need is a reasonably capable laptop:

Dual core CPU
8 GB RAM
256 GB hard-drive

When you transition to running Batfish on your own network, we recommend a server that at least has:

Quad-core CPU with 2 threads per CPU
32 GB RAM
256 GB hard-drive

Supported Network Device and Operating System List

Batfish supports configurations for a large and growing set of (physical and virtual) devices, including:

A10 Networks
Arista
AWS (VPCs, Network ACLs, VPN GW, NAT GW, Internet GW, Security Groups, etc…)
Cisco (All Cisco NX-OS, IOS, IOS-XE, IOS-XR and ASA devices)
Check Point
Cumulus
F5 BIG-IP
Fortinet
Free-Range Routing (FRR)
iptables (on hosts)
Juniper (All JunOS platforms: MX, EX, QFX, SRX, T-series, PTX)
Palo Alto Networks
SONiC

Batfish has limited support for the following platforms:

Aruba
Dell Force10
Foundry

If you'd like support for additional vendors or currently-unsupported configuration features, let us know via Slack or GitHub. We'll try to add support. Or, you can — we welcome pull requests! :)

License and Dependencies

Batfish is released under The Apache Software License, Version 2.0. All third-party dependencies are compatible with this licensing.

ansible's People

Contributors

Stargazers

Watchers

Forkers

saparikh marihanguirguis t-umeno th3architect jakezapper20 utkuozsan adirir nu107 vasu018 nitinaare hgade roare jvanderaa

ansible's Issues

Assertion doesn't handle failed check

The questions underlying Batfish assertions can fail, in which case the Answer returned will not be a TableAnswer. In at least one case, the check is not handling this case well: instead of getting the error, we get:

fatal: [localhost]: FAILED! => {"changed": false, "msg": "1 of 1 assertions failed", "result": [{"details": "'Answer' object has no attribute 'frame'", "name": "Confirm routers can ping each other", "status": "Error", "type": "assert_all_flows_succeed"}], "summary": "1 of 1 assertions failed"}

(see: https://networktocode.slack.com/archives/CCE02JK7T/p1582840013048800)

Batfish should always check the answer type before calling .frame().

Allow an option for strict or loose checking

In most production environments, there will be some config drift that aren't easily fixable, or in some network OSes, they sometimes change what would be an ordered list into an unordered list. While ordered lists are critical for things like ACLs, they can be seen as minor features for things like DNS or NTP servers. As such, an option to allow for loose ordering can be useful so we don't get false positives like this:

ok: [localhost] => {
    "msg": {
        "changed": false,
        "failed": true,
        "msg": "Validation failed for the following nodes: ['veos1'].",
        "result": {
            "veos1": {
                "NTP.NTP_Servers": {
                    "actual": [
                        "10.0.0.1",
                        "time-a-g.nist.gov",
                        "time-e-b.nist.gov"
                    ],
                    "expected": [
                        "time-a-g.nist.gov",
                        "10.0.0.1",
                        "time-e-b.nist.gov"
                    ]
                }
            }
        },
        "summary": "Validation failed for the following nodes: ['veos1']."
    }
}

I see something like this being definied in the category of a validation yml file like:

nodes:
  veos1:
    DNS:
      **non-strict**
      DNS_Servers:
        - 192.168.49.20
        - 8.8.8.8
    NTP:
      NTP_Servers:
        **non-strict**
        - 10.0.0.1
        - time-a-g.nist.gov
        - time-e-b.nist.gov

version: batfish_v0

Feature Request: Please allow validations lists to have wildcards

Right now, if you want to validate configurations on a bunch of devices, they need to be specified either on their own yml or in a master yml file. What would be very useful would be to have an option to use wildcards with device names to get a bunch of devices validated with the same dataset.

Example:

Nodes:
  *br*:
     DNS:
         DNS_Servers: [8.8.8.8]

Would cover any devices that are named:

us-sea-br1
us-sea-br2

but wouldn't cover devices that are named:

us-sea-core1
us-sea-access1

Use Pybatfish fact logic

Pybatfish now supports extracting and validating facts. Should use that logic instead of relying on duplicate logic in this role.

no option to input port # in bf_session

We have batfish deployed in K8 cluster with non-default ports, and require ansible playbooks to connect from other various hosts to batfish (not localhost)

bf_session script only takes host and name as input, would be useful if we can provide port # as well

Python 2.7 signature

Need a Python 2.7 workaround for inspect.signature.

This may help: https://stackoverflow.com/questions/45946051/signature-method-in-inspect-module-for-python-2

expose assert no loop

traceback when run in python3

Getting python traceback as following:

{ "module_stdout": "", "module_stderr": "Traceback (most recent call last):\n File \"/var/lib/awx/.ansible/tmp/ansible-local-31cdebbdf/ansible-tmp-1572372537.375614-50462794487074/AnsiballZ_bf_session.py\", line 114, in <module>\n _ansiballz_main()\n File \"/var/lib/awx/.ansible/tmp/ansible-local-31cdebbdf/ansible-tmp-1572372537.375614-50462794487074/AnsiballZ_bf_session.py\", line 106, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/var/lib/awx/.ansible/tmp/ansible-local-31cdebbdf/ansible-tmp-1572372537.375614-50462794487074/AnsiballZ_bf_session.py\", line 49, in invoke_module\n imp.load_module('__main__', mod, module, MOD_DESC)\n File \"/tmp/ansible_bf_session_payload_LjWdeN/__main__.py\", line 97, in <module>\n File \"/tmp/ansible_bf_session_payload_LjWdeN/ansible_bf_session_payload.zip/ansible/module_utils/bf_util.py\", line 16, in <module>\n File \"/opt/venv/ansible-python3/lib/python3.6/site-packages/pybatfish/client/_diagnostics.py\", line 22, in <module>\n from typing import Any, Dict, Iterable, Optional, TYPE_CHECKING # noqa: F401\n File \"/opt/venv/ansible-python3/lib/python3.6/site-packages/typing.py\", line 117\n def __new__(cls, name, bases, namespace, *, _root=False):\n ^\nSyntaxError: invalid syntax\n", "exception": "Traceback (most recent call last):\n File \"/var/lib/awx/.ansible/tmp/ansible-local-31cdebbdf/ansible-tmp-1572372537.375614-50462794487074/AnsiballZ_bf_session.py\", line 114, in <module>\n _ansiballz_main()\n File \"/var/lib/awx/.ansible/tmp/ansible-local-31cdebbdf/ansible-tmp-1572372537.375614-50462794487074/AnsiballZ_bf_session.py\", line 106, in _ansiballz_main\n invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n File \"/var/lib/awx/.ansible/tmp/ansible-local-31cdebbdf/ansible-tmp-1572372537.375614-50462794487074/AnsiballZ_bf_session.py\", line 49, in invoke_module\n imp.load_module('__main__', mod, module, MOD_DESC)\n File \"/tmp/ansible_bf_session_payload_LjWdeN/__main__.py\", line 97, in <module>\n File \"/tmp/ansible_bf_session_payload_LjWdeN/ansible_bf_session_payload.zip/ansible/module_utils/bf_util.py\", line 16, in <module>\n File \"/opt/venv/ansible-python3/lib/python3.6/site-packages/pybatfish/client/_diagnostics.py\", line 22, in <module>\n from typing import Any, Dict, Iterable, Optional, TYPE_CHECKING # noqa: F401\n File \"/opt/venv/ansible-python3/lib/python3.6/site-packages/typing.py\", line 117\n def __new__(cls, name, bases, namespace, *, _root=False):\n ^\nSyntaxError: invalid syntax\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1, "_ansible_no_log": false, "changed": false }

Environment details as following below, the playbook is run on an ansible tower in venv.

ansible 2.8.6 config file = /etc/ansible/ansible.cfg configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /opt/venv/ansible-python3/lib64/python3.6/site-packages/ansible executable location = /opt/venv/ansible-python3/bin/ansible python version = 3.6.8 (default, Aug 7 2019, 17:28:10) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]

(ansible-python3) [root@is-ansible01 ~]# ansible-galaxy info batfish.base Role: batfish.base description: Ansible modules for Batfish (www.batfish.org) active: True commit: b52e5de59f52b18182eaeffe7b51f69258f61a0a commit_message: Use and document new OSPF assert (#83) commit_url: https://api.github.com/repos/batfish/ansible/git/commits/b52e5de59f52b18182eaeffe7b51f69258f61a0a company: created: 2019-06-11T23:55:39.782028Z dependencies: [] download_count: 201 forks_count: 1 galaxy_info: author: Batfish galaxy_tags: ['batfish', 'networking', 'validation'] license: Apache min_ansible_version: 2.7 role_name: base github_branch: master github_repo: ansible github_user: batfish id: 41183 imported: 2019-09-19T16:45:49.835223-04:00 install_date: Tue Oct 29 17:05:44 2019 installed_version: master is_valid: True issue_tracker_url: https://github.com/batfish/ansible/issues license: Apache min_ansible_version: 2.7 modified: 2019-09-19T20:45:49.842435Z open_issues_count: 2 path: ('/root/.ansible/roles', '/usr/share/ansible/roles', '/etc/ansible/roles') role_type: ANS stargazers_count: 24 travis_status_url:

(ansible-python3) [root@is-ansible01 ~]# pip show pybatfish Name: pybatfish Version: 0.36.0 Summary: Python API and utilities for Batfish Home-page: https://github.com/batfish/pybatfish Author: The Batfish Open Source Project Author-email: [email protected] License: Apache License 2.0 Location: /opt/venv/ansible-python3/lib/python3.6/site-packages Requires: attrs, deepdiff, deprecated, netconan, pandas, python-dateutil, PyYAML, requests, requests-toolbelt, simplejson, six Required-by:

(ansible-python3) [root@is-ansible01 ~]# pip show typing Name: typing Version: 3.7.4 Summary: Type Hints for Python Home-page: https://docs.python.org/3/library/typing.html Author: Guido van Rossum, Jukka Lehtosalo, Łukasz Langa, Ivan Levkivskyi Author-email: [email protected] License: PSF Location: /opt/venv/ansible-python3/lib/python3.6/site-packages Requires: Required-by:

traceback in python 3.9 when trying to make a discord server

PS C:\Users\USER\codes> & C:/Users/USER/AppData/Local/Programs/Python/Python39/python.exe c:/Users/USER/codes/bot.py
Traceback (most recent call last):
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 936, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore # noqa
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1081, in create_connection
transport, protocol = await self._create_connection_transport(
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 1111, in _create_connection_transport
await waiter
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\asyncio\sslproto.py", line 528, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\asyncio\sslproto.py", line 188, in feed_ssldata
self._sslobj.do_handshake()
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\ssl.py", line 944, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1122)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "c:\Users\USER\codes\bot.py", line 11, in
client.run ('NzczMTgwMDc1ODMwODA0NDkw.X6FeDw.inwDm7CQhSBPNyf4GTHbK8M6L0c')
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\client.py", line 708, in run
return future.result()
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\client.py", line 687, in runner
await self.start(*args, **kwargs)
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\client.py", line 650, in start
await self.login(*args, bot=bot)
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\client.py", line 499, in login
await self.http.static_login(token.strip(), bot=bot)
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\http.py", line 291, in static_login
data = await self.request(Route('GET', '/users/@me'))
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\discord\http.py", line 185, in request
async with self.__session.request(method, url, **kwargs) as r:
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 1012, in aenter
self._resp = await self._coro
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\client.py", line 480, in _request
conn = await self._connector.connect(
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 523, in connect
proto = await self._create_connection(req, traces, timeout)
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 858, in _create_connection
_, proto = await self._create_direct_connection(
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 1004, in _create_direct_connection
raise last_exc
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 980, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "C:\Users\USER\AppData\Local\Programs\Python\Python39\lib\site-packages\aiohttp\connector.py", line 938, in _wrap_create_connection
raise ClientConnectorCertificateError(
aiohttp.client_exceptions.ClientConnectorCertificateError: Cannot connect to host discord.com:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1122)')]

im also using the code for it to power up the discord as this
import discord

class MyClient(discord.Client):
async def on_ready(self):
print('Logged on as {0}!'.format(self.user))

 async def on_message(self, message):
     print('Message from {0.author}: {0.content}'.format(message))

client = MyClient()
client.run ('discord.token')
discord token as in my discord bot token that im probably not gonna share