miaow2 / netbox-config-diff Goto Github PK

Netbox Version 3.6.5

When defining platform settings for driver juniper_junos we correctly configure the command as 'show configuration | display set' which displays the current junos configuration as set commands. After running the Config Compliance job - the actual configuration is NOT show in 'display set' format but instead as hierarchical format.

To Reproduce
Steps to reproduce the behavior:

1. Define platform settings for juniper_junos and set command as 'show configuration | display set'
2. Run ConfigDiffScripts which is successful
3. View Config Compliance against device - diff does not work as rendered config is in set format, collected config is not in set format

Expected behavior
Collected configuration in 'set' format

Is your feature request related to a problem? Please describe.
I do have two different groups of devices to manage: devices in production and devices in a lab setup for testing.
So far I had set all production devices to status active and devices in the lab as staging which means for the latter I cannot generate any diffs.

Describe the solution you'd like
Not being as strict and allowing devices with other status to be allowed in the device selection.

Describe alternatives you've considered
Maybe I am using the staging status wrong. I am unsure, what the general consensus is here about keeping multiple sets of devices online that allows for configuration, testing, etc, while still keeping production and testing separate. Maybe the answer is tags, but I find tags cumbersome to work with through the API.

NetBox version
What version of NetBox are you currently running?
3.6.6

Describe the bug
Upon a fresh installation and following the installation manual, the config diff script job gets stuck as running.

after restarting netbox with
sudo systemctl restart netbox netbox-rq
(note the additional netbox-rw), the script works as expected.

With scrapli-cfg you can make merging or replacing device configurations, I should try to integrate this into my plugin.

Main functions:

• Approve system (configuration process must be approved by user with specific permissions)
• Review config diff before approving
• Rollback config if any error is caught
• Schedule/cancel configuration job
• Merge/replace parts or full configuration

Add searching device config by name in selected DataSource instead of connecting to device

Hi. Great plugin, thanks for all your effort!

I am using netbox-config-diff with config files in a git data source. We use many virtual chassis in Netbox to represent switch stacks and VSS/SVL (Cisco virtual chassis) and we have an individual device in Netbox per member switch, with a naming convention like switch-name:1 (for switch 1 in the stack, and so on), and the Netbox virtual chassis name is the actual switch hostname. Also, we save our switch configs in the data source with the standard Cisco file naming, which is <switch hostname>-confg. Since netbox-config-diff looks for a file named "device name" it doesn't find our config files in the data source unless we rename them.

Would it be possible to implement some sort of file name template function to allow us to define which file to look for? Perhaps a jinja template with access to the device object data similar to the way that Netbox custom links can be defined?

Many thanks!

Is your feature request related to a problem? Please describe.
The selection of devices for the config diff generation by the script is somewhat cumbersome for me as I have about 20 devices spread across 10+ sites. Which means lots of clicks in the GUI when I want to reschedule the custom script.

Describe the solution you'd like
I'd like to use the device role for selecting devices.

Describe alternatives you've considered
So far I have been manually selecting each device by clicking on it.

the config diff in dark mode has a unreadable color scheme.

Describe the solution you'd like
a color scheme that is usable in dark mode

Describe alternatives you've considered
toggle to light mode when reading the config diff

It would be nice if we could choose between 'getting them direct from the device' or 'load from DataSource.
The login to the devices is not always just one username/password, and that makes the current implementation not flexible.

There are a dozen other tools out there that can pull the configs of devices and store them in a repo.
In my case, I have a rancid running that pulls the config (the authentication schema can be easily set per device/group) and store them in a git repo. That repo is then synced to netbox using the DataSources.

that makes the configs available for other plugins and custom scripts too

Is your feature request related to a problem? Please describe.
When I updated to 2.x I haven't been able to select devices anymore. I found out, that the device template is now mandatory on the device?
Whats the reason for this change? I thought getting at least some basic template inherited from the platform to be a good default.

The 'Netbox Config Diff Plugin' is now located under the 'Plugins' Section of the main menu
You can create your own section to move the plugin into.
That would make the accessibiliy of the plugin much easier
https://docs.netbox.dev/en/stable/plugins/development/navigation/#menus

Now plugin stores only diff in a unified format, I want to add missing and extra config lines to db

With hier_config plugin can build the remediation steps necessary to bring a device into spec with its intended configuration.

Is your feature request related to a problem? Please describe.
Running the ConfigDiffScript on a schedule creates one change log entry (/extras/changelog/) for each device and run of the script not considering if there was a change in the diff or not.

Describe the solution you'd like
If the result (meaning the content of diff) stays the same, do not create a log entry.

Is the changelog on the dashboard filterable in any way?

Check the plugin for compatibility with NetBox 3.7

NetBox version
v3.6.4

Describe the bug
When attempting to apply migrations using ./manage.py migrate I get the following stacktrace:

Traceback (most recent call last):
  File "/opt/netbox/netbox/./manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 106, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 100, in handle
    self.check(databases=[database])
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/management/base.py", line 485, in check
    all_issues = checks.run_checks(
                 ^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/checks/registry.py", line 88, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/checks/urls.py", line 14, in check_url_config
    return check_resolver(resolver)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/core/checks/urls.py", line 24, in check_resolver
    return check_method()
           ^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/urls/resolvers.py", line 494, in check
    for pattern in self.url_patterns:
                   ^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/utils/functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
                                         ^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/urls/resolvers.py", line 715, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
                       ^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/utils/functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
                                         ^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/urls/resolvers.py", line 708, in urlconf_module
    return import_module(self.urlconf_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/netbox/netbox/netbox/urls.py", line 9, in <module>
    from extras.plugins.urls import plugin_admin_patterns, plugin_patterns, plugin_api_patterns
  File "/opt/netbox/netbox/extras/plugins/urls.py", line 31, in <module>
    urlpatterns = import_string(f"{plugin_path}.urls.urlpatterns")
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/utils/module_loading.py", line 30, in import_string
    return cached_import(module_path, class_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/utils/module_loading.py", line 15, in cached_import
    module = import_module(module_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/netbox_config_diff/urls.py", line 4, in <module>
    from netbox_config_diff import views
  File "/opt/netbox/venv/lib/python3.11/site-packages/netbox_config_diff/views/__init__.py", line 9, in <module>
    from .configuration import (
  File "/opt/netbox/venv/lib/python3.11/site-packages/netbox_config_diff/views/configuration.py", line 302, in <module>
    class JobListView(generic.ObjectListView):
  File "/opt/netbox/venv/lib/python3.11/site-packages/netbox_config_diff/views/configuration.py", line 304, in JobListView
    object_type=ContentType.objects.get(app_label="netbox_config_diff", model="configurationrequest")
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/netbox/venv/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
django.contrib.contenttypes.models.ContentType.DoesNotExist: ContentType matching query does not exist.

I believe the issue is some kind of a "chicken and egg issue", where somehow JobListView defines a filterset on a ContentType that does not yet exist... and this class is attempted to be loaded before the migrations.

Quick and dirty workaround is to comment this line out before running migrations and then removing the comment again, this should allow the plugin to install.

This line is commented out to workaround:

  File "/opt/netbox/venv/lib/python3.11/site-packages/netbox_config_diff/views/configuration.py", line 304, in JobListView
    object_type=ContentType.objects.get(app_label="netbox_config_diff", model="configurationrequest")

To Reproduce

Install plugin on a fresh database and attempt to run db migrations.

Expected behavior
Migrations run successfully.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

NetBox version
What version of NetBox are you currently running?
3.6.2

Describe the bug
After run the script I get this message:

Traceback (most recent call last):
File "/opt/netbox/venv/lib/python3.10/site-packages/netbox_config_diff/compliance/models.py", line 65, in get_actual_config
async with AsyncScrapli(**self.to_scrapli()) as conn:
File "/opt/netbox/venv/lib/python3.10/site-packages/scrapli/driver/base/async_driver.py", line 42, in aenter
await self.open()
File "/opt/netbox/venv/lib/python3.10/site-packages/scrapli/driver/base/async_driver.py", line 84, in open
await self.transport.open()
File "/opt/netbox/venv/lib/python3.10/site-packages/scrapli/transport/plugins/asyncssh/transport.py", line 194, in open
self.session = await asyncio.wait_for(
File "/usr/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
return fut.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 650, in _wrap_awaitable
return (yield from awaitable.await())
File "/opt/netbox/venv/lib/python3.10/site-packages/asyncssh/connection.py", line 8093, in connect
return await asyncio.wait_for(
File "/usr/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
return await fut
File "/opt/netbox/venv/lib/python3.10/site-packages/asyncssh/connection.py", line 440, in _connect
await options.waiter
File "/opt/netbox/venv/lib/python3.10/site-packages/asyncssh/connection.py", line 1275, in data_received
while self._inpbuf and self._recv_handler():
File "/opt/netbox/venv/lib/python3.10/site-packages/asyncssh/connection.py", line 1522, in _recv_packet
processed = handler.process_packet(pkttype, seq, packet)
File "/opt/netbox/venv/lib/python3.10/site-packages/asyncssh/packet.py", line 237, in process_packet
self._packet_handlers[pkttype](self, pkttype, pktid, packet)
File "/opt/netbox/venv/lib/python3.10/site-packages/asyncssh/connection.py", line 2168, in _process_kexinit
self._enc_alg_cs = self._choose_alg('encryption', self._enc_algs,
File "/opt/netbox/venv/lib/python3.10/site-packages/asyncssh/connection.py", line 1349, in _choose_alg
raise KeyExchangeFailed(
asyncssh.misc.KeyExchangeFailed: No matching encryption algorithm found, sent [email protected],[email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr and received aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc

My switch is old, I can't changer the algorithm, I tried use telnet but I cant found the correct drive.

At this moment devices credentials are stored in the plugin config, рowever, not all devices have only one login and password.
I need to investigate the integration of netbox-secrets plugin

Is your feature request related to a problem? Please describe.
When I show the compliance diff to users there is confusion about which changes will be applied because the diff order does not match the expectations of the users. The diff shows how to patch the desired/rendered config to create the running config.
e.g. diff -u rendered.cfg running.cfg

Describe the solution you'd like
I believe it is more natural to have the compliance diff show the changes needed to patch the running config to create the desired config.
e.g. diff -u running.cfg rendered.cfg

Additional context

In the diff prefix: 10.101.16.0/23 is the prefix configured in netbox, the desired/rendered config and prefix: 10.101.16.0/24 is from the running config. A configuration request would deploy prefix: 10.101.16.0/23 but in the diff it seems like it would be removed.