ciscodevnet / dne-dna-code Goto Github PK

I just found 1 instance for simplifying the structure.. the code looks heavy - especially for engineers who are new to programming and are just getting started with python.. so, if possible can we look into reducing the complexity by removing unused and uncalled function blocks and variables?

-o option unnecessarily complicates the structure.

1. The strict check for -o will cause an error as the mission instruction does not ask to give the -o option.. hence the participants will face exit_with_print statement

In step 3 of DNE content as given here https://learninglabs.cisco.com/tracks/dnav3-track/dnav3-meraki/meraki-05-mission/step/3 - it is mentioned to execute this command :

python mxfirewallcontrol.py -k 6bec40cf957de430a6f1f2baa056b99a4fac9ea0 -c create-backup 

The below code will cause the command to exit with print always.. unless you add a dummy orgid or edit the code to keep only the api_key check.

line 471:

    if arg_apikey == "" or arg_org == "":

2. We are not using it for the mission example as we are parsing all the organization and the networks for taking the backup.

line 477:

# compile list of organizations to be processed
    orglist = []
    orgjson = getorglist(arg_apikey)

3. There is actually no standard api that will give orgid if we pass the orgname as an argument.. this will unnecessarily trip the participants and make the code more complex.. not to mention that it is not used at all for the mission output.

Line 134:

def getorgid(p_apikey, p_orgname):

intro-mdp/netconf/get_interface_list.py

Second last tag should be </interfaces> not </interface>.

netconf_filter = """
<filter>
  <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
    <interface></interface>
  </interface>
</filter>"""

Corrected version:

netconf_filter = """
<filter>
  <interfaces xmlns="urn:ietf:params:xml:ns:yang:ietf-interfaces">
    <interface></interface>
  </interfaces>
</filter>"""

Stacktrace of the error:

python get_interface_list.py 
Opening NETCONF Connection to ios-xe-mgmt.cisco.com
Sending a <get-config> operation to the device.

Traceback (most recent call last):
  File "get_interface_list.py", line 65, in <module>
    netconf_reply = m.get_config(source = 'running', filter = netconf_filter)
  File "/Users/taylor13/Sites/dnav3-code/venv/lib/python3.7/site-packages/ncclient/manager.py", line 170, in wrapper
    return self.execute(op_cls, *args, **kwds)
  File "/Users/taylor13/Sites/dnav3-code/venv/lib/python3.7/site-packages/ncclient/manager.py", line 240, in execute
    raise_mode=self._raise_mode).request(*args, **kwds)
  File "/Users/taylor13/Sites/dnav3-code/venv/lib/python3.7/site-packages/ncclient/operations/retrieve.py", line 129, in request
    node.append(util.build_filter(filter))
  File "/Users/taylor13/Sites/dnav3-code/venv/lib/python3.7/site-packages/ncclient/operations/util.py", line 61, in build_filter
    rep = validated_element(spec, ("filter", qualify("filter")))
  File "/Users/taylor13/Sites/dnav3-code/venv/lib/python3.7/site-packages/ncclient/xml_.py", line 131, in validated_element
    ele = to_ele(x)
  File "/Users/taylor13/Sites/dnav3-code/venv/lib/python3.7/site-packages/ncclient/xml_.py", line 111, in to_ele
    return x if etree.iselement(x) else etree.fromstring(x.encode('UTF-8'), parser=parser)
  File "src/lxml/etree.pyx", line 3213, in lxml.etree.fromstring
  File "src/lxml/parser.pxi", line 1877, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1765, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1127, in lxml.etree._BaseParser._parseDoc
  File "src/lxml/parser.pxi", line 601, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 711, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 640, in lxml.etree._raiseParseError
  File "<string>", line 5
lxml.etree.XMLSyntaxError: Opening and ending tag mismatch: interfaces line 3 and interface, line 5, column 24

I submitted a pull request to fix this already.

Need to add keyboard mappings for non-US keyboards. First one found/needed is Danish.

@swiatecki was nice enough to figure out how to do so, and has built the script that will make it happen. https://github.com/swiatecki/DevNetKeyboard

Would be good to poll other keyboards needed and apply them as well.

Based on the DNA Center documentation, this line should not work but it does, surprise! 🎉
This works in the current solution:

tmp_url = tmp_url + 'network-device/module?deviceId=%s' % device_id

Based on docs, this should work:

tmp_url = tmp_url + 'network-device/module/%s' % device_id

Eventually figure out which it is implemented as and make sure docs/code match.

If ansible module returns an error: "invalid connection specified, expected connection=local, got network_cli"
it is due to the venv not getting created properly (unknown why)

The solution is to delete or deactivate the virtual environment, and try again. Then, after installing the requirements the ansible version should get updated properly and change from:

(venv) [developer@centos intro-ansible]$ ansible --version
ansible 2.4.3.0
(...)
python version = 2.7.5 (default, Aug 4 2017, 00:39:18) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]

to this:

(venv2) [developer@centos intro-ansible]$ ansible --version
ansible 2.5.4
(...)
python version = 3.6.5 (default, Mar 31 2018, 15:47:21) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]

Check the Postman collection for RESTCONF -- seems to configure Gigabit Ethernet rather than loopback.

Hi.

If I connect to one of the sandbox XE devices, the filter works fine. (dne-dna-code/intro-mdp/netconf/get_interface_list.py)
If I connect to CSR running locally (XE 17.03.02), I get following error.

Sending a operation to the device.

Traceback (most recent call last):
File "get_interface_list.py", line 65, in
netconf_reply = m.get_config(source = 'running', filter=netconf_filter)
File "/home/heikki/.local/lib/python3.8/site-packages/ncclient/manager.py", line 212, in execute
return cls(self._session,
File "/home/heikki/.local/lib/python3.8/site-packages/ncclient/operations/retrieve.py", line 166, in request
return self._request(node)
File "/home/heikki/.local/lib/python3.8/site-packages/ncclient/operations/rpc.py", line 341, in _request
raise self._reply.error
ncclient.operations.rpc.RPCError: {'type': 'protocol', 'tag': 'unknown-element', 'severity': 'error', 'info': 'filter\n\n', 'path': '\n /rpc/get-config\n ', 'message': None}

If I remove filter, ncclient runs without errors.
I'm guessing I'm doing something very simple, very wrong. If somebody can tell me what it is, much appreciated.

Br,
Heikki

TODO 1 of the MDP mission is to build the parent XML tag correctly, but one of the inner tags has a typo. Specifically on line 40 of https://github.com/CiscoDevNet/dnav3-code/blob/master/intro-mdp/mission01/netconf_functions.py it currently reads as:

</interface>

should be

</interfaces>

An XML/Pyton noob might trip up on that. I'll submit a PR.

In the class workflow, we have the students import all the Postman collections and environments from the main Postman folder during initial setup.
The Postman files are missing from that folder. They're currently under the intro-dnac folder.
It would be better if they were moved to the main Postman folder and got imported all at once.

Hey guys! Would you mind checking line 40 on the NETCONF lab answer ? I believe it should be interfaces instead of interface. Thanks in advance!

Can someone direct me to where this module is located ..

Looks like the API call URL has been changed, as it's now called “Intent APIs”. Check DNAC API doc for DNAC v1.2.6 for more info.

For eg. Line 76 on dnac-nbapi-hello-world.py should be changed to the following - to represent the changes and match the official documentation:

    return "https://%s:%s/dna/intent/api/v1/%s" % (controller_ip, DNAC_PORT, path)

Credentials in this repo are outdated

https://github.com/CiscoDevNet/dne-dna-code

This file specifically,

https://github.com/CiscoDevNet/dne-dna-code/blob/master/env_lab.py

Incorrect

# Values for the Always On IOS XE Sandbox
IOS_XE_1 = {
    "host": "sandbox-iosxe-recomm-1.cisco.com",
    "username": "developer",
    "password": "C1sco12345",
    "netconf_port": 830,
    "restconf_port": 443,
    "ssh_port": 22
}

Correct

# Values for the Always On IOS XE Sandbox
IOS_XE_1 = {
    "host": "sandbox-iosxe-recomm-1.cisco.com",
    "username": "developer",
    "password": "lastorangerestoreball8876",
    "netconf_port": 830,
    "restconf_port": 443,
    "ssh_port": 22
}

See here for details: DevNet Sandbox - Workspace - Lab Catalog - IOS XE on CSR Recommended Code AlwaysOn (cisco.com)

This affects this LL (and probably more):

Python says hello with ncclient - Cisco DevNet Learning Center

In addition, this sandbox, which is listed twice in env_lab.py for a reason unknown to me, has the incorrect SSH port.

Incorrect

NXOS_1 = {
    "host": "sbx-nxos-mgmt.cisco.com",
    "username": "admin",
    "password": "Admin_1234!",
    "netconf_port": 10000,
    "restconf_port": 443,
    "nxapi_port": 80,
    "ssh_port": 8181
}

Correct

NXOS_1 = {
    "host": "sbx-nxos-mgmt.cisco.com",
    "username": "admin",
    "password": "Admin_1234!",
    "netconf_port": 10000,
    "restconf_port": 443,
    "nxapi_port": 80,
    "ssh_port": 22
}

See this page for details: DevNet Sandbox - Workspace - Lab Catalog - Open NX-OS Programmability (cisco.com)

Finally,

We have one more (as far as I can tell) that needs updating. However, this one I'm not sure about, because it's using the 'express' environment.

Incorrect

elif ENVIRONMENT_IN_USE == "express":
DNA_CENTER = {
"host": "sandboxdnac2.cisco.com",
"port": 443,
"username": "dnacdev",
"password": "D3v93T@wK!"
}

Correct

elif ENVIRONMENT_IN_USE == "express":
DNA_CENTER = {
"host": "sandboxdnac2.cisco.com",
"port": 443,
"username": "devnetuser",
"password": "Cisco123!"
}

Details: DevNet Sandbox - Workspace - Lab Catalog - Cisco DNA Center AO2 2.3.3.5

Ping command in verify fails because of a -c parameter that doesn't work in Windows

dnac-nbapi-hello-world.py
It looks like there are a few None values which will not be printed - I'm assuming this is due to the new devices added to the DNAC Always-on Sandbox.

To fix the issue I would suggest to explicitly convert the values to a string object by using !s, which translates into modifying line 105, so the code block would look as follows:

for device in response['response']:
        uptime = "N/A" if device['upTime'] is None else device['upTime']
        print("{0!s:42s}{1!s:17s}{2!s:17s}{3!s:18s}{4!s:12s}{5!s:16s}{6!s:15s}".
            format(device['hostname'],
                    device['managementIpAddress'],
                    device['serialNumber'],
                    device['platformId'],
                    device['softwareVersion'],
                    device['role'],uptime))

@denapom11

On this script [https://github.com/CiscoDevNet/dnav3-code/blob/master/verify/backend/cmx.py][https://github.com/CiscoDevNet/dnav3-code/blob/master/verify/backend/cmx.py]

Please silence the warning about the insecure certificate.

(venv) ubuntu@ubuntu1804:~/code/dne-dna-code/intro-ansible$ ansible-playbook ansible-02-ios-modules/02-ios_command_show.yaml

PLAY [Sample IOS show version for Ansible 2.5] *************************************************************************************************************************************************************

TASK [GATHERING FACTS] *************************************************************************************************************************************************************************************
fatal: [ios-xe-mgmt.cisco.com]: FAILED! => {"msg": "timed out"}
to retry, use: --limit @/home/ubuntu/code/dne-dna-code/intro-ansible/ansible-02-ios-modules/02-ios_command_show.retry

PLAY RECAP *************************************************************************************************************************************************************************************************
ios-xe-mgmt.cisco.com : ok=0 changed=0 unreachable=0 failed=1

Hi,

Here is the collection: https://github.com/CiscoDevNet/dne-dna-code/blob/master/intro-mdp/postman/DevNet%20Learning%20Labs-%20Intro%20to%20Model%20Driven%20Programmability.postman_collection.json

Every call which uses this auth...

Basic cm9vdDpEX1ZheSFfMTAm

...should replace it with this auth...

Basic ZGV2ZWxvcGVyOmxhc3RvcmFuZ2VyZXN0b3JlYmFsbDg4NzY=

This is based on the newish credentials in this sandbox: https://devnetsandbox.cisco.com/RM/Diagram/Index/27d9747a-db48-4565-8d44-df318fce37ad?diagramType=Topology

I have tested it

@aidevnet / @rshoemak

NFVIS Verify - https://github.com/CiscoDevNet/dnav3-code/blob/master/verify/backend/nfvis.py - is failing.

line 81
  return False

Indentation Error:

The CMX cmxnotificationreceiver.py refers to the Meraki documentation:

Cisco Meraki CMX Documentation
https://documentation.meraki.com/MR/Monitoring_and_Reporting/CMX_Analytics#CMX_Location_API

Should probably link to the CMX REST API guide instead:
https://www.cisco.com/c/en/us/td/docs/wireless/mse/10-2/api/b_cmx_102_api_reference.html

See this issue for ncclient: ncclient/ncclient#246

The ncclient module uses the async in its code and it becomes a reserved word in python 3.7. This was fixed in v0.6.x of ncclient.

This ends up breaking the verify.py script (and would cause more issues moving forward).

This causes a lot of hiccups in the class when setting up the dev environment as people download 3.7 or already have it. Most were fixing it by re-downloading/installing 3.6, but it is much simpler to update requirements.txt to ncclient 0.6.2.

I will be submitting a pull request for this.

Reported by Jose in Orlando. If attendees have mistakes in their lab, and re-run the configure-ips more than once they end up with a list of IPs in the try block around line 74. Workaround is to run the clear-ips function.

Would be good to replicate and build some code to error check into configure.

I am learning this lab Ansible IOS native modules. But the first task (GATHERING FACTS) of the playbook 02-ios_command_show.yaml is failed due to

fatal: [ios-xe-mgmt.cisco.com]: FAILED! => {"changed": false, "msg": "[Errno 60] Operation timed out"}.

I also tried different IOS XE sandbox but neither of them are working

fatal: [sandbox-iosxe-latest-1.cisco.com]: FAILED! => {"changed": false, "msg": "[Errno 60] Operation timed out"}
fatal: [sandbox-iosxe-recomm-1.cisco.com]: FAILED! => {"changed": false, "msg": "[Errno 60] Operation timed out"}

Some ansible playbooks won't run on Mac using Python 3.8.

This is solved by bumping the ansible version specified in requirements.txt to a modern version.

ansible-playbook ansible-03-netconf-config/03-ntp_create.yaml

PLAY [NETCONF create NTP server] ***********************************************

TASK [set_fact] ****************************************************************
ERROR! Unexpected Exception, this is probably a bug: cannot pickle '_io.TextIOWrapper' object

When running the solution, I get the following error:

Device: ios-xe-mgmt.cisco.com
  Interface: GigabitEthernet2
Traceback (most recent call last):
  File "netconf_configure_ips.py", line 62, in <module>
    check_ip(device)
  File "/Users/mbrainar/coding/DevNet/dnav3-code/intro-mdp/mission01/netconf_functions.py", line 81, in check_ip
    ipv4["ip"], ipv4["config"]["prefix-length"]
TypeError: list indices must be integers or slices, not str

I believe it is because the instance of ios-xe-mgmt.cisco.com has multiple addresses configured on GigabitEthernet2:

<ipv4 xmlns="http://openconfig.net/yang/interfaces/ip">
                                                        <addresses>
                                                                <address>
                                                                        <ip>10.255.255.1</ip>
                                                                        <config>
                                                                                <ip>10.255.255.1</ip>
                                                                                <prefix-length>24</prefix-length>
                                                                        </config>
                                                                </address>
                                                                <address>

                                                                        <ip>10.255.255.10</ip>
                                                                        <config>
                                                                                <ip>10.255.255.10</ip>
                                                                                <prefix-length>31</prefix-length>
                                                                        </config>
                                                                </address>
                                                                <address>
                                                                        <ip>10.255.255.12</ip>
                                                                        <config>
                                                                                <ip>10.255.255.12</ip>
                                                                                <prefix-length>24</prefix-length>
                                                                        </config>
                                                                </address>
                                                        </addresses>
                                                </ipv4>

The RESTCONF module described here:

https://developer.cisco.com/learning/tracks/dnav3-track/dnav3-intro-mdp/dnav3-intro-restconf/step/3

asks you to use the preloaded Postman collections in dnav3-code/postman, all of which use http to connect to the sample device in question. I find that yields:

but if you change the protocol to https it works:

I'll submit a PR for this and one other minor bug I found in the MDP modules.

Traceback:

(venv) JRIGLER-M-G5SZ:intro-dnac jrigler$ ./cmd-run/show_command.py Traceback (most recent call last): File "./cmd-run/show_command.py", line 112, in <module> results_fileId = execute_commands(token, deviceIdList, commandList) File "./cmd-run/show_command.py", line 47, in execute_commands task_result = post_and_wait(token,url, payload) File "./cmd-run/show_command.py", line 25, in post_and_wait taskid = response.json()['response']['taskId'] KeyError: 'response'

Associated code line (line 25):

taskid = response.json()['response']['taskId']

when parsing response from '/v1/network-device-poller/cli/read-request'.

Running as 'devnetuser' against Always On SandBox (1.2.6)
(
* URL: https://sandboxdnac.cisco.com/
* Username: devnetuser
* Password: Cisco123!
)

Executing with same credentials in POSTMAN returns results:
HTTP Status: 403 Forbidden
and
{
"message": "Role does not have valid permissions to access the API"
}

This sample code is used in Learning Lab "Execute Commands Across the Network with Cisco DNA Center Platform and Command Runner", so should not fail - with traceback but should handle invalid role.

When trying to run the lab I get errors around the inventory file/host file not being found:

ansible-playbook ansible-02-ios-modules/02-ios_command_show.yaml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'
[WARNING]: Could not match supplied host pattern, ignoring: iosxe

I believe the ansible.cfg file is incorrect.

When changed from:

[defaults]
hostfile = hosts
host_key_checking = False
deprecation_warnings=False
stdout_callback = skippy


[persistent_connection]
connect_timeout = 100
command_timeout = 80

to (changing line 2)

[defaults]
inventory = hosts
host_key_checking = False
deprecation_warnings=False
stdout_callback = skippy


[persistent_connection]
connect_timeout = 100
command_timeout = 80

all errors are solved and the playbooks run as intended.

I would create a pull request for this change but want someone to sanity check this first for me.

What's the pod name in hosts?