selfuryon / netdev Goto Github PK
View Code? Open in Web Editor NEWAsynchronous multi-vendor library for interacting with network devices
Home Page: http://netdev.readthedocs.io/
License: Apache License 2.0
Asynchronous multi-vendor library for interacting with network devices
Home Page: http://netdev.readthedocs.io/
License: Apache License 2.0
Hi,
This is netdev is great package. Thanks for writing.
I am writing AWS lambda function using netdev library . I zipped the package in local machine and upload to lambda layers but still not able to recognize. Anyone can tried it ?
Thanks
Ankush
Describe the bug
After yesterdays commits, MicrotikRouterOS`s start raising TimeoutError.
How to reproduce
Run /tests/test_mikrotik_routeros.py
Traceback
File "/home/kzhukov/scripts/netdev-zklevsha/tests/test_mikrotik_routeros.py", line 57, in test_show_several_commands
self.loop.run_until_complete(task())
File "/usr/lib64/python3.6/asyncio/base_events.py", line 468, in run_until_complete
return future.result()
File "/home/kzhukov/scripts/netdev-zklevsha/tests/test_mikrotik_routeros.py", line 51, in task
async with netdev.create(**dev) as mik:
File "/home/kzhukov/scripts/netdev-zklevsha/netdev/vendors/base.py", line 185, in __aenter__
await self.connect()
File "/home/kzhukov/scripts/netdev-zklevsha/netdev/vendors/mikrotik/mikrotik_routeros.py", line 53, in connect
await self._establish_connection()
File "/home/kzhukov/scripts/netdev-zklevsha/netdev/vendors/mikrotik/mikrotik_routeros.py", line 79, in _establish_connection
output = await self._read_until_prompt()
File "/home/kzhukov/scripts/netdev-zklevsha/netdev/vendors/base.py", line 347, in _read_until_prompt
return await self._read_until_pattern(self._base_pattern)
File "/home/kzhukov/scripts/netdev-zklevsha/netdev/vendors/base.py", line 361, in _read_until_pattern
raise TimeoutError(self._host)
netdev.exceptions.TimeoutError: Host 87.226.166.121 Timeout Error
Root cause
As far as I underenstand this situation caused by adding parameter term_size=(200, 24)
to self._conn.open_session() (microtik_routeros.py lines 74-76)
self._stdin, self._stdout, self._stderr = await self._conn.open_session(
term_type="Dumb", term_size=(200, 24)
)
Possible solution
Call function await self._conn.open_session() without term_size parameter.
In my opinion this parameter is redundand because term_size is set during init by prepening "+ct200w" to username.
From docstring:
Mikrotik duplicate prompt in connection, so we should use pattern like
prompt .* prompt.
For disabling colors in CLI output we should user this username = username+c
'+c' disables colors
'+t' disable auto term capabilities detection
'+200w' set terminal width to 200 rows
Proposed functionality
Support for Palo Alto Devices
Use case
Enable netdev for Palo Alto Devices which is already supported by Netmiko
Additional context
https://github.com/ktbyers/netmiko/search?q=paloalto_panos&unscoped_q=paloalto_panos
https://ktbyers.github.io/netmiko/docs/netmiko/paloalto/paloalto_panos.html
https://ktbyers.github.io/netmiko/docs/netmiko/paloalto/index.html
Can we add support for ubiquiti_edge please?
Proposed functionality
Use case
We have hundreds of SSG5 devices which run screenOs.
Would it be possible to to hae this device type support ?
Additional context
Add any other context about the problem here.
Describe the bug
The send_command function allows for a pattern to be input to stop the parse, however the default pattern is used as an optional stop point regardless.
This causes issues if you have a comment with the switch name and a > or # following it.
The regular expression should probably be anchored ^ $ to confirm that it is not found in the middle of a line, or other erronous location causing the send_command to prematurely return.
General Information
Debug information
Reading pattern "WAS-SF02-1-119-DC#" or "WAS-SF02-1-1.?((.?))?[>|#]" was found
^^^^ WHOOOPS!!!
My startup has this line...
description VPC Peerlink<Chassis1 WAS-SF01-1-119-DC | Chassis2 WAS-SF02-1-119-DC>
Additional context
So, without line anchors, I have to limit what descriptions I have, and I'm not sure if it's catching a truncating other commands.
The fix???
Well there are a few approaches. First if I override a pattern, it should respect that pattern, and timeout otherwise, not use both my pattern and the default pattern.
Second I think anchors in most cases would be appropriate to ensure the base prompt is found on a "clean" line, not as a mishap of some port description if I have some network admin getting fancy with their descriptions.
I haven't any exceptions/messages about the wrong secret password (the situation was described here #26 )
Question
Turned debugging on and added $
to the pattern, its finding it, but timing out after the command.
DEBUG:netdev:Host 192.168.16.1: Reading pattern '>|\#|\$' was found: '\r\nmyname@rtr0:~$ \r\n'
DEBUG:netdev:Host 192.168.16.1: Found Prompt: 'myname@rtr0:~$'
DEBUG:netdev:Host 192.168.16.1: Base Prompt: myname@rtr0
DEBUG:netdev:Host 192.168.16.1: Base Pattern: \(myname@rtr.*?\) (\(.*?\))?[>|\#|\$]
INFO:netdev:Host 192.168.16.1: Entering to privilege exec
INFO:netdev:Host 192.168.16.1: Checking privilege exec
INFO:netdev:Host 192.168.16.1: Reading until pattern
DEBUG:netdev:Host 192.168.16.1: Reading pattern: \(myname@rtr.*?\) (\(.*?\))?[>|\#|\$]
ERROR:asyncio:Task exception was never retrieved
netdev.exceptions.TimeoutError: Host 192.168.16.1 Timeout Error
Any insight? (running the command in ssh/cli works fine so i know its the command thats the issue. (let me know what you need) It looks like it finds the prompt fine but can't seem to read the pattern.
import asyncio
import netdev
import logging
logging.basicConfig(level=logging.INFO)
netdev.logger.setLevel(logging.DEBUG)
async def task(param):
async with netdev.create(**param) as ubi:
out = await ubi.send_command("?")
print(out)
async def run():
dev = {'username': 'admin',
'password': 'password',
'device_type': 'ubiquity_edge',
'host': '192.168.16.1',
}
devices = [dev]
tasks = [task(dev) for dev in devices]
await asyncio.wait(tasks)
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
Hello!
I want to make the module refactoring.
The first thing I want to separate the connection and the class which knows how to work with device.
The second thing I want to use the Strategy pattern instead of Inheritance due to lack of isolation.
Common class structure:
Try to show the concept of these changes.
Public factory method:
# Public Factory function for creating netdev classes
def create(device_type, connection_type, *args, **kwargs):
# Create IOConnection separately
if connection_type == "ssh":
IOConnection = SSHConnectiom(*args, **kwargs) # As Example for SSHConnection
# Create DeviceStream and all needed instances for a particular device
device_stream = DeviceStream(IOConnection)
# Create Device separately
if device_type == "cisco_ios":
layer_manager = LayerManager(device_stream, cisco_checker)
.add_layer(UserExecMode())
.add_layer(PrivilegeExecMode())
.add_layer(ConfigMode())
return Cisco_IOS(layer_manager)
IOConnection is an abstract class with a public interface which can be used by all particular device type classes.
class IOConnection(abc.ABC):
""" Abstract IO Connection class"""
async def connect(self):
""" Establish Connection """
pass
async def disconnect(self):
pass
def send(self, cmd):
""" Send command to stream """
pass
async def read(self):
""" Read from stream """
pass
IOConnection can be implemented in SSHConnection, TelnetConnection and SerialConnection.
DeviceStream adds some logic to IOConnection. It understands the end of the output for commands: it understands the prompt
class DeviceStream():
""" Class which know how to work with the device in a stream mode """
def __init__ (self, IOConnection, prompt_pattern = r"", ):
self._conn = IOConnection
self._prompt_pattern = prompt_pattern
async def send(self, cmd_list):
pass
async def read_until(self, pattern, re_flags, until_prompt=True):
pass
Device type classes are particular classes for working with network devices.
class CiscoIOS():
""" Abstract Device type"""
def __init__ (device_stream, layer_manager):
self._device_stream = device_stream
self._layer_manager = layer_manager
async def send_command(self, cmd_list, terminal_mode):
""" Go to specific terminal mode and run list of commands in there"""
self._layer_manager.switch_to_layer(terminal_mode)
self.device_stream(cmd_list)
async def send_config_set(self, cmd_list):
""" Go to configuration mode and run list of commands in there"""
self._layer_manager.switch_to_layer('config_mode')
self.device_stream(cmd_list)
self._layer_manager.switch_to_layer('privilege exec')
We have universal class LayerManager
for working with terminal modes:
class LayerManager():
def __init__(self, device_stream, checker_closure):
self._device_stream = device_stream
self._checker_closure = checker_closure
self._current_layer = None
def add_layer(layer):
self._layers[layer.name] = layer
return self
def switch_to_layer(layer_name):
if self._current_layer is None:
self._current_layer = self.checker_closure()
if self._current_layer == layer_name:
return
# switching to layer
def commit_transaction():
layer = get_layer(self._current_layer)
if layer.transactional:
layer.committer()
def get_layer(layer_name):
pass
Specific function for checking the cisco like terminal modes:
def cisco_checker(device_stream):
if ')#':
return 'config'
elif '#':
return 'privilege'
elif '>':
return 'user_exec'
else:
raise Exeption()
We have universal class Layer
for working with particular layer/terminal mode:
class Layer():
def __init__(name, device_stream, enter, exit, transactional=False, commiter=None):
self._name = name
self._device_stream = device_stream
self._enter_closure = enter
self._exit_closure = exit
self._committer_closure = committer
self.transactional = transactional
async def enter(cmd):
pass
async def exit(cmd):
pass
@atribute
def get_name():
return self._name
And cisco like layers:
class ConfigMode(Layer):
def __init__(name, device_stream, enter, exit, transactional=False, commiter=None):
super().__init()
async def enter(cmd='conf t'):
self._device_stream.send_command(cmd)
async def exit(cmd='end'):
self._device_stream.send_command(cmd)
@atribute
def get_name():
return self._name
Proposed functionality
send_config_from_file
Use case
same functionality as send_config_set but you can load the file directly.
Additional context
Add any other context about the problem here.
Describe the bug
MikrotikRouterOS during initialization raises
AttributeError: 'MikrotikRouterOS' object has no attribute '_username'
How to reproduce
Run tests/test_mikrotik_routeros.py
Root cause
Class parent class BaseDevice stores username in dictionary self._connect_params_dict['username']
Possible soluton
Replace
self._username+= "+ct200w" with self._connect_params_dict['username'] += "+ct200w"
Describe the bug
Timeout timer not working for mikrotik devices.
Timeout timer simply ignored. All unavailable devices causes asyncio to raise OSError after 120 sec timeout
How to reproduce
Try to connect to any unavailable Microtik devices.
Expected Behaviour
MicroticRouterOS must raise netdev.exeptions.TimeoutError after timeout timer expires (default=15 sec).
Current Behaviour
asyncio raises OSError after 120 sec timeout
Traceback
Task exception was never retrieved
future: <Task finished coro=<task() done, defined at examples/mikrotik_routeros.py:14> exception=TimeoutError(110, "Connect call failed ('8.8.8.8', 22)")>
Traceback (most recent call last):
File "examples/mikrotik_routeros.py", line 15, in task
async with netdev.create(**param) as routeros:
File "/home/kzhukov/scripts/netdev/netdev/vendors/base.py", line 185, in __aenter__
await self.connect()
File "/home/kzhukov/scripts/netdev/netdev/vendors/mikrotik/mikrotik_routeros.py", line 52, in connect
await self._establish_connection()
File "/home/kzhukov/scripts/netdev/netdev/vendors/mikrotik/mikrotik_routeros.py", line 64, in _establish_connection
self._conn = await asyncssh.connect(**self._connect_params_dict)
File "/home/kzhukov/scripts/envs/netdev-env/lib/python3.6/site-packages/asyncssh-1.17.0-py3.6.egg/asyncssh/misc.py", line 188, in __await__
return (yield from self._coro)
File "/home/kzhukov/scripts/envs/netdev-env/lib/python3.6/site-packages/asyncssh-1.17.0-py3.6.egg/asyncssh/connection.py", line 5745, in connect
'Opening SSH connection to'))
File "/home/kzhukov/scripts/envs/netdev-env/lib/python3.6/site-packages/asyncssh-1.17.0-py3.6.egg/asyncssh/connection.py", line 165, in _connect
local_addr=local_addr)
File "/usr/lib64/python3.6/asyncio/base_events.py", line 778, in create_connection
raise exceptions[0]
File "/usr/lib64/python3.6/asyncio/base_events.py", line 765, in create_connection
yield from self.sock_connect(sock, address)
File "/usr/lib64/python3.6/asyncio/selector_events.py", line 450, in sock_connect
return (yield from fut)
File "/usr/lib64/python3.6/asyncio/selector_events.py", line 480, in _sock_connect_cb
raise OSError(err, 'Connect call failed %s' % (address,))
TimeoutError: [Errno 110] Connect call failed ('8.8.8.8', 22)
Root cause
MicroticRouterOs uses it`s own _establish_connection function.
This function does not uses async.await_for() functionality.
Posible solution
Add async.await_for() to MicrotikRouterOs:
async def _establish_connection(self):
"""Establish SSH connection to the network device"""
logger.info(
"Host {}: Establishing connection to port {}".format(self._host, self._port)
)
output = ""
# initiate SSH connection
fut = asyncssh.connect(**self._connect_params_dict)
try:
self._conn = await asyncio.wait_for(fut, self._timeout)
except asyncssh.DisconnectError as e:
raise DisconnectError(self._host, e.code, e.reason)
except asyncio.TimeoutError:
raise TimeoutError(self._host)
self._stdin, self._stdout, self._stderr = await self._conn.open_session(
term_type="dumb"
)
logger.info("Host {}: Connection is established".format(self._host))
# Flush unnecessary data
output = await self._read_until_prompt()
logger.debug(
"Host {}: Establish Connection Output: {}".format(self._host, repr(output))
)
return output
While trying add use_textfsm=True to send_commad function it raises the below error.
Traceback (most recent call last):
File "async-ssh.py", line 51, in
main()
File "async-ssh.py", line 46, in main
print(task.result())
File "async-ssh.py", line 30, in collect_output
command_result = await connection.send_command(command,use_textfsm=True)
TypeError: send_command() got an unexpected keyword argument 'use_textfsm'
Task exception was never retrieved
future: <Task finished coro=<collect_output() done, defined at async-ssh.py:24> exception=TypeError("send_command() got an unexpected keyword argument 'use_textfsm'",)>
Traceback (most recent call last):
File "async-ssh.py", line 30, in collect_output
command_result = await connection.send_command(command,use_textfsm=True)
TypeError: send_command() got an unexpected keyword argument 'use_textfsm'
I'm running the following code on an IOS-XR device:
11 async with netdev.create(**param) as ios:
12 # Testing sending simple command
13 print("starting task\n {} {}".format(80 * '#', ios))
14 out = await ios.send_command("show run interface loopback0")
15 print(out)
16 # Testing sending configuration set
17 out += await ios.send_command("exit", strip_command=False, strip_prompt=False)
18 print(out)
19 print("task: {} done\n".format(ios))
Looking at the logs, it looks like the prompt is never received from the device, so the connection is never terminated. Any advice?
Hi,
I'm using netdev for a few months now, thanks for the great job!
Since asyncssh 1.16.0 is out, we have problems connecting to some of our devices. It looks like, that there was a change in the part, where the known_hosts are checked. It seems, that the script for these devices just reject the connection after KEX init.
What we see on some of the cisco devices:
SSH2: Session disconnect โ error 0x00
%SSH-4-SSH2_UNEXPECTED_MSG: Unexpected message type has arrived. Terminating the connection
What we see in the script output, when I enabled debugging for netdev and asyncssh:
2019-04-24 11:03:55,303 [conn=284] Requesting key exchange
2019-04-24 11:03:55,303 [conn=284] Key exchange algs: curve25519-sha256,[email protected],curve448-sha512,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,ecdh-sha2-1.3.132.0.10,diffie-hellman-group-exchange-sha256,di
ffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1,diffie-hellman-group14-sha1,diffie-hellman-group14-sha256,diffie-hellman-group15-sha512,diffie-hellman-group16-sha512,diffie-hellman-group17-sha512,diffie-hellman-group18-sha512,
rsa2048-sha256,rsa1024-sha1
2019-04-24 11:03:55,303 [conn=284] Host key algs: [email protected],[email protected],[email protected],[email protected],[email protected]
,[email protected],[email protected],[email protected],ssh-ed25519,ssh-ed448,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,ecdsa-sha2-1.3.132.0.10,rsa-sha2-256,rsa-sha2-512,ss
h-rsa,ssh-dss
2019-04-24 11:03:55,303 [conn=284] Encryption algs: [email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,arcfour256,arcfour128,arcfour
2019-04-24 11:03:55,304 [conn=284] MAC algs: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected]
m,[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-md5,hmac-sha2-256-96,hmac-sha2-512-96,hmac-sha1-96,hmac-md5-96
2019-04-24 11:03:55,304 [conn=284] Compression algs: [email protected],zlib,none
2019-04-24 11:03:55,306 [conn=284, pktid=0] Sent MSG_KEXINIT (20), 2217 bytes
00000000: 14 c8 bd 23 97 62 5b 90 7a 77 c3 b8 f0 a7 7f 54 ...#.b[.zw...T
00000010: 65 00 00 01 ca 63 75 72 76 65 32 35 35 31 39 2d e....curve25519-
00000020: 73 68 61 32 35 36 2c 63 75 72 76 65 32 35 35 31 sha256,curve2551
00000030: 39 2d 73 68 61 32 35 36 40 6c 69 62 73 73 68 2e 9-sha256@libssh.
00000040: 6f 72 67 2c 63 75 72 76 65 34 34 38 2d 73 68 61 org,curve448-sha
00000050: 35 31 32 2c 65 63 64 68 2d 73 68 61 32 2d 6e 69 512,ecdh-sha2-ni
00000060: 73 74 70 35 32 31 2c 65 63 64 68 2d 73 68 61 32 stp521,ecdh-sha2
00000070: 2d 6e 69 73 74 70 33 38 34 2c 65 63 64 68 2d 73 -nistp384,ecdh-s
00000080: 68 61 32 2d 6e 69 73 74 70 32 35 36 2c 65 63 64 ha2-nistp256,ecd
00000090: 68 2d 73 68 61 32 2d 31 2e 33 2e 31 33 32 2e 30 h-sha2-1.3.132.0
000000a0: 2e 31 30 2c 64 69 66 66 69 65 2d 68 65 6c 6c 6d .10,diffie-hellm
000000b0: 61 6e 2d 67 72 6f 75 70 2d 65 78 63 68 61 6e 67 an-group-exchang
000000c0: 65 2d 73 68 61 32 35 36 2c 64 69 66 66 69 65 2d e-sha256,diffie-
000000d0: 68 65 6c 6c 6d 61 6e 2d 67 72 6f 75 70 2d 65 78 hellman-group-ex
000000e0: 63 68 61 6e 67 65 2d 73 68 61 31 2c 64 69 66 66 change-sha1,diff
000000f0: 69 65 2d 68 65 6c 6c 6d 61 6e 2d 67 72 6f 75 70 ie-hellman-group
00000100: 31 2d 73 68 61 31 2c 64 69 66 66 69 65 2d 68 65 1-sha1,diffie-he
00000110: 6c 6c 6d 61 6e 2d 67 72 6f 75 70 31 34 2d 73 68 llman-group14-sh
00000120: 61 31 2c 64 69 66 66 69 65 2d 68 65 6c 6c 6d 61 a1,diffie-hellma
00000130: 6e 2d 67 72 6f 75 70 31 34 2d 73 68 61 32 35 36 n-group14-sha256
00000140: 2c 64 69 66 66 69 65 2d 68 65 6c 6c 6d 61 6e 2d ,diffie-hellman-
00000150: 67 72 6f 75 70 31 35 2d 73 68 61 35 31 32 2c 64 group15-sha512,d
00000160: 69 66 66 69 65 2d 68 65 6c 6c 6d 61 6e 2d 67 72 iffie-hellman-gr
00000170: 6f 75 70 31 36 2d 73 68 61 35 31 32 2c 64 69 66 oup16-sha512,dif
00000180: 66 69 65 2d 68 65 6c 6c 6d 61 6e 2d 67 72 6f 75 fie-hellman-grou
00000190: 70 31 37 2d 73 68 61 35 31 32 2c 64 69 66 66 69 p17-sha512,diffi
000001a0: 65 2d 68 65 6c 6c 6d 61 6e 2d 67 72 6f 75 70 31 e-hellman-group1
000001b0: 38 2d 73 68 61 35 31 32 2c 72 73 61 32 30 34 38 8-sha512,rsa2048
000001c0: 2d 73 68 61 32 35 36 2c 72 73 61 31 30 32 34 2d -sha256,rsa1024-
000001d0: 73 68 61 31 2c 65 78 74 2d 69 6e 66 6f 2d 63 00 sha1,ext-info-c.
000001e0: 00 01 b5 73 73 68 2d 65 64 32 35 35 31 39 2d 63 ...ssh-ed25519-c
000001f0: 65 72 74 2d 76 30 31 40 6f 70 65 6e 73 73 68 2e ert-v01@openssh.
00000200: 63 6f 6d 2c 73 73 68 2d 65 64 34 34 38 2d 63 65 com,ssh-ed448-ce
00000210: 72 74 2d 76 30 31 40 6f 70 65 6e 73 73 68 2e 63 [email protected]
00000220: 6f 6d 2c 65 63 64 73 61 2d 73 68 61 32 2d 6e 69 om,ecdsa-sha2-ni
00000230: 73 74 70 35 32 31 2d 63 65 72 74 2d 76 30 31 40 stp521-cert-v01@
00000240: 6f 70 65 6e 73 73 68 2e 63 6f 6d 2c 65 63 64 73 openssh.com,ecds
00000250: 61 2d 73 68 61 32 2d 6e 69 73 74 70 33 38 34 2d a-sha2-nistp384-
00000260: 63 65 72 74 2d 76 30 31 40 6f 70 65 6e 73 73 68 cert-v01@openssh
00000270: 2e 63 6f 6d 2c 65 63 64 73 61 2d 73 68 61 32 2d .com,ecdsa-sha2-
00000280: 6e 69 73 74 70 32 35 36 2d 63 65 72 74 2d 76 30 nistp256-cert-v0
00000290: 31 40 6f 70 65 6e 73 73 68 2e 63 6f 6d 2c 65 63 [email protected],ec
000002a0: 64 73 61 2d 73 68 61 32 2d 31 2e 33 2e 31 33 32 dsa-sha2-1.3.132
000002b0: 2e 30 2e 31 30 2d 63 65 72 74 2d 76 30 31 40 6f .0.10-cert-v01@o
000002c0: 70 65 6e 73 73 68 2e 63 6f 6d 2c 73 73 68 2d 72 penssh.com,ssh-r
000002d0: 73 61 2d 63 65 72 74 2d 76 30 31 40 6f 70 65 6e sa-cert-v01@open
000002e0: 73 73 68 2e 63 6f 6d 2c 73 73 68 2d 64 73 73 2d ssh.com,ssh-dss-
000002f0: 63 65 72 74 2d 76 30 31 40 6f 70 65 6e 73 73 68 cert-v01@openssh
00000300: 2e 63 6f 6d 2c 73 73 68 2d 65 64 32 35 35 31 39 .com,ssh-ed25519
00000310: 2c 73 73 68 2d 65 64 34 34 38 2c 65 63 64 73 61 ,ssh-ed448,ecdsa
00000320: 2d 73 68 61 32 2d 6e 69 73 74 70 35 32 31 2c 65 -sha2-nistp521,e
00000330: 63 64 73 61 2d 73 68 61 32 2d 6e 69 73 74 70 33 cdsa-sha2-nistp3
00000340: 38 34 2c 65 63 64 73 61 2d 73 68 61 32 2d 6e 69 84,ecdsa-sha2-ni
00000350: 73 74 70 32 35 36 2c 65 63 64 73 61 2d 73 68 61 stp256,ecdsa-sha
00000360: 32 2d 31 2e 33 2e 31 33 32 2e 30 2e 31 30 2c 72 2-1.3.132.0.10,r
00000370: 73 61 2d 73 68 61 32 2d 32 35 36 2c 72 73 61 2d sa-sha2-256,rsa-
00000380: 73 68 61 32 2d 35 31 32 2c 73 73 68 2d 72 73 61 sha2-512,ssh-rsa
00000390: 2c 73 73 68 2d 64 73 73 00 00 00 af 61 65 73 32 ,ssh-dss....aes2
000003a0: 35 36 2d 67 63 6d 40 6f 70 65 6e 73 73 68 2e 63 [email protected]
000003b0: 6f 6d 2c 61 65 73 31 32 38 2d 67 63 6d 40 6f 70 om,aes128-gcm@op
000003c0: 65 6e 73 73 68 2e 63 6f 6d 2c 61 65 73 32 35 36 enssh.com,aes256
000003d0: 2d 63 74 72 2c 61 65 73 31 39 32 2d 63 74 72 2c -ctr,aes192-ctr,
000003e0: 61 65 73 31 32 38 2d 63 74 72 2c 61 65 73 32 35 aes128-ctr,aes25
000003f0: 36 2d 63 62 63 2c 61 65 73 31 39 32 2d 63 62 63 6-cbc,aes192-cbc
00000400: 2c 61 65 73 31 32 38 2d 63 62 63 2c 33 64 65 73 ,aes128-cbc,3des
00000410: 2d 63 62 63 2c 62 6c 6f 77 66 69 73 68 2d 63 62 -cbc,blowfish-cb
00000420: 63 2c 63 61 73 74 31 32 38 2d 63 62 63 2c 61 72 c,cast128-cbc,ar
00000430: 63 66 6f 75 72 32 35 36 2c 61 72 63 66 6f 75 72 cfour256,arcfour
00000440: 31 32 38 2c 61 72 63 66 6f 75 72 00 00 00 af 61 128,arcfour....a
00000450: 65 73 32 35 36 2d 67 63 6d 40 6f 70 65 6e 73 73 es256-gcm@openss
00000460: 68 2e 63 6f 6d 2c 61 65 73 31 32 38 2d 67 63 6d h.com,aes128-gcm
00000470: 40 6f 70 65 6e 73 73 68 2e 63 6f 6d 2c 61 65 73 @openssh.com,aes
00000480: 32 35 36 2d 63 74 72 2c 61 65 73 31 39 32 2d 63 256-ctr,aes192-c
00000490: 74 72 2c 61 65 73 31 32 38 2d 63 74 72 2c 61 65 tr,aes128-ctr,ae
000004a0: 73 32 35 36 2d 63 62 63 2c 61 65 73 31 39 32 2d s256-cbc,aes192-
000004b0: 63 62 63 2c 61 65 73 31 32 38 2d 63 62 63 2c 33 cbc,aes128-cbc,3
000004c0: 64 65 73 2d 63 62 63 2c 62 6c 6f 77 66 69 73 68 des-cbc,blowfish
000004d0: 2d 63 62 63 2c 63 61 73 74 31 32 38 2d 63 62 63 -cbc,cast128-cbc
000004e0: 2c 61 72 63 66 6f 75 72 32 35 36 2c 61 72 63 66 ,arcfour256,arcf
000004f0: 6f 75 72 31 32 38 2c 61 72 63 66 6f 75 72 00 00 our128,arcfour..
00000500: 01 ad 75 6d 61 63 2d 36 34 2d 65 74 6d 40 6f 70 ..umac-64-etm@op
00000510: 65 6e 73 73 68 2e 63 6f 6d 2c 75 6d 61 63 2d 31 enssh.com,umac-1
00000520: 32 38 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e 63 [email protected]
00000530: 6f 6d 2c 68 6d 61 63 2d 73 68 61 32 2d 32 35 36 om,hmac-sha2-256
00000540: 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e 63 6f 6d [email protected]
00000550: 2c 68 6d 61 63 2d 73 68 61 32 2d 35 31 32 2d 65 ,hmac-sha2-512-e
00000560: 74 6d 40 6f 70 65 6e 73 73 68 2e 63 6f 6d 2c 68 [email protected],h
00000570: 6d 61 63 2d 73 68 61 31 2d 65 74 6d 40 6f 70 65 mac-sha1-etm@ope
00000580: 6e 73 73 68 2e 63 6f 6d 2c 68 6d 61 63 2d 6d 64 nssh.com,hmac-md
00000590: 35 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e 63 6f [email protected]
000005a0: 6d 2c 68 6d 61 63 2d 73 68 61 32 2d 32 35 36 2d m,hmac-sha2-256-
000005b0: 39 36 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e 63 [email protected]
000005c0: 6f 6d 2c 68 6d 61 63 2d 73 68 61 32 2d 35 31 32 om,hmac-sha2-512
000005d0: 2d 39 36 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e -96-etm@openssh.
000005e0: 63 6f 6d 2c 68 6d 61 63 2d 73 68 61 31 2d 39 36 com,hmac-sha1-96
000005f0: 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e 63 6f 6d [email protected]
00000600: 2c 68 6d 61 63 2d 6d 64 35 2d 39 36 2d 65 74 6d ,hmac-md5-96-etm
00000610: 40 6f 70 65 6e 73 73 68 2e 63 6f 6d 2c 75 6d 61 @openssh.com,uma
00000620: 63 2d 36 34 40 6f 70 65 6e 73 73 68 2e 63 6f 6d [email protected]
00000630: 2c 75 6d 61 63 2d 31 32 38 40 6f 70 65 6e 73 73 ,umac-128@openss
00000640: 68 2e 63 6f 6d 2c 68 6d 61 63 2d 73 68 61 32 2d h.com,hmac-sha2-
00000650: 32 35 36 2c 68 6d 61 63 2d 73 68 61 32 2d 35 31 256,hmac-sha2-51
00000660: 32 2c 68 6d 61 63 2d 73 68 61 31 2c 68 6d 61 63 2,hmac-sha1,hmac
00000670: 2d 6d 64 35 2c 68 6d 61 63 2d 73 68 61 32 2d 32 -md5,hmac-sha2-2
00000680: 35 36 2d 39 36 2c 68 6d 61 63 2d 73 68 61 32 2d 56-96,hmac-sha2-
00000690: 35 31 32 2d 39 36 2c 68 6d 61 63 2d 73 68 61 31 512-96,hmac-sha1
000006a0: 2d 39 36 2c 68 6d 61 63 2d 6d 64 35 2d 39 36 00 -96,hmac-md5-96.
000006b0: 00 01 ad 75 6d 61 63 2d 36 34 2d 65 74 6d 40 6f ...umac-64-etm@o
000006c0: 70 65 6e 73 73 68 2e 63 6f 6d 2c 75 6d 61 63 2d penssh.com,umac-
000006d0: 31 32 38 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e 128-etm@openssh.
000006e0: 63 6f 6d 2c 68 6d 61 63 2d 73 68 61 32 2d 32 35 com,hmac-sha2-25
000006f0: 36 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e 63 6f [email protected]
00000700: 6d 2c 68 6d 61 63 2d 73 68 61 32 2d 35 31 32 2d m,hmac-sha2-512-
00000710: 65 74 6d 40 6f 70 65 6e 73 73 68 2e 63 6f 6d 2c [email protected],
00000720: 68 6d 61 63 2d 73 68 61 31 2d 65 74 6d 40 6f 70 hmac-sha1-etm@op
00000730: 65 6e 73 73 68 2e 63 6f 6d 2c 68 6d 61 63 2d 6d enssh.com,hmac-m
00000740: 64 35 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e 63 [email protected]
00000750: 6f 6d 2c 68 6d 61 63 2d 73 68 61 32 2d 32 35 36 om,hmac-sha2-256
00000760: 2d 39 36 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e -96-etm@openssh.
00000770: 63 6f 6d 2c 68 6d 61 63 2d 73 68 61 32 2d 35 31 com,hmac-sha2-51
00000780: 32 2d 39 36 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2-96-etm@openssh
00000790: 2e 63 6f 6d 2c 68 6d 61 63 2d 73 68 61 31 2d 39 .com,hmac-sha1-9
000007a0: 36 2d 65 74 6d 40 6f 70 65 6e 73 73 68 2e 63 6f [email protected]
000007b0: 6d 2c 68 6d 61 63 2d 6d 64 35 2d 39 36 2d 65 74 m,hmac-md5-96-et
000007c0: 6d 40 6f 70 65 6e 73 73 68 2e 63 6f 6d 2c 75 6d [email protected],um
000007d0: 61 63 2d 36 34 40 6f 70 65 6e 73 73 68 2e 63 6f [email protected]
000007e0: 6d 2c 75 6d 61 63 2d 31 32 38 40 6f 70 65 6e 73 m,umac-128@opens
000007f0: 73 68 2e 63 6f 6d 2c 68 6d 61 63 2d 73 68 61 32 sh.com,hmac-sha2
00000800: 2d 32 35 36 2c 68 6d 61 63 2d 73 68 61 32 2d 35 -256,hmac-sha2-5
00000810: 31 32 2c 68 6d 61 63 2d 73 68 61 31 2c 68 6d 61 12,hmac-sha1,hma
00000820: 63 2d 6d 64 35 2c 68 6d 61 63 2d 73 68 61 32 2d c-md5,hmac-sha2-
00000830: 32 35 36 2d 39 36 2c 68 6d 61 63 2d 73 68 61 32 256-96,hmac-sha2
00000840: 2d 35 31 32 2d 39 36 2c 68 6d 61 63 2d 73 68 61 -512-96,hmac-sha
00000850: 31 2d 39 36 2c 68 6d 61 63 2d 6d 64 35 2d 39 36 1-96,hmac-md5-96
00000860: 00 00 00 1a 7a 6c 69 62 40 6f 70 65 6e 73 73 68 ....zlib@openssh
00000870: 2e 63 6f 6d 2c 7a 6c 69 62 2c 6e 6f 6e 65 00 00 .com,zlib,none..
00000880: 00 1a 7a 6c 69 62 40 6f 70 65 6e 73 73 68 2e 63 [email protected]
00000890: 6f 6d 2c 7a 6c 69 62 2c 6e 6f 6e 65 00 00 00 00 om,zlib,none....
000008a0: 00 00 00 00 00 00 00 00 00 .........
2019-04-24 11:03:55,306 [conn=284, pktid=0] Received MSG_KEXINIT (20), 271 bytes
00000000: 14 78 4c 9d 24 47 05 21 54 3b 6f bc 59 c2 1d 70 .xL.$G.!T;o.Y..p
00000010: 1e 00 00 00 1a 64 69 66 66 69 65 2d 68 65 6c 6c .....diffie-hell
00000020: 6d 61 6e 2d 67 72 6f 75 70 31 2d 73 68 61 31 00 man-group1-sha1.
00000030: 00 00 07 73 73 68 2d 72 73 61 00 00 00 29 61 65 ...ssh-rsa...)ae
00000040: 73 31 32 38 2d 63 62 63 2c 33 64 65 73 2d 63 62 s128-cbc,3des-cb
00000050: 63 2c 61 65 73 31 39 32 2d 63 62 63 2c 61 65 73 c,aes192-cbc,aes
00000060: 32 35 36 2d 63 62 63 00 00 00 29 61 65 73 31 32 256-cbc...)aes12
00000070: 38 2d 63 62 63 2c 33 64 65 73 2d 63 62 63 2c 61 8-cbc,3des-cbc,a
00000080: 65 73 31 39 32 2d 63 62 63 2c 61 65 73 32 35 36 es192-cbc,aes256
00000090: 2d 63 62 63 00 00 00 2b 68 6d 61 63 2d 73 68 61 -cbc...+hmac-sha
000000a0: 31 2c 68 6d 61 63 2d 73 68 61 31 2d 39 36 2c 68 1,hmac-sha1-96,h
000000b0: 6d 61 63 2d 6d 64 35 2c 68 6d 61 63 2d 6d 64 35 mac-md5,hmac-md5
000000c0: 2d 39 36 00 00 00 2b 68 6d 61 63 2d 73 68 61 31 -96...+hmac-sha1
000000d0: 2c 68 6d 61 63 2d 73 68 61 31 2d 39 36 2c 68 6d ,hmac-sha1-96,hm
000000e0: 61 63 2d 6d 64 35 2c 68 6d 61 63 2d 6d 64 35 2d ac-md5,hmac-md5-
000000f0: 39 36 00 00 00 04 6e 6f 6e 65 00 00 00 04 6e 6f 96....none....no
00000100: 6e 65 00 00 00 00 00 00 00 00 00 00 00 00 00 ne.............
2019-04-24 11:03:55,306 [conn=284] Received key exchange request
2019-04-24 11:03:55,306 [conn=284] Key exchange algs: diffie-hellman-group1-sha1
2019-04-24 11:03:55,306 [conn=284] Host key algs: ssh-rsa
2019-04-24 11:03:55,306 [conn=284] Client to server:
2019-04-24 11:03:55,306 [conn=284] Encryption algs: aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
2019-04-24 11:03:55,306 [conn=284] MAC algs: hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96
2019-04-24 11:03:55,306 [conn=284] Compression algs: none
2019-04-24 11:03:55,306 [conn=284] Server to client:
2019-04-24 11:03:55,307 [conn=284] Encryption algs: aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
2019-04-24 11:03:55,307 [conn=284] MAC algs: hmac-sha1,hmac-sha1-96,hmac-md5,hmac-md5-96
2019-04-24 11:03:55,307 [conn=284] Compression algs: none
2019-04-24 11:03:55,307 [conn=284] Beginning key exchange
2019-04-24 11:03:55,307 [conn=284] Key exchange alg: diffie-hellman-group1-sha1
2019-04-24 11:03:55,311 [conn=284, pktid=1] Sent MSG_KEXDH_INIT (30), 133 bytes
00000000: 1e 00 00 00 80 3e 65 5c af d2 40 65 d0 dc 07 ff .....>e\..@e....
00000010: 91 cc 82 6f 47 02 46 67 92 98 ae 3b ae 64 16 d4 ...oG.Fg...;.d..
00000020: a5 6d 9a e2 a0 c7 43 ff c9 b9 32 f3 eb 7c 02 8d .m....C...2..|..
00000030: 02 6e bb 2d e1 32 45 7e 3a 68 ab 47 c7 14 1b b4 .n.-.2E~:h.G....
00000040: 20 6a 61 45 95 f9 35 61 8a 25 96 5e 76 23 e3 46 jaE..5a.%.^v#.F
00000050: 5f 57 10 5e 27 63 ef f1 ea 30 84 07 ef 91 ce c4 _W.^'c...0......
00000060: 86 ba e1 57 6a 30 2b 33 c1 1e 66 83 98 72 f3 ed ...Wj0+3..f..r..
00000070: 2b 7d 05 54 cf af 2e 1f 96 d7 29 92 62 87 2a a6 +}.T......).b.*.
00000080: 25 e8 b2 06 7c %...|
2019-04-24 11:03:55,746 [conn=284] Connection failure: [Errno 104] Connection reset by peer.
Temp solution was to fix the asyncssh version: asyncssh==1.15.1
An issue occurs when a commit on IOS-XR results in an error. I see this with CiscoIOSXR.send_config_set()
. After the code sends the commit
, it then runs IOSLikeDevice._read_until_prompt()
, but if the commit fails, then _read_until_prompt()
will wait forever.
Here is an example of an interactive session that produces a failed commit (I presume it doesn't really matter what produces the commit failure, as long as it fails):
RP/0/RSP0/CPU0:xxx#conf t
Fri Feb 9 19:26:29.685 GMT
RP/0/RSP0/CPU0:xxx(config)#interface GigabitEthernet0/0/1/2.2859
RP/0/RSP0/CPU0:xxx(config-subif)#service-policy input 03692d7b-9acd-4a14-b4ed-345f513518$
RP/0/RSP0/CPU0:xxx(config-subif)#service-policy output 03692d7b-9acd-4a14-b4ed-345f51351$
RP/0/RSP0/CPU0:xxx(config-subif)#
RP/0/RSP0/CPU0:xxx(config-subif)#
RP/0/RSP0/CPU0:xxx(config-subif)#commit
Fri Feb 9 19:26:53.050 GMT
% Failed to commit one or more configuration items during a pseudo-atomic operation. All changes made have been reverted. Please issue 'show configuration failed [inheritance]' from this session to view the errors
RP/0/RSP0/CPU0:xxx(config-subif)#end
Uncommitted changes found, commit them before exiting(yes/no/cancel)? [cancel]:no
(It fails because the policies mentioned in the config do not exist)
Note that the prompt does not return after "end" is entered: instead, the returned bytes are Uncommitted changes found, commit them before exiting(yes/no/cancel)? [cancel]:
. This is the cause of the problem.
This is the code in CiscoIOSXR.send_config_set()
:
async def send_config_set(self, config_commands=None, with_commit=True, commit_comment='', exit_config_mode=True):
# Send config commands by IOS Like Device
output = await super().send_config_set(config_commands=config_commands, exit_config_mode=False)
if with_commit:
commit = type(self)._commit_command
if commit_comment:
commit = type(self)._commit_comment_command.format(commit_comment)
self._stdin.write(self._normalize_cmd(commit))
output += await self._read_until_prompt()
if exit_config_mode:
output += await self.exit_config_mode() # <<-------- HERE
The commit
, and then the end
, is issued on the line marked above. This runs the method in the IOSLikeDevice
superclass:
async def exit_config_mode(self):
logger.info('Host {}: Exiting from configuration mode'.format(self._host))
output = ''
exit_config = type(self)._config_exit
if await self.check_config_mode():
self._stdin.write(self._normalize_cmd(exit_config))
output = await self._read_until_prompt() # <<------------ HERE
if await self.check_config_mode():
raise ValueError("Failed to exit from configuration mode")
return output
In the line marked above, if the commit has failed, and then end
is sent (inside exit_config()
), the prompt will never be found, because read_until_prompt()
simply calls read_until_pattern(self._base_pattern)
:
async def _read_until_pattern(self, pattern='', re_flags=0):
output = ''
logger.info("Host {}: Reading until pattern".format(self._host))
if not pattern:
pattern = self._base_pattern
logger.debug("Host {}: Reading pattern: {}".format(self._host, pattern))
while True:
output += await self._stdout.read(self._MAX_BUFFER)
if re.search(pattern, output, flags=re_flags): # <<----------- HERE
logger.debug("Host {}: Reading pattern '{}' was found: {}".format(self._host, pattern, repr(output)))
return output
Eventually, the device returns all the bytes up to the end of the confirmation request:
Uncommitted changes found, commit them before exiting(yes/no/cancel)? [cancel]:
And of course no prompt can be detected in this, and no more bytes are coming.
I will do a bit more debugging soon, but I just wanted to raise this issue now to get your thoughts. I'm happy to do the work to fix or improve the handling of errors during commit, if you could give your thoughts, or make any suggestions of how best to proceed?
One idea that I had while looking at the code was to add timeout handling inside _read_until_pattern()
:
while True:
output += await self._stdout.read(self._MAX_BUFFER) # <<------ on this line
only so that the output can be captured and logged. Currently, if the code waits here, and an asyncio.wait_for()
triggers due to timeout in an outer scope, any bytes currently pending in the output
identifier will be lost. By capturing and logging those bytes (and then possibly reraising a TimeoutError), it should make it easier to diagnose why a timeout occurred.
But for the specific case where we get the Uncommitted changes found
problem, it is probably better to try to match that also in read_until_pattern
, so that it can be handled directly, rather than waiting for a timeout. I'm not sure yet, and would like to know what your thoughts are before I do any work on this.
By the way, thanks for making this library!
I need to rename this library for avoiding collision with linux netdev. So if you have some good idea about new name - write here!
Hello,
here is the logs:
INFO:netdev:Host 192.168.47.6: Trying to connect to the device
INFO:netdev:Host 192.168.47.6: Establishing connection to port 22
INFO:asyncssh:Opening SSH connection to 192.168.47.6, port 22
INFO:asyncssh:[conn=0] Connection to 192.168.47.6, port 22 succeeded
INFO:asyncssh:[conn=0] Local address: 192.168.175.160, port 28247
INFO:asyncssh:[conn=0] Beginning auth for user xxxx
INFO:asyncssh:[conn=0] Auth for user xxxx succeeded
INFO:asyncssh:[conn=0, chan=0] Requesting new SSH session
INFO:asyncssh:[conn=0, chan=0] Interactive shell requested
INFO:netdev:Host 192.168.47.6: Connection is established
INFO:netdev:Host 192.168.47.6: Reading until pattern
DEBUG:netdev:Host 192.168.47.6: Reading pattern: \>|\]
DEBUG:netdev:Host 192.168.47.6: Reading pattern '\>|\]' was found: "\r\n*************************************************************************\r\n* Copyright (C) 2007-2017 *\r\n* Huawei Technologies Co., Ltd. *\r\n* All rights reserved. *\r\n* Without the owner's prior written consent, *\r\n* no decompiling or reverse-engineering shall be allowed. *\r\n*************************************************************************\r\n\r\n\r\nInfo: The max number of VTY users is 10, and the number\r\n of current VTY users on line is 3.\r\n The current login time is 2020-09-13 12:16:56+00:00.\r\nHRP_M<WLWZW-ST-B31-E13-E8000E-C01>"
DEBUG:netdev:Host 192.168.47.6: Establish Connection Output: "\r\n*************************************************************************\r\n* Copyright (C) 2007-2017 *\r\n* Huawei Technologies Co., Ltd. *\r\n* All rights reserved. *\r\n* Without the owner's prior written consent, *\r\n* no decompiling or reverse-engineering shall be allowed. *\r\n*************************************************************************\r\n\r\n\r\nInfo: The max number of VTY users is 10, and the number\r\n of current VTY users on line is 3.\r\n The current login time is 2020-09-13 12:16:56+00:00.\r\nHRP_M<WLWZW-ST-B31-E13-E8000E-C01>"
INFO:netdev:Host 192.168.47.6: Setting base prompt
INFO:netdev:Host 192.168.47.6: Finding prompt
INFO:netdev:Host 192.168.47.6: Reading until pattern
DEBUG:netdev:Host 192.168.47.6: Reading pattern: \>|\]
DEBUG:netdev:Host 192.168.47.6: Reading pattern '\>|\]' was found: '\r\nHRP_M<WLWZW-ST-B31-E13-E8000E-C01>'
DEBUG:netdev:Host 192.168.47.6: Found Prompt: 'HRP_M<WLWZW-ST-B31-E13-E8000E-C01>'
DEBUG:netdev:Host 192.168.47.6: Base Prompt: RP_M<WLWZW-ST-B31-E13-E8000E-C01
DEBUG:netdev:Host 192.168.47.6: Base Pattern: [\<|\[]RP_M\<WLWZW\-S[\-\w]*[\>|\]]
INFO:netdev:Host 192.168.47.6: Trying to disable paging
DEBUG:netdev:Host 192.168.47.6: Disable paging command: 'screen-length disable\n'
INFO:netdev:Host 192.168.47.6: Reading until pattern
DEBUG:netdev:Host 192.168.47.6: Reading pattern: [\<|\[]RP_M\<WLWZW\-S[\-\w]*[\>|\]]
ERROR:asyncio:Task exception was never retrieved
future: <Task finished coro=<task() done, defined at yibu.py:5> exception=TimeoutError('Host 192.168.47.6 Timeout Error',)>
Traceback (most recent call last):
File "/root/cli/dev/lib/python3.6/site-packages/netdev/vendors/base.py", line 359, in _read_until_pattern
output += await asyncio.wait_for(fut, self._timeout)
File "/usr/lib/python3.6/asyncio/tasks.py", line 362, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
I think that the problem might be my hostname begins with "HRP_M" befor "<" , because of the HRP hot standby. I just don`t know how to fix it
Need to add pattern arg to init for manual managing _pattern for devices
I have this problem
netdev with asyncssh<2.0,>=1.15
and scrapli with asyncssh<3.0.0,>=2.2.1
Could you upgrade netdev with new version Asyncssh and make compatibility
new to netdev. just wanted to test it with an existing script and it kept pausing after initial connecting and then waiting indefitnley, so I tried using the sample script to see if it was something I was doing. and I get the same result.
sample script:
#!/usr/bin/env python
from django_setup import setup
setup()
import re, argparse, time, ipaddress, asyncio, netdev, logging
from home.models import Credentials
logging.basicConfig(level=logging.INFO)
netdev.logger.setLevel(logging.DEBUG)
async def task(param):
async with netdev.create(**param) as ios:
# Testing sending simple command
out = await ios.send_command("show ver")
print(out)
# Testing sending configuration set
commands = ["line console 0", "exit"]
out = await ios.send_config_set(commands)
print(out)
# Testing sending simple command with long output
out = await ios.send_command("show run")
print(out)
# Testing interactive dialog
out = await ios.send_command("conf", pattern=r'\[terminal\]\?', strip_command=False)
out += await ios.send_command("term", strip_command=False)
out += await ios.send_command("exit", strip_command=False, strip_prompt=False)
print(out)
async def run():
dev1 = {
'username': 'autoconfiguration',
'password': 'xxxxxxxx',
'device_type': 'cisco_ios',
'host': '10.10.10.2',
}
devices = [dev1]
tasks = [task(dev) for dev in devices]
await asyncio.wait(tasks)
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
output from the script is:
INFO:netdev.vendors:Host 10.10.10.2: Trying to connect to the device
INFO:netdev.vendors:Host 10.10.10.2: Establishing connection to port 22
INFO:asyncssh:Opening SSH connection to 10.10.10.2, port 22
INFO:asyncssh:[conn=0] Connection to 10.10.10.2, port 22 succeeded
INFO:asyncssh:[conn=0] Local address: 172.17.0.2, port 48790
INFO:asyncssh:[conn=0] Beginning auth for user autoconfiguration
INFO:asyncssh:[conn=0] Auth for user autoconfiguration succeeded
INFO:asyncssh:[conn=0, chan=0] Requesting new SSH session
INFO:asyncssh:[conn=0, chan=0] Interactive shell requested
INFO:netdev.vendors:Host 10.10.10.2: Connection is established
INFO:netdev.vendors:Host 10.10.10.2: Reading until pattern
DEBUG:netdev.vendors:Host 10.10.10.2: Reading pattern: \>|\#
DEBUG:netdev.vendors:Host 10.10.10.2: Reading pattern '\>|\#xxxxx
DEBUG:netdev.vendors:Host 110.10.10.2: Establish Connection Output: xxxx
INFO:netdev.vendors:Host 110.10.10.2: Setting base prompt
INFO:netdev.vendors:Host 10.10.10.2 Finding prompt
INFO:netdev.vendors:Host 10.10.10.2: Reading until pattern
DEBUG:netdev.vendors:Host 10.10.10.2: Reading pattern: \>|\#
DEBUG:netdev.vendors:Host 10.10.10.2: Reading pattern '\>|\xxxx
DEBUG:netdev.vendors:Host 10.10.10.2: Base Prompt: xxxx
STR-LAB-01-RTR-01
DEBUG:netdev.vendors:Host 10.10.10.2: Base Pattern: \$\$\$\$\$\$\ \ \ \ \ \ .*?(\(.*?\))?[\>|\#]
INFO:netdev.vendors:Host 10.10.10.2: Entering to privilege exec
INFO:netdev.vendors:Host 10.10.10.2: Checking privilege exec
INFO:netdev.vendors:Host 10.10.10.2: Reading until pattern
DEBUG:netdev.vendors:Host 10.10.10.2: Reading pattern: \$\$\$\$\$\$\ \ \ \ \ \ .*?(\(.*?\))?[\>|\#]
it stops here and never continues
Hi,
I am able to connect device manually. but using netdev. I am getting Timeout. do i increase any timeout value in script ? if yes , how ?. Enable password is same. device are in active/standby.
enable mode prompt looks like this. We don't have any context
Hostname/pri/act>
Task exception was never retrieved
future: <Task finished coro=<backup_config() done, defined at C:/Users/anarang/PycharmProjects/lambda/lambda-asa.py:10> exception=TimeoutError('Host 10.33.224.19 Timeout Error')>
Traceback (most recent call last):
File "C:\Users\anarang\PycharmProjects\venv\lib\site-packages\netdev\vendors\base.py", line 243, in _read_until_pattern
output += await asyncio.wait_for(fut, self._timeout)
File "C:\Users\anarang\AppData\Local\Programs\Python\Python37\lib\asyncio\tasks.py", line 419, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
raceback (most recent call last):
File "C:/Users/anarang/PycharmProjects/lambda/lambda-asa.py", line 13, in backup_config
await io.connect()
File "C:\Users\anarang\PycharmProjects\venv\lib\site-packages\netdev\vendors\cisco\cisco_asa.py", line 54, in connect
await self.enable_mode()
File "C:\Users\anarang\PycharmProjects\venv\lib\site-packages\netdev\vendors\ios_like.py", line 100, in enable_mode
output += await self._read_until_prompt()
File "C:\Users\anarang\PycharmProjects\venv\lib\site-packages\netdev\vendors\base.py", line 231, in _read_until_prompt
return await self._read_until_pattern(self._base_pattern)
File "C:\Users\anarang\PycharmProjects\venv\lib\site-packages\netdev\vendors\base.py", line 245, in _read_until_pattern
raise TimeoutError(self._host)
netdev.exceptions.TimeoutError: Host 10.33.224.19 Timeout Error
Task exception was never retrieved
future: <Task finished coro=<backup_config() done, defined at C:/Users/anarang/PycharmProjects/lambda/lambda-asa.py:10> exception=TimeoutError('Host 10.33.224.18 Timeout Error')>
Traceback (most recent call last):
File "C:\Users\anarang\PycharmProjects\venv\lib\site-packages\netdev\vendors\base.py", line 243, in _read_until_pattern
output += await asyncio.wait_for(fut, self._timeout)
File "C:\Users\anarang\AppData\Local\Programs\Python\Python37\lib\asyncio\tasks.py", line 419, in wait_for
raise futures.TimeoutError()
concurrent.futures._base.TimeoutError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/anarang/PycharmProjects/lambda/lambda-asa.py", line 13, in backup_config
await io.connect()
File "C:\Users\anarang\PycharmProjects\venv\lib\site-packages\netdev\vendors\cisco\cisco_asa.py", line 54, in connect
await self.enable_mode()
File "C:\Users\anarang\PycharmProjects\venv\lib\site-packages\netdev\vendors\ios_like.py", line 100, in enable_mode
output += await self._read_until_prompt()
File "C:\Users\anarang\PycharmProjects\venv\lib\site-packages\netdev\vendors\base.py", line 231, in _read_until_prompt
return await self._read_until_pattern(self._base_pattern)
File "C:\Users\anarang\PycharmProjects\venv\lib\site-packages\netdev\vendors\base.py", line 245, in _read_until_pattern
raise TimeoutError(self._host)
netdev.exceptions.TimeoutError: Host 10.33.224.18 Timeout Error
During handling of the above exception, another exception occurred:
Hi,
vendors/base.py has a dict, with all the arguments for asyncssh:
self._connect_params_dict = {
"host": self._host,
"port": self._port,
"username": username,
"password": password,
"known_hosts": known_hosts,
"local_addr": local_addr,
"client_keys": client_keys,
"passphrase": passphrase,
"tunnel": tunnel,
"agent_forwarding": agent_forwarding,
"loop": loop,
"family": family,
"agent_path": agent_path,
"client_version": client_version,
"kex_algs": kex_algs,
"encryption_algs": encryption_algs,
"mac_algs": mac_algs,
"compression_algs": compression_algs,
"signature_algs": signature_algs,
}
But asyncssh is not allowing passing the loop argument anymore.
class SSHConnectionOptions(Options):
"""SSH connection options"""
def __init__(self, options=None, **kwargs):
last_config = options.config if options else None
super().__init__(options=options, last_config=last_config, **kwargs)
# pylint: disable=arguments-differ
def prepare(self, config, protocol_factory, version, host, port, tunnel,
family, local_addr, tcp_keepalive, kex_algs, encryption_algs,
mac_algs, compression_algs, signature_algs, host_based_auth,
public_key_auth, kbdint_auth, password_auth,
x509_trusted_certs, x509_trusted_cert_paths, x509_purposes,
rekey_bytes, rekey_seconds, login_timeout, keepalive_interval,
keepalive_count_max):
This causes the following exception:
File "/usr/lib/python3.6/site-packages/asyncssh/misc.py", line 254, in __init__
self.prepare(**self.kwargs)
TypeError: prepare() got an unexpected keyword argument 'loop'
As I see, asyncssh is getting the current loop, so probably the solution would be to just remove the loop argument from the dictionary.
asyncssh/connection.py:
def conn_factory():
"""Return an SSH client connection factory"""
return SSHClientConnection(loop, options, wait='auth')
loop = asyncio.get_event_loop()
options = SSHClientConnectionOptions(options, config=config, host=host,
port=port, tunnel=tunnel,
family=family, local_addr=local_addr,
**kwargs)
return await _connect(options.host, options.port, loop, options.tunnel,
options.family, flags, options.local_addr,
conn_factory, 'Opening SSH connection to')
Hi, is there any way to create a connection pool?
Describe the bug
Prompt pattern brn\-boll111\-.*?(\(.*?\))?[\>|\#]
matches in show interfaces description
on an interface that has brn-boll111-ap-02 Gi0 <LS:LIR>
in its description.
General Information
OS:
RHEL7, python3.6
Netdev version:
asyncssh==1.18.0
netdev==0.9.3
Device OS:
Cisco IOS XE Software, Version 16.12.03s
C9300
Debug information
[2020-12-10 10:30:09,152 DEBUG base.py:388 - _read_until_prompt_or_pattern() pn:MainProcess pid:31799 t:MainThread ] Host <SNIP>: Reading pattern 'brn\-boll111\-.*?(\(.*?\))?[\>|\#]' or 'brn\-boll111\-.*?(\(.*?\))?[\>|\#]' was found: 'show interfaces description\r\nInterface Status Protocol Description\r\nVl1 up up not used <LS:C>\r\n<SNIP> <LS:CV>\r\nTe3/0/20 down down Access-Port with 802.1x - lan <LS:CV>\r\nTe3/0/21 down down Access-Port with 802.1x - lan <LS:CV>\r\nTe3/0/22 down down Access-Port with 802.1x - lan <LS:CV>\r\nTe3/0/23 up up Fabric Physical Link - brn-boll111-ap-02 Gi0 <LS:LIR>\r\nTe3/0/24 up up Fabric Physical Link - brn-boll111-ap-02 Gi0 <LS:LIR>\r\nGi3/1/1 admin down down not used <LS:C>\r\nGi3/1/2 admin down down not used <LS:C>\r\nGi3/1/3 admin down down not used <LS:C>\r\nGi3/1/4 admin down down not used <LS:C>\r\nTe3'
Additional context
Any ideas for a quick fix until the problem can be solved permanently?
It would be awesome if you can add support for terminal_server & aruba
https://github.com/ktbyers/netmiko/tree/develop/netmiko/aruba
https://github.com/ktbyers/netmiko/tree/develop/netmiko/terminal_server
These are already supported by netmiko. Looking forward to netdev magic on these.
Hello,
I am trying netdev with the following code:
async def task(param):
print('before async with')
async with netdev.create(**param) as ios:
print('after async with')
out = await ios.send_config_set([CONFIG])
async def run():
devices = []
for ip, driver in DISPATCHER:
devices.append({
'username': USERNAME,
'password': PASSWORD,
'device_type': driver,
'host': ip
})
tasks = [task(dev) for dev in devices]
await asyncio.wait(tasks)
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
My issue is that I don't get past the "async with" step, i.e the print('before async with')
is displayed, but not the print('after async with')
. Netdev is connected to the device (ASK1K, IOS-XE 15.3(3)S3), because I see the VTY after starting the script, but the VTY / the script stays idle.
When I KeyboardInterrupt, it says the task is pending:
afourmy@afourmy-virtual-machine:~/automation$ python3.5 async_netmiko.py
before async with
^CTraceback (most recent call last):
File "async_netmiko.py", line 34, in <module>
loop.run_until_complete(run())
File "/usr/lib/python3.5/asyncio/base_events.py", line 375, in run_until_complete
self.run_forever()
File "/usr/lib/python3.5/asyncio/base_events.py", line 345, in run_forever
self._run_once()
File "/usr/lib/python3.5/asyncio/base_events.py", line 1276, in _run_once
event_list = self._selector.select(timeout)
File "/usr/lib/python3.5/selectors.py", line 441, in select
fd_event_list = self._epoll.poll(timeout, max_ev)
KeyboardInterrupt
Task was destroyed but it is pending!
task: <Task pending coro=<task() done, defined at async_netmiko.py:13> wait_for=<Future pending cb=[Task._wakeup()]> cb=[_wait.<locals>._on_completion() at /usr/lib/python3.5/asyncio/tasks.py:414]>
Environment: afourmy@afourmy-virtual-machine:~/automation$ uname -a Linux afourmy-virtual-machine 4.10.0-28-generic #32~16.04.2-Ubuntu SMP Thu Jul 20 10:19:48 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Any idea what's wrong ?
Thanks
Describe the bug
Hi,
We are observing a condition where the asyncio task getting stuck at send_command method in Netdev library. This condition happens for the device models:
When we enabled the netdev logs, we saw that the library hangs in:
2021-06-22T20:28:40.233974439Z Host : Stripping prompt
2021-06-22T20:28:40.234000040Z Stripping command
2021-06-22T20:28:40.235203004Z Host Disconnecting
2021-06-22T20:28:40.235248606Z Host Cleanup session
2021-06-22T20:28:40.235309610Z Host Exiting from configuration mode
2021-06-22T20:28:40.235333911Z Host Checking configuration mode
2021-06-22T20:28:40.235739232Z Host Reading until pattern
A clear and concise description of what the bug is.
We observe that when the condition happens, the code hangs and the control is not coming to the main method. While following the library code, on where the code hangs, the netdev library code hangs in: _read_until_pattern method, inside the while True block, more specifically on the re.search line(if re.search(pattern, output, flags=re_flags): in vendors/base.py:362. Since it is happening only for few models, we suspect the re.search method hangs because of catastrophic backtracking while searching on the base_pattern. If possible, can you please add a functionality to raise an exception if this condition occurs?
Thanks for your help in advance.
Regards,
Mani
Hello!
I found out a problem with rtfd and outdated docs. Need to 'make it great again'
Hi,
what is default timeout value to connecting to device and what is retry number if fails to connect in one time ?
How can i change retry or timeout value ?
Thanks
Hello, netdev execution fails in case ASA is configured with non-default prompt. Like "(config)# prompt hostname priority state". There is a small fix which is not ideal, but helps to avoid exceptions in some longer prompts.
prompt = await self._find_prompt()
context = 'system'
# Cut off prompt from "prompt/context"
splitted_prompt = prompt[:-1].split('/')
# In case prompt is "hostname/context"
if len(splitted_prompt) == 2:
prompt = splitted_prompt[0]
context = splitted_prompt[1]
# In other cases like "hostname" or "hostname/priority/state" or whatever else
else:
prompt = splitted_prompt[0]
self._base_prompt = prompt
self._current_context = context
delimiters = map(re.escape, type(self)._delimiter_list)
After that fix module works like a charm for me!
Thank you very much for your project.
I decided to change my license to Apache 2.0
I think the issue I open for async ssh fits here because #I am using this to fetch the output.
Need to create mock tests for fast developing otherwise I need to test all classes after each significant commit to base classes. It's too hard and I can't use it in CI/CD
Using hostname instead of IP Address breaks the script. Can you add support for hostname please?
Need to create a proxy class for ssh tunneling (I've already added support tunneling to init param but still haven't proxy class for doing that)
Hello,
I am trying netdev with the following code:
import asyncio
import netdev
import logging
import netdev
logging.basicConfig(level=logging.INFO)
netdev.logger.setLevel(logging.DEBUG)
async def task(param):
async with netdev.create(**param) as ios:
# Testing sending simple command
out = await ios.send_command("show ip int br")
print(out)
async def run():
dev1 = { 'username' : 'calvin',
'password' : 'password',
'device_type': 'cisco_ios',
'host': '192.168.129.132',
}
devices = [dev1]
tasks = [task(dev) for dev in devices]
await asyncio.wait(tasks)
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
I seem to getting an error where it is not executing, I have enabled logging and I am receiving the following error:
INFO:netdev:Host 192.168.129.132: Trying to connect to the device
INFO:netdev:Host 192.168.129.132: Establishing connection to port 22
INFO:asyncssh:Opening SSH connection to 192.168.129.132, port 22
INFO:asyncssh:[conn=0] Connection to 192.168.129.132, port 22 succeeded
INFO:asyncssh:[conn=0] Local address: 192.168.129.1, port 52907
INFO:asyncssh:[conn=0] Beginning auth for user calvin
INFO:asyncssh:[conn=0] Auth for user calvin succeeded
INFO:asyncssh:[conn=0, chan=0] Requesting new SSH session
INFO:asyncssh:[conn=0, chan=0] Interactive shell requested
INFO:netdev:Host 192.168.129.132: Connection is established
INFO:netdev:Host 192.168.129.132: Reading until pattern
DEBUG:netdev:Host 192.168.129.132: Reading pattern: >|\#
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-2' coro=<task() done, defined
Can I please receive help?
I don't see any reference to telnet support in the code. It would be good if you could add it.
I regularly use telnet to test connections/and my code in an emulated env (using eve-ng).
Currently I've been using Netmiko, which supports this, I would love it if you could add the support in.
Thanks.
How do we handle errors when a device cannot be connected to? currently I et
File "/usr/local/lib/python3.6/asyncio/selector_events.py", line 480, in _sock_connect_cb
raise OSError(err, 'Connect call failed %s' % (address,))
ConnectionRefusedError: [Errno 111] Connect call failed ('10.10.10.2', 22)
I would just like to print an exception i.e. couldn't connect to X. at the moment it aborts my script
I can see in base.py the timeout is set to 15 seconds, I tried to set the timeout in my params but got an error. how do we set timeout value?
Thanks
EDIT, resolve issue by upgrading net dev and having the following:
async def task(param):
try:
async with netdev.create(**param, timeout=5) as ios:
# Testing sending simple command
out = await ios.send_command("show ver")
print(out)
except:
print('unable to connect to device')
Hi.
In the documentation I would suggest to replace:
###############################
import netdev
netdev_logger = netdev.logger
netdev_logger.setLevel(logging.INFO)
netdev_logger.addHandler(logging.StreamHandler())
#Your own code
###############################
With;
###############################
import netdev
import logging
netdev_logger = netdev.logger
netdev_logger.setLevel(logging.INFO)
netdev_logger.addHandler(logging.StreamHandler())
#Your own code
###############################
Since that code would fail for the 2 last instructions.
Is it possible to download an IOS for example from an ftp server using netdev? ive tried with the below
try:
async with netdev.create(**conn[0], timeout=5) as ios:
print('starting process on device {}.'.format(router_hostname))
upload_ios = await ios.send_command('copy ftp://user:[email protected]/test.txt flash:')
except:
error = 'unable to connect to {} - {}'.format(conn[0]['host'], router_hostname)
print(error)
results.append(error)
debugging gives the below output
INFO:netdev:Host 10.10.10.10: Reading until prompt or pattern
INFO:netdev:Host 10.10.10.10: Disconnecting
INFO:netdev:Host 10.10.10.10: Cleanup session
INFO:netdev:Host 10.10.10.10: Exiting from configuration mode
INFO:netdev:Host 10.10.10.10: Checking configuration mode
INFO:netdev:Host 10.10.10.10: Reading until pattern
DEBUG:netdev:Host 10.10.10.10: Reading pattern: LAB\-01\-R.*?(\(.*?\))?[\>|\#]
unable to connect to 10.10.10.10 - LAB 01
im guessing at this stage its not possible?
Thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.