pixieofhugs / cs410agile Goto Github PK

Our tests are located in the tests folder. To run the tests through the terminal, just run "pytest" from the project's root folder. (Be sure to create tests/test_server.py with the constants HOSTNAME, USERNAME, and PASSWORD first.)

Libraries

We heavily relied on the following libraries:

Software Development: pysftp and paramiko.

Testing: pytest.

cs410agile's People

Contributors

Watchers

Forkers

shwetapaliwal28 edev danc2050

cs410agile's Issues

Missing tests (and probably exception-handling code) for permissions issues

Across all user stories up to PR #18, we are most likely missing both test cases and exception cases for permissions issues. For instance, what if we try to put a file onto the remote server, but we don't have permission to write to the containing folder? Most likely outcome: our client crashes.

If time allows, we should go back and add test code (and exception handling as required).

Sporadic test failures over campus Wi-Fi

Tests sometimes generate errors over campus Wi-Fi that are most likely due to either lag or network security policies. The tests pass when run on team members' home connections and pass when run individually, but opening 20+ SFTP connections to the same server in a short time span, on campus Wi-Fi, tends to cause issues. The root causes appears to be a race condition in paramiko or pysftp. Example output is below. Any E in the tests that corresponds to a Paramiko exception of the form

E               paramiko.ssh_exception.SSHException: Error reading SSH protocol banner

is an issue of this kind.

[dylan@localhost git]$ pytest
========================================== test session starts ==========================================
platform linux -- Python 3.7.2, pytest-4.2.0, py-1.7.0, pluggy-0.8.1
rootdir: /home/dylan/Dropbox/Computer Science/Classes/CS 510 - Agile Software Development/git, inifile:
collected 21 items                                                                                      

tests/integration/test_basic_skeleton.py .E                                                       [  9%]
tests/integration/test_close.py EE                                                                [ 19%]
tests/integration/test_list_local_files.py ..                                                     [ 28%]
tests/integration/test_login.py ..                                                                [ 38%]
tests/integration/test_put_file_onto_remote_server.py ..                                          [ 47%]
tests/unit/test_close.py .                                                                        [ 52%]
tests/unit/test_input_handler.py ...                                                              [ 66%]
tests/unit/test_list_files_local.py .                                                             [ 71%]
tests/unit/test_login.py ..                                                                       [ 80%]
tests/unit/test_put_file_onto_remote_server.py ..                                                 [ 90%]
tests/unit/test_sftp_fixture.py ..                                                                [100%]

================================================ ERRORS =================================================
__________________________________ ERROR at setup of test_end_of_file ___________________________________

self = <paramiko.Transport at 0x258caa20 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
>               buf = self.packetizer.readline(timeout)

/usr/lib/python3.7/site-packages/paramiko/transport.py:2138: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f862472c828>, timeout = 15

    def readline(self, timeout):
        """
        Read a line from the socket.  We assume no data is pending after the
        line, so it's okay to attempt large reads.
        """
        buf = self.__remainder
        while linefeed_byte not in buf:
>           buf += self._read_timeout(timeout)

/usr/lib/python3.7/site-packages/paramiko/packet.py:367: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f862472c828>, timeout = 15

    def _read_timeout(self, timeout):
        start = time.time()
        while True:
            try:
                x = self.__socket.recv(128)
                if len(x) == 0:
                    raise EOFError()
                break
            except socket.timeout:
                pass
            except EnvironmentError as e:
                if first_arg(e) == errno.EINTR:
                    pass
                else:
                    raise
            if self.__closed:
                raise EOFError()
            now = time.time()
            if now - start >= timeout:
>               raise socket.timeout()
E               socket.timeout

/usr/lib/python3.7/site-packages/paramiko/packet.py:576: timeout

During handling of the above exception, another exception occurred:

    @pytest.fixture(scope="function")
    def sftp():
        """This fixture provides a pysftp.Connection object that's shared
        across the entire tests package. If there's a problem connecting,
        it will raise an exception that will terminate testing."""
        return pysftp.Connection(
            host=HOSTNAME,
            username=USERNAME,
>           password=PASSWORD
        )

tests/conftest.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.7/site-packages/pysftp/__init__.py:143: in __init__
    self._transport.connect(**self._tconnect)
/usr/lib/python3.7/site-packages/paramiko/transport.py:1218: in connect
    self.start_client()
/usr/lib/python3.7/site-packages/paramiko/transport.py:587: in start_client
    raise e
/usr/lib/python3.7/site-packages/paramiko/transport.py:1966: in run
    self._check_banner()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.Transport at 0x258caa20 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
                buf = self.packetizer.readline(timeout)
            except ProxyCommandFailure:
                raise
            except Exception as e:
                raise SSHException(
>                   "Error reading SSH protocol banner" + str(e)
                )
E               paramiko.ssh_exception.SSHException: Error reading SSH protocol banner

/usr/lib/python3.7/site-packages/paramiko/transport.py:2143: SSHException
------------------------------------------ Captured log setup -------------------------------------------
transport.py              1746 ERROR    Exception: Error reading SSH protocol banner
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2138, in _check_banner
transport.py              1744 ERROR        buf = self.packetizer.readline(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 367, in readline
transport.py              1744 ERROR        buf += self._read_timeout(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 576, in _read_timeout
transport.py              1744 ERROR        raise socket.timeout()
transport.py              1744 ERROR    socket.timeout
transport.py              1744 ERROR    
transport.py              1744 ERROR    During handling of the above exception, another exception occurred:
transport.py              1744 ERROR    
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 1966, in run
transport.py              1744 ERROR        self._check_banner()
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2143, in _check_banner
transport.py              1744 ERROR        "Error reading SSH protocol banner" + str(e)
transport.py              1744 ERROR    paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
transport.py              1744 ERROR
______________________ ERROR at setup of test_close_commands_invoke_close_function ______________________

self = <paramiko.Transport at 0x23485390 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
>               buf = self.packetizer.readline(timeout)

/usr/lib/python3.7/site-packages/paramiko/transport.py:2138: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f862358b630>, timeout = 15

    def readline(self, timeout):
        """
        Read a line from the socket.  We assume no data is pending after the
        line, so it's okay to attempt large reads.
        """
        buf = self.__remainder
        while linefeed_byte not in buf:
>           buf += self._read_timeout(timeout)

/usr/lib/python3.7/site-packages/paramiko/packet.py:367: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f862358b630>, timeout = 15

    def _read_timeout(self, timeout):
        start = time.time()
        while True:
            try:
                x = self.__socket.recv(128)
                if len(x) == 0:
                    raise EOFError()
                break
            except socket.timeout:
                pass
            except EnvironmentError as e:
                if first_arg(e) == errno.EINTR:
                    pass
                else:
                    raise
            if self.__closed:
                raise EOFError()
            now = time.time()
            if now - start >= timeout:
>               raise socket.timeout()
E               socket.timeout

/usr/lib/python3.7/site-packages/paramiko/packet.py:576: timeout

During handling of the above exception, another exception occurred:

    @pytest.fixture(scope="function")
    def sftp():
        """This fixture provides a pysftp.Connection object that's shared
        across the entire tests package. If there's a problem connecting,
        it will raise an exception that will terminate testing."""
        return pysftp.Connection(
            host=HOSTNAME,
            username=USERNAME,
>           password=PASSWORD
        )

tests/conftest.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.7/site-packages/pysftp/__init__.py:143: in __init__
    self._transport.connect(**self._tconnect)
/usr/lib/python3.7/site-packages/paramiko/transport.py:1218: in connect
    self.start_client()
/usr/lib/python3.7/site-packages/paramiko/transport.py:587: in start_client
    raise e
/usr/lib/python3.7/site-packages/paramiko/transport.py:1966: in run
    self._check_banner()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.Transport at 0x23485390 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
                buf = self.packetizer.readline(timeout)
            except ProxyCommandFailure:
                raise
            except Exception as e:
                raise SSHException(
>                   "Error reading SSH protocol banner" + str(e)
                )
E               paramiko.ssh_exception.SSHException: Error reading SSH protocol banner

/usr/lib/python3.7/site-packages/paramiko/transport.py:2143: SSHException
------------------------------------------ Captured log setup -------------------------------------------
transport.py              1746 ERROR    Exception: Error reading SSH protocol banner
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2138, in _check_banner
transport.py              1744 ERROR        buf = self.packetizer.readline(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 367, in readline
transport.py              1744 ERROR        buf += self._read_timeout(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 576, in _read_timeout
transport.py              1744 ERROR        raise socket.timeout()
transport.py              1744 ERROR    socket.timeout
transport.py              1744 ERROR    
transport.py              1744 ERROR    During handling of the above exception, another exception occurred:
transport.py              1744 ERROR    
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 1966, in run
transport.py              1744 ERROR        self._check_banner()
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2143, in _check_banner
transport.py              1744 ERROR        "Error reading SSH protocol banner" + str(e)
transport.py              1744 ERROR    paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
transport.py              1744 ERROR
_________________ ERROR at setup of test_invalid_commands_do_not_invoke_close_function __________________

self = <paramiko.Transport at 0x233a6eb8 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
>               buf = self.packetizer.readline(timeout)

/usr/lib/python3.7/site-packages/paramiko/transport.py:2138: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f86233a6278>, timeout = 15

    def readline(self, timeout):
        """
        Read a line from the socket.  We assume no data is pending after the
        line, so it's okay to attempt large reads.
        """
        buf = self.__remainder
        while linefeed_byte not in buf:
>           buf += self._read_timeout(timeout)

/usr/lib/python3.7/site-packages/paramiko/packet.py:367: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f86233a6278>, timeout = 15

    def _read_timeout(self, timeout):
        start = time.time()
        while True:
            try:
                x = self.__socket.recv(128)
                if len(x) == 0:
                    raise EOFError()
                break
            except socket.timeout:
                pass
            except EnvironmentError as e:
                if first_arg(e) == errno.EINTR:
                    pass
                else:
                    raise
            if self.__closed:
                raise EOFError()
            now = time.time()
            if now - start >= timeout:
>               raise socket.timeout()
E               socket.timeout

/usr/lib/python3.7/site-packages/paramiko/packet.py:576: timeout

During handling of the above exception, another exception occurred:

    @pytest.fixture(scope="function")
    def sftp():
        """This fixture provides a pysftp.Connection object that's shared
        across the entire tests package. If there's a problem connecting,
        it will raise an exception that will terminate testing."""    [dylan@localhost git]$ pytest
========================================== test session starts ==========================================
platform linux -- Python 3.7.2, pytest-4.2.0, py-1.7.0, pluggy-0.8.1
rootdir: /home/dylan/Dropbox/Computer Science/Classes/CS 510 - Agile Software Development/git, inifile:
collected 21 items                                                                                      

tests/integration/test_basic_skeleton.py .E                                                       [  9%]
tests/integration/test_close.py EE                                                                [ 19%]
tests/integration/test_list_local_files.py ..                                                     [ 28%]
tests/integration/test_login.py ..                                                                [ 38%]
tests/integration/test_put_file_onto_remote_server.py ..                                          [ 47%]
tests/unit/test_close.py .                                                                        [ 52%]
tests/unit/test_input_handler.py ...                                                              [ 66%]
tests/unit/test_list_files_local.py .                                                             [ 71%]
tests/unit/test_login.py ..                                                                       [ 80%]
tests/unit/test_put_file_onto_remote_server.py ..                                                 [ 90%]
tests/unit/test_sftp_fixture.py ..                                                                [100%]

================================================ ERRORS =================================================
__________________________________ ERROR at setup of test_end_of_file ___________________________________

self = <paramiko.Transport at 0x258caa20 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
>               buf = self.packetizer.readline(timeout)

/usr/lib/python3.7/site-packages/paramiko/transport.py:2138: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f862472c828>, timeout = 15

    def readline(self, timeout):
        """
        Read a line from the socket.  We assume no data is pending after the
        line, so it's okay to attempt large reads.
        """
        buf = self.__remainder
        while linefeed_byte not in buf:
>           buf += self._read_timeout(timeout)

/usr/lib/python3.7/site-packages/paramiko/packet.py:367: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f862472c828>, timeout = 15

    def _read_timeout(self, timeout):
        start = time.time()
        while True:
            try:
                x = self.__socket.recv(128)
                if len(x) == 0:
                    raise EOFError()
                break
            except socket.timeout:
                pass
            except EnvironmentError as e:
                if first_arg(e) == errno.EINTR:
                    pass
                else:
                    raise
            if self.__closed:
                raise EOFError()
            now = time.time()
            if now - start >= timeout:
>               raise socket.timeout()
E               socket.timeout

/usr/lib/python3.7/site-packages/paramiko/packet.py:576: timeout

During handling of the above exception, another exception occurred:

    @pytest.fixture(scope="function")
    def sftp():
        """This fixture provides a pysftp.Connection object that's shared
        across the entire tests package. If there's a problem connecting,
        it will raise an exception that will terminate testing."""
        return pysftp.Connection(
            host=HOSTNAME,
            username=USERNAME,
>           password=PASSWORD
        )

tests/conftest.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.7/site-packages/pysftp/__init__.py:143: in __init__
    self._transport.connect(**self._tconnect)
/usr/lib/python3.7/site-packages/paramiko/transport.py:1218: in connect
    self.start_client()
/usr/lib/python3.7/site-packages/paramiko/transport.py:587: in start_client
    raise e
/usr/lib/python3.7/site-packages/paramiko/transport.py:1966: in run
    self._check_banner()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.Transport at 0x258caa20 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
                buf = self.packetizer.readline(timeout)
            except ProxyCommandFailure:
                raise
            except Exception as e:
                raise SSHException(
>                   "Error reading SSH protocol banner" + str(e)
                )
E               paramiko.ssh_exception.SSHException: Error reading SSH protocol banner

/usr/lib/python3.7/site-packages/paramiko/transport.py:2143: SSHException
------------------------------------------ Captured log setup -------------------------------------------
transport.py              1746 ERROR    Exception: Error reading SSH protocol banner
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2138, in _check_banner
transport.py              1744 ERROR        buf = self.packetizer.readline(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 367, in readline
transport.py              1744 ERROR        buf += self._read_timeout(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 576, in _read_timeout
transport.py              1744 ERROR        raise socket.timeout()
transport.py              1744 ERROR    socket.timeout
transport.py              1744 ERROR    
transport.py              1744 ERROR    During handling of the above exception, another exception occurred:
transport.py              1744 ERROR    
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 1966, in run
transport.py              1744 ERROR        self._check_banner()
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2143, in _check_banner
transport.py              1744 ERROR        "Error reading SSH protocol banner" + str(e)
transport.py              1744 ERROR    paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
transport.py              1744 ERROR
______________________ ERROR at setup of test_close_commands_invoke_close_function ______________________

self = <paramiko.Transport at 0x23485390 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
>               buf = self.packetizer.readline(timeout)

/usr/lib/python3.7/site-packages/paramiko/transport.py:2138: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f862358b630>, timeout = 15

    def readline(self, timeout):
        """
        Read a line from the socket.  We assume no data is pending after the
        line, so it's okay to attempt large reads.
        """
        buf = self.__remainder
        while linefeed_byte not in buf:
>           buf += self._read_timeout(timeout)

/usr/lib/python3.7/site-packages/paramiko/packet.py:367: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f862358b630>, timeout = 15

    def _read_timeout(self, timeout):
        start = time.time()
        while True:
            try:
                x = self.__socket.recv(128)
                if len(x) == 0:
                    raise EOFError()
                break
            except socket.timeout:
                pass
            except EnvironmentError as e:
                if first_arg(e) == errno.EINTR:
                    pass
                else:
                    raise
            if self.__closed:
                raise EOFError()
            now = time.time()
            if now - start >= timeout:
>               raise socket.timeout()
E               socket.timeout

/usr/lib/python3.7/site-packages/paramiko/packet.py:576: timeout

During handling of the above exception, another exception occurred:

    @pytest.fixture(scope="function")
    def sftp():
        """This fixture provides a pysftp.Connection object that's shared
        across the entire tests package. If there's a problem connecting,
        it will raise an exception that will terminate testing."""
        return pysftp.Connection(
            host=HOSTNAME,
            username=USERNAME,
>           password=PASSWORD
        )

tests/conftest.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.7/site-packages/pysftp/__init__.py:143: in __init__
    self._transport.connect(**self._tconnect)
/usr/lib/python3.7/site-packages/paramiko/transport.py:1218: in connect
    self.start_client()
/usr/lib/python3.7/site-packages/paramiko/transport.py:587: in start_client
    raise e
/usr/lib/python3.7/site-packages/paramiko/transport.py:1966: in run
    self._check_banner()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.Transport at 0x23485390 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
                buf = self.packetizer.readline(timeout)
            except ProxyCommandFailure:
                raise
            except Exception as e:
                raise SSHException(
>                   "Error reading SSH protocol banner" + str(e)
                )
E               paramiko.ssh_exception.SSHException: Error reading SSH protocol banner

/usr/lib/python3.7/site-packages/paramiko/transport.py:2143: SSHException
------------------------------------------ Captured log setup -------------------------------------------
transport.py              1746 ERROR    Exception: Error reading SSH protocol banner
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2138, in _check_banner
transport.py              1744 ERROR        buf = self.packetizer.readline(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 367, in readline
transport.py              1744 ERROR        buf += self._read_timeout(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 576, in _read_timeout
transport.py              1744 ERROR        raise socket.timeout()
transport.py              1744 ERROR    socket.timeout
transport.py              1744 ERROR    
transport.py              1744 ERROR    During handling of the above exception, another exception occurred:
transport.py              1744 ERROR    
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 1966, in run
transport.py              1744 ERROR        self._check_banner()
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2143, in _check_banner
transport.py              1744 ERROR        "Error reading SSH protocol banner" + str(e)
transport.py              1744 ERROR    paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
transport.py              1744 ERROR
_________________ ERROR at setup of test_invalid_commands_do_not_invoke_close_function __________________

self = <paramiko.Transport at 0x233a6eb8 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
>               buf = self.packetizer.readline(timeout)

/usr/lib/python3.7/site-packages/paramiko/transport.py:2138: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f86233a6278>, timeout = 15

    def readline(self, timeout):
        """
        Read a line from the socket.  We assume no data is pending after the
        line, so it's okay to attempt large reads.
        """
        buf = self.__remainder
        while linefeed_byte not in buf:
>           buf += self._read_timeout(timeout)

/usr/lib/python3.7/site-packages/paramiko/packet.py:367: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.packet.Packetizer object at 0x7f86233a6278>, timeout = 15

    def _read_timeout(self, timeout):
        start = time.time()
        while True:
            try:
                x = self.__socket.recv(128)
                if len(x) == 0:
                    raise EOFError()
                break
            except socket.timeout:
                pass
            except EnvironmentError as e:
                if first_arg(e) == errno.EINTR:
                    pass
                else:
                    raise
            if self.__closed:
                raise EOFError()
            now = time.time()
            if now - start >= timeout:
>               raise socket.timeout()
E               socket.timeout

/usr/lib/python3.7/site-packages/paramiko/packet.py:576: timeout

During handling of the above exception, another exception occurred:

    @pytest.fixture(scope="function")
    def sftp():
        """This fixture provides a pysftp.Connection object that's shared
        across the entire tests package. If there's a problem connecting,
        it will raise an exception that will terminate testing."""
        return pysftp.Connection(
            host=HOSTNAME,
            username=USERNAME,
>           password=PASSWORD
        )

tests/conftest.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.7/site-packages/pysftp/__init__.py:143: in __init__
    self._transport.connect(**self._tconnect)
/usr/lib/python3.7/site-packages/paramiko/transport.py:1218: in connect
    self.start_client()
/usr/lib/python3.7/site-packages/paramiko/transport.py:587: in start_client
    raise e
/usr/lib/python3.7/site-packages/paramiko/transport.py:1966: in run
    self._check_banner()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.Transport at 0x233a6eb8 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
                buf = self.packetizer.readline(timeout)
            except ProxyCommandFailure:
                raise
            except Exception as e:
                raise SSHException(
>                   "Error reading SSH protocol banner" + str(e)
                )
E               paramiko.ssh_exception.SSHException: Error reading SSH protocol banner

/usr/lib/python3.7/site-packages/paramiko/transport.py:2143: SSHException
------------------------------------------ Captured log setup -------------------------------------------
transport.py              1746 ERROR    Exception: Error reading SSH protocol banner
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2138, in _check_banner
transport.py              1744 ERROR        buf = self.packetizer.readline(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 367, in readline
transport.py              1744 ERROR        buf += self._read_timeout(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 576, in _read_timeout
transport.py              1744 ERROR        raise socket.timeout()
transport.py              1744 ERROR    socket.timeout
transport.py              1744 ERROR    
transport.py              1744 ERROR    During handling of the above exception, another exception occurred:
transport.py              1744 ERROR    
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 1966, in run
transport.py              1744 ERROR        self._check_banner()
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2143, in _check_banner
transport.py              1744 ERROR        "Error reading SSH protocol banner" + str(e)
transport.py              1744 ERROR    paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
transport.py              1744 ERROR
================================== 18 passed, 3 error in 64.55 seconds ==================================
[dylan@localhost git]$ 
        return pysftp.Connection(
            host=HOSTNAME,
            username=USERNAME,
>           password=PASSWORD
        )

tests/conftest.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.7/site-packages/pysftp/__init__.py:143: in __init__
    self._transport.connect(**self._tconnect)
/usr/lib/python3.7/site-packages/paramiko/transport.py:1218: in connect
    self.start_client()
/usr/lib/python3.7/site-packages/paramiko/transport.py:587: in start_client
    raise e
/usr/lib/python3.7/site-packages/paramiko/transport.py:1966: in run
    self._check_banner()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <paramiko.Transport at 0x233a6eb8 (unconnected)>

    def _check_banner(self):
        # this is slow, but we only have to do it once
        for i in range(100):
            # give them 15 seconds for the first line, then just 2 seconds
            # each additional line.  (some sites have very high latency.)
            if i == 0:
                timeout = self.banner_timeout
            else:
                timeout = 2
            try:
                buf = self.packetizer.readline(timeout)
            except ProxyCommandFailure:
                raise
            except Exception as e:
                raise SSHException(
>                   "Error reading SSH protocol banner" + str(e)
                )
E               paramiko.ssh_exception.SSHException: Error reading SSH protocol banner

/usr/lib/python3.7/site-packages/paramiko/transport.py:2143: SSHException
------------------------------------------ Captured log setup -------------------------------------------
transport.py              1746 ERROR    Exception: Error reading SSH protocol banner
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2138, in _check_banner
transport.py              1744 ERROR        buf = self.packetizer.readline(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 367, in readline
transport.py              1744 ERROR        buf += self._read_timeout(timeout)
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/packet.py", line 576, in _read_timeout
transport.py              1744 ERROR        raise socket.timeout()
transport.py              1744 ERROR    socket.timeout
transport.py              1744 ERROR    
transport.py              1744 ERROR    During handling of the above exception, another exception occurred:
transport.py              1744 ERROR    
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 1966, in run
transport.py              1744 ERROR        self._check_banner()
transport.py              1744 ERROR      File "/usr/lib/python3.7/site-packages/paramiko/transport.py", line 2143, in _check_banner
transport.py              1744 ERROR        "Error reading SSH protocol banner" + str(e)
transport.py              1744 ERROR    paramiko.ssh_exception.SSHException: Error reading SSH protocol banner
transport.py              1744 ERROR
================================== 18 passed, 3 error in 64.55 seconds ==================================
[dylan@localhost git]$

Uncaught library edge/corner cases crash the client unnecessarily

As far as we know, the pysftp library remains in a valid state any time it encounters an error. We catch and print errors in the controller, but we then close the application, which can be inconvenient. Wouldn't it be preferable to catch exceptions, print them, and then continue?

If we want to be a bit more cautious, we could specifically catch OSError (which is a synonym of IOError) and possibly one or two other, specific error classes that we know our libraries can raise, and we can continue execution after printing the error messages. However, if we receive any other types of exceptions, we can print and exit.

pysftp throws exception during teardown after receiving invalid inputs

If you start the application, passing in credentials that don't make sense, like "a@boopboop", then pysftp causes a traceback to be printed as the program closes.

Steps to reproduce:

Run: python3 main.py hamsandwich@beepboop
Enter any password, such as: asdf

Results:
python3 main.py hamsandwich@beepboop
hamsandwich@beepboop's Password:
No hostkey for host beepboop found.
Exception ignored in: <bound method Connection.del of <pysftp.Connection object at 0x7f8670899e48>>
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pysftp/init.py", line 1013, in del
self.close()
File "/usr/local/lib/python3.6/site-packages/pysftp/init.py", line 784, in close
if self._sftp_live:
AttributeError: 'Connection' object has no attribute '_sftp_live'

Cause:
When Python cleans up pysftp during end-of-program teardown, pysftp raises an exception. Since we have no code in scope at that point, it's not possible to catch said exception. Thus, Python prints that the exception was ignored.

put_r does not put a file with a path containing the home folder

The tilde does not get parsed. Instead, our implementation causes a folder with the '~' to be included in the name.

put_r does not put folders with spaces

There is no implementation in pysftp or our implementation of the put_r function that captures a folder that has a space in its name.

put_r does not allow a full path

Ideal behavior should be /a/b/c -> ./c (put this folder in the ./c folder).

pysftp and our implementation don't allow for this.

put -r with a relative or absolute path tries to match path on server

If I write put /home/dylan/myfile our program will try to upload myfile to /home/dylan on the server. This is unlikely to work and diverges from common SFTP behavior. Correct behavior is to always upload to the current folder. We can easily accomplish this with os.path.basename().

Put crashes if destination already exists as a folder.

How to reproduce:

Create a local file named foo
Create a remote folder named foo
Run put foo from the client

Output:

sftp> put main.py
Failure
[dylan@localhost git]$

Cause:

In the put action, we catch only FileNotFoundError. This is a subclass of OSError, but we don't catch any other subclasses that might arise, nor the parent class itself.

Solution:

Either (a) add an OSError handler after the FileNotFoundError handler or (b) if the OSError messages are clear enough, replace the FileNotFoundError handler with an OSError handler that looks something like:

except OSError as e:
print("Error:", e.strerror)

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.