======================================= ERRORS =======================================
_________________________ ERROR at setup of test_constructor _________________________

item = <Function test_constructor>

    @hookimpl(tryfirst=True)
    def pytest_runtest_setup(item):
        """
        pytest hook to setup test before run.
        """
>       test_id_marker = item.get_marker('test_id')
E       AttributeError: 'Function' object has no attribute 'get_marker'

/anaconda3/envs/vaak/lib/python3.6/site-packages/topology/pytest/plugin.py:311: AttributeError
============================== 1 error in 0.04 seconds ===============================

$ pip list | grep pytest

pytest              5.0.1      
pytest-remotedata   0.3.1      

Is there any fix for this? Thanks.

Documentation is failing to render after sphinx release 2.0.1

topology/doc/reference/topology.logging.rst:98:Could not lex literal_block as "guess". Highlighting skipped.

In the example below the parser fails to parse _wait_timeout_s, If I remove the leading underscore the parser succeeds, but of course, a different variable is set.

topology.parser.ParseException: Unable to parse line #9: "[type=clint name="Clint 1" _wait_timeout_s=60] clnt1"

When typing exit() or ctrl+D in the interactive shell, the topology is not explicitly un-built (or destroyed).

Even though this behavior may be ok in platforms where the associated processes are local to the computer running topology, this is an error when it is necessary to clean up remote resources (e.g. stop a remote HTTP web service).

One way to solve is by catching the SystemExit exception when topology.interact.interact() is called and
instruct the Topology manager to "unbuild" the topology.

As of Mar 10 2016 the marker test_id depends on pytest internal call add_custom_property. This worked in pytest===2.8.4 but later version broke the feature:

Update the marker to allow using it in newer pytest versions.

It would be nice to display the shell that was used to call a command in the output.

Right now output looks like this:

hs1('ls')
2016-01-28T20:31:14.597933 [hs1].send_command(ls) ::
...

It would be nice if it looked like this:

hs1('ls')
2016-01-28T20:31:14.597933 [hs1].send_command(ls, shell='bash') ::
...

Currently, when launching the topology executable the first time and if the history file doesn't exists an error is shown. This can confuse new users and cleary is not really an error but at most a warning. We need to change this:

Pexpect supports sending control characters, as explained here.

Make send_command capable of doing so.

Last release published in Pypi is 1.9.5. we are at 1.9.8.

https://pypi.python.org/pypi/topology

This responsibility can be easily left for the child class, and one argument against is that if this is implemented, the user of the PExpectShell class loses the control of these exceptions, but since the PExpectShell class is already wrapping some pexpect methods, the user is already losing the control of them (which is necessary since the purpose of this class is to make it easy to use pexpect with shells).

Are these methods and the EOF or TIMEOUT exceptions in an analogous situation? Should the PExpectShell class handle these exceptions in some way?

The new parser that would be introduced with #55 among other things support parsing environment attributes on the form of:

[attr1=value1]  #environment attribute
# Rest of szn string
[type= host] h1 
... 

In order to do something useful with this environment attributes we need to pass them along to the platform engines somehow. The two alternatives I can come up with in order to do this are:

1. Modify pynml in order to support holding this attributes somewhere
2. Passing the environment to the platform's engine init.

Option 1 has the benefit would be the easiest and would not break topology's API, however the environment is not part of nml and it would be questionable whether this should go there. Option 2 seems more correct but it would break topology's API to the platform engine since we would be passing a new parameter along that they are not prepared to receive. This means that option 2 requires to update all platform engines' init functions to accept an environment parameter, even if they do nothing with them.

Thoughts and suggestions on how we would proceed with this?

This API will allow users to connect to a node services, like RESTfull API's, databases, etc.

Used the following attribute injector file:

*****************************************************************
[
    {
        "files": ["/pnb/software/at/sand00/saenzpa/repos/openswitch_tests/test/lacp/*"],
        "modifiers": [
            {
                "nodes": ["type=openswitch"],
                "attributes": {
                    "image": "onie-installer-x86_64-as5712_54x-0.2.0-rc0+2016012518"
                }
            }
        ]
    }
]
*******************************************************************

The logicalXML file generated had the attirbute injected to the wrong (type=host) nodes.

<?xml version="1.0" ?>
<topology version="3">
        <device group="NULL" name="dut01" target="true">
                <attribute name="system-category" value="switch"/>
                <attribute name="system-family" value="AS5712"/>
        </device>
        <device group="NULL" name="dut02" target="true">
                <attribute name="system-category" value="switch"/>
                <attribute name="system-family" value="AS5712"/>
        </device>
        <device group="NULL" name="wrkston01" target="false">
                <attribute name="system-category" value="workstation"/>
                <attribute name="system-profile" value="onie-installer-x86_64-as5712_54x-0.2.0-rc0+2016012518"/>
        </device>
        <device group="NULL" name="wrkston02" target="false">
                <attribute name="system-category" value="workstation"/>
                <attribute name="system-profile" value="onie-installer-x86_64-as5712_54x-0.2.0-rc0+2016012518"/>
        </device>
        <link device1="wrkston01" device2="dut01" name="lnk01" rate="any"/>
        <link device1="dut01" device2="dut02" name="lnk02" rate="any"/>
        <link device1="dut01" device2="dut02" name="lnk03" rate="any"/>
        <link device1="dut02" device2="wrkston02" name="lnk04" rate="any"/>
</topology>

pytest plugin

Relevant lines are:

# Get a list of all testing directories
search_paths = [
    abspath(arg) for arg in config.args if isdir(arg)
]

injected_attr = parse_attribute_injection(
    injection_file,
    search_paths=search_paths
)

In this case only arguments that are directory are considered for the search path. But if we pass a single test as an argument pytest ... test/test_my.py then it will never be injected. In this case, we could also do:

search_paths = [
    abspath(arg) for arg in config.args
    if isdir(arg) else abspath(dirname(arg))
]

Executable

Relevant lines are:

if args.inject is not None:
    injection_spec = parse_attribute_injection(args.inject)

In this case injection is basically broken unless we call the topology executable from where do we expect the root be, as the default search_paths is used (cwd).

In this case too, we could get the search path from the input:

if args.inject is not None:
    injection_spec = parse_attribute_injection(
        args.inject,
        search_paths=dirname(args.topology)
    )

What do you think?

Use instead pytest-randomly:

https://pypi.python.org/pypi/pytest-randomly

The main issue is that pytest-random suffles all tests, without option to group by module / suite. So, consider:

test_topo_b.py::test1
test_topo_b.py::test2
test_topo_b.py::test3
test_topo_a.py::test1
test_topo_a.py::test2
test_topo_a.py::test3

With pytest-random this can happen:

test_topo_b.py::test2
test_topo_a.py::test1
test_topo_b.py::test1
test_topo_a.py::test2
test_topo_b.py::test3
test_topo_a.py::test3

In those cases, because the topology is a level module fixture, the topology will be rebuilt.

pytest-randomly states:

Randomly shuffles the order of test items. This is done first at the level of modules, then at the level of test classes (if you have them), then at the order of functions. This also works with things like doctests.

After testing, we confirmed that this is the correct behavior.

There are some test cases on which you will need to perform a logout from vtysh and then log again. Once you perform a logout a authentication prompt will be displayed. If you access using admin/admin credentials you will have access to vtysh, if you use root/root you will access bash. Another common is when we perform a reboot on the switch. After the switch boots it will prompt the login.

Login context after performing a reboot:
OpenSwitch 0.3.0 ops-as5712 ttyS1
ops-as5712 login:

https://github.com/HPENetworking/topology/blob/master/lib/topology/platforms/shell.py#L510

This raises an error when decode fails to decode with the specified encoding. Change this to work like we do in:

https://github.com/HPENetworking/topology/blob/master/lib/topology/logging.py#L233

step used to show up in the end user console in the same way as send_command and get_response show up now. This behavior has changed after the introduction of logging capabilities. Make step show up in the console as it used to show.

$ python3 -m topology --plot-dir=/tmp/ ./my_topology.szn

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/ubuntu/shitcluster/network/.venv/lib/python3.10/site-packages/topology/__main__.py", line 135, in <module>
    args = parse_args()
  File "/home/ubuntu/shitcluster/network/.venv/lib/python3.10/site-packages/topology/args.py", line 334, in parse_args
    args = validate_args(args)
  File "/home/ubuntu/shitcluster/network/.venv/lib/python3.10/site-packages/topology/args.py", line 210, in validate_args
    mkpath(args.nml_dir)
  File "/usr/lib/python3.10/distutils/dir_util.py", line 32, in mkpath
    raise DistutilsInternalError(
distutils.errors.DistutilsInternalError: mkpath: 'name' must be a string (got None)

Right now, there is only the option of blacklisting platform engines, it would be helpful to have a whitelisting feature too.

Right now a command can be called in a node while specifying its shell. Some commands may take unusually long to complete so it would be useful to be able to specify the desired timeout there in the same way as the shell can be specified.

I have an old telnet shell that needs a \r at the end of the line for the command to actually execute.

This causes the connection logger to not log the command

>>> telnet_shell.send_command('/off {}\r'.format(port))
', matches=['bla> '], newline=True, timeout=None)

This can be solved by making the logger do repr(command) when logging, but not sure if it is the correct approach... What do you guys think?

When a TOPOLOGY is defined in a external python module and then imported on a test, attribute injection fails with

15:47:51,0556 [topology.injection:248 ][WARNING ] Skipping node expansion for attribute injection in filename [----]/tests/test_duck.py in the lookup path as it does not contain a TOPOLOGY definition.

test file goes something like:

from topologies.duck import TOPOLOGY  # noqa

def test_duck(topology):
    pass