open-traffic-generator / abstract-python-package Goto Github PK

Following snippet in result.py[421] generated by abstract open traffic generator is causing issues when de-serializing JSON str to result.Flow:

        if isinstance(additionalProperties, **additional_properties) is True:
            self.additionalProperties = additionalProperties
        else:
            raise TypeError('additionalProperties must be an instance of **additional_properties')

isinstance doesn't accept key-val args (**var). More so, additionalProperties should be additional_properties.

will be nice to have a way to get the json (or other text formats) of the config object
maybe we can add a .json for abstract_open_traffic_generator.config that will return the whole config in json format.

Currently the classes in abstract python package do not hold type information for each of its members.
Hence I couldn't find any other way to de-serialize nested JSON or dict to config.Config.

Use cases:

• De-serialize stat results received as JSON to result.Port or result.Flow
• Load config.Config stored as JSON file somewhere in tests

Since 1st use case requires converting to objects that are only one level deep, I was able to use this snippet (but it won't work for nested objects):

    def _dict_to_abstract(self, src_dict, abstract_cls):
        abstract_obj = abstract_cls()
        abstract_obj.__dict__.update(src_dict)
        return abstract_obj

I see device.py and flow.pyfrom ...\abstract_open_traffic_generator have Pattern defined, code looks almost similar but not quite the same.

will be good if we could have a unified Pattern object in the end if I configure the IP on an interface or in a packet I could make use of the same pattern object.

also if we follow the samples and do

from abstract_open_traffic_generator.flow import *
from abstract_open_traffic_generator.device import *

it creates confusion on what Pattern is imported

and doing like this for Pattern, counter, random,..... seems too much of a hassle.

from abstract_open_traffic_generator.device import Pattern as DevicePattern
from abstract_open_traffic_generator.flow import Pattern as FlowPattern

ipv4,vlan..will suffer for same name collisions even if code is different and may not make sense to consolidate in a generic ipv4 VLAN object

1. modeling (should be the source of truth)

• model repository
• data models
• api model

2. abstract python class package (current source of truth)

• repository (private to start on github)
• data model classes
• api class
• data model test cases (unittest)
• package pypi
• travis job
• create traffic top of NGPF IP stack or raw traffic item – full mesh / bidirectional / uni-directional
• Multiple traffic item (Each item–we need to set packet field from template–DSCP/ECN )
• capture packet and retrieve needed field of IP/Header ( cap verification )
  – we may need to selectively start some traffic items from a set of multiple configured traffic items.
• delay/latency measurement may be needed
• Traffic stats verification of different types of stats available eg. traffic item/flow stats etc.

3. concrete python class package

• repository (Andy to setup on github, admin rights to other, sample)
• handcrafted using the abstract pkg and ixnetwork_restpy pkg (Andy)
• build process to produce pypi package (Andy)
• translate abstract data structures to importconfig data structures, call importconfig
• https://openixia.github.io/ixnetwork_openapi/

4. pytest fixtures (provide sample, unittests from abstract)

• chooses concrete python package
• returns abstract package

5. test case

• uses pytest fixtures

Vertical Solution

• provide 2) to developer writing test case so that 3), 4), 5) can proceed
• backfill all of 1), 2)

import abstract_open_traffic_generator as aotg
aotg.port.Port(name='pname', location='localuhd/1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'abstract_open_traffic_generator' has no attribute 'port'

below code works but we should not have to explicitly import every single component they should be loaded if we import the main package.

import abstract_open_traffic_generator as aotg
import abstract_open_traffic_generator.port <----- if we add this explictly it makes it work with no other change
aotg.port.Port(name='pname', location='localuhd/1')
<abstract_open_traffic_generator.port.Port object at 0x7f0dc5181160>

I ran the the test case pfc_lossy configuration test case. The test cases is failing with below error while configuring the IxNetwork probably DHCP priority.
Reference: https://github.com/Azure/sonic-mgmt/blob/e52e520e96384089326fcaa38b381d0effa81d37/tests/pfc/files/configs/pfc.py

The test_dscp_list == [0, 1, 2, 5, 6, 7] and background_dhcp_list == [3, 4]

The test cases was running with the abstract-open-traffic-generator version 0.0.46. The test cases is failing with the abstract-open-traffic-generator version 0.0.47. Please look into this.

pfc/test_pfc_pause_lossy.py:56:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python2.7/dist-packages/ixnetwork_open_traffic_generator/ixnetworkapi.py:91: in set_config
    self.traffic_item.config()
/usr/local/lib/python2.7/dist-packages/ixnetwork_open_traffic_generator/trafficitem.py:129: in config
    self._configure_stack(ixn_stream, flow.packet)
/usr/local/lib/python2.7/dist-packages/ixnetwork_open_traffic_generator/trafficitem.py:185: in _configure_stack
    self._configure_field(stack.Field, header)
/usr/local/lib/python2.7/dist-packages/ixnetwork_open_traffic_generator/trafficitem.py:210: in _configure_field
    self._configure_pattern(ixn_field, field_type_id, pattern)
/usr/local/lib/python2.7/dist-packages/ixnetwork_open_traffic_generator/trafficitem.py:218: in _configure_pattern
    custom_field(ixn_field, pattern)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <ixnetwork_open_traffic_generator.trafficitem.TrafficItem object at 0x7f28d620ef10>
ixn_field = <ixnetwork_restpy.testplatform.sessions.ixnetwork.traffic.trafficitem.configelement.stack.field.field.Field object at 0x7f28d6168b90>
priority = <abstract_open_traffic_generator.flow_ipv4.Priority object at 0x7f28d5ed7a10>

    def _ipv4_priority(self, ixn_field, priority):
        if priority.choice == 'dscp':
            field_type_id = None
            if priority.dscp.phb.choice == 'fixed':
                field_type_id = CustomField._IPv4_DSCP_PHB[priority.dscp.phb.fixed]
            elif priority.dscp.phb.choice == 'list':
>               field_type_id = CustomField._IPv4_DSCP_PHB[priority.dscp.phb.list[0]]
E               KeyError: 0

field_type_id = None
ixn_field  = <ixnetwork_restpy.testplatform.sessions.ixnetwork.traffic.trafficitem.configelement.stack.field.field.Field object at 0x7f28d6168b90>
priority   = <abstract_open_traffic_generator.flow_ipv4.Priority object at 0x7f28d5ed7a10>
self       = <ixnetwork_open_traffic_generator.trafficitem.TrafficItem object at 0x7f28d620ef10>

/usr/local/lib/python2.7/dist-packages/ixnetwork_open_traffic_generator/customfield.py:51: KeyError

In Flow Statistics Tx_Bytes/Rx_Bytes are required. Only Tx_Farme/Rx_frame is not enough.
We want to calculate estimated bytes to be received at the RX end, and match that should closely match actual Rx byte within certain tolerance. Something like below.

TOLERANCE_THRESHOLD = .98
exp_tx_bytes = (exp_dur * port_bw * 1000000 * (RATE_PERCENTAGE / 100.0)) / 8
tolerance_ratio = rx_bytes / exp_tx_bytes
if ((tolerance_ratio < TOLERANCE_THRESHOLD) or (tolerance_ratio > 1)) :
assert(False)

See the diagram below: