kytos-ng / coloring Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
coloring is packing whatever dpid value and filling out with 0xee
https://github.com/kytos-ng/coloring/blob/master/main.py#L115-L127
This can result in cases where it sets the reserved multicast bit in the most significant byte, and it should set the most significant second bit as 0 (locally administered). So, far this hasn't resulted in a crash when sdntrace
generates a PacketOut but it's using invalid/reserved values and can lead to issues.
This NApp doesn't have an openapi.yml spec yet, although the existing API doesn't expect to be changed in the future. It's great to have the API to facilitate for existing integrations with sdntrace, e2e testing and the Stable shield that we plan to add.
This is for replacing Travis CI with GH Actions
If a switch has a loop link, the switch is being considered a neighbor of itself, so a flow with its color is wrongly installed.
As, of now, coloring isn't handling errors (other than logging it) from flow_manager so it should use the force mode at least:
https://github.com/kytos-ng/coloring/blob/master/main.py#L99-L105
As it is, this can lead to issues if switches are temporarily disconnected since it only going to try o install again when it received another kytos/topology.updated
event. To make this NApp more reliable, at least it use the force: True
option on flow_manager since it doesn't need that it gets installed right away, that way in the worst case, consistency check will still install it simplifying the error handling here. The rest of error handling should still be handled like cleaning up or something else in the future too if it gets an ofpt_error, although an ofpt_error is highly unlikely to ever happen considering how simple the match/action is.
For more information, check out kytos-ng/kytos#337. Where applicable, the following libs should also be updated ipython, requests/certifi, pytest 7.2+. Ideally, issue kytos-ng/kytos#338 should be handled to simplify our team upgrade workflow too.
Python is getting updated, related. Some dependencies from this NApp need to be updated.
Eventually, this NApp should have a v1 on its routes.
By default https://github.com/kytos-ng/coloring/blob/master/settings.py#L3, COLOR_FIELD
uses dl_src
as a field type to match, which been the field mostly tested so far. However, lookig into the code, there was an initial attempt to try to support other fieds like nw_src
and so on, but these will result in ErrorType.OFPET_BAD_MATCH
https://github.com/kytos-ng/coloring/blob/master/main.py#L118-L135:
2023-02-20 18:46:02,550 - WARNING [kytos.napps.kytos/flow_manager] [main.py:828:handle_errors] (thread_pool_sb_5) Deleting flow: {'switch': '00:00:00:00:00:00:00:01', 'table_id': 0, 'ma
tch': {'nw_src': '0.0.0.2'}, 'priority': 50000, 'idle_timeout': 0, 'hard_timeout': 0, 'cookie': 12393906174523604993, 'id': '3c0f44c428e9201fa467b05c527cca87', 'stats': {}, 'cookie_mask
': 0, 'instructions': [{'instruction_type': 'apply_actions', 'actions': [{'port': 4294967293, 'action_type': 'output'}]}]}, xid: 252102323, cookie: 12393906174523604993, error: {'error_
command': 'add', 'error_type': UBInt16(<ErrorType.OFPET_BAD_MATCH: 4>), 'error_code': <BadMatchCode.OFPBMC_BAD_PREREQ: 9>}
2023-02-20 18:46:02,557 - WARNING [kytos.napps.kytos/flow_manager] [main.py:828:handle_errors] (thread_pool_sb_0) Deleting flow: {'switch': '00:00:00:00:00:00:00:02', 'table_id': 0, 'ma
tch': {'nw_src': '0.0.0.3'}, 'priority': 50000, 'idle_timeout': 0, 'hard_timeout': 0, 'cookie': 12393906174523604994, 'id': '0d24898d66eef3ff8c216d6737ce0187', 'stats': {}, 'cookie_mask
': 0, 'instructions': [{'instruction_type': 'apply_actions', 'actions': [{'port': 4294967293, 'action_type': 'output'}]}]}, xid: 343664476, cookie: 12393906174523604994, error: {'error_
command': 'add', 'error_type': UBInt16(<ErrorType.OFPET_BAD_MATCH: 4>), 'error_code': <BadMatchCode.OFPBMC_BAD_PREREQ: 9>}
2023-02-20 18:46:02,574 - ERROR [kytos.napps.kytos/of_core] [main.py:273:on_raw_in] (MainThread) OFPT_ERROR: type ErrorType.OFPET_BAD_MATCH, error code 9, from switch 00:00:00:00:00:00:
00:02, xid 3946840149/0xeb400055
If for some reason other fields are needed, this will needed to be augmented accordingly. We also should better document which ones are expected to work. Looking in the code, I'd expect only dl_*
to work.
Coloring is supposed to also handle kytos/flow_manager.flow.error
, similarly to how mef_eline handles https://github.com/kytos-ng/mef_eline/blob/master/main.py#L781-L795, however this would require also keeping track of the flows at a class level in order to clean up errors. This will be addressed in the future.
This is the root cause of issue kytos-ng/of_core#102 that @italovalcy has caught when he was also using higher dpid values.
Check out my comment here for more information.
2023-02-17 09:52:42,976 - INFO [kytos.napps.kytos/flow_manager] [main.py:601:_send_flow_mods_from_request] (Thread-96) Send FlowMod from request dpid: cc:4e:24:4b:11:00:00:00, command:
add, force: True, flows_dict: {'flows': [{'table_id': 0, 'match': {'dl_src': '38:94:06:ee:ee:ee'}, 'priority': 50000, 'actions': [{'action_type': 'output', 'port': 4294967293}], 'cookie
': 27115650311270694912}], 'force': True}
2023-02-17 09:52:43,000 - ERROR [kytos.core.controller] [controller.py:629:msg_out_event_handler] (MainThread) FlowMod.cookie - Expected UBInt64, found value "27115650311270694912"
of type int
kytos $> 27115650311270694912 > 2 ** 64
Out[1]: True
kytos $> 27115650311270694912 > 2 ** 65
Out[2]: False
We are seeing that each switch has a different color for our Noviflow topology, which should not happen. We have to try to have the minimum number of colors possible for the following topology:
{
"colors": {
"00:00:00:00:00:00:00:01": {
"color_field": "dl_src",
"color_value": "ee:ee:ee:ee:ee:01"
},
"00:00:00:00:00:00:00:02": {
"color_field": "dl_src",
"color_value": "ee:ee:ee:ee:ee:02"
},
"00:00:00:00:00:00:00:03": {
"color_field": "dl_src",
"color_value": "ee:ee:ee:ee:ee:03"
},
"00:00:00:00:00:00:00:04": {
"color_field": "dl_src",
"color_value": "ee:ee:ee:ee:ee:04"
},
"00:00:00:00:00:00:00:05": {
"color_field": "dl_src",
"color_value": "ee:ee:ee:ee:ee:05"
},
"00:00:00:00:00:00:00:06": {
"color_field": "dl_src",
"color_value": "ee:ee:ee:ee:ee:06"
}
}
}
The kytos/topology.updated
is a recurring event that could update a dictionary, The possibility of out of order event is a problem. To make it more reliable the listeners need to be replaced with alisten_to
and locks with asyncio
locks.
The installation of a table_group
could result in errors (e.g. KeyError
).
Hi,
When loading the amlight/coloring napp, the following exception is raised:
2021-02-11 18:31:05,170 - INFO [kytos.napps] (coloring) Running NApp: <Main(coloring, started 139726963078912)>
Exception in thread coloring:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/urllib3/connection.py", line 157, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/usr/local/lib/python3.7/dist-packages/urllib3/util/connection.py", line 84, in create_connection
raise err
File "/usr/local/lib/python3.7/dist-packages/urllib3/util/connection.py", line 74, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.7/http/client.py", line 1244, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/usr/local/lib/python3.7/dist-packages/urllib3/connection.py", line 184, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.7/dist-packages/urllib3/connection.py", line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f14d413bb70>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/usr/local/lib/python3.7/dist-packages/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8181): Max retries exceeded with url: /api/kytos/topology/v3/links (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14d413bb70>: Failed to establish a new connection: [Errno 111] Connection refused'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
File "/usr/local/lib/python3.7/dist-packages/kytos/core/napps/base.py", line 247, in run
self.execute()
File "//var/lib/kytos/napps/amlight/coloring/main.py", line 42, in execute
response = requests.get(settings.TOPOLOGY_URL)
File "/usr/local/lib/python3.7/dist-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.7/dist-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8181): Max retries exceeded with url: /api/kytos/topology/v3/links (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f14d413bb70>: Failed to establish a new connection: [Errno 111] Connection refused'))
Steps to reproduce:
python3 -m pip install -e git+https://github.com/kytos/storehouse#egg=kytos-storehouse
python3 -m pip install -e git+https://github.com/kytos/of_core#egg=kytos-of_core
python3 -m pip install -e git+https://github.com/kytos/flow_manager#egg=kytos-flow_manager
python3 -m pip install -e git+https://github.com/kytos/topology#egg=kytos-topology
python3 -m pip install -e git+https://github.com/kytos/of_lldp#egg=kytos-of_lldp
python3 -m pip install -e git+https://github.com/kytos/pathfinder#egg=kytos-pathfinder
python3 -m pip install -e git+https://github.com/kytos/mef_eline#egg=kytos-mef_eline
python3 -m pip install -e git+https://github.com/amlight/coloring#egg=amlight-coloring
python3 -m pip install -e git+https://github.com/amlight/sdntrace#egg=amlight-sdntrace
kytosd -f -E
Coloring push its own FlowMods, we have a cookie_prefix pattern similarly to how mef_eline and of_lldp kytos-ng/kytos-ng.github.io#13, this helps to identify ownership, plus e2e testability since they can be filtered out accordingly in the output later on. Once the prefix is defined, it should also be documented in the linked issue just so it becomes reserved for this NApp
Same as kytos-ng/flow_stats#32
This NApp is supposed to be moved to kytos-ng org
Add a setting configuration on which table flows should be installed, by default it should still use table 0, buf it network operators have a different pipeline they should set accordingly.
Update:
Add listener to event kytos/multi_table.enable_table
which content looks like:
"content": {
"mef_eline": {"epl": 3, "evpl": 2},
"of_lldp": {"base": 0},
"coloring": {"base": 0}
}
Add response event kytos/coloring.enable_table
with content:
"content": {
"coloring": {"base": 0}
}
Related to kytos-ng/kytos#186
Hi,
If you try to install coloring through PIP it will fail with some inconsistency on its metadata.
Steps to reproduce:
0. Get the most updated kytos installation, for instance:
docker pull amlight/kytos
docker run -d --name k1 amlight/kytos
docker exec -it k1 python3 -m pip install -e git+https://github.com/amlight/coloring#egg=coloring
Expected behavior:
Actual behavior:
$ docker exec -it k1 python3 -m pip install -e git+https://github.com/amlight/coloring#egg=coloring
Obtaining coloring from git+https://github.com/amlight/coloring#egg=coloring
Cloning https://github.com/amlight/coloring to /src/coloring
Running command git clone -q https://github.com/amlight/coloring /src/coloring
WARNING: Generating metadata for package coloring produced metadata for project name amlight-coloring. Fix your #egg=coloring fragments.
WARNING: Discarding git+https://github.com/amlight/coloring#egg=coloring. Requested amlight-coloring from git+https://github.com/amlight/coloring#egg=coloring has inconsistent name: filename has 'coloring', but metadata has 'amlight-coloring'
ERROR: Could not find a version that satisfies the requirement coloring (unavailable)
ERROR: No matching distribution found for coloring (unavailable)
Even if you try to use the name according to the metadata:
$ docker run -it --rm amlight/kytos python3 -m pip install -e git+https://github.com/amlight/coloring#egg=amlight-coloring
[ ok ] Starting enhanced syslogd: rsyslogd.
Starting Kytos controller: done
Obtaining amlight-coloring from git+https://github.com/amlight/coloring#egg=amlight-coloring
Cloning https://github.com/amlight/coloring to /src/amlight-coloring
Running command git clone -q https://github.com/amlight/coloring /src/amlight-coloring
Requirement already satisfied: setuptools>=36.0.1 in /usr/local/lib/python3.7/dist-packages (from amlight-coloring) (53.0.0)
Installing collected packages: amlight-coloring
Running setup.py develop for amlight-coloring
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/src/amlight-coloring/setup.py'"'"'; __file__='"'"'/src/amlight-coloring/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
cwd: /src/amlight-coloring/
Complete output (41 lines):
running develop
running egg_info
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from -r requirements/run.in (line 1)) (2.23.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->-r requirements/run.in (line 1)) (1.25.8)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->-r requirements/run.in (line 1)) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->-r requirements/run.in (line 1)) (2019.11.28)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->-r requirements/run.in (line 1)) (2.9)
Installing dependencies...
creating amlight_coloring.egg-info
writing amlight_coloring.egg-info/PKG-INFO
writing dependency_links to amlight_coloring.egg-info/dependency_links.txt
writing requirements to amlight_coloring.egg-info/requires.txt
writing top-level names to amlight_coloring.egg-info/top_level.txt
writing manifest file 'amlight_coloring.egg-info/SOURCES.txt'
reading manifest file 'amlight_coloring.egg-info/SOURCES.txt'
writing manifest file 'amlight_coloring.egg-info/SOURCES.txt'
running build_ext
Creating /usr/local/lib/python3.7/dist-packages/amlight-coloring.egg-link (link to .)
Adding amlight-coloring 0.1 to easy-install.pth file
Installed /src/amlight-coloring
fatal: destination path '/var/lib/kytos/napps/.installed/kytos/of_core' already exists and is not an empty directory.
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/src/amlight-coloring/setup.py", line 321, in <module>
'Topic :: System :: Networking',
File "/usr/local/lib/python3.7/dist-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/src/amlight-coloring/setup.py", line 243, in run
KytosInstall.install_core_napps()
File "/src/amlight-coloring/setup.py", line 197, in install_core_napps
check_call(install_cmd, shell=True)
File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'git clone https://github.com/kytos/of_core.git /var/lib/kytos/napps/.installed/kytos/of_core' returned non-zero exit status 128.
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/src/amlight-coloring/setup.py'"'"'; __file__='"'"'/src/amlight-coloring/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
Coloring napp leaves its flows installed at flow_manager after shutdown, instead of properly cleaning up.
How to reproduce:
kytos napps disable amlight/coloring
mininet> sh for s in s1 s2 s3; do echo ====== $s; ovs-ofctl dump-flows $s; done
Expected result:
You should only see the LLDP flows
Actual result:
mininet> sh for s in s1 s2 s3; do echo ====== $s; ovs-ofctl dump-flows $s; done
====== s1
cookie=0x0, duration=378.356s, table=0, n_packets=0, n_bytes=0, priority=50000,dl_src=ee:ee:ee:ee:ee:03 actions=CONTROLLER:65535
cookie=0x0, duration=321.275s, table=0, n_packets=0, n_bytes=0, priority=50000,dl_src=ee:ee:ee:ee:ee:02 actions=CONTROLLER:65535
cookie=0x0, duration=321.280s, table=0, n_packets=5630, n_bytes=236460, priority=1000,dl_vlan=3799,dl_type=0x88cc actions=CONTROLLER:65535
====== s2
cookie=0x0, duration=321.302s, table=0, n_packets=0, n_bytes=0, priority=50000,dl_src=ee:ee:ee:ee:ee:01 actions=CONTROLLER:65535
cookie=0x0, duration=321.297s, table=0, n_packets=0, n_bytes=0, priority=50000,dl_src=ee:ee:ee:ee:ee:03 actions=CONTROLLER:65535
cookie=0x0, duration=321.307s, table=0, n_packets=5630, n_bytes=236460, priority=1000,dl_vlan=3799,dl_type=0x88cc actions=CONTROLLER:65535
====== s3
cookie=0x0, duration=378.390s, table=0, n_packets=0, n_bytes=0, priority=50000,dl_src=ee:ee:ee:ee:ee:01 actions=CONTROLLER:65535
cookie=0x0, duration=378.385s, table=0, n_packets=0, n_bytes=0, priority=50000,dl_src=ee:ee:ee:ee:ee:02 actions=CONTROLLER:65535
cookie=0x0, duration=380.748s, table=0, n_packets=5630, n_bytes=236460, priority=1000,dl_vlan=3799,dl_type=0x88cc actions=CONTROLLER:65535
As you can see, the flows related to amlight/coloring remain at the switch's flow table.
Looks like the napp needs to actually use the shutdown() method to run the cleanup process, including remove the flows previously created.
This NApps doesn't depend on flow_stats, update kytos.json and readme.rst accordingly to avoid confusion
https://github.com/kytos-ng/coloring/blob/master/kytos.json#L7
After this PR lands, links are going to be able to be deleted but coloring
does not update its flows accordingly. An event kytos/topology.link.deleted
with reason="link deleted" is enough to know about a deleted link.
Same as kytos-ng/mef_eline#185 for a future opportunity
This is related to kytos-ng/kytos#347
self.switches and controller.switches being iterated on are vulnerable to thread RuntimeError iterable changed size
main.py
5:# Check for import order disabled in pylint due to conflict
45: [link.as_dict() for link in topology.links.values()]
56: for switch in self.controller.switches.values():
65: for link in links:
72: # Create the flows for each neighbor of each switch and installs it
74: for dpid, switch_dict in self.switches.items():
82: for neighbor in switch_dict['neighbors']:
125: color_value = ':'.join([f'{b:02x}' for b in int_mac])
141: for dpid, switch_dict in self.switches.items():
Related to kytos
issue #186
Add event kytos/coloring.cookie
with content:
"content": {
"coloring": "0x0"
}
When kytos starts, coloring
installs flows before the event of_multi_table.enable_table
was sent causing some flows to be installed in the wrong table. of_multi_table
can verify the flows installation afterwards but it will be better if we do not allow flows to be installed in the wrong table in the first place.
Hi,
The way coloring creates the flows, matching only for the ethernet source, can lead to a vulnerability of Deneil of Service in the Controller. Basically, the Flow only matches on the eth_src, which means that if we run any traffic generator in a customer port and specify the source MAC address as being the same standard as the coloring utilizes (ee:ee:ee:ee:ee:01, ee:ee:ee:ee:ee:02, etc), all packets will be sent to the controller and overload the controller.
Since those flows are only meaningful for the SDN-Trace Napp, there is no impact on adding the IN_PORT as the match field (for each NNI). The number of flows would increase from 1 to N, where N is the number of NNIs of a switch (which tends not to be so high).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.