sachinites / tcpip_stack Goto Github PK

View Code? Open in Web Editor NEW

165.0 10.0 122.0 3.63 MB

Implement your own TCP IP Stack

Home Page: https://www.csepracticals.com/

License: GNU General Public License v3.0

Makefile 1.14% C 98.86%

tcpip_stack's Introduction

tcpip_stack

Implement your own TCP IP Stack

In this course, We shall be implementing a TCP/IP Stack demo through 5 Networking Projects. All Below Projects should be done in the same sequence as listed.

Project 1 : Build a MultiNode Topology Emulation of Routers and Switches

Project 2 : Implement DataLink Layer (L2 routing), including ARP

Project 3 : Implement L2 Switching (Mac-based Learning and Forwarding)

Project 4 : Implement Vlan Based Mac learning and Forwarding

Project 5 : Implement Network Layer (L3 routing)

After Doing These Projects, you shall be able to :

Tell why you need Data link layer and Network Layer
How to design a new Application protocol on a TCP/IP stack (just like ICMP, HTTP, etc all work on TCP/IP Stack)
Get your hands dirty with industry-level network programming.
Learn cooking up, parsing and reading the packet buffers
Understand End-To-End Architecture and Design of Network Application and TCP/IP Stack
Conquer Interviews for the role of Network Developer Engineer
Decorate your GitHub, and add a strong project to your HAT, Expected LOCs of this course shall exceed 10k !

This project will fill up the gap between theoretical knowledge and the Implementation version of it. How does it sound that you have written code by your own hands to resolve ARP, packet forwarding, etc. Decorate your resume and GitHub with this project.

NOTE : This Project has been Carved out as a full Udemy Video Course, divided into two parts : Part A and Part B.

Link to the Course :

Part A : https://www.udemy.com/course/tcpipstack/?referralCode=83CF8FB4DE8E4C0D42E4

Part B : https://www.udemy.com/course/tcpipstack_b/?referralCode=210EED3DDB079FA45AAF

For other courses offered, visit : www.csepracticals.com

tcpip_stack's People

Contributors

Stargazers

Watchers

Forkers

appdev87 newottoman abinnj009 swetacodes bmorettodama svaaz amingad seemaknavada manojkumar1053 reeser leonweb33 g3yang okayfly fadinglr tlhconsulting parth1614 dirkhammersley thiagoblima fanszoro golisrikanth1989 swapnilbarde007 marjun777 cwhuang888 ramanbank thuraiselvaraj nhk1993 neeraj2296 pinaki2 imarunsinghal brmanuel reshma-immaculate manibalansubramani arasheg xmonader junye-lu trprajapati161 zhangbiyun kk-kishorekumar rkirann9 ganeshbecse alugalashravankumar bharatvohra nguyendanny010 joboliveira babajr caiocrux aradait yashwanth-manjegowda sebnyberg harshal1603 jcraffael yopcat-is-pwnd beneslami vn-os yuanhui-yang ujwal-deep iamsandeepvs mofelee yusufaye n0lll zohebshaikh ravi-teja-konda ethanhuang-wisc thisisanaccount6840 ragvijay still-learner leyouhong markqian gianpaolof yuchaozheng88 mcoghlan constructingthereality vinay-peddisetti topguntrader007 ashleebryant ekpenyong-esu martelleur kerrylinux smithe65 shogunz786 quantum179 sumantrad mahi123palagati nam-ashish skv99 lookharm abhi0608 ojhghkjnhbghj mark-song0618 kadensungbincho kkennymore hitek-financials-ltd createtoohard chaaochieh skaiser hieunguyenbn samirhosseini pkennedy67 akula rawatrajeev15

tcpip_stack's Issues

Define the pkt_block structure, analogues to skbuff in linux kernel.

Define the pkt_block structure, analogus to skbuff in linux kernel.

Add packet Tracer Functionality

Streamline the hdr/protocol numbers used throughout the project

Protocol numbers/Hdr enums are defined in adhoc manner throughout the project.
Properly define the values and use them uniformly throughout he project.

address locking issues of Routing Table throughout the project

is_interface_l3_bidirectional() enhancement

Currently, this API does not check the strict subnet mismatch checking. Add a missing piece.

Make TCP/IP Stack Robust by handling all Error Scenarios in Core code

Make TCP/Ip Stack Robust by handling all Error Scenarios in Core code

Crash is seen on 3rd ecmp node when same route is exported on other two nodes of the topology in ISIS

Configs :
run ut Layer5/isis/isis_config_sample_cross_link_topology.txt 0
conf node R2 pref 100.1.1.1-reject deny 0 100.1.1.1 32
conf node R2 pref 100.1.1.1-reject permit 0 0.0.0.0 0 ge 0 le 32
conf node R4 pref 100.1.1.1-reject deny 0 100.1.1.1 32
conf node R4 pref 100.1.1.1-reject permit 0 0.0.0.0 0 ge 0 le 32
conf node R2 pro isis impo 100.1.1.1-reject
conf node R4 pro isis impo 100.1.1.1-reject
conf node R2 route 100.1.1.1 32
conf node R4 route 100.1.1.1 32
conf node R2 pref export-100.1.1.1 permit 0 100.1.1.1 32
conf node R2 pref export-100.1.1.1 deny 0 0.0.0.0 0 ge 0 le 32
conf node R4 pref export-100.1.1.1 permit 0 100.1.1.1 32
conf node R4 pref export-100.1.1.1 deny 0 0.0.0.0 0 ge 0 le 3
conf node R2 interface eth3 down
conf node R2 interface eth4 down
conf node R2 interface eth7 down
conf node R4 interface eth6 down
conf node R4 interface eth13 down
conf node R2 pro isis expo export-100.1.1.1

Then configuring below additional config causes crash on node R3. Always happening.
conf node R4 pro isis expo export-100.1.1.1 <<< config that causes crash

Topology used : cross_link_topology

Thread 9 "tcpstack.exe" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff4c8c700 (LWP 2568092)]
0x000055555557f315 in isis_spf_install_routes (spf_root=0x5555555de000, ted_spf_root=0x7fffdc018430) at Layer5/isis/isis_spf.c:165
165 if (!l3route || !l3route->nexthops[nxthop_proto][0] ) {

Crash reported when running the testcase

Topology loaded : cross_link_topology

Crash happens like 2 times out of 5 times.

Soft-Firewall>$ run ut Layer5/isis/ut/isis_tc.ut 2

free(): invalid pointer
Thread 6 "tcpstack.exe" received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffff517e640 (LWP 26561)]
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737305372224) at ./nptl/pthread_kill.c:44
44      ./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737305372224) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737305372224) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737305372224, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff7adb476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7ac17f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007ffff7b226f6 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff7c74b8c "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#6  0x00007ffff7b39d7c in malloc_printerr (str=str@entry=0x7ffff7c72764 "free(): invalid pointer") at ./malloc/malloc.c:5664
#7  0x00007ffff7b3bac4 in _int_free (av=<optimized out>, p=<optimized out>, have_lock=0) at ./malloc/malloc.c:4439
#8  0x00007ffff7b3e4d3 in __GI___libc_free (mem=<optimized out>) at ./malloc/malloc.c:3391
#9  0x00005555555b2f1f in tcp_ip_free_pkt_buffer (pkt=0x7fffe4001315 "\377\377\377\377\377\377", pkt_size=93) at ../../net.h:183
#10 0x00005555555b4989 in isis_deref_isis_pkt (node=0x7ffff6b8f760, lsp_pkt=0x7fffe4000b70) at isis_pkt.c:681
#11 0x00005555555b53b3 in isis_lsp_xmit_job (ev_dis=0x7ffff6b8fab8, arg=0x7ffff6ba7740, arg_size=0) at isis_flood.c:99
#12 0x000055555557d5f2 in event_dispatcher_thread (arg=0x7ffff6b8fab8) at EventDispatcher/event_dispatcher.c:220
#13 0x00007ffff7b2db43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#14 0x00007ffff7bbfa00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb)

Add Object Network and Object Group Functionality to the Project

ACLs should use Object Networks and Object Network Groups seamlessly to ease ACL configurations and grouping.

Crash on disabling ISIS protocol

Branch : NewLsdb
Topology : build_dualswitch_topo
Steps to repro Crash :
conf node H1 protocol isis
conf node H1 protocol isis interface all
conf node H2 protocol isis
conf node H2 protocol isis interface all
conf node H2 no protocol isis << repro the crash.

The crash is related to memory corruption. Issue is reproducible even when Mem Mgr Lib is disintegrated with the application.

BT : (when Mem Mgr is integrated )
Repro 1 :
Thread 6 "tcpstack.exe" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff52c4700 (LWP 11865)]
0x0000555555574edd in mm_free_blocks (to_be_free_block=0x7fffe898c82e) at LinuxMemoryManager/mm.c:680
680 if(next_block && next_block->is_free == MM_TRUE){
(gdb) bt
#0 0x0000555555574edd in mm_free_blocks (to_be_free_block=0x7fffe898c82e) at LinuxMemoryManager/mm.c:680
#1 0x000055555557500d in xfree (app_data=0x7fffe898c85e) at LinuxMemoryManager/mm.c:720
#2 0x000055555559eed8 in tcp_ip_free_pkt_buffer (pkt=0x7fffe898cfbd "\377\377\377\377\377\377\210\343", <incomplete sequence \307>, pkt_size=33) at ../../net.h:175
#3 0x00005555555a0dc1 in isis_deref_isis_pkt (node=0x55555566ecb0, lsp_pkt=0x7fffe8999050) at isis_pkt.c:894
#4 0x00005555555ab77e in isis_fragment_dealloc_lsp_pkt (node=0x55555566ecb0, fragment=0x7fffe899c150) at isis_advt.cpp:597
#5 0x00005555555ab1ec in isis_discard_fragment (node=0x55555566ecb0, fragment=0x7fffe899c150) at isis_advt.cpp:440
#6 0x00005555555aafa0 in isis_destroy_advt_db (node=0x55555566ecb0, pn_no=0 '\000') at isis_advt.cpp:391
#7 0x000055555559c639 in isis_protocol_shutdown_now (node=0x55555566ecb0) at isis_rtr.c:121
#8 0x000055555559c707 in isis_check_and_shutdown_protocol_now (node=0x55555566ecb0, work_completed_flag=1) at isis_rtr.c:148
#9 0x00005555555a180f in isis_lsp_xmit_job (ev_dis=0x55555566eff0, arg=0x7ffff49fded0, arg_size=0) at isis_flood.c:133
#10 0x0000555555571cf0 in event_dispatcher_thread (arg=0x55555566eff0) at EventDispatcher/event_dispatcher.c:217
#11 0x00007ffff7fa3609 in start_thread (arg=) at pthread_create.c:477
#12 0x00007ffff7c71133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb) run

BT when MEM mgr is disintegrated.
Repro 2 :
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1 0x00007ffff7b74859 in __GI_abort () at abort.c:79
#2 0x00007ffff7bdf26e in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff7d09298 "%s\n") at ../sysdeps/posix/libc_fatal.c:155
#3 0x00007ffff7be72fc in malloc_printerr (str=str@entry=0x7ffff7d074c1 "free(): invalid pointer") at malloc.c:5347
#4 0x00007ffff7be8b2c in _int_free (av=, p=, have_lock=0) at malloc.c:4173
#5 0x000055555559ee03 in tcp_ip_free_pkt_buffer (pkt=0x7fffe8004dbd "\377\377\377\377\377\377\210\343", <incomplete sequence \307>, pkt_size=33) at ../../net.h:175
#6 0x00005555555a0ce1 in isis_deref_isis_pkt (node=0x55555566f0e0, lsp_pkt=0x7fffe8006610) at isis_pkt.c:894
#7 0x00005555555ab65b in isis_fragment_dealloc_lsp_pkt (node=0x55555566f0e0, fragment=0x7fffe8003be0) at isis_advt.cpp:597
#8 0x00005555555ab0c9 in isis_discard_fragment (node=0x55555566f0e0, fragment=0x7fffe8003be0) at isis_advt.cpp:440
#9 0x00005555555aae7d in isis_destroy_advt_db (node=0x55555566f0e0, pn_no=0 '\000') at isis_advt.cpp:391
#10 0x000055555559c57d in isis_protocol_shutdown_now (node=0x55555566f0e0) at isis_rtr.c:121
#11 0x000055555559c64b in isis_check_and_shutdown_protocol_now (node=0x55555566f0e0, work_completed_flag=1) at isis_rtr.c:148
#12 0x00005555555a172f in isis_lsp_xmit_job (ev_dis=0x55555566f420, arg=0x7ffff5210c80, arg_size=0) at isis_flood.c:133
#13 0x0000555555571c83 in event_dispatcher_thread (arg=0x55555566f420) at EventDispatcher/event_dispatcher.c:217
#14 0x00007ffff7fa3609 in start_thread (arg=) at pthread_create.c:477
#15 0x00007ffff7c71133 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
(gdb)

Tried again without gdb attached (Mem Mgr Disintegrated)
Repro 3:
Soft-FireWall> $ conf node H2 no pro isis
Parse Success.
CLI returned

free(): invalid pointer

Use Atomic Variables to replace spinlocks and mutexes in a data path

Move all Notification Chains to node specific instead of global NFCs

move all Notification Chains to node-specific instead of global NFCs

Segregate data path with control plane

Segregate data path with the control plane. Also, Run each node of a topology as a separate thread.

Implement a Generic pkt Trapping infrastructure

Implement a Generic pkt Trapping infrastructure.

Currently, pkt trap by application is supported based on protocol value in L2 and L3 hdr.
Protocol may wish to trap pkts based on other criteria, not necessary a protocol value. For ex, trapping all multicast pkts, or trapping pkts recvd on particular vlan.
Re-implement pkt trap infrastructure to meet this need.

Add Route Filters functionality.

Implement Route filters ( Also called Prefix List ) . Use them as import and export policy for routing protocols, besides access-list.

Optimization

Optimize fn tcp_ip_expand_buffer_ethernet_hdr( )
DP to CP call : node_get_matching_subnet_interface( )
DP to CP Call : gre_lookup_tunnel_intf( )
Covert all packet counters which are fetched by show command as Atomic.
Remove hdr_type_t argument to tcp_dump_recv_logger( ) ( and more such fns ) since starting hdr type info is present in pkt_block_t itself.
Convert all interface members to atomic as interface attributes are accessed extensively in data-path
Remove direct call to XFREE(pkt_block); everywhere.
Introduce cp2dp... version of ARP Crud Operations and let ISIS use those.
Remove node->print_buff as it is no more used.

While installing access-lists concurrently ( async ), limit the number of access-list installation job triggered.

Currently the fn og_update_acls_task( ) triggeres as many jobs as number of access-lists are present in Hashtable. Number of access-list could be many, we cannot afford to fork these many jobs as it will drastically increase CPU load.
Put the limit on a maximum number of jobs to be forked ( say 10 ), and install access-list in batches using max 10 jobs.

Timer undergo Deadlock Situation

on DCB, When Routers generated ddcp queries, all other router's timers freezes (deadlocked). Timer for ARP table entries do not decrease.

ERO ping is broken

tcp-ip-stack> $ run node R0 ping 122.1.1.5 ero 122.1.1.2
Parse Success.
Router R2 : Cannot Route IP : 0.0.0.0
CLI returned

tcp-ip-stack> $

RLI : Integrate Logging through out TCP/IP Code

We need to integrate logging through out tcp/ip code library so that it make it easier to debug. Before making part-C of the course, integrating tracing/logging library is essential.

Add a functionality of Administratively & protocol interface up-down

An interface is said to be admin up/down if it is shut/no-shut by the admin. This we already support.
Local interface is said to be protocol up/down, if machine on the other side is up/down.

Add Vlan Support to L2 Mac Learning

Make Interface DB Thread Safe

Make Interface DB Thread Safe as it is used by CP and DP thread extensively.

Display issue : Using filter exclude add extra leading space to next line in the output

show node H1 protocol isis | exclude Delete
...
Adjacencies :                             
    Nbr : H6(122.1.1.6)   priority : 6           
    Nbr Sys-id : 122.1.1.6-0                          
    Nbr Lan-id : 122.1.1.6-1                          
    Nbr intf ip : 10.1.1.6  ifindex : 113             
    Nbr Mac Addr : ba:43:a2:3d:00:00                   
    State : Up   HT : 6 sec   Cost : 10
    Expiry Timer Remaining : 5000 msec                          
            Up Time : 0::4::9                   <<<<<< Extra Space                                                                                                                                                                           
                                                                                                                                                                                                                           
    Nbr : H5(122.1.1.5)   priority : 5                      
    Nbr Sys-id : 122.1.1.5-0                             
    Nbr Lan-id : 122.1.1.5-1                            
    Nbr intf ip : 10.1.1.5  ifindex : 109                                                                                                                                                                                  
    Nbr Mac Addr : b5:5b:3e:52:00:00                                                                                                                                                                                       
    State : Up   HT : 6 sec   Cost : 10                                                                                                                                                                                    
    Expiry Timer Remaining : 6000 msec                                                                                                                                                                                     
            Up Time : 0::4::9                                                                                                                                                                                              
                                                                                                                                                                                                                           
    Nbr : H4(122.1.1.4)   priority : 4                                                                                                                                                                                     
    Nbr Sys-id : 122.1.1.4-0                                                                                                                                                                                               
    Nbr Lan-id : 122.1.1.4-1                                                                                                                                                                                               
    Nbr intf ip : 10.1.1.4  ifindex : 111                                                                                                                                                                                  
    Nbr Mac Addr : f4:64:4d:2c:00:00                                                                                                                                                                                       
    State : Up   HT : 6 sec   Cost : 10                                                                                                                                                                                    
    Expiry Timer Remaining : 6000 msec                                                                                                                                                                                     
            Up Time : 0::4::9                                                                                                                                                                                              
                                                                                                                                                                                                                           
    Nbr : H3(122.1.1.3)   priority : 3                                                                                                                                                                                     
    Nbr Sys-id : 122.1.1.3-0                                                                                                                                                                                               
    Nbr Lan-id : 122.1.1.3-1                                                                                                                                                                                               
    Nbr intf ip : 10.1.1.3  ifindex : 105                                                                                                                                                                                  
    Nbr Mac Addr : 97:ce:cb:b3:00:00                                                                                                                                                                                       
    State : Up   HT : 6 sec   Cost : 10                                                                                                                                                                                    
    Expiry Timer Remaining : 6000 msec                                                                                                                                                                                     
            Up Time : 0::4::9                                                                                                                                                                                              
                                                                                                                                                                                                                           
    Nbr : H2(122.1.1.2)   priority : 2                                                                                                                                                                                     
    Nbr Sys-id : 122.1.1.2-0                                                                                                                                                                                               
    Nbr Lan-id : 122.1.1.2-1                                                                                                                                                                                               
    Nbr intf ip : 10.1.1.2  ifindex : 103                                                                                                                                                                                  
    Nbr Mac Addr : 88:e3:59:c7:00:00                                                                                                                                                                                       
    State : Up   HT : 6 sec   Cost : 10                                                                                                                                                                                    
    Expiry Timer Remaining : 6000 msec                                                                                                                                                                                     
            Up Time : 0::4::9

Add a Flow Support to data path

Add a Flow Support to data path. It will optimize per packet processing per flow.

Deploy the Emulated Topology in the Real Network

Add Vlan-interface mapping Database in Layer 2

Add FireWall Support

Enabling DDCP Queries with interval 1 sec on all routers result in L2 Frame Rejected Error

Below errors are seen :

tcp-ip-project> $ L2 Frame Rejected on node R3(eth4)
L2 Frame Rejected on node R2(eth1)
L2 Frame Rejected on node R2(eth2)
L2 Frame Rejected on node R4(eth6)

steps to Repro -

load build_square_topo() topology
trigger below commands :
run node R1 dd per 1
run node R2 dd per 1
run node R3 dd per 1
run node R4 dd per 1

and error is seen only once, thereafter error is not seen. Issue is seen in DCB also.

Enhance glthread library to insert the element in the end with O(1) time complexity.

Enhance glthread library to insert the element in the end with O(1) time complexity.
This will enhance the performance of the project throughout at several places.