travelping / upg-vpp Goto Github PK
View Code? Open in Web Editor NEWUser Plane Gateway (UPG) based on VPP
License: Apache License 2.0
User Plane Gateway (UPG) based on VPP
License: Apache License 2.0
The following invalid SessionSetupRequest
has caused a crash in the e2e tests (IPv6 mode TDF setup):
Frame 13: 503 bytes on wire (4024 bits), 503 bytes captured (4024 bits)
Ethernet II, Src: 4a:f8:a5:c3:85:ea (4a:f8:a5:c3:85:ea), Dst: fa:8a:78:4d:5b:5b (fa:8a:78:4d:5b:5b)
Internet Protocol Version 6, Src: 2001:db8:10::3, Dst: 2001:db8:10::2
User Datagram Protocol, Src Port: 8805, Dst Port: 8805
Packet Forwarding Control Protocol
Flags: 0x21, SEID (S)
001. .... = Version: 1
...0 .... = Spare: 0
.... 0... = Spare: 0
.... .0.. = Spare: 0
.... ..0. = Message Priority (MP): False
.... ...1 = SEID (S): True
Message Type: PFCP Session Establishment Request (50)
Length: 437
SEID: 0x0000000000000000
Sequence Number: 2
Spare: 0
Node ID : FQDN: pfcpstub
IE Type: Node ID (60)
IE Length: 10
0000 .... = Spare: 0
.... 0010 = Node ID Type: FQDN (2)
Node ID FQDN: pfcpstub
F-SEID : SEID: 0x3c04951aa42655d9, IPv4 0.0.0.0
IE Type: F-SEID (57)
IE Length: 13
Flags: 0x02, V4 (IPv4)
0... .... = Spare: 0
.0.. .... = Spare: 0
..0. .... = Spare: 0
...0 .... = Spare: 0
.... 0... = Spare: 0
.... .0.. = Spare: 0
.... ..1. = V4 (IPv4): Present
.... ...0 = V6 (IPv6): Not Present
SEID: 0x3c04951aa42655d9
IPv4 address: 0.0.0.0
Create PDR : [Grouped IE]
IE Type: Create PDR (1)
IE Length: 113
Packet Detection Rule ID : 1
IE Type: Packet Detection Rule ID (56)
IE Length: 2
Rule ID: 1
FAR ID : Dynamic by CP 1
IE Type: FAR ID (108)
IE Length: 4
0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
.000 0000 0000 0000 0000 0000 0000 0001 = FAR ID: 1
Precedence : 200
IE Type: Precedence (29)
IE Length: 4
Precedence: 200
PDI : [Grouped IE]
IE Type: PDI (2)
IE Length: 79
Network Instance : access
IE Type: Network Instance (22)
IE Length: 7
Network Instance: access
SDF Filter :
IE Type: SDF Filter (23)
IE Length: 38
Flags: 0x01, FD (Flow Description)
0000 .... = Spare: 0
...0 .... = BID (Bidirectional SDF Filter): False
.... 0... = FL (Flow Label): False
.... .0.. = SPI (Security Parameter Index): False
.... ..0. = TTC (ToS Traffic Class): False
.... ...1 = FD (Flow Description): True
Spare: 0
Length of Flow Description: 34
Flow Description: permit out ip from any to assigned
Source Interface : Access
IE Type: Source Interface (20)
IE Length: 1
0000 .... = Spare: 0
.... 0000 = Source Interface: Access (0)
UE IP Address :
IE Type: UE IP Address (93)
IE Length: 17
Flags: 0x01, V6 (IPv6)
0000 .... = Spare: 0
.... 0... = IPv6D: Source IP address
.... .0.. = S/D: Source IP address
.... ..0. = V4 (IPv4): Not Present
.... ...1 = V6 (IPv6): Present
IPv6 address: ::
URR ID : Dynamic by CP 1
IE Type: URR ID (81)
IE Length: 4
0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
.000 0000 0000 0000 0000 0000 0000 0001 = URR ID: 1
Create PDR : [Grouped IE]
IE Type: Create PDR (1)
IE Length: 110
Packet Detection Rule ID : 2
IE Type: Packet Detection Rule ID (56)
IE Length: 2
Rule ID: 2
FAR ID : Dynamic by CP 2
IE Type: FAR ID (108)
IE Length: 4
0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
.000 0000 0000 0000 0000 0000 0000 0010 = FAR ID: 2
PDI : [Grouped IE]
IE Type: PDI (2)
IE Length: 76
SDF Filter :
IE Type: SDF Filter (23)
IE Length: 38
Flags: 0x01, FD (Flow Description)
0000 .... = Spare: 0
...0 .... = BID (Bidirectional SDF Filter): False
.... 0... = FL (Flow Label): False
.... .0.. = SPI (Security Parameter Index): False
.... ..0. = TTC (ToS Traffic Class): False
.... ...1 = FD (Flow Description): True
Spare: 0
Length of Flow Description: 34
Flow Description: permit out ip from any to assigned
Network Instance : sgi
IE Type: Network Instance (22)
IE Length: 4
Network Instance: sgi
Source Interface : SGi-LAN/N6-LAN
IE Type: Source Interface (20)
IE Length: 1
0000 .... = Spare: 0
.... 0010 = Source Interface: SGi-LAN/N6-LAN (2)
UE IP Address :
IE Type: UE IP Address (93)
IE Length: 17
Flags: 0x05, S/D, V6 (IPv6)
0000 .... = Spare: 0
.... 0... = IPv6D: Source IP address
.... .1.. = S/D: Destination IP address
.... ..0. = V4 (IPv4): Not Present
.... ...1 = V6 (IPv6): Present
IPv6 address: ::
Precedence : 200
IE Type: Precedence (29)
IE Length: 4
Precedence: 200
URR ID : Dynamic by CP 1
IE Type: URR ID (81)
IE Length: 4
0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
.000 0000 0000 0000 0000 0000 0000 0001 = URR ID: 1
Create FAR : [Grouped IE]
IE Type: Create FAR (3)
IE Length: 76
FAR ID : Dynamic by CP 1
IE Type: FAR ID (108)
IE Length: 4
0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
.000 0000 0000 0000 0000 0000 0000 0001 = FAR ID: 1
Apply Action :
IE Type: Apply Action (44)
IE Length: 1
Flags: 0x02, FORW (Forward)
000. .... = Spare: 0
...0 .... = DUPL (Duplicate): False
.... 0... = NOCP (Notify the CP function): False
.... .0.. = BUFF (Buffer): False
.... ..1. = FORW (Forward): True
.... ...0 = DROP (Drop): False
Forwarding Parameters : [Grouped IE]
IE Type: Forwarding Parameters (4)
IE Length: 59
Destination Interface : SGi-LAN/N6-LAN
IE Type: Destination Interface (42)
IE Length: 1
0000 .... = Spare: 0
.... 0010 = Interface: SGi-LAN/N6-LAN (2)
Network Instance : sgi
IE Type: Network Instance (22)
IE Length: 4
Network Instance: sgi
Redirect Information :
IE Type: Redirect Information (38)
IE Length: 42
0000 .... = Spare: 0
.... 0010 = Redirect Address Type: URL (2)
Redirect Server Address Length: 37
Redirect Server Address: http://127.0.0.1/this-is-my-redirect/
IE data not decoded by WS yet
[Expert Info (Note/Undecoded): IE data not decoded by WS yet]
[IE data not decoded by WS yet]
[Severity level: Note]
[Group: Undecoded]
Create FAR : [Grouped IE]
IE Type: Create FAR (3)
IE Length: 33
FAR ID : Dynamic by CP 2
IE Type: FAR ID (108)
IE Length: 4
0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
.000 0000 0000 0000 0000 0000 0000 0010 = FAR ID: 2
Apply Action :
IE Type: Apply Action (44)
IE Length: 1
Flags: 0x02, FORW (Forward)
000. .... = Spare: 0
...0 .... = DUPL (Duplicate): False
.... 0... = NOCP (Notify the CP function): False
.... .0.. = BUFF (Buffer): False
.... ..1. = FORW (Forward): True
.... ...0 = DROP (Drop): False
Forwarding Parameters : [Grouped IE]
IE Type: Forwarding Parameters (4)
IE Length: 16
Destination Interface : Access
IE Type: Destination Interface (42)
IE Length: 1
0000 .... = Spare: 0
.... 0000 = Interface: Access (0)
Network Instance : access
IE Type: Network Instance (22)
IE Length: 7
Network Instance: access
Create URR : [Grouped IE]
IE Type: Create URR (6)
IE Length: 19
URR ID : Dynamic by CP 1
IE Type: URR ID (81)
IE Length: 4
0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
.000 0000 0000 0000 0000 0000 0000 0001 = URR ID: 1
Measurement Method :
IE Type: Measurement Method (62)
IE Length: 1
Flags: 0x03, VOLUM (Volume), DURAT (Duration)
0000 0... = Spare: 0
.... .0.. = EVENT (Event): False
.... ..1. = VOLUM (Volume): True
.... ...1 = DURAT (Duration): True
Reporting Triggers :
IE Type: Reporting Triggers (37)
IE Length: 2
0... .... = LIUSA (Linked Usage Reporting): False
.0.. .... = DROTH (Dropped DL Traffic Threshold): False
..0. .... = STOPT (Stop of Traffic): False
...0 .... = START (Start of Traffic): False
.... 0... = QUHTI (Quota Holding Time): False
.... .0.. = TIMTH (Time Threshold): False
.... ..0. = VOLTH (Volume Threshold): False
.... ...0 = PERIO (Periodic Reporting): False
000. .... = Spare: 0
..0. .... = EVEQU (Event Quota): False
...0 .... = EVETH (Event Threshold): False
.... 0... = MACAR (MAC Addresses Reporting): False
.... .0.. = ENVCL (Envelope Closure): False
.... ..0. = TIMQU (Time Quota): False
.... ...0 = VOLQU (Volume Quota): False
Create URR : [Grouped IE]
IE Type: Create URR (6)
IE Length: 19
URR ID : Dynamic by CP 2
IE Type: URR ID (81)
IE Length: 4
0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
.000 0000 0000 0000 0000 0000 0000 0010 = URR ID: 2
Measurement Method :
IE Type: Measurement Method (62)
IE Length: 1
Flags: 0x03, VOLUM (Volume), DURAT (Duration)
0000 0... = Spare: 0
.... .0.. = EVENT (Event): False
.... ..1. = VOLUM (Volume): True
.... ...1 = DURAT (Duration): True
Reporting Triggers :
IE Type: Reporting Triggers (37)
IE Length: 2
0... .... = LIUSA (Linked Usage Reporting): False
.0.. .... = DROTH (Dropped DL Traffic Threshold): False
..0. .... = STOPT (Stop of Traffic): False
...0 .... = START (Start of Traffic): False
.... 0... = QUHTI (Quota Holding Time): False
.... .0.. = TIMTH (Time Threshold): False
.... ..0. = VOLTH (Volume Threshold): False
.... ...0 = PERIO (Periodic Reporting): False
000. .... = Spare: 0
..0. .... = EVEQU (Event Quota): False
...0 .... = EVETH (Event Threshold): False
.... 0... = MACAR (MAC Addresses Reporting): False
.... .0.. = ENVCL (Envelope Closure): False
.... ..0. = TIMQU (Time Quota): False
.... ...0 = VOLQU (Volume Quota): False
Stack trace:
/src/vpp/src/vnet/fib/fib_table.c:35 (fib_table_get) assertion `! pool_is_free (ip4_main.fibs, _e)' fails
Program received signal SIGABRT, Aborted.
0x00007ffff4719f47 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#0 0x00007ffff4719f47 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff471b8b1 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x0000000000407193 in os_panic () at /src/vpp/src/vpp/vnet/main.c:371
#3 0x00007ffff55fa619 in debugger () at /src/vpp/src/vppinfra/error.c:84
#4 0x00007ffff55fa397 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7ffff76dc130 "%s:%d (%s) assertion `%s' fails") at /src/vpp/src/vppinfra/error.c:143
#5 0x00007ffff74f7cef in fib_table_get (index=4, proto=FIB_PROTOCOL_IP4) at /src/vpp/src/vnet/fib/fib_table.c:35
#6 0x00007ffff74f88ea in fib_table_entry_special_dpo_add (fib_index=4, prefix=0x7ffff7fc7fe0, source=FIB_SOURCE_FIRST, flags=(FIB_ENTRY_FLAG_EXCLUSIVE | FIB_ENTRY_FLAG_LOOSE_URPF_EXEMPT), dpo=0x7ffff7fc7fd8) at /src/vpp/src/vnet/fib/fib_table.c:333
#7 0x00007fffabdd88cf in pfcp_add_del_ue_ip (ip=0x7fffe6ebeb00, si=0x7fffe6ebdd00, is_add=1) at /src/vpp/src/plugins/upf/upf_pfcp.c:1197
#8 0x00007fffabddc766 in pfcp_update_apply (sx=0x7fffe6ebdd00) at /src/vpp/src/plugins/upf/upf_pfcp.c:1836
#9 0x00007fffabdf4f31 in handle_session_establishment_request (req=0x7fffe6ea63c0, msg=0x7ffff7fc8a20) at /src/vpp/src/plugins/upf/upf_pfcp_api.c:2444
somewhat related to #34
For Update PDR
, handle_f_teid is called with create = false
. Despite that setting, a new TEID would still be allocated, only the registration of the new TEID would be skipped. This is clearly wrong, when a new TEID is allocated is must also registered.
The other question is if a Update PDR
should be permitted to allocate a new TEID. There is no clear guidance spelled out in the specification. All references to F-TEID allocation always put this into relation of PDR creation. The is no mentioning of PDR modification in this context. It seems therefore safe to assume that a Update PDR
might only reference an already allocated F-TEID, but might not by itself allocated a new F-TEID.
The choose mechanism in handle_f_teid therefore needs to be adjusted to only allocate a new F-TEID when create
is true.
Note: it would be legal in a modify request to have a Create PDR
that creates a new F-TEID and a Update PDR
that references that new F-TEID. There is no order requirement for PFCP IEs, the Update PDR
could therefore be encoded in the PFCP request before the Update PDR
. This is not a problem for our current implementation that uses a fixed ordering when processing IE and will always process the Create PDR
before the Update PDR
I am getting this issue of session establishment with cause "no established pfcp association".
In pcap it can be seein that pfcp node has associated with upg.
$ docker exec -it vpp-upf vppctl show upf association
Node: 192.168.61.196
Recovery Time Stamp: 2021/03/02 14:19:02:000
Sessions: 0
UPG crashes if there's duplicate UE IP between several sessions. Most likely, this only concerns duplicate UE IP Address between several sessions on PDIs for SGi an also possibly for Access in TDF mode, but duplicate UE IPs in PGW mode must also be checked.
Hi,
I am currently trying to use your UPG plugin in a LTE architecture with erGW.
The UE is connecting to the network and an IP address is attributed to it but it can't reach anything nor be reached by any interface. Moreover, VPP crashes whenever I try to ping the SGi interface of VPP (10.0.0.2 on the diagram below).
My network configuration is the following
VPP and erGW are placed in a Ubuntu 20.04 docker container on a computer.
The MME and HSS are both from the Open5GS project and are hosted on the same computer.
I use srsLTE for the eNodeB.
I can see from the command sh upf session
that my UE is recognised by VPP as the IP address appears there.
CP F-SEID: 0xffde7210bfdc4048 (18437299340765315144) @ 192.168.1.2
UP F-SEID: 0xffde7210bfdc4048 (18437299340765315144) @ 192.168.1.1
PFCP Association: 0
PDR: 1 @ 0x7fd9f4ef3740
Precedence: 100
PDI:
Fields: 00000001
Source Interface: CP-function
Network Instance: cp
Local F-TEID: 2271652686 (0x8766ab4e)
IPv4: 192.168.1.1
Outer Header Removal: GTP-U/UDP/IPv4
FAR Id: 1
URR Ids: [] @ 0x0
QER Ids: [] @ 0x0
FAR: 1
Apply Action: 00000002 == [FORWARD]
Forward:
Network Instance: epc
Destination Interface: 0
CP F-SEID: 0xffde7210c5b30007 (18437299340863275015) @ 192.168.1.2
UP F-SEID: 0xffde7210c5b30007 (18437299340863275015) @ 192.168.1.1
PFCP Association: 0
UP Inactivity Timer: 28800 secs, inactive 8.6304 secs (0x00000c05)
PDR: 1 @ 0x7fd9f49a1700
Precedence: 100
PDI:
Fields: 0000000d
Source Interface: Access
Network Instance: epc
Local F-TEID: 3559416503 (0xd42862b7)
IPv4: 172.20.17.2
UE IP address (source):
IPv4 address: 10.180.0.136
SDF Filter [1]:
permit out ip from any to assigned
Outer Header Removal: GTP-U/UDP/IPv4
FAR Id: 1
URR Ids: [1] @ 0x7fd9f4ef6960
QER Ids: [] @ 0x0
PDR: 2 @ 0x7fd9f49a1780
Precedence: 100
PDI:
Fields: 0000000c
Source Interface: SGi-LAN
Network Instance: sgi
UE IP address (destination):
IPv4 address: 10.180.0.136
SDF Filter [1]:
permit out ip from any to assigned
Outer Header Removal: no
FAR Id: 2
URR Ids: [1] @ 0x7fd9f4ef6a00
QER Ids: [] @ 0x0
FAR: 1
Apply Action: 00000002 == [FORWARD]
Forward:
Network Instance: sgi
Destination Interface: 2
FAR: 2
Apply Action: 00000002 == [FORWARD]
Forward:
Network Instance: epc
Destination Interface: 0
Outer Header Creation: [GTP-U/UDP/IPv4],TEID:00000001,IP:172.20.17.1
URR: 1
Measurement Method: 0003 == [TIME,VOLUME]
Reporting Triggers: 0000 == []
Status: 0 == []
Start Time: 2021/03/03 16:43:21:707
vTime of First Usage: 0.0000
vTime of Last Usage: 0.0000
Volume
Up: Measured: 0, Theshold: 0, Pkts: 0
Consumed: 0, Quota: 0
Down: Measured: 0, Theshold: 0, Pkts: 0
Consumed: 0, Quota: 0
Total: Measured: 0, Theshold: 0, Pkts: 0
Consumed: 0, Quota: 0
Time
Quota: 0 secs @ 2021/03/03 16:43:30:326, in -16147862 secs, handle 0xffffffff
Threshold: 0 secs @ 2021/03/03 16:43:30:326, in -16147862 secs, handle 0xffffffff
Here is my startup.conf
unix {
nodaemon
log /tmp/vpp.log
full-coredump
gid vpp
interactive
cli-listen localhost:5002
exec init.conf
}
api-trace {
on
}
api-segment {
gid vpp
}
plugins {
path /usr/src/upg-vpp/vpp/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins/
plugin dpdk_plugin.so { disable }
plugin gtpu_plugin.so { disable }
plugin upf_plugin.so { enable }
}
And my init.conf
ip table add 1
ip table add 2
ip6 table add 1
ip6 table add 2
create host-interface name grx
set interface mac address host-grx 02:42:ac:14:11:02
set interface mtu 1500 host-grx
set interface ip table host-grx 1
set interface ip address host-grx 172.20.17.2/24
set interface state host-grx up
create host-interface name sgi
set interface mac address host-sgi 02:42:0a:00:00:02
set interface mtu 1500 host-sgi
set interface ip table host-sgi 2
set interface ip address host-sgi 10.0.0.2/24
set interface state host-sgi up
create tap host-ip4-addr 192.168.1.2/24
set int ip address tap0 192.168.1.1/24
set int state tap0 up
ip route add 0.0.0.0/0 table 2 via 10.0.0.1 host-sgi
upf pfcp endpoint ip 192.168.1.1 vrf 0
upf nwi name cp vrf 0
upf nwi name epc vrf 1
upf nwi name sgi vrf 2
upf gtpu endpoint ip 192.168.1.1 nwi cp teid 0x80000000/2
upf gtpu endpoint ip 172.20.17.2 nwi epc teid 0x80000000/2
Thank you in advance
Hi,
I set up upg-vpp as UPF with free5GC SMF, and used UERANSIM to test UE traffic by sending ping requests.
The uplink tunnel worked well, but the downlink did not.
Below is my setting and if there was anything wrong?
My startup.conf file
unix {
interactive
cli-listen /run/vpp/cli-vpp-upf.sock
exec /home/upf/init.conf
}
cpu {
main-core 1
corelist-workers 2-3
}
dpdk {
socket-mem 2048,2048,2048
dev 0000:0b:00.0 {
num-rx-queues 2
num-rx-desc 1024
num-tx-queues 2
num-tx-desc 1024
}
dev 0000:13:00.0 {
num-rx-queues 2
num-rx-desc 1024
num-tx-queues 2
num-tx-desc 1024
}
dev 0000:1b:00.0 {
num-rx-queues 2
num-rx-desc 1024
num-tx-queues 2
num-tx-desc 1024
}
}
My init.conf file
set int state GigabitEthernet13/0/0 up
set int state GigabitEthernetb/0/0 up
set int state GigabitEthernet1b/0/0 up
set int ip table GigabitEthernet13/0/0 0
set int ip table GigabitEthernetb/0/0 0
set int ip table GigabitEthernet1b/0/0 0
set int ip address GigabitEthernet13/0/0 192.168.1.10/24
set int ip address GigabitEthernetb/0/0 192.168.0.30/24
set int ip address GigabitEthernet1b/0/0 192.168.2.1/24
ip route add 0.0.0.0/0 table 0 via 192.168.2.1 GigabitEthernet1b/0/0
upf pfcp endpoint ip 192.168.1.10 vrf 0
upf nwi name access vrf 0
upf nwi name core vrf 0
upf gtpu endpoint ip 192.168.0.30 nwi access teid 0x00000001/2
upf node-id ip4 192.168.1.10
After tracing with vpp debug tools, I found that upg-vpp did not have encap node after interface-output node and thus DL packets were dropped. (vpp gtup plugin has this node)
------------------- Start of thread 2 vpp_wk_1 -------------------
Packet 1
00:07:21:817644: dpdk-input
GigabitEthernetb/0/0 rx queue 1
buffer 0x6a2e6: current data 0, length 142, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x2000001
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 0, nb_segs 1, pkt_len 142
buf_len 2176, data_len 142, ol_flags 0x182, data_off 128, phys_addr 0x9108ba00
packet_type 0x291 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x9e668c47 fdir.hi 0x0 fdir.lo 0x9e668c47
Packet Offload Flags
PKT_RX_RSS_HASH (0x0002) RX packet with RSS hash result
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers
RTE_PTYPE_L4_UDP (0x0200) UDP packet
IP4: 00:0c:29:5e:21:45 -> 00:0c:29:5d:4b:f1
UDP: 192.168.0.21 -> 192.168.0.30
tos 0x00, ttl 64, length 128, checksum 0x941d dscp CS0 ecn NON_ECN
fragment id 0x24cc, flags DONT_FRAGMENT
UDP: 2152 -> 2152
length 108, checksum 0x2cd1
00:07:21:817657: ethernet-input
frame: flags 0x1, hw-if-index 1, sw-if-index 1
IP4: 00:0c:29:5e:21:45 -> 00:0c:29:5d:4b:f1
00:07:21:817664: ip4-input
UDP: 192.168.0.21 -> 192.168.0.30
tos 0x00, ttl 64, length 128, checksum 0x941d dscp CS0 ecn NON_ECN
fragment id 0x24cc, flags DONT_FRAGMENT
UDP: 2152 -> 2152
length 108, checksum 0x2cd1
00:07:21:817666: ip4-lookup
fib 0 dpo-idx 8 flow hash: 0x00000000
UDP: 192.168.0.21 -> 192.168.0.30
tos 0x00, ttl 64, length 128, checksum 0x941d dscp CS0 ecn NON_ECN
fragment id 0x24cc, flags DONT_FRAGMENT
UDP: 2152 -> 2152
length 108, checksum 0x2cd1
00:07:21:817669: ip4-local
UDP: 192.168.0.21 -> 192.168.0.30
tos 0x00, ttl 64, length 128, checksum 0x941d dscp CS0 ecn NON_ECN
fragment id 0x24cc, flags DONT_FRAGMENT
UDP: 2152 -> 2152
length 108, checksum 0x2cd1
00:07:21:817670: ip4-udp-lookup
UDP: src-port 2152 dst-port 2152
00:07:21:817672: upf-gtpu4-input
GTPU decap from gtpu_session0 teid 0x00000001 next 1 error 0
00:07:21:817676: upf-ip4-flow-process
upf_session0 cp-seid 0x0000000000000001
FlowInfo - sw_if_index 1, next_index = 2
proto 0x1, 60.60.0.1:0 <-> 192.168.2.10:0, seid 0x0000000000000001
00000000: 01000000000000000000000000000000000000003c3c00010000000000000000
00000020: 00000000c0a8020a0000000001000000
ICMP: 60.60.0.1 -> 192.168.2.10
tos 0x00, ttl 64, length 84, checksum 0x9829 dscp CS0 ecn NON_ECN
fragment id 0xa390, flags DONT_FRAGMENT
ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817680: upf-ip4-input
upf_session0 cp-seid 0x0000000000000001 pdr 1 far 1
ICMP: 60.60.0.1 -> 192.168.2.10
tos 0x00, ttl 64, length 84, checksum 0x9829 dscp CS0 ecn NON_ECN
fragment id 0xa390, flags DONT_FRAGMENT
ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817682: upf-ip4-forward
upf_session0 cp-seid 0x0000000000000001 pdr 1 far 1
ICMP: 60.60.0.1 -> 192.168.2.10
tos 0x00, ttl 64, length 84, checksum 0x9829 dscp CS0 ecn NON_ECN
fragment id 0xa390, flags DONT_FRAGMENT
ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817684: ip4-input
ICMP: 60.60.0.1 -> 192.168.2.10
tos 0x00, ttl 64, length 84, checksum 0x9829 dscp CS0 ecn NON_ECN
fragment id 0xa390, flags DONT_FRAGMENT
ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817684: ip4-lookup
fib 0 dpo-idx 9 flow hash: 0x00000000
ICMP: 60.60.0.1 -> 192.168.2.10
tos 0x00, ttl 64, length 84, checksum 0x9829 dscp CS0 ecn NON_ECN
fragment id 0xa390, flags DONT_FRAGMENT
ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817685: ip4-rewrite
tx_sw_if_index 3 dpo-idx 9 : ipv4 via 192.168.2.10 GigabitEthernet1b/0/0: mtu:9000 next:5 000c29232347000c295d4b050800 flow hash: 0x00000000
00000000: 000c29232347000c295d4b05080045000054a39040003f0199293c3c0001c0a8
00000020: 020a08003d3340bd0002eb80c1600000000006590800000000001011
00:07:21:817688: GigabitEthernet1b/0/0-output
GigabitEthernet1b/0/0
IP4: 00:0c:29:5d:4b:05 -> 00:0c:29:23:23:47
ICMP: 60.60.0.1 -> 192.168.2.10
tos 0x00, ttl 63, length 84, checksum 0x9929 dscp CS0 ecn NON_ECN
fragment id 0xa390, flags DONT_FRAGMENT
ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817690: GigabitEthernet1b/0/0-tx
GigabitEthernet1b/0/0 tx queue 2
buffer 0x6a2e6: current data 44, length 98, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x2000001
ext-hdr-valid
l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 l3-hdr-offset 58
PKT MBUF: port 0, nb_segs 1, pkt_len 98
buf_len 2176, data_len 98, ol_flags 0x182, data_off 172, phys_addr 0x9108ba00
packet_type 0x291 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x9e668c47 fdir.hi 0x0 fdir.lo 0x9e668c47
Packet Offload Flags
PKT_RX_RSS_HASH (0x0002) RX packet with RSS hash result
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers
RTE_PTYPE_L4_UDP (0x0200) UDP packet
IP4: 00:0c:29:5d:4b:05 -> 00:0c:29:23:23:47
ICMP: 60.60.0.1 -> 192.168.2.10
tos 0x00, ttl 63, length 84, checksum 0x9929 dscp CS0 ecn NON_ECN
fragment id 0xa390, flags DONT_FRAGMENT
ICMP echo_request checksum 0x3d33 id 16573
Packet 2
00:07:21:818087: dpdk-input
GigabitEthernet1b/0/0 rx queue 1
buffer 0x3dd18: current data 0, length 98, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x2000002
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 2, nb_segs 1, pkt_len 98
buf_len 2176, data_len 98, ol_flags 0x82, data_off 128, phys_addr 0xa4974680
packet_type 0x91 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0xb8ab52c9 fdir.hi 0x0 fdir.lo 0xb8ab52c9
Packet Offload Flags
PKT_RX_RSS_HASH (0x0002) RX packet with RSS hash result
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers
IP4: 00:0c:29:23:23:47 -> 00:0c:29:5d:4b:05
ICMP: 192.168.2.10 -> 60.60.0.1
tos 0x00, ttl 64, length 84, checksum 0x1f9c dscp CS0 ecn NON_ECN
fragment id 0x5c1e
ICMP echo_reply checksum 0x4533 id 16573
00:07:21:818097: ethernet-input
frame: flags 0x1, hw-if-index 3, sw-if-index 3
IP4: 00:0c:29:23:23:47 -> 00:0c:29:5d:4b:05
00:07:21:818099: ip4-input
ICMP: 192.168.2.10 -> 60.60.0.1
tos 0x00, ttl 64, length 84, checksum 0x1f9c dscp CS0 ecn NON_ECN
fragment id 0x5c1e
ICMP echo_reply checksum 0x4533 id 16573
00:07:21:818099: ip4-lookup
fib 0 dpo-idx 6 flow hash: 0x00000000
ICMP: 192.168.2.10 -> 60.60.0.1
tos 0x00, ttl 64, length 84, checksum 0x1f9c dscp CS0 ecn NON_ECN
fragment id 0x5c1e
ICMP echo_reply checksum 0x4533 id 16573
00:07:21:818106: ip4-arp
ICMP: 192.168.2.10 -> 60.60.0.1
tos 0x00, ttl 64, length 84, checksum 0x1f9c dscp CS0 ecn NON_ECN
fragment id 0x5c1e
ICMP echo_reply checksum 0x4533 id 16573
00:07:21:818111: ip4-drop
ICMP: 192.168.2.10 -> 60.60.0.1
tos 0x00, ttl 64, length 84, checksum 0x1f9c dscp CS0 ecn NON_ECN
fragment id 0x5c1e
ICMP echo_reply checksum 0x4533 id 16573
00:07:21:818131: error-drop
rx:GigabitEthernet1b/0/0
00:07:21:818133: drop
ip4-arp: ARP requests sent
PFCP create PDR message
0: handle_session_modification_request:2502: CP F-SEID: 0x0000000000000001 (1) @ 192.168.1.3
UP F-SEID: 0x0000000000000001 (1) @ 192.168.1.10
SIdx: 0
Pointer: 0x7fffb8ad10c0
PDR: 0x7fffb5b46c70
FAR: 0x7fffbc9f0b60
PFCP Association: 0
(prev:4294967295,next:4294967295)
PDR: 1 @ 0x7fffb5b46c70
Precedence: 255
PDI:
Fields: 0000000d
Source Interface: Access
Network Instance: access
Local F-TEID: 1 (0x00000001)
IPv4: 192.168.0.30
UE IP address (source):
IPv4 address: 60.60.0.1
SDF Filter [1]:
permit out ip from any to assigned
Outer Header Removal: GTP-U/UDP/IPv4
FAR Id: 1
URR Ids: [] @ 0x0
QER Ids: [1] @ 0x7fffbcc389b0
PDR: 2 @ 0x7fffb5b46cf0
Precedence: 255
PDI:
Fields: 0000000c
Source Interface: Core
Network Instance: access
UE IP address (source):
IPv4 address: 60.60.0.1
SDF Filter [1]:
permit out ip from any to assigned
Outer Header Removal: no
FAR Id: 2
URR Ids: [] @ 0x0
QER Ids: [1] @ 0x7fffbcc38a50
FAR: 1 @ 0x7fffbc9f0b60
Apply Action: 00000002 == [FORWARD]
Forward:
Network Instance: access
Destination Interface: 2
FAR: 2 @ 0x7fffbc9f0bd0
Apply Action: 00000002 == [FORWARD]
Forward:
Network Instance: access
Destination Interface: 0
Outer Header Creation: [GTP-U/UDP/IPv4],TEID:00000001,IP:192.168.0.21
Rewrite Header: UDP: 192.168.0.30 -> 192.168.0.21
tos 0x00, ttl 254, length 0, checksum 0x3b69 dscp CS0 ecn NON_ECN
fragment id 0x0000
UDP: 2152 -> 2152
length 0, checksum 0x0000
0: encode_ie:7693: PFCP: Cause: 1.
0: enqueue_response:464: Msg Seq No: 3, idx 2
ping pacp file
ping.zip
There was a crash with the following stack trace:
Core was generated by `/usr/bin/vpp -c /etc/vpp/startup.conf'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 __GI_abort () at abort.c:107
107 abort.c: No such file or directory.
(gdb) bt
#0 __GI_abort () at abort.c:107
#1 0x000055d60dd13715 in os_exit (code=<optimized out>) at /src/vpp/src/vpp/vnet/main.c:395
#2 0x00007fd4ad59a135 in unix_signal_handler (signum=11, si=<optimized out>, uc=<optimized out>) at /src/vpp/src/vlib/unix/main.c:184
#3 <signal handler called>
#4 upf_pfcp_send_data (msg=0x7fd49ec0b980) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:82
#5 0x00007fd463532b5f in upf_pfcp_server_rx_msg (msg=0x7fd49fe71340) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:263
#6 pfcp_process (vm=0x7fd4ad7b46c0 <vlib_global_main>, rt=<optimized out>, f=<optimized out>) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:1142
#7 0x00007fd4ad53a1a7 in vlib_process_bootstrap (_a=<optimized out>) at /src/vpp/src/vlib/main.c:1454
#8 0x00007fd4ac980ec4 in clib_calljmp () from /usr/lib/x86_64-linux-gnu/libvppinfra.so.20.09
#9 0x00007fd46c2e7550 in ?? ()
#10 0x00007fd4ad52fcd2 in vlib_process_startup (vm=0x7fd4ad7b46c0 <vlib_global_main>, p=0x7fd46bdc8900, f=0x0) at /src/vpp/src/vlib/main.c:1479
#11 dispatch_process (vm=0x7fd4ad7b46c0 <vlib_global_main>, p=0x7fd46bdc8900, f=0x0, last_time_stamp=<optimized out>) at /src/vpp/src/vlib/main.c:1535
Apparently, there is a problem in PFCP resend code that is causing UPG to crash when it re-sends a response to a control plane request.
The line related to re-send:
Line 263 in e95bbb8
first use clito config appid,suck as
create upf application proxy name 111
upf application 111 rule 3000 add l7 regex ^https?://(.\.)(example)\.com/
upf application 111 rule 3001 add ipfilter permit out ip from any to assigned
then contrust pdr with application 111 ie .
then eatablish session ,
all is ok
Simulate a mobile phone to visit an app。
but show trace ,i find pdrindex is error.No proxy node,i think It must pass the node of proxy
Avoid gitlab.com mirroring which sometimes breaks
Automate release process
Implement R16 UE IP address Pool Information AVP.
R16 clarified the UE IP address pool handling and remove some older AVP. Having the R16 AVP supported makes handling the UE pools in the SMF simpler.
maybe don't support send echo message ?
Scapy-based tests broke after #12. Also, they're actually currently skipped in CI, perhaps due to an unnoticed issue when migrating the repo.
This was verified for Session Creation Requests with duplicate URR Ids, but likely applies to other IEs and to Session Modification Requests, too
When a session is deleted, all of the PFCP responses in the retransmit queue for its SEID are dropped.
This happens to include Session Deletion Response for the request that caused the session to be deleted in the first place.
Consequently, if a Session Deletion Response is lost, the retransmission from the SMF side will cause a "Session Context Not Found" error.
flows that have expired
counts rescheduled flows as well as flows that have really expired
When CH field in F-TEID is set then UPF should handle TEID allocation and report teid in response (TS 29.244, 8.2.3, bit3 CH).
But I see error of mandatory IE is missing.
Am I correct and this is to be handled ?
Frame 14: 164 bytes on wire (1312 bits), 164 bytes captured (1312 bits) on interface eth0, id 0
Ethernet II, Src: 02:42:c0:a8:3d:c4 (02:42:c0:a8:3d:c4), Dst: 02:42:c0:a8:3d:c5 (02:42:c0:a8:3d:c5)
Internet Protocol Version 4, Src: 192.168.61.196, Dst: 172.21.16.2
User Datagram Protocol, Src Port: 56768, Dst Port: 8805
Packet Forwarding Control Protocol
Flags: 0x21, SEID (S)
001. .... = Version: 1
...0 .... = Spare: 0
.... 0... = Spare: 0
.... .0.. = Spare: 0
.... ..0. = Message Priority (MP): False
.... ...1 = SEID (S): True
Message Type: PFCP Session Establishment Request (50)
Length: 118
SEID: 0x0000000000000000
Sequence Number: 601419
Spare: 0
Node ID : IPv4 address: 192.168.61.196
F-SEID : SEID: 0x0000000000000001, IPv4 192.168.61.196
Create PDR : [Grouped IE]
IE Type: Create PDR (1)
IE Length: 50
PDR ID : 1
Precedence : 15
PDI : [Grouped IE]
IE Type: PDI (2)
IE Length: 19
Source Interface : Access
IE Type: Source Interface (20)
IE Length: 1
0000 .... = Spare: 0
.... 0000 = Source Interface: Access (0)
F-TEID :
IE Type: F-TEID (21)
IE Length: 1
Flags: 0x04, CH (CHOOSE)
0000 .... = Spare: 0
.... 0... = CHID (CHOOSE_ID): False
.... .1.. = CH (CHOOSE): True
.... ..0. = V6 (IPv6): Not Present
.... ...0 = V4 (IPv4): Not Present
UE IP Address :
Outer Header Removal : GTP-U/UDP/IPv4
FAR ID : Dynamic by CP 1
Create FAR : [Grouped IE]
[Response In: 15]
Response message -
Frame 15: 69 bytes on wire (552 bits), 69 bytes captured (552 bits) on interface eth0, id 0
Ethernet II, Src: 02:42:c0:a8:3d:c5 (02:42:c0:a8:3d:c5), Dst: 02:42:c0:a8:3d:c4 (02:42:c0:a8:3d:c4)
Internet Protocol Version 4, Src: 172.21.16.2, Dst: 192.168.61.196
User Datagram Protocol, Src Port: 8805, Dst Port: 56768
Packet Forwarding Control Protocol
Flags: 0x21, SEID (S)
001. .... = Version: 1
...0 .... = Spare: 0
.... 0... = Spare: 0
.... .0.. = Spare: 0
.... ..0. = Message Priority (MP): False
.... ...1 = SEID (S): True
Message Type: PFCP Session Establishment Response (51)
Length: 23
SEID: 0x0000000000000000
Sequence Number: 601419
Spare: 0
Cause : Mandatory IE missing
Offending IE : PDI
IE Type: Offending IE (40)
IE Length: 2
Type of the offending IE: PDI (2)
[Response To: 14]
[Response Time: 0.000559919 seconds]
The PFCP inherent error reporting is not very verbose and makes it hard to troubleshoot failed PFCP requests.
Add a TP vendor IE to give a verbose error description. Either a string or a structured IE that indicates file and line where the error occurred. The inclusion of that IE could be restricted to debug builds.
$ git commit -m"test"
Build image tag not up to date (.gitlab-ci.yml), please run
hack/update-build-image-tag.sh
(without arguments)
$ hack/update-build-image-tag.sh
$ git commit -m"test"
Build image tag not up to date (.gitlab-ci.yml), please run
hack/update-build-image-tag.sh
(without arguments)
eithe hook is missdetecting that or the hack/update-build-image-tag.sh is not solving the problem as expected.
VPPs checkstyle should be run for every PR, somehow.
Hi,
Please I am trying to setup upg-vpp as UPF for open5gs SMF.
The error I get from the open5gs SMF is:
03/20 23:01:14.209: [smf] WARNING: cannot handle PFCP message type[1] (../src/smf/pfcp-sm.c:140)
03/20 23:01:14.729: [pfcp] DEBUG: [3] REMOTE Create peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:156)
03/20 23:01:14.729: [pfcp] DEBUG: [3] REMOTE Receive peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:699)
03/20 23:01:14.729: [pfcp] DEBUG: [3] REMOTE UPD RX-1 peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:294)
03/20 23:01:14.729: [smf] WARNING: cannot handle PFCP message type[1] (../src/smf/pfcp-sm.c:140)
03/20 23:01:15.207: [smf] WARNING: Retry to association with peer [10.0.0.149]:8805 failed (../src/smf/pfcp-sm.c:107)
03/20 23:01:15.207: [pfcp] DEBUG: Association Setup Request (../lib/pfcp/build.c:64)
03/20 23:01:15.207: [pfcp] DEBUG: [83] LOCAL UPD TX-5 peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:193)
03/20 23:01:15.207: [pfcp] DEBUG: [83] LOCAL Create peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:113)
03/20 23:01:15.207: [pfcp] DEBUG: [83] LOCAL Commit peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:473)
03/20 23:01:15.207: [core] ERROR: Unexpected TLV type:32770 (../lib/core/ogs-tlv-msg.c:454)
03/20 23:01:15.207: [smf] ERROR: ogs_pfcp_parse_msg() failed (../src/smf/smf-sm.c:228)
After a while upg-vpp crashes..
DBGvpp# /usr/src/upg-vpp/vpp/src/vlib/counter.h:101 (vlib_decrement_simple_counter) assertion `my_counters[index] >= decrement' fails
Aborted (core dumped)
Following are my config:
ip table add 1
ip table add 2
ip6 table add 1
ip6 table add 2
create host-interface name ens6
set interface mac address host-ens6 02:7e:e1:cc:04:a7
set interface mtu 1500 host-ens6
set interface ip table host-ens6 0
set interface ip address host-ens6 10.0.0.149/24
set interface state host-ens6 up
create host-interface name ens7
set interface mac address host-ens7 02:68:23:7e:1d:bf
set interface mtu 1500 host-ens7
set interface ip table host-ens7 1
set interface ip address host-ens7 10.0.6.133/24
set interface state host-ens7 up
create host-interface name ens8
set interface mac address host-ens8 02:bc:29:ee:b2:63
set interface mtu 1500 host-ens8
set interface ip table host-ens8 2
set interface ip address host-ens8 10.0.4.246/24
set interface state host-ens8 up
ip route add 0.0.0.0/0 table 0 via 10.0.0.1 host-ens6
ip route add 0.0.0.0/0 table 1 via 10.0.6.1 host-ens7
ip route add 0.0.0.0/0 table 2 via 10.0.4.1 host-ens8
upf pfcp endpoint ip 10.0.0.149 vrf 0
upf nwi name cp vrf 0
upf nwi name epc vrf 1
upf nwi name sgi vrf 2
upf gtpu endpoint ip 10.0.0.149 nwi cp teid 0x80000000/2
upf gtpu endpoint ip 10.0.6.133 nwi epc teid 0x80000000/2
unix {
nodaemon
log /tmp/vpp.log
full-coredump
gid vpp
interactive
cli-listen localhost:5002
exec init.conf
}
api-trace {
on
}
api-segment {
gid vpp
}
plugins {
path /usr/src/upg-vpp/vpp/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins/
plugin dpdk_plugin.so { disable }
plugin gtpu_plugin.so { disable }
plugin upf_plugin.so { enable }
}
Steps that I used in installing upg-vpp:
cd /usr/src
git clone https://github.com/travelping/upg-vpp
cd upg-vpp
make update-vpp
cd vpp
make install-deps
make install-ext-deps
make build
Also it will be great if someone can point me to where the documentation is, I was unable to locate this in this repo.
Debug build (v1.1.1
):
pfcp_msg_pool_elt_at_index:261: ERROR: accessing a PFCP msg that was freed at: /src/vpp/src/plugins/upf/upf_pfcp_server.c:473 response_expired
/src/vpp/src/plugins/upf/upf_pfcp_server.h:262 (pfcp_msg_pool_elt_at_index) assertion `0' fails
...
[New LWP 7]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/vpp -c /etc/vpp/startup.conf'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f9c0c7dbfb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#0 0x00007f9c0c7dbfb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f9c0c7dd921 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00000000004071f5 in os_exit (code=1) at /src/vpp/src/vpp/vnet/main.c:395
#3 0x00007f9c0e314707 in unix_signal_handler (signum=6, si=0x7f9c10090bb0, uc=0x7f9c10090a80) at /src/vpp/src/vlib/unix/main.c:187
#4 <signal handler called>
#5 0x00007f9c0c7dbfb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#6 0x00007f9c0c7dd921 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#7 0x0000000000407193 in os_panic () at /src/vpp/src/vpp/vnet/main.c:371
#8 0x00007f9c0d6bc619 in debugger () at /src/vpp/src/vppinfra/error.c:84
#9 0x00007f9c0d6bc397 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7f9bc417e027 "%s:%d (%s) assertion `%s' fails") at /src/vpp/src/vppinfra/error.c:143
#10 0x00007f9bc4126614 in pfcp_msg_pool_elt_at_index (psm=0x7f9bc43bb460 <pfcp_server_main>, index=6414) at /src/vpp/src/plugins/upf/upf_pfcp_server.h:262
#11 0x00007f9bc41254b2 in response_expired (id=6414) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:467
#12 0x00007f9bc41166ac in pfcp_process (vm=0x7f9c0e547b80 <vlib_global_main>, rt=0x7f9bcca6a0c0, f=0x0) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:1252
#13 0x00007f9c0e28deed in vlib_process_bootstrap (_a=140307136063496) at /src/vpp/src/vlib/main.c:1454
#14 0x00007f9c0d6e07a4 in clib_calljmp () at /src/vpp/src/vppinfra/longjmp.S:123
#15 0x00007f9bcd012800 in ?? ()
#16 0x00007f9c0e28d92f in vlib_process_startup (vm=0x1afee8f1e8, p=0x874ad741c34756, f=0x95) at /src/vpp/src/vlib/main.c:1479
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
Release build (v1.1.4
; due to lack of ASSERTs in the release build, the issue is not caught early enough, leads to crash in another place later):
warning: core file may not match specified executable file.
[New LWP 8]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/vpp -c /etc/vpp/startup.conf'.
Program terminated with signal SIGABRT, Aborted.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007f1ea4a7d921 in __GI_abort () at abort.c:79
#2 0x0000561a8cc38715 in os_exit (code=<optimized out>) at /src/vpp/src/vpp/vnet/main.c:395
#3 0x00007f1ea6568135 in unix_signal_handler (signum=11, si=<optimized out>, uc=<optimized out>) at /src/vpp/src/vlib/unix/main.c:184
#4 <signal handler called>
#5 upf_pfcp_send_data (msg=0x7f1ce2867800) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:83
#6 0x00007f1e5c4fcb08 in upf_pfcp_server_rx_msg (msg=0x7f1cecaa7180) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:272
#7 pfcp_process (vm=<optimized out>, rt=<optimized out>, f=<optimized out>) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:1162
#8 0x00007f1ea65081a7 in vlib_process_bootstrap (_a=<optimized out>) at /src/vpp/src/vlib/main.c:1454
#9 0x00007f1ea594eec4 in clib_calljmp () from /usr/lib/x86_64-linux-gnu/libvppinfra.so.20.09
#10 0x00007f1e6528a550 in ?? ()
#11 0x00007f1ea64fdcd2 in vlib_process_startup (vm=0x7f1ea67826c0 <vlib_global_main>, p=0x7f1e64ce20c0, f=0x0) at /src/vpp/src/vlib/main.c:1479
#12 dispatch_process (vm=0x7f1ea67826c0 <vlib_global_main>, p=0x7f1e64ce20c0, f=0x0, last_time_stamp=<optimized out>) at /src/vpp/src/vlib/main.c:1535
Backtrace stopped: Cannot access memory at address 0x7f1ea899c0b8```
It should contain UPG build id instead
UPG may hang after receiving a corrupt GTP-U datagram such as one generated by UPG itself due to #42
--label quay.expires-after=7d
should be used for all non tagged builds
GTP-U header values from the incoming GTP-U traffic can sometimes "leak" to the GTP-U packets created by GTP-U, leading to corrupt GTP-U datagrams. E.g. E
flag.
Right now, there's no proper "sync" between proxy sessions and the flows.
A flow can be deleted upon a timeout while there's still a proxy session entry for that flow. The unused proxy sessions may be kept indefinitely. Also, a flow can "respawn" due to packets sent by the proxy after the flow has expired.
There is a check for flow index pointing to a non-free flowtable entry e.g. in proxy_start_connect_fn()
, but that is not correct either as that function can be invoked after quite a bit of time after the entry has been removed, in which case there'll be another flow in its place, leading to undefined behavior.
Trying to delete a proxy session right away poses some problems, namely, it's hard to avoid "late" callbacks from the VPP session layer, e.g. it's very hard to get rid of pending half-open connections properly (e.g. active connections in TCP_STATE_SYN_SENT
state).
Suggested solution: add 'flow expiration' hook. When a flow is about to expire, the hook is invoked, and if it returns 0
, the flow is removed. Otherwise, the flow expiration is rescheduled.
The flows should contain proxy session index instead of connection index that it has now. If there's no proxy session associated with the flow, the proxy session index contains ~0
.
The proxy installs the expiration hook that checks if the flow has proxy session index other than ~0
. If so, it "nudges" the proxy session via proxy_session_try_close_unlocked()
(with proper locking around it), and returns -1
to tell the flowtable that the flow can't be removed just yet. proxy_session_free()
sets proxy session index to ~0
in the corresponding flow, after which 'flow expiration hook' will succeed.
Alternatives: 'refcount' for flows. Perhaps not really necessary till only proxy sessions keep the flows from being freed. Also, this doesn't cover 'nudging' the proxy to clean up its session.
Also: not using the hook, just calling a proxy function from flowtable. But that's too much coupling IMO.
The symptoms are described in #94
The buffer is reused for ICMP echo response by VPP, and thus re-enters UPG subgraph, causing assertion failure.
This does not affect release builds
The proxy is only needed when there are URL-based rules in use
ip table add 100
ip table add 200
ip6 table add 100
ip6 table add 200
upf nwi name cp table 0
upf nwi name epc table 100
upf nwi name apn table 200
After switching to a newer VPP version (21.01), the master branch started consuming too much heap in some cases (possibly allocated for session structures).
https://lists.fd.io/g/vpp-dev/message/18768
Also, build a base VPP image separately from the rest of UPG
$ git commit -m"test"
.git/hooks/pre-commit: 10: set: Illegal option -o pipefail
It seems the hock assumes that /bin/sh is bash, which is not always the case
The flows may take a very long time to expire if packets don't go through UPG every second.
With latest update in the code for node_id (association request), I see that node id is 0.0.0.0 instead of pfcp endpoint ip.
Is it problem in the configuration or something else ?
Message Type: PFCP Association Setup Response (6)
:
:
Node ID : IPv4 address: 0.0.0.0
IE Type: Node ID (60)
IE Length: 5
0000 .... = Spare: 0
.... 0000 = Node ID Type: IPv4 address (0)
Node ID IPv4: 0.0.0.0
This is the API
https://github.com/travelping/upg-vpp/blob/5129d0d9fd79cd1e0b7e143d96b0b3c5a5997567/upf/upf.api
I think the UPG code supports matching prefix and layer-4 port range. But, I don't see layer-4 port info in the API.
Implement support for quota validity time feature in UPG.
This is required to support Gy quota validity time in a state less C node.
The following crash has been observed when running e2e tests:
INFO[0827] VPP: /src/vpp/src/plugins/upf/upf_pfcp_server.h:240 (pfcp_msg_pool_elt_at_index) assertion `m->is_valid_pool_item' fails pid=5954 what=stdout
INFO[0827] VPP: pid=5954 what=stdout
INFO[0827] VPP: Program received signal SIGABRT, Aborted. pid=5954 what=stdout
INFO[0827] VPP: 0x00007ffff4717fb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6 pid=5954 what=stdout
INFO[0827] VPP: #0 0x00007ffff4717fb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6 pid=5954 what=stdout
INFO[0827] VPP: #1 0x00007ffff4719921 in abort () from /lib/x86_64-linux-gnu/libc.so.6 pid=5954 what=stdout
INFO[0827] VPP: #2 0x0000000000407193 in os_panic () at /src/vpp/src/vpp/vnet/main.c:371 pid=5954 what=stdout
INFO[0827] VPP: #3 0x00007ffff55f8619 in debugger () at /src/vpp/src/vppinfra/error.c:84 pid=5954 what=stdout
INFO[0827] VPP: #4 0x00007ffff55f8397 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7fffac0ba8b7 "%s:%d (%s) assertion `%s' fails") at /src/vpp/src/vppinfra/error.c:143 pid=5954 what=stdout
INFO[0827] VPP: #5 0x00007fffac063138 in pfcp_msg_pool_elt_at_index (psm=0x7fffac2f7250 <pfcp_server_main>, index=11880) at /src/vpp/src/plugins/upf/upf_pfcp_server.h:240 pid=5954 what=stdout
INFO[0827] VPP: #6 0x00007fffac062052 in response_expired (id=11880) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:462 pid=5954 what=stdout
INFO[0827] VPP: #7 0x00007fffac05352a in pfcp_process (vm=0x7ffff6483b80 <vlib_global_main>, rt=0x7fffb4cb3580, f=0x0) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:1250 pid=5954 what=stdout
INFO[0827] VPP: #8 0x00007ffff61c9eed in vlib_process_bootstrap (_a=140736242026504) at /src/vpp/src/vlib/main.c:1454 pid=5954 what=stdout
INFO[0827] VPP: #9 0x00007ffff561c7a4 in clib_calljmp () at /src/vpp/src/vppinfra/longjmp.S:123 pid=5954 what=stdout
In a configuration where all of the traffic that UPG passes in PGW-U mode should match one of the IP rules it was observed that the buffers are dropped after upf-ip[46]-flow-process
.
i can't understand it.
I have a Rule creation/modification failure. I use free5gc SMF.
A reflection track would be welcome
test2.zip
my init file:
create host-interface name vpp-pfcp-peer
set interface state host-vpp-pfcp-peer up
set interface ip address host-vpp-pfcp-peer 172.26.0.50/24
create host-interface name vpp-gtp-peer
set interface state host-vpp-gtp-peer up
set interface ip address host-vpp-gtp-peer 172.16.0.1/24
create host-interface name vpp-sgi-peer
set interface state host-vpp-sgi-peer up
set interface ip address host-vpp-sgi-peer 192.168.0.2/24
upf pfcp endpoint ip 172.26.0.50 vrf 0
upf nwi name cp vrf 0
upf nwi name access vrf 100
upf nwi name sgi vrf 200
upf gtpu endpoint ip 172.16.0.1 nwi access teid 0x00001234/2
There's a memory leak in UPG when using proxy, possibly mostly for non-stitched TCP connections.
The leak is not visible in show memory
, but vpp RSS keeps increasing.
It is likely related to SVM FIFOs being not properly freed.
Below is a flame graph constructed from page fault tracing with perf (based on [1]).
[1] http://www.brendangregg.com/FlameGraphs/memoryflamegraphs.html
I see use of pfcp_add_del_v4_teid (v4_teid, sx, 0);
for delete as shown on the line below.
Line 991 in ca2791c
For better code maintenance, pfcp_add_del_v4_teid() should be used to add as well, but the code does not. Instead the code uses hash_set()
I see use of pfcp_add_del_v4_teid (v4_teid, sx, 0);
for delete as shown on the line below.
Line 991 in ca2791c
For better code maintenance, pfcp_add_del_v4_teid() should be used to add as well, but the code does not. Instead the code uses hash_set()
So far the problem was only observed in the CI environment. It is not clear at this point if it's a test issue or a real bug. Lowering the rate of session modifications appears to make this problem more rare.
This problem appears only to happen with release builds.
Test:
[Fail] Clearing message queue during PFCP Association Release upon timeout [It] should work correcty
Diagnostics:
Program received signal SIGABRT, Aborted.
0x00007ffff5cb618b in raise () from /lib/x86_64-linux-gnu/libc.so.6
#0 0x00007ffff5cb618b in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007ffff5c95859 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007ffff6385d86 in os_panic () at /src/vpp/src/vppinfra/unix-misc.c:177
#3 0x00007ffff6389637 in clib_mem_alloc_aligned_at_offset (size=2056, align=<optimized out>, align_offset=<optimized out>, os_out_of_memory_on_failure=1) at /src/vpp/src/vppinfra/mem.h:243
#4 vec_resize_allocate_memory (v=<optimized out>, length_increment=2048, data_bytes=2056, header_bytes=8, data_align=<optimized out>, numa_id=<optimized out>) at /src/vpp/src/vppinfra/vec.c:67
#5 0x00007fffb3258a45 in _vec_resize_inline (v=0x0, length_increment=<optimized out>, data_bytes=<optimized out>, header_bytes=<optimized out>, data_align=<optimized out>, numa_id=<optimized out>) at /src/vpp/src/vppinfra/vec.h:170
#6 upf_pfcp_make_response (resp=0x7ffff3f04580, req=0x7fffefaf9ec0, len=2048) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:520
#7 upf_pfcp_send_response (req=0x7fffefaf9ec0, cp_seid=18299514082375153098, type=<optimized out>, grp=<optimized out>) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:532
#8 0x00007fffb32531e3 in session_msg (msg=<optimized out>) at /src/vpp/src/plugins/upf/upf_pfcp_api.c:2644
#9 upf_pfcp_handle_msg (msg=<optimized out>) at /src/vpp/src/plugins/upf/upf_pfcp_api.c:181
Previous report: travelping/vpp#85
quay.io/travelping/upf:fix-2009-traffic-timer_v20.09-upfbase-9-g9a520fa75_debug
The result of having stale PDRs is this crash:
/src/vpp/src/plugins/upf/upf_input.c:152 (upf_input) assertion `(((upf_buffer_opaque_t *)((u8 *)((b)->opaque2) + ((uword) & (((vnet_buffer_opaque_t *) 0)->unused))))->gtpu.pdr_idx) < vec_len (active->pdr)' fails
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.