upg-vpp doesn't encap gtpu header for downlink traffic about upg-vpp HOT 12 CLOSED

HI @infinitydon
The performance bottleneck is on UERANSIM, you can try free5gc's gtp5g and libgtp5gnl tool to simulate UE.
Or using T-rex, It powerful test tool.
Two way will get better throughput via my test.

Good luck

from upg-vpp.

Looks like a bug in your SMF, PDR 2 is wrong:

• Source Interface need to be SGi-LAN/N6-LAN
• UE IP Address needs to have S/D flag set to make it a destination IP

from upg-vpp.

Thanks, it's work. :)

from upg-vpp.

@redbag-2 can you share please how you changed these two settings on the smf ? thank you in advance

from upg-vpp.

@redbag-2 - Can you kindly share how you resolved the networkInstance encoding format on Free5gc SMF?

from upg-vpp.

Sorry reply so late..
I modify free5gc/NFs/smf/context/datapath.go file, to add Sd flag in UEIPAddress IE.

			ULPDR.PDI = PDI{
				SourceInterface: pfcpType.SourceInterface{InterfaceValue: pfcpType.SourceInterfaceAccess},
				NetworkInstance: []byte(smContext.Dnn),
				LocalFTeid: &pfcpType.FTEID{
					V4:          true,
					Ipv4Address: upIP,
					Teid:        curULTunnel.TEID,
				},
				UEIPAddress: &pfcpType.UEIPAddress{
					Sd:          false,
					V4:          true,
					Ipv4Address: smContext.PDUAddress.To4(),
				},

				DLPDR.PDI = PDI{
					SourceInterface: pfcpType.SourceInterface{InterfaceValue: pfcpType.SourceInterfaceCore},
					LocalFTeid: &pfcpType.FTEID{
						V4:          true,
						Ipv4Address: upIP,
						Teid:        curDLTunnel.TEID,
					},

					UEIPAddress: &pfcpType.UEIPAddress{
					 	Sd:          true,
					 	V4:          true,
					 	Ipv4Address: smContext.PDUAddress.To4(),
					 },
				}
			}

	if curDataPathNode.DownLinkTunnel != nil {
		if curDataPathNode.DownLinkTunnel.SrcEndPoint == nil {
			DNDLPDR := curDataPathNode.DownLinkTunnel.PDR
			DNDLPDR.PDI = PDI{
				SourceInterface: pfcpType.SourceInterface{InterfaceValue: pfcpType.SourceInterfaceCore},
				NetworkInstance: util_3gpp.Dnn(smContext.Dnn),
				UEIPAddress: &pfcpType.UEIPAddress{
					Sd:          true,
					V4:          true,
					Ipv4Address: smContext.PDUAddress.To4(),
				},
			}
		}
	}

from upg-vpp.

@redbag-2 - Can you kindly share how you resolved the networkInstance encoding format on Free5gc SMF?

Sorry, I didn't encounter any problems about networkInstance encoding format on Free5gc SMF.
Can you explain your problems more clearly?

from upg-vpp.

@redbag-2 - First of all, thanks for sharing how you got it working.

With regards the NWI encoding please see the issue below:

#118 (comment)

Can you also kindly share the upg-vpp and Freee5gc software version (or commit) that you used? Also if possible the Free5gc SMF config, I will like to compare it with what is currently available in the Free5gc repo.

from upg-vpp.

@infinitydon
Hi,
UPG-VPP NWI setting must be equal to smfcfg.conf networkInstance.

And I use ip format in node_id, so UPG-VPP need to setup node_id address.
Below is my init.conf and smfcfg.conf, you can refer it.

init.conf

set int state GigabitEthernet13/0/0 up // data_plane
set int state GigabitEthernetb/0/0 up // control_plane

set int ip table GigabitEthernet13/0/0 0
set int ip table GigabitEthernetb/0/0 0

set int ip address GigabitEthernetb/0/0 192.168.1.50/24
set int ip address GigabitEthernet13/0/0 192.168.0.50/24

ip route add 0.0.0.0/0 table 0 via <gw_ip> GigabitEthernet13/0/0

upf pfcp endpoint ip 192.168.1.50 vrf 0
upf nwi name internet vrf 0
upf gtpu endpoint ip 192.168.0.50 nwi internet teid 0x00000001/2

upf node-id ip4 192.168.1.50

smfcfg.conf

userplane_information: # list of userplane information
up_nodes: # information of userplane node (AN or UPF)
gNB1: # the name of the node
type: AN # the type of the node (AN or UPF)
UPF: # the name of the node
type: UPF # the type of the node (AN or UPF)
node_id: 192.168.1.50 # the IP/FQDN of N4 interface on this UPF (PFCP) -> It must equal to upg-vpp node-id
sNssaiUpfInfos: # S-NSSAI information list for this UPF
- sNssai: # S-NSSAI (Single Network Slice Selection Assistance Information)
sst: 1
sd: 010203
dnnUpfInfoList: # DNN information list for this S-NSSAI
- dnn: internet
pools:
- cidr: 60.60.0.0/16
- sNssai: # S-NSSAI (Single Network Slice Selection Assistance Information)
sst: 1
sd: 112233 # Slice Differentiator
dnnUpfInfoList: # DNN information list for this S-NSSAI
- dnn: internet
pools:
- cidr: 60.61.0.0/16
interfaces: # Interface list for this UPF
- interfaceType: N3 # the type of the interface (N3 or N9)
endpoints: # the IP address of this N3/N9 interface on this UPF
- 192.168.0.50 -> It must equal to upg-vpp gtpu ip address
networkInstance: internet # Data Network Name (DNN) -> It must equal to upg-vpp NWI

Finally,I test upg-vpp and free5gc latest version, it all work well :) .

Best regards

from upg-vpp.

Thanks @redbag-2 - Now SMF is able to create PDU session successfully on the upg-vpp:

vpp# show upf association
Node: 10.0.4.13
  Recovery Time Stamp: 2021/08/25 09:44:54:000
  Sessions: 0
vpp#
vpp# show upf association
Node: 10.0.4.13
  Recovery Time Stamp: 2021/08/25 09:44:54:000
  Sessions: 1
vpp# show upf session
CP F-SEID: 0x0000000000000001 (1) @ 10.0.4.13
UP F-SEID: 0x0000000000000001 (1) @ 10.0.4.10
  PFCP Association: 0
PDR: 1 @ 0x7fe45dd3cd80
  Precedence: 255
  PDI:
    Fields: 0000000d
    Source Interface: Access
    Network Instance: internet
    Local F-TEID: 1 (0x00000001)
            IPv4: 10.0.6.10
    UE IP address (source):
      IPv4 address: 60.60.0.1
    SDF Filter [1]:
      permit out ip from any to assigned
  Outer Header Removal: GTP-U/UDP/IPv4
  FAR Id: 1
  URR Ids: [] @ 0x0
  QER Ids: [1] @ 0x7fe45dd6c7f0
PDR: 2 @ 0x7fe45dd3ce00
  Precedence: 255
  PDI:
    Fields: 0000000c
    Source Interface: Core
    Network Instance: internet
    UE IP address (destination):
      IPv4 address: 60.60.0.1
    SDF Filter [1]:
      permit out ip from any to assigned
  Outer Header Removal: no
  FAR Id: 2
  URR Ids: [] @ 0x0
  QER Ids: [1] @ 0x7fe45dd6c880
FAR: 1
  Apply Action: 00000002 == [FORWARD]
  Forward:
    Network Instance: internet
    Destination Interface: 2
FAR: 2
  Apply Action: 00000002 == [FORWARD]
  Forward:
    Network Instance: internet
    Destination Interface: 0
    Outer Header Creation: [GTP-U/UDP/IPv4],TEID:00000001,IP:10.0.6.158

from upg-vpp.

@redbag-2 - Sorry to bother you again, please I want to ask about the throughput speed you were able to achieve with the DPDK deployment?

I have used Kubernetes to deploy the vpp-upf with the DPDK interfaces (via sriov-dp) and everything looks fine, I am able to create sessions on the vpp-upf but the iperf throughput is very low:

ubuntu@ip-10-0-2-51:~/UERANSIM/build$ ./nr-binder 12.1.1.7 iperf3 -c 10.0.7.167 -i 1 -t 10
Connecting to host 10.0.7.167, port 5201
[  5] local 12.1.1.7 port 42151 connected to 10.0.7.167 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  7.74 MBytes  64.9 Mbits/sec  498   55.3 KBytes
[  5]   1.00-2.00   sec  2.50 MBytes  21.0 Mbits/sec  305   18.4 KBytes
[  5]   2.00-3.00   sec  2.50 MBytes  21.0 Mbits/sec  319   11.8 KBytes
[  5]   3.00-4.00   sec  1.25 MBytes  10.5 Mbits/sec  299   25.0 KBytes
[  5]   4.00-5.00   sec  2.50 MBytes  21.0 Mbits/sec  208   21.1 KBytes
[  5]   5.00-6.00   sec  2.50 MBytes  21.0 Mbits/sec  259   18.4 KBytes
[  5]   6.00-7.00   sec  2.50 MBytes  21.0 Mbits/sec  219   9.21 KBytes
[  5]   7.00-8.00   sec  2.50 MBytes  21.0 Mbits/sec  292   22.4 KBytes
[  5]   8.00-9.00   sec  1.25 MBytes  10.5 Mbits/sec  216   39.5 KBytes
[  5]   9.00-10.00  sec  3.75 MBytes  31.5 Mbits/sec  301   3.95 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  29.0 MBytes  24.3 Mbits/sec  2916             sender
[  5]   0.00-10.00  sec  25.1 MBytes  21.1 Mbits/sec                  receiver

iperf Done.

I am using ueransim to test.. Any idea what could be causing the low throughput? I tried running the ueransim on a system with 16 CPU cores but that did not help.

from upg-vpp.

Sorry reply so late.. I modify free5gc/NFs/smf/context/datapath.go file, to add Sd flag in UEIPAddress IE.

			ULPDR.PDI = PDI{
				SourceInterface: pfcpType.SourceInterface{InterfaceValue: pfcpType.SourceInterfaceAccess},
				NetworkInstance: []byte(smContext.Dnn),
				LocalFTeid: &pfcpType.FTEID{
					V4:          true,
					Ipv4Address: upIP,
					Teid:        curULTunnel.TEID,
				},
				UEIPAddress: &pfcpType.UEIPAddress{
					Sd:          false,
					V4:          true,
					Ipv4Address: smContext.PDUAddress.To4(),
				},

				DLPDR.PDI = PDI{
					SourceInterface: pfcpType.SourceInterface{InterfaceValue: pfcpType.SourceInterfaceCore},
					LocalFTeid: &pfcpType.FTEID{
						V4:          true,
						Ipv4Address: upIP,
						Teid:        curDLTunnel.TEID,
					},

					UEIPAddress: &pfcpType.UEIPAddress{
					 	Sd:          true,
					 	V4:          true,
					 	Ipv4Address: smContext.PDUAddress.To4(),
					 },
				}
			}

	if curDataPathNode.DownLinkTunnel != nil {
		if curDataPathNode.DownLinkTunnel.SrcEndPoint == nil {
			DNDLPDR := curDataPathNode.DownLinkTunnel.PDR
			DNDLPDR.PDI = PDI{
				SourceInterface: pfcpType.SourceInterface{InterfaceValue: pfcpType.SourceInterfaceCore},
				NetworkInstance: util_3gpp.Dnn(smContext.Dnn),
				UEIPAddress: &pfcpType.UEIPAddress{
					Sd:          true,
					V4:          true,
					Ipv4Address: smContext.PDUAddress.To4(),
				},
			}
		}
	}

fixed in free5gc/smf#57

from upg-vpp.