Giter VIP home page Giter VIP logo

upg-vpp's Introduction

User Plane Gateway (UPG) based on VPP

CI

UPG implements a GTP-U user plane based on 3GPP TS 23.214 and 3GPP TS 29.244 Release 15. It is implemented as an out-of-tree plugin for FD.io VPP.

The possible uses for UPG are:

  • User Plane Function (UPF) in 5G networks
  • Packet Data Network Gateway User plane (PGW-U)
  • Traffic Detection Function User plane (TDF-U)

Current State

UPG is used in production in conjunction with erGW as GGSN/PGW in multiple installation in several telecom operators (Tier 1 and smaller).

For the list of known issues, see KNOWN_ISSUES document.

Working features

  • PFCP protocol
    • en/decoding of most IEs
    • heartbeat
    • node related messages
    • session related messages
  • Uplink and Downlink Packet Detection Rules (PDR) and Forward Action Rules (FAR) -- (some parts)
  • IPv4 -- inner and outer
  • IPv6 -- inner and outer
  • Usage Reporting Rules (URR)
  • PFCP Session Reports
  • Linked Usage Reports

No yet working

  • Buffer Action Rules (BAR)
  • QoS Enforcement Rule (QER)

Limitations

  • FAR action with destination LI are not implemented
  • Ethernet bearer support

Development setup

Design rationale for the development environment is this:

  • provide an easily reproducible development and build environment usable both on CI and locally
  • provide quick commands for common tasks
  • simplify bisecting against upstream VPP
  • discourage downstream VPP changes, as we should make effort to upstream them

Relevant parts of the source tree layout:

  • hack/ contains helper scripts most of which are wrapped in make commands
  • Makefile provides user interface for the environment
  • upf/ contains the source code of the plugin1
  • upf/test/ contains the integration tests1
  • vpp.spec contains the info on VPP-base repo, branch and commit to use

There's a simple dockerized environment wrapped in 'make' commands.

The "build image" which is used for the devenv is tagged with a hash of Dockerfile.build as well as VPP's external dependencies.

The following make commands are supported:

  • make image builds UPG Docker image
  • make test build VPP and run UPG integration tests. The compilation results are retained under vpp/
  • make retest run UPG integration tests without building VPP. These can be run under make test to re-run the tests quickly if there were no UPG / VPP code changes
  • make ensure-build-image checks if the build image exists or can be pulled and builds it otherwise
  • make update-build-image-tag updates the build image tag in the Dockerfiles according to the hash calculated from Dockerfile.build and VPP external dependencies
  • make install-hooks installs git hooks in the repo which prevent the user from making commits that contain ZZZZZ: substring. This is handy for debug print like clib_warning("ZZZZZ: i %d", i);
  • make update-vpp re-clones VPP into vpp/ directory
  • make buildenv runs an interactive shell inside the build environment with UPG and VPP sources mounted into the container
  • make e2e build UPG and run E2E tests for it. For more information, see E2E test documentation
  • make checkstyle performs style checks on the source code

Commands for building images and running tests default to debug builds. To do release build instead, pass BUILD_TYPE=release to make:

make e2e BUILD_TYPE=release

If docker is used, one should set the following environment variable to enable wrapping the internally run commands in a docker container:

export UPG_BUILDENV=docker

It is also possible to use a k8s cluster to run the build container in a pod:

export UPG_BUILDENV=k8s
# optional: specify the node to run the build pod
export UPG_BUILDENV_NODE=somenode

In this case, the buildenv is run as statefulset inside the cluster. It can be removed using

hack/buildenv.sh clean

CI and releases

The CI for UPG-VPP is based on GitHub Actions. Currently, the CI only runs for pushes to branches in the repository itself. The jobs include:

  • prepare: make sure build image is available for the commit
  • build (debug + release): build the docker images and binaries / packages
  • checkstyle: check for style errors in the code
  • test: unit and e2e tests for release and debug builds
  • conclude: intermediate job used for sync by the release workflow
  • slack: internal notification job

The images built per-commit expire within 7 days.

When a tag is pushed, the release workflow is also run for it, re-tagging the images built as part of normal build process (preserving the old tags too). In case if the tag doesn't have test substring in it, it is also published as a release. The release notes list the PRs with the following tags:

  • feature, enhancement: features
  • fix: fixes
  • test: tests

The releases for tags that contain pre substring are marked as pre-releases.

1: Historically, the project was named simply "UPF". There may be more UPF->UPG renames later

VS Code

It is possible to attach to running buildenv container with VS Code to get full intellisense.

To do that run make code.

Note: this command leaves the buildenv running in the background.

After attaching for the first time, some vscode plugins may not be enabled. To fix that open: F1 -> "Dev Containers: Open Named Container Configuration File" And specify what plugins you'd like loaded at start.

Here are some nice plugins to work with this repo:

{
	"workspaceFolder": "/src",
	"extensions": [
		"eamodio.gitlens",
		"EditorConfig.EditorConfig",
		"golang.go",
		"ms-azuretools.vscode-docker",
		"ms-vscode.cmake-tools",
		"ms-vscode.cpptools",
		"ms-vscode.cpptools-extension-pack",
		"ms-vscode.cpptools-themes",
		"ms-vscode.makefile-tools",
		"llvm-vs-code-extensions.vscode-clangd",
		"xaver.clang-format",
		"twxs.cmake",
	],
}

upg-vpp's People

Contributors

demo-exe avatar ivan4th avatar korroot avatar lazyhippogriff avatar miguelbf-alb avatar mitmitmitm avatar mogaika avatar roadrunnr avatar sergeymatov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

upg-vpp's Issues

"Out of memory" crash in PFCP Association Release E2E test

So far the problem was only observed in the CI environment. It is not clear at this point if it's a test issue or a real bug. Lowering the rate of session modifications appears to make this problem more rare.
This problem appears only to happen with release builds.

Test:

[Fail] Clearing message queue during PFCP Association Release upon timeout [It] should work correcty

Diagnostics:

Program received signal SIGABRT, Aborted.
0x00007ffff5cb618b in raise () from /lib/x86_64-linux-gnu/libc.so.6
#0  0x00007ffff5cb618b in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007ffff5c95859 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00007ffff6385d86 in os_panic () at /src/vpp/src/vppinfra/unix-misc.c:177
#3  0x00007ffff6389637 in clib_mem_alloc_aligned_at_offset (size=2056, align=<optimized out>, align_offset=<optimized out>, os_out_of_memory_on_failure=1) at /src/vpp/src/vppinfra/mem.h:243
#4  vec_resize_allocate_memory (v=<optimized out>, length_increment=2048, data_bytes=2056, header_bytes=8, data_align=<optimized out>, numa_id=<optimized out>) at /src/vpp/src/vppinfra/vec.c:67
#5  0x00007fffb3258a45 in _vec_resize_inline (v=0x0, length_increment=<optimized out>, data_bytes=<optimized out>, header_bytes=<optimized out>, data_align=<optimized out>, numa_id=<optimized out>) at /src/vpp/src/vppinfra/vec.h:170
#6  upf_pfcp_make_response (resp=0x7ffff3f04580, req=0x7fffefaf9ec0, len=2048) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:520
#7  upf_pfcp_send_response (req=0x7fffefaf9ec0, cp_seid=18299514082375153098, type=<optimized out>, grp=<optimized out>) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:532
#8  0x00007fffb32531e3 in session_msg (msg=<optimized out>) at /src/vpp/src/plugins/upf/upf_pfcp_api.c:2644
#9  upf_pfcp_handle_msg (msg=<optimized out>) at /src/vpp/src/plugins/upf/upf_pfcp_api.c:181

either precommit or hack/update-build-image-tag.sh are broken

$ git commit -m"test"
Build image tag not up to date (.gitlab-ci.yml), please run
hack/update-build-image-tag.sh
(without arguments)
$ hack/update-build-image-tag.sh
$ git commit -m"test"
Build image tag not up to date (.gitlab-ci.yml), please run
hack/update-build-image-tag.sh
(without arguments)

eithe hook is missdetecting that or the hack/update-build-image-tag.sh is not solving the problem as expected.

Session Deletion Responses dropped from retransmit queue immediately

When a session is deleted, all of the PFCP responses in the retransmit queue for its SEID are dropped.
This happens to include Session Deletion Response for the request that caused the session to be deleted in the first place.
Consequently, if a Session Deletion Response is lost, the retransmission from the SMF side will cause a "Session Context Not Found" error.

`pfcp_add_del_v4_teid is not used to add

I see use of pfcp_add_del_v4_teid (v4_teid, sx, 0); for delete as shown on the line below.

vec_foreach (v4_teid, active->v4_teid) pfcp_add_del_v4_teid (v4_teid, sx,

For better code maintenance, pfcp_add_del_v4_teid() should be used to add as well, but the code does not. Instead the code uses hash_set()

Failed to test app detect,What's wrong with me。

first use clito config appid,suck as
create upf application proxy name 111
upf application 111 rule 3000 add l7 regex ^https?://(.\.)(example)\.com/
upf application 111 rule 3001 add ipfilter permit out ip from any to assigned
then contrust pdr with application 111 ie .
then eatablish session ,
all is ok

Simulate a mobile phone to visit an app。
but show trace ,i find pdrindex is error.No proxy node,i think It must pass the node of proxy

Crash on duplicate UE IP

UPG crashes if there's duplicate UE IP between several sessions. Most likely, this only concerns duplicate UE IP Address between several sessions on PDIs for SGi an also possibly for Access in TDF mode, but duplicate UE IPs in PGW mode must also be checked.

`create` in handle_f_teid is wrong

somewhat related to #34

For Update PDR, handle_f_teid is called with create = false. Despite that setting, a new TEID would still be allocated, only the registration of the new TEID would be skipped. This is clearly wrong, when a new TEID is allocated is must also registered.

The other question is if a Update PDR should be permitted to allocate a new TEID. There is no clear guidance spelled out in the specification. All references to F-TEID allocation always put this into relation of PDR creation. The is no mentioning of PDR modification in this context. It seems therefore safe to assume that a Update PDR might only reference an already allocated F-TEID, but might not by itself allocated a new F-TEID.

The choose mechanism in handle_f_teid therefore needs to be adjusted to only allocate a new F-TEID when create is true.

Note: it would be legal in a modify request to have a Create PDR that creates a new F-TEID and a Update PDR that references that new F-TEID. There is no order requirement for PFCP IEs, the Update PDR could therefore be encoded in the PFCP request before the Update PDR. This is not a problem for our current implementation that uses a fixed ordering when processing IE and will always process the Create PDR before the Update PDR

Session establishment failed - No PFCP association

I am getting this issue of session establishment with cause "no established pfcp association".
In pcap it can be seein that pfcp node has associated with upg.

session_fail.zip

$ docker exec -it vpp-upf vppctl show upf association
Node: 192.168.61.196
  Recovery Time Stamp: 2021/03/02 14:19:02:000
  Sessions: 0

Debug builds crash when pinging SGi IP

The symptoms are described in #94
The buffer is reused for ICMP echo response by VPP, and thus re-enters UPG subgraph, causing assertion failure.
This does not affect release builds

Crash in response_expired()

The following crash has been observed when running e2e tests:

INFO[0827] VPP: /src/vpp/src/plugins/upf/upf_pfcp_server.h:240 (pfcp_msg_pool_elt_at_index) assertion `m->is_valid_pool_item' fails  pid=5954 what=stdout
INFO[0827] VPP:                                          pid=5954 what=stdout
INFO[0827] VPP: Program received signal SIGABRT, Aborted.  pid=5954 what=stdout
INFO[0827] VPP: 0x00007ffff4717fb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6  pid=5954 what=stdout
INFO[0827] VPP: #0  0x00007ffff4717fb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6  pid=5954 what=stdout
INFO[0827] VPP: #1  0x00007ffff4719921 in abort () from /lib/x86_64-linux-gnu/libc.so.6  pid=5954 what=stdout
INFO[0827] VPP: #2  0x0000000000407193 in os_panic () at /src/vpp/src/vpp/vnet/main.c:371  pid=5954 what=stdout
INFO[0827] VPP: #3  0x00007ffff55f8619 in debugger () at /src/vpp/src/vppinfra/error.c:84  pid=5954 what=stdout
INFO[0827] VPP: #4  0x00007ffff55f8397 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7fffac0ba8b7 "%s:%d (%s) assertion `%s' fails") at /src/vpp/src/vppinfra/error.c:143  pid=5954 what=stdout
INFO[0827] VPP: #5  0x00007fffac063138 in pfcp_msg_pool_elt_at_index (psm=0x7fffac2f7250 <pfcp_server_main>, index=11880) at /src/vpp/src/plugins/upf/upf_pfcp_server.h:240  pid=5954 what=stdout
INFO[0827] VPP: #6  0x00007fffac062052 in response_expired (id=11880) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:462  pid=5954 what=stdout
INFO[0827] VPP: #7  0x00007fffac05352a in pfcp_process (vm=0x7ffff6483b80 <vlib_global_main>, rt=0x7fffb4cb3580, f=0x0) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:1250  pid=5954 what=stdout
INFO[0827] VPP: #8  0x00007ffff61c9eed in vlib_process_bootstrap (_a=140736242026504) at /src/vpp/src/vlib/main.c:1454  pid=5954 what=stdout
INFO[0827] VPP: #9  0x00007ffff561c7a4 in clib_calljmp () at /src/vpp/src/vppinfra/longjmp.S:123  pid=5954 what=stdout

A crash in PFCP resend

There was a crash with the following stack trace:

Core was generated by `/usr/bin/vpp -c /etc/vpp/startup.conf'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  __GI_abort () at abort.c:107
107     abort.c: No such file or directory.
(gdb) bt
#0  __GI_abort () at abort.c:107
#1  0x000055d60dd13715 in os_exit (code=<optimized out>) at /src/vpp/src/vpp/vnet/main.c:395
#2  0x00007fd4ad59a135 in unix_signal_handler (signum=11, si=<optimized out>, uc=<optimized out>) at /src/vpp/src/vlib/unix/main.c:184
#3  <signal handler called>
#4  upf_pfcp_send_data (msg=0x7fd49ec0b980) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:82
#5  0x00007fd463532b5f in upf_pfcp_server_rx_msg (msg=0x7fd49fe71340) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:263
#6  pfcp_process (vm=0x7fd4ad7b46c0 <vlib_global_main>, rt=<optimized out>, f=<optimized out>) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:1142
#7  0x00007fd4ad53a1a7 in vlib_process_bootstrap (_a=<optimized out>) at /src/vpp/src/vlib/main.c:1454
#8  0x00007fd4ac980ec4 in clib_calljmp () from /usr/lib/x86_64-linux-gnu/libvppinfra.so.20.09
#9  0x00007fd46c2e7550 in ?? ()
#10 0x00007fd4ad52fcd2 in vlib_process_startup (vm=0x7fd4ad7b46c0 <vlib_global_main>, p=0x7fd46bdc8900, f=0x0) at /src/vpp/src/vlib/main.c:1479
#11 dispatch_process (vm=0x7fd4ad7b46c0 <vlib_global_main>, p=0x7fd46bdc8900, f=0x0, last_time_stamp=<optimized out>) at /src/vpp/src/vlib/main.c:1535

Apparently, there is a problem in PFCP resend code that is causing UPG to crash when it re-sends a response to a control plane request.

The line related to re-send:

upf_pfcp_send_data (resp);

Crash on invalid PFCP data

The following invalid SessionSetupRequest has caused a crash in the e2e tests (IPv6 mode TDF setup):

Frame 13: 503 bytes on wire (4024 bits), 503 bytes captured (4024 bits)
Ethernet II, Src: 4a:f8:a5:c3:85:ea (4a:f8:a5:c3:85:ea), Dst: fa:8a:78:4d:5b:5b (fa:8a:78:4d:5b:5b)
Internet Protocol Version 6, Src: 2001:db8:10::3, Dst: 2001:db8:10::2
User Datagram Protocol, Src Port: 8805, Dst Port: 8805
Packet Forwarding Control Protocol
    Flags: 0x21, SEID (S)
        001. .... = Version: 1
        ...0 .... = Spare: 0
        .... 0... = Spare: 0
        .... .0.. = Spare: 0
        .... ..0. = Message Priority (MP): False
        .... ...1 = SEID (S): True
    Message Type: PFCP Session Establishment Request (50)
    Length: 437
    SEID: 0x0000000000000000
    Sequence Number: 2
    Spare: 0
    Node ID : FQDN: pfcpstub
        IE Type: Node ID (60)
        IE Length: 10
        0000 .... = Spare: 0
        .... 0010 = Node ID Type: FQDN (2)
        Node ID FQDN: pfcpstub
    F-SEID : SEID: 0x3c04951aa42655d9, IPv4 0.0.0.0
        IE Type: F-SEID (57)
        IE Length: 13
        Flags: 0x02, V4 (IPv4)
            0... .... = Spare: 0
            .0.. .... = Spare: 0
            ..0. .... = Spare: 0
            ...0 .... = Spare: 0
            .... 0... = Spare: 0
            .... .0.. = Spare: 0
            .... ..1. = V4 (IPv4): Present
            .... ...0 = V6 (IPv6): Not Present
        SEID: 0x3c04951aa42655d9
        IPv4 address: 0.0.0.0
    Create PDR : [Grouped IE]
        IE Type: Create PDR (1)
        IE Length: 113
        Packet Detection Rule ID : 1
            IE Type: Packet Detection Rule ID (56)
            IE Length: 2
            Rule ID: 1
        FAR ID : Dynamic by CP 1
            IE Type: FAR ID (108)
            IE Length: 4
            0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
            .000 0000 0000 0000 0000 0000 0000 0001 = FAR ID: 1
        Precedence : 200
            IE Type: Precedence (29)
            IE Length: 4
            Precedence: 200
        PDI : [Grouped IE]
            IE Type: PDI (2)
            IE Length: 79
            Network Instance : access
                IE Type: Network Instance (22)
                IE Length: 7
                Network Instance: access
            SDF Filter : 
                IE Type: SDF Filter (23)
                IE Length: 38
                Flags: 0x01, FD (Flow Description)
                    0000 .... = Spare: 0
                    ...0 .... = BID (Bidirectional SDF Filter): False
                    .... 0... = FL (Flow Label): False
                    .... .0.. = SPI (Security Parameter Index): False
                    .... ..0. = TTC (ToS Traffic Class): False
                    .... ...1 = FD (Flow Description): True
                Spare: 0
                Length of Flow Description: 34
                Flow Description: permit out ip from any to assigned
            Source Interface : Access
                IE Type: Source Interface (20)
                IE Length: 1
                0000 .... = Spare: 0
                .... 0000 = Source Interface: Access (0)
            UE IP Address : 
                IE Type: UE IP Address (93)
                IE Length: 17
                Flags: 0x01, V6 (IPv6)
                    0000 .... = Spare: 0
                    .... 0... = IPv6D: Source IP address
                    .... .0.. = S/D: Source IP address
                    .... ..0. = V4 (IPv4): Not Present
                    .... ...1 = V6 (IPv6): Present
                IPv6 address: ::
        URR ID : Dynamic by CP 1
            IE Type: URR ID (81)
            IE Length: 4
            0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
            .000 0000 0000 0000 0000 0000 0000 0001 = URR ID: 1
    Create PDR : [Grouped IE]
        IE Type: Create PDR (1)
        IE Length: 110
        Packet Detection Rule ID : 2
            IE Type: Packet Detection Rule ID (56)
            IE Length: 2
            Rule ID: 2
        FAR ID : Dynamic by CP 2
            IE Type: FAR ID (108)
            IE Length: 4
            0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
            .000 0000 0000 0000 0000 0000 0000 0010 = FAR ID: 2
        PDI : [Grouped IE]
            IE Type: PDI (2)
            IE Length: 76
            SDF Filter : 
                IE Type: SDF Filter (23)
                IE Length: 38
                Flags: 0x01, FD (Flow Description)
                    0000 .... = Spare: 0
                    ...0 .... = BID (Bidirectional SDF Filter): False
                    .... 0... = FL (Flow Label): False
                    .... .0.. = SPI (Security Parameter Index): False
                    .... ..0. = TTC (ToS Traffic Class): False
                    .... ...1 = FD (Flow Description): True
                Spare: 0
                Length of Flow Description: 34
                Flow Description: permit out ip from any to assigned
            Network Instance : sgi
                IE Type: Network Instance (22)
                IE Length: 4
                Network Instance: sgi
            Source Interface : SGi-LAN/N6-LAN
                IE Type: Source Interface (20)
                IE Length: 1
                0000 .... = Spare: 0
                .... 0010 = Source Interface: SGi-LAN/N6-LAN (2)
            UE IP Address : 
                IE Type: UE IP Address (93)
                IE Length: 17
                Flags: 0x05, S/D, V6 (IPv6)
                    0000 .... = Spare: 0
                    .... 0... = IPv6D: Source IP address
                    .... .1.. = S/D: Destination IP address
                    .... ..0. = V4 (IPv4): Not Present
                    .... ...1 = V6 (IPv6): Present
                IPv6 address: ::
        Precedence : 200
            IE Type: Precedence (29)
            IE Length: 4
            Precedence: 200
        URR ID : Dynamic by CP 1
            IE Type: URR ID (81)
            IE Length: 4
            0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
            .000 0000 0000 0000 0000 0000 0000 0001 = URR ID: 1
    Create FAR : [Grouped IE]
        IE Type: Create FAR (3)
        IE Length: 76
        FAR ID : Dynamic by CP 1
            IE Type: FAR ID (108)
            IE Length: 4
            0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
            .000 0000 0000 0000 0000 0000 0000 0001 = FAR ID: 1
        Apply Action : 
            IE Type: Apply Action (44)
            IE Length: 1
            Flags: 0x02, FORW (Forward)
                000. .... = Spare: 0
                ...0 .... = DUPL (Duplicate): False
                .... 0... = NOCP (Notify the CP function): False
                .... .0.. = BUFF (Buffer): False
                .... ..1. = FORW (Forward): True
                .... ...0 = DROP (Drop): False
        Forwarding Parameters : [Grouped IE]
            IE Type: Forwarding Parameters (4)
            IE Length: 59
            Destination Interface : SGi-LAN/N6-LAN
                IE Type: Destination Interface (42)
                IE Length: 1
                0000 .... = Spare: 0
                .... 0010 = Interface: SGi-LAN/N6-LAN (2)
            Network Instance : sgi
                IE Type: Network Instance (22)
                IE Length: 4
                Network Instance: sgi
            Redirect Information : 
                IE Type: Redirect Information (38)
                IE Length: 42
                0000 .... = Spare: 0
                .... 0010 = Redirect Address Type: URL (2)
                Redirect Server Address Length: 37
                Redirect Server Address: http://127.0.0.1/this-is-my-redirect/
                IE data not decoded by WS yet
                    [Expert Info (Note/Undecoded): IE data not decoded by WS yet]
                        [IE data not decoded by WS yet]
                        [Severity level: Note]
                        [Group: Undecoded]
    Create FAR : [Grouped IE]
        IE Type: Create FAR (3)
        IE Length: 33
        FAR ID : Dynamic by CP 2
            IE Type: FAR ID (108)
            IE Length: 4
            0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
            .000 0000 0000 0000 0000 0000 0000 0010 = FAR ID: 2
        Apply Action : 
            IE Type: Apply Action (44)
            IE Length: 1
            Flags: 0x02, FORW (Forward)
                000. .... = Spare: 0
                ...0 .... = DUPL (Duplicate): False
                .... 0... = NOCP (Notify the CP function): False
                .... .0.. = BUFF (Buffer): False
                .... ..1. = FORW (Forward): True
                .... ...0 = DROP (Drop): False
        Forwarding Parameters : [Grouped IE]
            IE Type: Forwarding Parameters (4)
            IE Length: 16
            Destination Interface : Access
                IE Type: Destination Interface (42)
                IE Length: 1
                0000 .... = Spare: 0
                .... 0000 = Interface: Access (0)
            Network Instance : access
                IE Type: Network Instance (22)
                IE Length: 7
                Network Instance: access
    Create URR : [Grouped IE]
        IE Type: Create URR (6)
        IE Length: 19
        URR ID : Dynamic by CP 1
            IE Type: URR ID (81)
            IE Length: 4
            0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
            .000 0000 0000 0000 0000 0000 0000 0001 = URR ID: 1
        Measurement Method : 
            IE Type: Measurement Method (62)
            IE Length: 1
            Flags: 0x03, VOLUM (Volume), DURAT (Duration)
                0000 0... = Spare: 0
                .... .0.. = EVENT (Event): False
                .... ..1. = VOLUM (Volume): True
                .... ...1 = DURAT (Duration): True
        Reporting Triggers : 
            IE Type: Reporting Triggers (37)
            IE Length: 2
            0... .... = LIUSA (Linked Usage Reporting): False
            .0.. .... = DROTH (Dropped DL Traffic Threshold): False
            ..0. .... = STOPT (Stop of Traffic): False
            ...0 .... = START (Start of Traffic): False
            .... 0... = QUHTI (Quota Holding Time): False
            .... .0.. = TIMTH (Time Threshold): False
            .... ..0. = VOLTH (Volume Threshold): False
            .... ...0 = PERIO (Periodic Reporting): False
            000. .... = Spare: 0
            ..0. .... = EVEQU (Event Quota): False
            ...0 .... = EVETH (Event Threshold): False
            .... 0... = MACAR (MAC Addresses Reporting): False
            .... .0.. = ENVCL (Envelope Closure): False
            .... ..0. = TIMQU (Time Quota): False
            .... ...0 = VOLQU (Volume Quota): False
    Create URR : [Grouped IE]
        IE Type: Create URR (6)
        IE Length: 19
        URR ID : Dynamic by CP 2
            IE Type: URR ID (81)
            IE Length: 4
            0... .... .... .... .... .... .... .... = Allocation type: Dynamic by CP
            .000 0000 0000 0000 0000 0000 0000 0010 = URR ID: 2
        Measurement Method : 
            IE Type: Measurement Method (62)
            IE Length: 1
            Flags: 0x03, VOLUM (Volume), DURAT (Duration)
                0000 0... = Spare: 0
                .... .0.. = EVENT (Event): False
                .... ..1. = VOLUM (Volume): True
                .... ...1 = DURAT (Duration): True
        Reporting Triggers : 
            IE Type: Reporting Triggers (37)
            IE Length: 2
            0... .... = LIUSA (Linked Usage Reporting): False
            .0.. .... = DROTH (Dropped DL Traffic Threshold): False
            ..0. .... = STOPT (Stop of Traffic): False
            ...0 .... = START (Start of Traffic): False
            .... 0... = QUHTI (Quota Holding Time): False
            .... .0.. = TIMTH (Time Threshold): False
            .... ..0. = VOLTH (Volume Threshold): False
            .... ...0 = PERIO (Periodic Reporting): False
            000. .... = Spare: 0
            ..0. .... = EVEQU (Event Quota): False
            ...0 .... = EVETH (Event Threshold): False
            .... 0... = MACAR (MAC Addresses Reporting): False
            .... .0.. = ENVCL (Envelope Closure): False
            .... ..0. = TIMQU (Time Quota): False
            .... ...0 = VOLQU (Volume Quota): False

Stack trace:

/src/vpp/src/vnet/fib/fib_table.c:35 (fib_table_get) assertion `! pool_is_free (ip4_main.fibs, _e)' fails

Program received signal SIGABRT, Aborted.
0x00007ffff4719f47 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#0  0x00007ffff4719f47 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007ffff471b8b1 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x0000000000407193 in os_panic () at /src/vpp/src/vpp/vnet/main.c:371
#3  0x00007ffff55fa619 in debugger () at /src/vpp/src/vppinfra/error.c:84
#4  0x00007ffff55fa397 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7ffff76dc130 "%s:%d (%s) assertion `%s' fails") at /src/vpp/src/vppinfra/error.c:143
#5  0x00007ffff74f7cef in fib_table_get (index=4, proto=FIB_PROTOCOL_IP4) at /src/vpp/src/vnet/fib/fib_table.c:35
#6  0x00007ffff74f88ea in fib_table_entry_special_dpo_add (fib_index=4, prefix=0x7ffff7fc7fe0, source=FIB_SOURCE_FIRST, flags=(FIB_ENTRY_FLAG_EXCLUSIVE | FIB_ENTRY_FLAG_LOOSE_URPF_EXEMPT), dpo=0x7ffff7fc7fd8) at /src/vpp/src/vnet/fib/fib_table.c:333
#7  0x00007fffabdd88cf in pfcp_add_del_ue_ip (ip=0x7fffe6ebeb00, si=0x7fffe6ebdd00, is_add=1) at /src/vpp/src/plugins/upf/upf_pfcp.c:1197
#8  0x00007fffabddc766 in pfcp_update_apply (sx=0x7fffe6ebdd00) at /src/vpp/src/plugins/upf/upf_pfcp.c:1836
#9  0x00007fffabdf4f31 in handle_session_establishment_request (req=0x7fffe6ea63c0, msg=0x7ffff7fc8a20) at /src/vpp/src/plugins/upf/upf_pfcp_api.c:2444

Node id

With latest update in the code for node_id (association request), I see that node id is 0.0.0.0 instead of pfcp endpoint ip.
Is it problem in the configuration or something else ?

Message Type: PFCP Association Setup Response (6)
:
:
Node ID : IPv4 address: 0.0.0.0
    IE Type: Node ID (60)
    IE Length: 5
    0000 .... = Spare: 0
    .... 0000 = Node ID Type: IPv4 address (0)
    Node ID IPv4: 0.0.0.0

Limitations of IP-based application rules

In a configuration where all of the traffic that UPG passes in PGW-U mode should match one of the IP rules it was observed that the buffers are dropped after upf-ip[46]-flow-process.

Bad expired flow count

flows that have expired counts rescheduled flows as well as flows that have really expired

UE can't ping anything and crashes VPP

Hi,

I am currently trying to use your UPG plugin in a LTE architecture with erGW.
The UE is connecting to the network and an IP address is attributed to it but it can't reach anything nor be reached by any interface. Moreover, VPP crashes whenever I try to ping the SGi interface of VPP (10.0.0.2 on the diagram below).

My network configuration is the following
image

VPP and erGW are placed in a Ubuntu 20.04 docker container on a computer.
The MME and HSS are both from the Open5GS project and are hosted on the same computer.
I use srsLTE for the eNodeB.

I can see from the command sh upf session that my UE is recognised by VPP as the IP address appears there.

CP F-SEID: 0xffde7210bfdc4048 (18437299340765315144) @ 192.168.1.2
UP F-SEID: 0xffde7210bfdc4048 (18437299340765315144) @ 192.168.1.1
  PFCP Association: 0
PDR: 1 @ 0x7fd9f4ef3740
  Precedence: 100
  PDI:
    Fields: 00000001
    Source Interface: CP-function
    Network Instance: cp
    Local F-TEID: 2271652686 (0x8766ab4e)
            IPv4: 192.168.1.1
  Outer Header Removal: GTP-U/UDP/IPv4
  FAR Id: 1
  URR Ids: [] @ 0x0
  QER Ids: [] @ 0x0
FAR: 1
  Apply Action: 00000002 == [FORWARD]
  Forward:
    Network Instance: epc
    Destination Interface: 0
CP F-SEID: 0xffde7210c5b30007 (18437299340863275015) @ 192.168.1.2
UP F-SEID: 0xffde7210c5b30007 (18437299340863275015) @ 192.168.1.1
  PFCP Association: 0
  UP Inactivity Timer: 28800 secs, inactive       8.6304 secs (0x00000c05)
PDR: 1 @ 0x7fd9f49a1700
  Precedence: 100
  PDI:
    Fields: 0000000d
    Source Interface: Access
    Network Instance: epc
    Local F-TEID: 3559416503 (0xd42862b7)
            IPv4: 172.20.17.2
    UE IP address (source):
      IPv4 address: 10.180.0.136
    SDF Filter [1]:
      permit out ip from any to assigned 
  Outer Header Removal: GTP-U/UDP/IPv4
  FAR Id: 1
  URR Ids: [1] @ 0x7fd9f4ef6960
  QER Ids: [] @ 0x0
PDR: 2 @ 0x7fd9f49a1780
  Precedence: 100
  PDI:
    Fields: 0000000c
    Source Interface: SGi-LAN
    Network Instance: sgi
    UE IP address (destination):
      IPv4 address: 10.180.0.136
    SDF Filter [1]:
      permit out ip from any to assigned 
  Outer Header Removal: no
  FAR Id: 2
  URR Ids: [1] @ 0x7fd9f4ef6a00
  QER Ids: [] @ 0x0
FAR: 1
  Apply Action: 00000002 == [FORWARD]
  Forward:
    Network Instance: sgi
    Destination Interface: 2
FAR: 2
  Apply Action: 00000002 == [FORWARD]
  Forward:
    Network Instance: epc
    Destination Interface: 0
    Outer Header Creation: [GTP-U/UDP/IPv4],TEID:00000001,IP:172.20.17.1
URR: 1
  Measurement Method: 0003 == [TIME,VOLUME]
  Reporting Triggers: 0000 == []
  Status: 0 == []
  Start Time: 2021/03/03 16:43:21:707
  vTime of First Usage:       0.0000 
  vTime of Last Usage:        0.0000 
  Volume
    Up:    Measured:                    0, Theshold:                    0, Pkts:          0
           Consumed:                    0, Quota:                       0
    Down:  Measured:                    0, Theshold:                    0, Pkts:          0
           Consumed:                    0, Quota:                       0
    Total: Measured:                    0, Theshold:                    0, Pkts:          0
           Consumed:                    0, Quota:                       0
  Time
    Quota:                        0 secs @ 2021/03/03 16:43:30:326, in -16147862 secs, handle 0xffffffff
    Threshold:                    0 secs @ 2021/03/03 16:43:30:326, in -16147862 secs, handle 0xffffffff

Here is my startup.conf

unix {
  nodaemon
  log /tmp/vpp.log
  full-coredump
  gid vpp
  interactive
  cli-listen localhost:5002
  exec init.conf
}

api-trace {
  on
}

api-segment {
  gid vpp
}

plugins {
    path /usr/src/upg-vpp/vpp/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins/
    plugin dpdk_plugin.so { disable }
    plugin gtpu_plugin.so { disable }
    plugin upf_plugin.so { enable }
}

And my init.conf

ip table add 1
ip table add 2
ip6 table add 1
ip6 table add 2

create host-interface name grx
set interface mac address host-grx 02:42:ac:14:11:02
set interface mtu 1500 host-grx
set interface ip table host-grx 1
set interface ip address host-grx 172.20.17.2/24
set interface state host-grx up

create host-interface name sgi
set interface mac address host-sgi 02:42:0a:00:00:02
set interface mtu 1500 host-sgi
set interface ip table host-sgi 2
set interface ip address host-sgi 10.0.0.2/24
set interface state host-sgi up

create tap host-ip4-addr 192.168.1.2/24
set int ip address tap0 192.168.1.1/24
set int state tap0 up

ip route add 0.0.0.0/0 table 2 via 10.0.0.1 host-sgi

upf pfcp endpoint ip 192.168.1.1 vrf 0

upf nwi name cp vrf 0
upf nwi name epc vrf 1
upf nwi name sgi vrf 2

upf gtpu endpoint ip 192.168.1.1 nwi cp teid 0x80000000/2
upf gtpu endpoint ip 172.20.17.2 nwi epc teid 0x80000000/2

Thank you in advance

Rule creation/modification failure

I have a Rule creation/modification failure. I use free5gc SMF.
A reflection track would be welcome
test2.zip

my init file:

create host-interface name vpp-pfcp-peer

set interface state host-vpp-pfcp-peer up

set interface ip address host-vpp-pfcp-peer 172.26.0.50/24

create host-interface name vpp-gtp-peer

set interface state host-vpp-gtp-peer up

set interface ip address host-vpp-gtp-peer 172.16.0.1/24

create host-interface name vpp-sgi-peer

set interface state host-vpp-sgi-peer up

set interface ip address host-vpp-sgi-peer 192.168.0.2/24

upf pfcp endpoint ip 172.26.0.50 vrf 0

upf nwi name cp vrf 0

upf nwi name access vrf 100

upf nwi name sgi vrf 200

upf gtpu endpoint ip 172.16.0.1 nwi access teid 0x00001234/2

upg-vpp doesn't encap gtpu header for downlink traffic

Hi,
I set up upg-vpp as UPF with free5GC SMF, and used UERANSIM to test UE traffic by sending ping requests.
The uplink tunnel worked well, but the downlink did not.
Below is my setting and if there was anything wrong?

My startup.conf file

unix {
  interactive
  cli-listen /run/vpp/cli-vpp-upf.sock
  exec /home/upf/init.conf
}

cpu {
  main-core 1
  corelist-workers 2-3
}

dpdk {
  socket-mem 2048,2048,2048

  dev 0000:0b:00.0 {
    num-rx-queues 2
    num-rx-desc 1024
    num-tx-queues 2
    num-tx-desc 1024
  }
  
  dev 0000:13:00.0 {
    num-rx-queues 2
    num-rx-desc 1024
    num-tx-queues 2
    num-tx-desc 1024
  }
  
  dev 0000:1b:00.0 {
    num-rx-queues 2
    num-rx-desc 1024
    num-tx-queues 2
    num-tx-desc 1024    
  }
}

My init.conf file

set int state GigabitEthernet13/0/0 up
set int state GigabitEthernetb/0/0 up
set int state GigabitEthernet1b/0/0 up

set int ip table GigabitEthernet13/0/0 0
set int ip table GigabitEthernetb/0/0 0
set int ip table GigabitEthernet1b/0/0 0

set int ip address GigabitEthernet13/0/0 192.168.1.10/24
set int ip address GigabitEthernetb/0/0 192.168.0.30/24
set int ip address GigabitEthernet1b/0/0 192.168.2.1/24

ip route add 0.0.0.0/0 table 0 via 192.168.2.1 GigabitEthernet1b/0/0

upf pfcp endpoint ip 192.168.1.10 vrf 0

upf nwi name access vrf 0
upf nwi name core vrf 0

upf gtpu endpoint ip 192.168.0.30 nwi access teid 0x00000001/2

upf node-id ip4 192.168.1.10

After tracing with vpp debug tools, I found that upg-vpp did not have encap node after interface-output node and thus DL packets were dropped. (vpp gtup plugin has this node)

------------------- Start of thread 2 vpp_wk_1 -------------------
Packet 1

00:07:21:817644: dpdk-input
  GigabitEthernetb/0/0 rx queue 1
  buffer 0x6a2e6: current data 0, length 142, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x2000001
                  ext-hdr-valid 
                  l4-cksum-computed l4-cksum-correct 
  PKT MBUF: port 0, nb_segs 1, pkt_len 142
    buf_len 2176, data_len 142, ol_flags 0x182, data_off 128, phys_addr 0x9108ba00
    packet_type 0x291 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x9e668c47 fdir.hi 0x0 fdir.lo 0x9e668c47
    Packet Offload Flags
      PKT_RX_RSS_HASH (0x0002) RX packet with RSS hash result
      PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
      PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
    Packet Types
      RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
      RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers
      RTE_PTYPE_L4_UDP (0x0200) UDP packet
  IP4: 00:0c:29:5e:21:45 -> 00:0c:29:5d:4b:f1
  UDP: 192.168.0.21 -> 192.168.0.30
    tos 0x00, ttl 64, length 128, checksum 0x941d dscp CS0 ecn NON_ECN
    fragment id 0x24cc, flags DONT_FRAGMENT
  UDP: 2152 -> 2152
    length 108, checksum 0x2cd1
00:07:21:817657: ethernet-input
  frame: flags 0x1, hw-if-index 1, sw-if-index 1
  IP4: 00:0c:29:5e:21:45 -> 00:0c:29:5d:4b:f1
00:07:21:817664: ip4-input
  UDP: 192.168.0.21 -> 192.168.0.30
    tos 0x00, ttl 64, length 128, checksum 0x941d dscp CS0 ecn NON_ECN
    fragment id 0x24cc, flags DONT_FRAGMENT
  UDP: 2152 -> 2152
    length 108, checksum 0x2cd1
00:07:21:817666: ip4-lookup
  fib 0 dpo-idx 8 flow hash: 0x00000000
  UDP: 192.168.0.21 -> 192.168.0.30
    tos 0x00, ttl 64, length 128, checksum 0x941d dscp CS0 ecn NON_ECN
    fragment id 0x24cc, flags DONT_FRAGMENT
  UDP: 2152 -> 2152
    length 108, checksum 0x2cd1
00:07:21:817669: ip4-local
    UDP: 192.168.0.21 -> 192.168.0.30
      tos 0x00, ttl 64, length 128, checksum 0x941d dscp CS0 ecn NON_ECN
      fragment id 0x24cc, flags DONT_FRAGMENT
    UDP: 2152 -> 2152
      length 108, checksum 0x2cd1
00:07:21:817670: ip4-udp-lookup
  UDP: src-port 2152 dst-port 2152
00:07:21:817672: upf-gtpu4-input
  GTPU decap from gtpu_session0 teid 0x00000001 next 1 error 0
00:07:21:817676: upf-ip4-flow-process
  upf_session0 cp-seid 0x0000000000000001
  FlowInfo - sw_if_index 1, next_index = 2
  proto 0x1, 60.60.0.1:0 <-> 192.168.2.10:0, seid 0x0000000000000001
  00000000: 01000000000000000000000000000000000000003c3c00010000000000000000
  00000020: 00000000c0a8020a0000000001000000
  ICMP: 60.60.0.1 -> 192.168.2.10
    tos 0x00, ttl 64, length 84, checksum 0x9829 dscp CS0 ecn NON_ECN
    fragment id 0xa390, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817680: upf-ip4-input
  upf_session0 cp-seid 0x0000000000000001 pdr 1 far 1
  ICMP: 60.60.0.1 -> 192.168.2.10
    tos 0x00, ttl 64, length 84, checksum 0x9829 dscp CS0 ecn NON_ECN
    fragment id 0xa390, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817682: upf-ip4-forward
  upf_session0 cp-seid 0x0000000000000001 pdr 1 far 1
  ICMP: 60.60.0.1 -> 192.168.2.10
    tos 0x00, ttl 64, length 84, checksum 0x9829 dscp CS0 ecn NON_ECN
    fragment id 0xa390, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817684: ip4-input
  ICMP: 60.60.0.1 -> 192.168.2.10
    tos 0x00, ttl 64, length 84, checksum 0x9829 dscp CS0 ecn NON_ECN
    fragment id 0xa390, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817684: ip4-lookup
  fib 0 dpo-idx 9 flow hash: 0x00000000
  ICMP: 60.60.0.1 -> 192.168.2.10
    tos 0x00, ttl 64, length 84, checksum 0x9829 dscp CS0 ecn NON_ECN
    fragment id 0xa390, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817685: ip4-rewrite
  tx_sw_if_index 3 dpo-idx 9 : ipv4 via 192.168.2.10 GigabitEthernet1b/0/0: mtu:9000 next:5 000c29232347000c295d4b050800 flow hash: 0x00000000
  00000000: 000c29232347000c295d4b05080045000054a39040003f0199293c3c0001c0a8
  00000020: 020a08003d3340bd0002eb80c1600000000006590800000000001011
00:07:21:817688: GigabitEthernet1b/0/0-output
  GigabitEthernet1b/0/0 
  IP4: 00:0c:29:5d:4b:05 -> 00:0c:29:23:23:47
  ICMP: 60.60.0.1 -> 192.168.2.10
    tos 0x00, ttl 63, length 84, checksum 0x9929 dscp CS0 ecn NON_ECN
    fragment id 0xa390, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x3d33 id 16573
00:07:21:817690: GigabitEthernet1b/0/0-tx
  GigabitEthernet1b/0/0 tx queue 2
  buffer 0x6a2e6: current data 44, length 98, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x2000001
                  ext-hdr-valid 
                  l4-cksum-computed l4-cksum-correct l2-hdr-offset 0 l3-hdr-offset 58 
  PKT MBUF: port 0, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x182, data_off 172, phys_addr 0x9108ba00
    packet_type 0x291 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0x9e668c47 fdir.hi 0x0 fdir.lo 0x9e668c47
    Packet Offload Flags
      PKT_RX_RSS_HASH (0x0002) RX packet with RSS hash result
      PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
      PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
    Packet Types
      RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
      RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers
      RTE_PTYPE_L4_UDP (0x0200) UDP packet
  IP4: 00:0c:29:5d:4b:05 -> 00:0c:29:23:23:47
  ICMP: 60.60.0.1 -> 192.168.2.10
    tos 0x00, ttl 63, length 84, checksum 0x9929 dscp CS0 ecn NON_ECN
    fragment id 0xa390, flags DONT_FRAGMENT
  ICMP echo_request checksum 0x3d33 id 16573

Packet 2

00:07:21:818087: dpdk-input
  GigabitEthernet1b/0/0 rx queue 1
  buffer 0x3dd18: current data 0, length 98, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x2000002
                  ext-hdr-valid 
                  l4-cksum-computed l4-cksum-correct 
  PKT MBUF: port 2, nb_segs 1, pkt_len 98
    buf_len 2176, data_len 98, ol_flags 0x82, data_off 128, phys_addr 0xa4974680
    packet_type 0x91 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
    rss 0xb8ab52c9 fdir.hi 0x0 fdir.lo 0xb8ab52c9
    Packet Offload Flags
      PKT_RX_RSS_HASH (0x0002) RX packet with RSS hash result
      PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
    Packet Types
      RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
      RTE_PTYPE_L3_IPV4_EXT_UNKNOWN (0x0090) IPv4 packet with or without extension headers
  IP4: 00:0c:29:23:23:47 -> 00:0c:29:5d:4b:05
  ICMP: 192.168.2.10 -> 60.60.0.1
    tos 0x00, ttl 64, length 84, checksum 0x1f9c dscp CS0 ecn NON_ECN
    fragment id 0x5c1e
  ICMP echo_reply checksum 0x4533 id 16573
00:07:21:818097: ethernet-input
  frame: flags 0x1, hw-if-index 3, sw-if-index 3
  IP4: 00:0c:29:23:23:47 -> 00:0c:29:5d:4b:05
00:07:21:818099: ip4-input
  ICMP: 192.168.2.10 -> 60.60.0.1
    tos 0x00, ttl 64, length 84, checksum 0x1f9c dscp CS0 ecn NON_ECN
    fragment id 0x5c1e
  ICMP echo_reply checksum 0x4533 id 16573
00:07:21:818099: ip4-lookup
  fib 0 dpo-idx 6 flow hash: 0x00000000
  ICMP: 192.168.2.10 -> 60.60.0.1
    tos 0x00, ttl 64, length 84, checksum 0x1f9c dscp CS0 ecn NON_ECN
    fragment id 0x5c1e
  ICMP echo_reply checksum 0x4533 id 16573
00:07:21:818106: ip4-arp
    ICMP: 192.168.2.10 -> 60.60.0.1
      tos 0x00, ttl 64, length 84, checksum 0x1f9c dscp CS0 ecn NON_ECN
      fragment id 0x5c1e
    ICMP echo_reply checksum 0x4533 id 16573
00:07:21:818111: ip4-drop
    ICMP: 192.168.2.10 -> 60.60.0.1
      tos 0x00, ttl 64, length 84, checksum 0x1f9c dscp CS0 ecn NON_ECN
      fragment id 0x5c1e
    ICMP echo_reply checksum 0x4533 id 16573
00:07:21:818131: error-drop
  rx:GigabitEthernet1b/0/0
00:07:21:818133: drop
  ip4-arp: ARP requests sent

PFCP create PDR message

0: handle_session_modification_request:2502: CP F-SEID: 0x0000000000000001 (1) @ 192.168.1.3
UP F-SEID: 0x0000000000000001 (1) @ 192.168.1.10
  SIdx: 0
  Pointer: 0x7fffb8ad10c0
  PDR: 0x7fffb5b46c70
  FAR: 0x7fffbc9f0b60
  PFCP Association: 0
                  (prev:4294967295,next:4294967295)
PDR: 1 @ 0x7fffb5b46c70
  Precedence: 255
  PDI:
    Fields: 0000000d
    Source Interface: Access
    Network Instance: access
    Local F-TEID: 1 (0x00000001)
            IPv4: 192.168.0.30
    UE IP address (source):
      IPv4 address: 60.60.0.1
    SDF Filter [1]:
      permit out ip from any to assigned 
  Outer Header Removal: GTP-U/UDP/IPv4
  FAR Id: 1
  URR Ids: [] @ 0x0
  QER Ids: [1] @ 0x7fffbcc389b0
PDR: 2 @ 0x7fffb5b46cf0
  Precedence: 255
  PDI:
    Fields: 0000000c
    Source Interface: Core
    Network Instance: access
    UE IP address (source):
      IPv4 address: 60.60.0.1
    SDF Filter [1]:
      permit out ip from any to assigned 
  Outer Header Removal: no
  FAR Id: 2
  URR Ids: [] @ 0x0
  QER Ids: [1] @ 0x7fffbcc38a50
FAR: 1 @ 0x7fffbc9f0b60
  Apply Action: 00000002 == [FORWARD]
  Forward:
    Network Instance: access
    Destination Interface: 2
FAR: 2 @ 0x7fffbc9f0bd0
  Apply Action: 00000002 == [FORWARD]
  Forward:
    Network Instance: access
    Destination Interface: 0
    Outer Header Creation: [GTP-U/UDP/IPv4],TEID:00000001,IP:192.168.0.21
    Rewrite Header: UDP: 192.168.0.30 -> 192.168.0.21
                      tos 0x00, ttl 254, length 0, checksum 0x3b69 dscp CS0 ecn NON_ECN
                      fragment id 0x0000
                    UDP: 2152 -> 2152
                      length 0, checksum 0x0000
0: encode_ie:7693: PFCP: Cause: 1.
0: enqueue_response:464: Msg Seq No: 3, idx 2

ping pacp file
ping.zip

add verbose cause for failed PFCP request

The PFCP inherent error reporting is not very verbose and makes it hard to troubleshoot failed PFCP requests.

Add a TP vendor IE to give a verbose error description. Either a string or a structured IE that indicates file and line where the error occurred. The inclusion of that IE could be restricted to debug builds.

Clean up proxy sessions together with flows

Right now, there's no proper "sync" between proxy sessions and the flows.
A flow can be deleted upon a timeout while there's still a proxy session entry for that flow. The unused proxy sessions may be kept indefinitely. Also, a flow can "respawn" due to packets sent by the proxy after the flow has expired.
There is a check for flow index pointing to a non-free flowtable entry e.g. in proxy_start_connect_fn(), but that is not correct either as that function can be invoked after quite a bit of time after the entry has been removed, in which case there'll be another flow in its place, leading to undefined behavior.

Trying to delete a proxy session right away poses some problems, namely, it's hard to avoid "late" callbacks from the VPP session layer, e.g. it's very hard to get rid of pending half-open connections properly (e.g. active connections in TCP_STATE_SYN_SENT state).

Suggested solution: add 'flow expiration' hook. When a flow is about to expire, the hook is invoked, and if it returns 0, the flow is removed. Otherwise, the flow expiration is rescheduled.

The flows should contain proxy session index instead of connection index that it has now. If there's no proxy session associated with the flow, the proxy session index contains ~0.

The proxy installs the expiration hook that checks if the flow has proxy session index other than ~0. If so, it "nudges" the proxy session via proxy_session_try_close_unlocked() (with proper locking around it), and returns -1 to tell the flowtable that the flow can't be removed just yet. proxy_session_free() sets proxy session index to ~0 in the corresponding flow, after which 'flow expiration hook' will succeed.

Alternatives: 'refcount' for flows. Perhaps not really necessary till only proxy sessions keep the flows from being freed. Also, this doesn't cover 'nudging' the proxy to clean up its session.
Also: not using the hook, just calling a proxy function from flowtable. But that's too much coupling IMO.

implement VTIME support

Implement support for quota validity time feature in UPG.

This is required to support Gy quota validity time in a state less C node.

Why pfcp_add_del_v4_teid is not used for add?

I see use of pfcp_add_del_v4_teid (v4_teid, sx, 0); for delete as shown on the line below.

vec_foreach (v4_teid, active->v4_teid) pfcp_add_del_v4_teid (v4_teid, sx,

For better code maintenance, pfcp_add_del_v4_teid() should be used to add as well, but the code does not. Instead the code uses hash_set()

git precommit hook is broken

$ git commit -m"test"
.git/hooks/pre-commit: 10: set: Illegal option -o pipefail

It seems the hock assumes that /bin/sh is bash, which is not always the case

5G PFCP Error

Hi,

Please I am trying to setup upg-vpp as UPF for open5gs SMF.

The error I get from the open5gs SMF is:

03/20 23:01:14.209: [smf] WARNING: cannot handle PFCP message type[1] (../src/smf/pfcp-sm.c:140)
03/20 23:01:14.729: [pfcp] DEBUG: [3] REMOTE Create  peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:156)
03/20 23:01:14.729: [pfcp] DEBUG: [3] REMOTE Receive peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:699)
03/20 23:01:14.729: [pfcp] DEBUG: [3] REMOTE UPD RX-1  peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:294)
03/20 23:01:14.729: [smf] WARNING: cannot handle PFCP message type[1] (../src/smf/pfcp-sm.c:140)
03/20 23:01:15.207: [smf] WARNING: Retry to association with peer [10.0.0.149]:8805 failed (../src/smf/pfcp-sm.c:107)
03/20 23:01:15.207: [pfcp] DEBUG: Association Setup Request (../lib/pfcp/build.c:64)
03/20 23:01:15.207: [pfcp] DEBUG: [83] LOCAL  UPD TX-5  peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:193)
03/20 23:01:15.207: [pfcp] DEBUG: [83] LOCAL  Create  peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:113)
03/20 23:01:15.207: [pfcp] DEBUG: [83] LOCAL  Commit  peer [10.0.0.149]:8805 (../lib/pfcp/xact.c:473)
03/20 23:01:15.207: [core] ERROR: Unexpected TLV type:32770 (../lib/core/ogs-tlv-msg.c:454)
03/20 23:01:15.207: [smf] ERROR: ogs_pfcp_parse_msg() failed (../src/smf/smf-sm.c:228)

After a while upg-vpp crashes..

DBGvpp# /usr/src/upg-vpp/vpp/src/vlib/counter.h:101 (vlib_decrement_simple_counter) assertion `my_counters[index] >= decrement' fails
Aborted (core dumped)

Following are my config:

ip table add 1
ip table add 2
ip6 table add 1
ip6 table add 2

create host-interface name ens6
set interface mac address host-ens6 02:7e:e1:cc:04:a7
set interface mtu 1500 host-ens6
set interface ip table host-ens6 0
set interface ip address host-ens6 10.0.0.149/24
set interface state host-ens6 up

create host-interface name ens7
set interface mac address host-ens7 02:68:23:7e:1d:bf
set interface mtu 1500 host-ens7
set interface ip table host-ens7 1
set interface ip address host-ens7 10.0.6.133/24
set interface state host-ens7 up

create host-interface name ens8
set interface mac address host-ens8 02:bc:29:ee:b2:63
set interface mtu 1500 host-ens8
set interface ip table host-ens8 2
set interface ip address host-ens8 10.0.4.246/24
set interface state host-ens8 up

ip route add 0.0.0.0/0 table 0 via 10.0.0.1 host-ens6
ip route add 0.0.0.0/0 table 1 via 10.0.6.1 host-ens7
ip route add 0.0.0.0/0 table 2 via 10.0.4.1 host-ens8

upf pfcp endpoint ip 10.0.0.149 vrf 0

upf nwi name cp vrf 0
upf nwi name epc vrf 1
upf nwi name sgi vrf 2

upf gtpu endpoint ip 10.0.0.149 nwi cp teid 0x80000000/2
upf gtpu endpoint ip 10.0.6.133 nwi epc teid 0x80000000/2
unix {
  nodaemon
  log /tmp/vpp.log
  full-coredump
  gid vpp
  interactive
  cli-listen localhost:5002
  exec init.conf
}

api-trace {
  on
}

api-segment {
  gid vpp
}

plugins {
    path /usr/src/upg-vpp/vpp/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins/
    plugin dpdk_plugin.so { disable }
    plugin gtpu_plugin.so { disable }
    plugin upf_plugin.so { enable }
}

Steps that I used in installing upg-vpp:

cd /usr/src
git clone https://github.com/travelping/upg-vpp
cd upg-vpp
make update-vpp
cd vpp
make install-deps
make install-ext-deps
make build

Also it will be great if someone can point me to where the documentation is, I was unable to locate this in this repo.

Another crash due to PFCP response timers issue

Debug build (v1.1.1):

pfcp_msg_pool_elt_at_index:261: ERROR: accessing a PFCP msg that was freed at: /src/vpp/src/plugins/upf/upf_pfcp_server.c:473 response_expired                    
/src/vpp/src/plugins/upf/upf_pfcp_server.h:262 (pfcp_msg_pool_elt_at_index) assertion `0' fails
...
[New LWP 7]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/vpp -c /etc/vpp/startup.conf'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f9c0c7dbfb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#0  0x00007f9c0c7dbfb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00007f9c0c7dd921 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00000000004071f5 in os_exit (code=1) at /src/vpp/src/vpp/vnet/main.c:395
#3  0x00007f9c0e314707 in unix_signal_handler (signum=6, si=0x7f9c10090bb0, uc=0x7f9c10090a80) at /src/vpp/src/vlib/unix/main.c:187
#4  <signal handler called>
#5  0x00007f9c0c7dbfb7 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#6  0x00007f9c0c7dd921 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#7  0x0000000000407193 in os_panic () at /src/vpp/src/vpp/vnet/main.c:371
#8  0x00007f9c0d6bc619 in debugger () at /src/vpp/src/vppinfra/error.c:84
#9  0x00007f9c0d6bc397 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7f9bc417e027 "%s:%d (%s) assertion `%s' fails") at /src/vpp/src/vppinfra/error.c:143
#10 0x00007f9bc4126614 in pfcp_msg_pool_elt_at_index (psm=0x7f9bc43bb460 <pfcp_server_main>, index=6414) at /src/vpp/src/plugins/upf/upf_pfcp_server.h:262
#11 0x00007f9bc41254b2 in response_expired (id=6414) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:467
#12 0x00007f9bc41166ac in pfcp_process (vm=0x7f9c0e547b80 <vlib_global_main>, rt=0x7f9bcca6a0c0, f=0x0) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:1252
#13 0x00007f9c0e28deed in vlib_process_bootstrap (_a=140307136063496) at /src/vpp/src/vlib/main.c:1454
#14 0x00007f9c0d6e07a4 in clib_calljmp () at /src/vpp/src/vppinfra/longjmp.S:123
#15 0x00007f9bcd012800 in ?? ()
#16 0x00007f9c0e28d92f in vlib_process_startup (vm=0x1afee8f1e8, p=0x874ad741c34756, f=0x95) at /src/vpp/src/vlib/main.c:1479
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

Release build (v1.1.4; due to lack of ASSERTs in the release build, the issue is not caught early enough, leads to crash in another place later):

warning: core file may not match specified executable file.
[New LWP 8]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/bin/vpp -c /etc/vpp/startup.conf'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007f1ea4a7d921 in __GI_abort () at abort.c:79
#2  0x0000561a8cc38715 in os_exit (code=<optimized out>) at /src/vpp/src/vpp/vnet/main.c:395
#3  0x00007f1ea6568135 in unix_signal_handler (signum=11, si=<optimized out>, uc=<optimized out>) at /src/vpp/src/vlib/unix/main.c:184
#4  <signal handler called>
#5  upf_pfcp_send_data (msg=0x7f1ce2867800) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:83
#6  0x00007f1e5c4fcb08 in upf_pfcp_server_rx_msg (msg=0x7f1cecaa7180) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:272
#7  pfcp_process (vm=<optimized out>, rt=<optimized out>, f=<optimized out>) at /src/vpp/src/plugins/upf/upf_pfcp_server.c:1162
#8  0x00007f1ea65081a7 in vlib_process_bootstrap (_a=<optimized out>) at /src/vpp/src/vlib/main.c:1454
#9  0x00007f1ea594eec4 in clib_calljmp () from /usr/lib/x86_64-linux-gnu/libvppinfra.so.20.09
#10 0x00007f1e6528a550 in ?? ()
#11 0x00007f1ea64fdcd2 in vlib_process_startup (vm=0x7f1ea67826c0 <vlib_global_main>, p=0x7f1e64ce20c0, f=0x0) at /src/vpp/src/vlib/main.c:1479
#12 dispatch_process (vm=0x7f1ea67826c0 <vlib_global_main>, p=0x7f1e64ce20c0, f=0x0, last_time_stamp=<optimized out>) at /src/vpp/src/vlib/main.c:1535
Backtrace stopped: Cannot access memory at address 0x7f1ea899c0b8```

Flows may have stale PDR ids after session modification

Previous report: travelping/vpp#85

quay.io/travelping/upf:fix-2009-traffic-timer_v20.09-upfbase-9-g9a520fa75_debug

The result of having stale PDRs is this crash:

/src/vpp/src/plugins/upf/upf_input.c:152 (upf_input) assertion `(((upf_buffer_opaque_t *)((u8 *)((b)->opaque2) + ((uword) & (((vnet_buffer_opaque_t *) 0)->unused))))->gtpu.pdr_idx) < vec_len (active->pdr)' fails

session_establishment failed due to missing PDI information F-TEID

When CH field in F-TEID is set then UPF should handle TEID allocation and report teid in response (TS 29.244, 8.2.3, bit3 CH).
But I see error of mandatory IE is missing.
Am I correct and this is to be handled ?

Frame 14: 164 bytes on wire (1312 bits), 164 bytes captured (1312 bits) on interface eth0, id 0
Ethernet II, Src: 02:42:c0:a8:3d:c4 (02:42:c0:a8:3d:c4), Dst: 02:42:c0:a8:3d:c5 (02:42:c0:a8:3d:c5)
Internet Protocol Version 4, Src: 192.168.61.196, Dst: 172.21.16.2
User Datagram Protocol, Src Port: 56768, Dst Port: 8805
Packet Forwarding Control Protocol
    Flags: 0x21, SEID (S)
        001. .... = Version: 1
        ...0 .... = Spare: 0
        .... 0... = Spare: 0
        .... .0.. = Spare: 0
        .... ..0. = Message Priority (MP): False
        .... ...1 = SEID (S): True
    Message Type: PFCP Session Establishment Request (50)
    Length: 118
    SEID: 0x0000000000000000
    Sequence Number: 601419
    Spare: 0
    Node ID : IPv4 address: 192.168.61.196
    F-SEID : SEID: 0x0000000000000001, IPv4 192.168.61.196
    Create PDR : [Grouped IE]
        IE Type: Create PDR (1)
        IE Length: 50
        PDR ID : 1
        Precedence : 15
        PDI : [Grouped IE]
            IE Type: PDI (2)
            IE Length: 19
            Source Interface : Access
                IE Type: Source Interface (20)
                IE Length: 1
                0000 .... = Spare: 0
                .... 0000 = Source Interface: Access (0)
            F-TEID : 
                IE Type: F-TEID (21)
                IE Length: 1
                Flags: 0x04, CH (CHOOSE)
                    0000 .... = Spare: 0
                    .... 0... = CHID (CHOOSE_ID): False
                    .... .1.. = CH (CHOOSE): True
                    .... ..0. = V6 (IPv6): Not Present
                    .... ...0 = V4 (IPv4): Not Present
            UE IP Address : 
        Outer Header Removal : GTP-U/UDP/IPv4
        FAR ID : Dynamic by CP 1
    Create FAR : [Grouped IE]
    [Response In: 15]

Response message -

Frame 15: 69 bytes on wire (552 bits), 69 bytes captured (552 bits) on interface eth0, id 0
Ethernet II, Src: 02:42:c0:a8:3d:c5 (02:42:c0:a8:3d:c5), Dst: 02:42:c0:a8:3d:c4 (02:42:c0:a8:3d:c4)
Internet Protocol Version 4, Src: 172.21.16.2, Dst: 192.168.61.196
User Datagram Protocol, Src Port: 8805, Dst Port: 56768
Packet Forwarding Control Protocol
    Flags: 0x21, SEID (S)
        001. .... = Version: 1
        ...0 .... = Spare: 0
        .... 0... = Spare: 0
        .... .0.. = Spare: 0
        .... ..0. = Message Priority (MP): False
        .... ...1 = SEID (S): True
    Message Type: PFCP Session Establishment Response (51)
    Length: 23
    SEID: 0x0000000000000000
    Sequence Number: 601419
    Spare: 0
    Cause : Mandatory IE missing
    Offending IE : PDI
        IE Type: Offending IE (40)
        IE Length: 2
        Type of the offending IE: PDI (2)
    [Response To: 14]
    [Response Time: 0.000559919 seconds]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.