Comments (18)
That was the problem. I can now reach 8.8.8.8 from the UE.
Thank you very much for your time.
from upg-vpp.
I started off with this walkthrough but the configurations are outdated.
For the erGW configuration, you can refer to this issue which provides a suitable ergw-c-node.config
file.
For the UPG-VPP part, you need to run this to install VPP with this plugin:
cd /usr/src
git clone https://github.com/travelping/upg-vpp
cd upg-vpp
make update-vpp
cd vpp
make install-deps
make install-ext-deps
make build
The startup.conf
and init.conf
I use are the one from here.
Also, you don't need to run the sudo ip addr add 192.168.1.2/24 dev vpptap
from the walkthrough.
from upg-vpp.
Thanks for the report. Do you have a coredump for the crash or could you please run VPP under gdb to get the stack trace:
gdb --args /path/to/vpp -c /path/to/startup.conf
r
# after crash:
bt full 30
from upg-vpp.
I apologise for the delay.
When I try to ping the SGi interface from the UE, VPP with gdb returns this
DBGvpp# /usr/src/upg-vpp/vpp/src/plugins/upf/upf_session_dpo.c:426 (upf_ip4_session_dpo_node_fn_hsw) assertion `!((b)->flags & UPF_BUFFER_F_GTPU_INITIALIZED)' fails
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
Here is the output of bt full 30
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
set = {__val = {0, 140736272781584, 2, 9223372036854775822, 0, 0, 0, 0, 0, 0, 5335729, 0, 0, 140737329983501, 0, 93824993305248}}
pid = <optimized out>
tid = <optimized out>
ret = <optimized out>
#1 0x00007ffff688a859 in __GI_abort () at abort.c:79
save_stage = 1
act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction = 0x0}, sa_mask = {__val = {140737329930307, 140736256225216, 140736260557040, 93824992260704, 140737353822485,
140736270511584, 3076570368, 140737353899776, 93824993305248, 7597679427751605295, 8029178664428990559, 3546082429144233074, 140736269958400, 140736269958416,
140736269958424, 140737333840448}}, sa_flags = -154513920, sa_restorer = 0x7fffb6d14910}
sigs = {__val = {32, 0 <repeats 15 times>}}
#2 0x000055555555cef2 in os_panic () at /usr/src/upg-vpp/vpp/src/vpp/vnet/main.c:371
No locals.
#3 0x00007ffff6a8bb3b in debugger () at /usr/src/upg-vpp/vpp/src/vppinfra/error.c:84
No locals.
#4 0x00007ffff6a8bf32 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7fffb47a8818 "%s:%d (%s) assertion `%s' fails")
at /usr/src/upg-vpp/vpp/src/vppinfra/error.c:143
msg = 0x0
va = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0x7fffb6d14a60, reg_save_area = 0x7fffb6d14980}}
#5 0x00007fffb465a5d4 in upf_ip4_session_dpo_node_fn_hsw (vm=0x7ffff6ca4a40 <vlib_global_main>, node=0x7fffb670c240, from_frame=0x7fffb789d380)
at /usr/src/upg-vpp/vpp/src/plugins/upf/upf_session_dpo.c:426
ip0 = 0x10024c7c32
error0 = 1
n_left_to_next = 255
b = 0x10024c7b00
bi = 602604
error_node = 0x7fffb671b240
n_left_from = 0
next_index = 0
from = 0x7fffb789d394
to_next = 0x7fffb76aa4d4
gtm = 0x7fffb4819440 <upf_main>
next = 0
sidx = 1
__FUNCTION__ = "upf_ip4_session_dpo_node_fn_hsw"
#6 0x00007ffff6bf97d4 in dispatch_node (vm=0x7ffff6ca4a40 <vlib_global_main>, node=0x7fffb670c240, type=VLIB_NODE_TYPE_INTERNAL, dispatch_state=VLIB_NODE_STATE_POLLING,
frame=0x7fffb789d380, last_time_stamp=6795155973138) at /usr/src/upg-vpp/vpp/src/vlib/main.c:1194
n = 1
v = 0
t = 6795155973138
nm = 0x7ffff6ca4ba8 <vlib_global_main+360>
nf = 0x7fffb7229cb8
__FUNCTION__ = "dispatch_node"
i = <optimized out>
from = <optimized out>
b = <optimized out>
#7 0x00007ffff6bf9ee2 in dispatch_pending_node (vm=0x7ffff6ca4a40 <vlib_global_main>, pending_frame_index=16, last_time_stamp=6795155973138)
at /usr/src/upg-vpp/vpp/src/vlib/main.c:1353
nm = 0x7ffff6ca4ba8 <vlib_global_main+360>
f = 0x7fffb789d380
nf = 0x7fffb72348a8
nf_placeholder = {frame = 0x7fffb7615050, node_runtime_index = 8, flags = 0, vectors_since_last_overflow = 3076608080}
--Type <RET> for more, q to quit, c to continue without paging--
n = 0x7fffb670c240
restore_frame = 0x7fffb789d380
p = 0x7fffb7684000
__FUNCTION__ = "dispatch_pending_node"
#8 0x00007ffff6bfbf42 in vlib_main_or_worker_loop (vm=0x7ffff6ca4a40 <vlib_global_main>, is_main=1) at /usr/src/upg-vpp/vpp/src/vlib/main.c:1848
n = 0x7fffb66e2600
nm = 0x7ffff6ca4ba8 <vlib_global_main+360>
tm = 0x7ffff6ca5340 <vlib_thread_main>
i = 16
cpu_time_now = 6795155973138
now = 29.880849768809789
fqm = 0x2
frame_queue_check_counter = 0
__FUNCTION__ = "vlib_main_or_worker_loop"
#9 0x00007ffff6bfc9a8 in vlib_main_loop (vm=0x7ffff6ca4a40 <vlib_global_main>) at /usr/src/upg-vpp/vpp/src/vlib/main.c:1976
No locals.
#10 0x00007ffff6bfd7ae in vlib_main (vm=0x7ffff6ca4a40 <vlib_global_main>, input=0x7fffb6d14fb0) at /usr/src/upg-vpp/vpp/src/vlib/main.c:2222
error = 0x0
nm = 0x7ffff6ca4ba8 <vlib_global_main+360>
#11 0x00007ffff6c674b2 in thread0 (arg=140737333840448) at /usr/src/upg-vpp/vpp/src/vlib/unix/main.c:660
vm = 0x7ffff6ca4a40 <vlib_global_main>
input = {
buffer = 0x7fffb8648820 "plugins { path /usr/src/upg-vpp/vpp/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins/ plugin dpdk_plugin.so { disable } plugin gtpu_plugin.so { disable } plugin upf_plugin.so { enable } }", index = 18446744073709551615, buffer_marks = 0x7fffb697e8f0, fill_buffer = 0x0, fill_buffer_arg = 0x0}
i = 320017171
#12 0x00007ffff6aaaa4c in clib_calljmp () at /usr/src/upg-vpp/vpp/src/vppinfra/longjmp.S:123
No locals.
#13 0x00007fffffffd1e0 in ?? ()
No symbol table info available.
#14 0x00007ffff6c67a5b in vlib_unix_main (argc=44, argv=0x55555565c9a0) at /usr/src/upg-vpp/vpp/src/vlib/unix/main.c:733
vm = 0x7ffff6ca4a40 <vlib_global_main>
input = {buffer = 0x0, index = 0, buffer_marks = 0x0, fill_buffer = 0x0, fill_buffer_arg = 0x0}
e = 0x0
i = 0
__FUNCTION__ = "vlib_unix_main"
#15 0x000055555555c825 in main (argc=44, argv=0x55555565c9a0) at /usr/src/upg-vpp/vpp/src/vpp/vnet/main.c:296
tmp = {cacheline0 = 0x7fffffffd440 "", wait_at_barrier = 0x7000, workers_at_barrier = 0x8000, cacheline1 = 0x7fffffffd480 "x\242", thread_mheap = 0xa278,
thread_stack = 0xabe0 <error: Cannot access memory at address 0xabe0>, thread_function = 0x8000, thread_function_arg = 0x3, recursion_level = 745472, elog_track = {
name = 0x1 <error: Cannot access memory at address 0x1>, track_index_plus_one = 1363968}, instance_id = 1372160, registration = 0x14e104,
name = 0x14e110 <error: Cannot access memory at address 0x14e110>, barrier_sync_count = 1359872, barrier_elog_enabled = 3 '\003',
barrier_caller = 0x2127ad <error: Cannot access memory at address 0x2127ad>, barrier_context = 0x2127ad <error: Cannot access memory at address 0x2127ad>,
node_reforks_required = 0x78000, lwp = 5, cpu_id = 2174976, core_id = 1, numa_id = 0, thread_id = 2765856}
i = 43
vm = 0x7ffff6ca4a40 <vlib_global_main>
main_heap_size = 1073741824
sizep = 0x0
size = 0
main_core = 1
cpuset = {__bits = {2, 0 <repeats 15 times>}}
main_heap = 0x7fffb63e1010
msg = 0x5555556291e8 "ERROR: This binary requires CPU with %s extensions.\n"
from upg-vpp.
Hi, thanks for the stack trace. Looks like you're hitting an interesting edge case here that we were unable to observe in our environment so far. If this is not too much to ask, could you please do the following:
- Do a release build or comment out this ASSERT and rebuild
- Enable dispatch trace:
pcap dispatch trace on max 10000 file vppcapture
- Try passing the traffic after initiating the session. That might not work but hopefully VPP will not crash without the ASSERT in place.
- Disable VPP dispatch trace to have it saved:
pcap dispatch trace off
. It will be written to/tmp/vppcapture
- Attach the dispatch trace file here.
Thanks!
from upg-vpp.
Also, grabbing textual packet trace wouldn't harm either:
trace add af-packet input 10
then after trying to pass the traffic
show trace
from upg-vpp.
After deleting the ASSEST line and rebuilding, I can now ping the SGi interface without VPP crashing and it is responding to the ICMP requests.
However the UE still cannot access to the other addresses (I don't have a second UE for the moment to test if they can see each other) nor the Internet.
It also seems that I can't ping the UE from the VPP CLI nor the docker container or the computer.
The following zip contains three files:
vppcapture
from thepcap dispatch
command after testing several IP addresses to pingtrace_sgi_interface
, the output oftrace add
after pinging 10.0.0.2, the SGi interfacetrace_internet_fail
, the output oftrace add
after pinging 8.8.8.8 which is not responding
Also, I updated my version of VPP to 21.01.0-12 and the plugin to run these tests and I noticed several things:
- VPP still crashes with the ASSERT line up
- I have those errors when I start VPP
vat-plug/load [error ]: vat_plugin_register: gtpu plugin not loaded...
vat-plug/load [error ]: vat_plugin_register: oddbuf plugin not loaded...
- I can't ping 8.8.8.8 anymore from the VPP CLI
DBGvpp# ping 8.8.8.8 table-id 2
Failed: no source address for egress interface
Edit: with VPP version 21.01.0-14 I can ping 8.8.8.8 from the VPP CLI again.
from upg-vpp.
Hi,
I tested the architecture with two UEs connected to the eNodeB. They both can ping each other and the SGi interface of VPP but cannot access anything else.
I also bumped into another VPP crash when trying to run the command nmap -sn 10.0.0.1/30
from the UE. It doesn't occur when I try to nmap any other network.
DBGvpp# /usr/src/upg-vpp/vpp/src/plugins/upf/upf_flow_node.c:328 (upf_flow_process) assertion `b0->flags & UPF_BUFFER_F_GTPU_INITIALIZED' fails
Aborted (core dumped)
(gdb) bt full 30
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
set = {__val = {0 <repeats 11 times>, 1, 70414772434196, 140736250691648, 140736192005568, 140737332278973}}
pid = <optimized out>
tid = <optimized out>
ret = <optimized out>
#1 0x00007ffff6945859 in __GI_abort () at abort.c:79
save_stage = 1
act = {__sigaction_handler = {sa_handler = 0x7fffffffe680, sa_sigaction = 0x7fffffffe680}, sa_mask = {__val = {0, 0, 140736306982736, 140736250692248, 140736306982896,
140736306940152, 140736306982880, 140736192005648, 140737332276177, 140736306982904, 140736250691648, 140736250691648, 140736306982900, 140736306982896, 140736306982904,
8}}, sa_flags = 208, sa_restorer = 0x7fffba561dc0}
sigs = {__val = {32, 0 <repeats 15 times>}}
#2 0x00007ffff6be47e6 in os_panic () at /usr/src/upg-vpp/vpp/src/vppinfra/unix-misc.c:177
No locals.
#3 0x00007ffff6b45e5c in debugger () at /usr/src/upg-vpp/vpp/src/vppinfra/error.c:84
No locals.
#4 0x00007ffff6b46253 in _clib_error (how_to_die=2, function_name=0x0, line_number=0, fmt=0x7fffb39784f0 "%s:%d (%s) assertion `%s' fails")
at /usr/src/upg-vpp/vpp/src/vppinfra/error.c:143
msg = 0x0
va = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0x7fffb2bb4790, reg_save_area = 0x7fffb2bb46b0}}
#5 0x00007fffb3823b9c in upf_flow_process (vm=0x7ffff6d73bc0 <vlib_global_main>, node=0x7fffba202580, frame=0x7fffb9955d40, is_ip4=1 '\001')
at /usr/src/upg-vpp/vpp/src/plugins/upf/upf_flow_node.c:328
bi0 = 602219
next0 = 0
flow_idx = 0
is_reverse = 0
kv = {key = {140736316056960, 140737334688704, 140736192006848, 0, 38532544, 11791424602000}, value = 0}
p = 0x7fffba0c24d0 "\\E\202\263\377\177"
sx0 = 0x8
active0 = 0x7fffb76f0130
created = 0
flow = 0x0
b0 = 0x10024c1ac0
gtm = 0x7fffb39e2e00 <upf_main>
n_left_from = 2
from = 0x7fffb9955d50
next_index = 1
to_next = 0x7fffb9959490
n_left_to_next = 256
fm = 0x7fffb39e85c0 <flowtable_main>
cpu_index = 0
fmt = 0x7fffb71ada90
CPT_HIT = 0
CPT_THRU = 0
CPT_CREATED = 0
CPT_UNHANDLED = 0
CPT_TIMER_EXPIRE = 0
CPT_COLLISION = 0
CPT_RECYCLE = 0
current_time = 29
__FUNCTION__ = "upf_flow_process"
#6 0x00007fffb382458d in upf_ip4_flow_node_fn_hsw (vm=0x7ffff6d73bc0 <vlib_global_main>, node=0x7fffba202580, from_frame=0x7fffb9955d40)
at /usr/src/upg-vpp/vpp/src/plugins/upf/upf_flow_node.c:444
No locals.
#7 0x00007ffff6cc621c in dispatch_node (vm=0x7ffff6d73bc0 <vlib_global_main>, node=0x7fffba202580, type=VLIB_NODE_TYPE_INTERNAL, dispatch_state=VLIB_NODE_STATE_POLLING,
frame=0x7fffb9955d40, last_time_stamp=11791424602000) at /usr/src/upg-vpp/vpp/src/vlib/main.c:1217
n = 2
v = 0
t = 11791424602000
nm = 0x7ffff6d73d28 <vlib_global_main+360>
nf = 0x7fffb703ddc8
__FUNCTION__ = "dispatch_node"
i = <optimized out>
from = <optimized out>
b = <optimized out>
#8 0x00007ffff6cc692a in dispatch_pending_node (vm=0x7ffff6d73bc0 <vlib_global_main>, pending_frame_index=26, last_time_stamp=11791424602000)
at /usr/src/upg-vpp/vpp/src/vlib/main.c:1376
nm = 0x7ffff6d73d28 <vlib_global_main+360>
f = 0x7fffb9955d40
nf = 0x7fffb703db70
nf_placeholder = {frame = 0x0, node_runtime_index = 4140560227, flags = 0, vectors_since_last_overflow = 3079135472}
n = 0x7fffba202580
restore_frame = 0x7fffb9955d40
p = 0x7fffb76f03a0
__FUNCTION__ = "dispatch_pending_node"
#9 0x00007ffff6cc8cab in vlib_main_or_worker_loop (vm=0x7ffff6d73bc0 <vlib_global_main>, is_main=1) at /usr/src/upg-vpp/vpp/src/vlib/main.c:1904
n = 0x7fffb7985c00
nm = 0x7ffff6d73d28 <vlib_global_main+360>
tm = 0x7ffff6d744c0 <vlib_thread_main>
i = 26
cpu_time_now = 11791424602000
now = 29.033688491622456
fqm = 0x1
frame_queue_check_counter = 0
empty_int_list = 0x7fffb787e0f0
__FUNCTION__ = "vlib_main_or_worker_loop"
#10 0x00007ffff6cc9711 in vlib_main_loop (vm=0x7ffff6d73bc0 <vlib_global_main>) at /usr/src/upg-vpp/vpp/src/vlib/main.c:2032
No locals.
#11 0x00007ffff6cca534 in vlib_main (vm=0x7ffff6d73bc0 <vlib_global_main>, input=0x7fffb2bb4fb0) at /usr/src/upg-vpp/vpp/src/vlib/main.c:2278
error = 0x0
nm = 0x7ffff6d73d28 <vlib_global_main+360>
#12 0x00007ffff6d35641 in thread0 (arg=140737334688704) at /usr/src/upg-vpp/vpp/src/vlib/unix/main.c:670
vm = 0x7ffff6d73bc0 <vlib_global_main>
input = {
buffer = 0x7fffba5ca1c0 "plugins { path /usr/src/upg-vpp/vpp/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins/ plugin dpdk_plugin.so { disable } plugin gtpu_plugin.so { disable } plugin upf_plugin.so { enable } }", index = 18446744073709551615, buffer_marks = 0x7fffba21e570, fill_buffer = 0x0, fill_buffer_arg = 0x0}
i = 0
#13 0x00007ffff6b64ee8 in clib_calljmp () at /usr/src/upg-vpp/vpp/src/vppinfra/longjmp.S:123
No locals.
#14 0x00007fffffffd2b0 in ?? ()
No symbol table info available.
#15 0x00007ffff6d35c11 in vlib_unix_main (argc=44, argv=0x5555556329a0) at /usr/src/upg-vpp/vpp/src/vlib/unix/main.c:747
vm = 0x7ffff6d73bc0 <vlib_global_main>
input = {buffer = 0x0, index = 0, buffer_marks = 0x0, fill_buffer = 0x0, fill_buffer_arg = 0x0}
e = 0x0
i = 0
__FUNCTION__ = "vlib_unix_main"
#16 0x000055555555cd5d in main (argc=44, argv=0x5555556329a0) at /usr/src/upg-vpp/vpp/src/vpp/vnet/main.c:338
i = 43
vm = 0x7ffff6d73bc0 <vlib_global_main>
main_heap_size = 1073741824
sizep = 0x7fffffffd740 ""
size = 4294957416
main_heap_log2_page_sz = CLIB_MEM_PAGE_SZ_DEFAULT
input = {buffer = 0x0, index = 0, buffer_marks = 0x0, fill_buffer = 0x0, fill_buffer_arg = 0x0}
sub_input = {buffer = 0xa278 <error: Cannot access memory at address 0xa278>, index = 44000, buffer_marks = 0x8000, fill_buffer = 0x3, fill_buffer_arg = 0xb6000}
s = 0x0
v = 0x0
main_core = 1
cpuset = {__bits = {2, 0 <repeats 15 times>}}
main_heap = 0x7fffb63ac000
msg = 0x5555556071e8 "ERROR: This binary requires CPU with %s extensions.\n"
__FUNCTION__ = "main"
from upg-vpp.
Hi again, sorry for delay. There's indeed a bug in UPG related to how pinging SGi interface is handled, and we will work on it.
It is related to the UPG subgraph check mechanism and only affects debug images.
Your connectivity problem doesn't seem to be related to it directly, though. Looking at the dispatch trace, the packets e.g. for 8.8.8.8
are sent out from the SGi interface just fine, but no replies from 8.8.8.8
ever reach it. Did you try tcpdump
on SGi side to see if there's some problem outside UPG?
from upg-vpp.
VPP can ping 8.8.8.8 directly through the SGi interface.
The sgi.pcapng file is the output when I listen to the SGi interface of the machine hosting the erGW+VPP container.
When I try to ping any other address from the UE (10.180.0.136), it just doesn't respond.
I also did ping the SGi interface 10.0.0.2 and the other UE (10.180.0.166) but it does not show up in the pcap file.
Do I need to add any additional rule on the host machine or in the container? Also, could the problem come from my erGW configuration?
from upg-vpp.
Looks like the traffic destined for 10.180.0.136
(from outside) doesn't reach SGi interface at all. This appears to be a routing/NAT problem on the host machine. Maybe you need to set up NAT on the box? Looks like 10.0.0.2
is covered by NAT, while 10.180.0.166
is not. Maybe it's a subnet mask problem?
from upg-vpp.
I tried to change the PDN address pool from 10.180.0.1/24 to 10.0.0.1/24 since the host machine has a nat rule for the 10.0.0.0/24 subnet but the problem seems to persist.
from upg-vpp.
Well you can't have UE IPs in SGi subnet ... Maybe changing host nat rule to 10.0.0.0/8 and keeping 10.180.0.1/24 PDN address pool will work better?
from upg-vpp.
I still can't reach anything from the UE. Maybe the problem is coming from the docker network?
from upg-vpp.
It's possible. You may need to add route to 10.180.0.0/24
via the container IP on the host.
from upg-vpp.
You're welcome. I've made a separate issue for the problem with pinging SGi IP
from upg-vpp.
@gantonie Please I can't find the documentation on how to configure upg-vpp, I am also trying to integrate this with open5gs..
If you don't mind, can you share both the config of the erGW and upg-vpp that you used?
from upg-vpp.
Thanks @gantonie!
from upg-vpp.
Related Issues (20)
- any plan for Network Triggered Service Request? HOT 1
- UPG crash when an UE pings another UE in the same UPF. HOT 2
- Build in arm-architecture HOT 3
- Help Needed HOT 2
- Whether Support Guaranteed Bitrate and Maximum Bitrate HOT 1
- support for non-root non-privileged mode HOT 2
- Support to launch travelPing using N3,N4 and N6 adresses
- About using upg-vpp with free5GC v3.3.0 HOT 2
- format_user_id function error HOT 1
- integrate oai 5g amf smf with travel ping UPF HOT 1
- Connection loss after approx 30 Seconds HOT 1
- Compilation failure for upg-vpp on host system (outside docker) HOT 4
- should keep VLIB_BUFFER_IS_TRACED flag HOT 3
- UPG changes Ethernet Header type field HOT 1
- Integration of upg-vpp inside PacketRusher HOT 3
- Help Needed HOT 1
- ICMP reply doesn't reach UE HOT 23
- Data is not working with UPG-VPP integrated with Open5GS HOT 10
- Does UPG-VPP support QoS and network slicing functions? HOT 1
- Does UPG-VPP support to forward end-marker from eNB? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from upg-vpp.