ccp-project / libccp Goto Github PK
View Code? Open in Web Editor NEWA library for implementing CCP-compatible datapaths.
Home Page: https://ccp-project.github.io/
License: ISC License
A library for implementing CCP-compatible datapaths.
Home Page: https://ccp-project.github.io/
License: ISC License
currently, datapath program ids are assigned in portus, meaning the datapath has to loop through the array of programs every time it wants to look one up. it would be better if the datapath assigned the id based on the available slot and then told portus to use this id going forward. the main reason this isn't trivial is that it requires the datapath sending a response to portus, which isn't currently implemented.
see comments in #48
If the first create message fails to send, the integration test currently fails.
osx can't find pthread_spinlock_t. not sure if it's not defined or it's not being included from the right place. don't have time now but I'll fix it soon.
Let's get it working with a spin lock for now, but I'll look at the code and see if there's a better way to do it that doesn't require one.
for now, we've assumed only one ccp process will run at a time.
new programs are only installed when a ccp first starts, so if there's only one ccp, then we can use this as a signal that any ccp-specific state stored in the datapath (such as datapath programs) is no longer needed and can be overwritten.
however, if there are multiple ccps, obviously we don't want to remove state related to ccp1 when ccp2 starts if ccp1 is still running. we can generalize the above solution by having ccps give themselves an ID and then provide that ID when they install new programs.
During the travis build of libccp, run this integration test: https://github.com/ccp-project/portus/tree/master/integration_tests/libccp_integration
Possible ways to do this:
When running ccp on linux, libccp
attempts to de-reference a null pointer when freeing a connection, resulting in the following in syslog:
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.759642] ccp: freeing connection 1
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.759786] libccp: index mismatch: sid 1, index 0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.759810] BUG: unable to handle kernel NULL pointer dereference at 00000000000000b8
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.759891] IP: get_ccp_priv_state+0x1/0x10 [ccp]
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.759910] PGD 0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.759911] P4D 0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.759922]
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.759956] Oops: 0000 [#1] SMP PTI
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.759974] Modules linked in: ccp(OE) tcp_probe tcp_bbr vboxsf(OE) binfmt_misc ppdev intel_rapl_perf input_leds serio_raw parport_pc parport mac_hid i2c_piix4 vboxguest(OE) ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc vboxvideo(OE) ttm drm_kms_helper syscopyarea aesni_intel sysfillrect aes_x86_64 sysimgblt crypto_simd glue_helper cryptd fb_sys_fops drm psmouse e1000 pata_acpi video [last unloaded: tcp_probe]
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760214] CPU: 1 PID: 3108 Comm: cubic Tainted: G OE 4.13.0-46-generic #51-Ubuntu
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760258] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760288] task: ffff8eb7f4761740 task.stack: ffffb342806f0000
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760316] RIP: 0010:get_ccp_priv_state+0x1/0x10 [ccp]
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760339] RSP: 0018:ffffb342806f3bd0 EFLAGS: 00010246
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760367] RAX: 0000000000000026 RBX: ffff8eb7f935c818 RCX: 0000000000000000
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760395] RDX: 0000000000000000 RSI: ffff8eb7ffd16578 RDI: 0000000000000000
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760422] RBP: ffffb342806f3c20 R08: 0000000000000001 R09: 00000000000001d7
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760449] R10: 0000000000000200 R11: 0000000000000000 R12: 0000000000000008
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760478] R13: 0000000000000029 R14: ffff8eb7f6700000 R15: 0000000000000000
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760505] FS: 00007fe9cd55b840(0000) GS:ffff8eb7ffd00000(0000) knlGS:0000000000000000
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760541] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760564] CR2: 00000000000000b8 CR3: 0000000076d74003 CR4: 00000000000606e0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760600] Call Trace:
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760621] ? ccp_read_msg+0x4bf/0x530 [ccp]
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760643] nl_recv+0x27/0x50 [ccp]
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760667] netlink_unicast+0x194/0x230
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760686] netlink_sendmsg+0x2c5/0x3c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760705] sock_sendmsg+0x3e/0x50
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.760719] ___sys_sendmsg+0x2a0/0x2f0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.761499] ? apic_timer_interrupt+0xb4/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.761941] ? apic_timer_interrupt+0xad/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.762407] ? apic_timer_interrupt+0xa6/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.762878] ? apic_timer_interrupt+0x9f/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.763321] ? apic_timer_interrupt+0x98/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.763760] ? apic_timer_interrupt+0x91/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.764102] ? apic_timer_interrupt+0x8a/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.764443] ? apic_timer_interrupt+0x83/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.764782] ? apic_timer_interrupt+0x7c/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.765194] ? apic_timer_interrupt+0x75/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.765516] ? apic_timer_interrupt+0x6e/0x1c0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.765824] ? __fget_light+0x1b/0x60
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.766118] ? entry_SYSCALL_64_after_hwframe+0x11a/0x139
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.766405] ? entry_SYSCALL_64_after_hwframe+0x113/0x139
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.766677] __sys_sendmsg+0x54/0x90
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.766936] ? __sys_sendmsg+0x54/0x90
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.767275] SyS_sendmsg+0x12/0x20
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.767512] entry_SYSCALL_64_fastpath+0x24/0xab
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.767741] RIP: 0033:0x7fe9ccd2def0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.767960] RSP: 002b:00007ffe47cc75e0 EFLAGS: 00000293 ORIG_RAX: 000000000000002e
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.768190] RAX: ffffffffffffffda RBX: 00007fe9ccafec78 RCX: 00007fe9ccd2def0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.768423] RDX: 0000000000000000 RSI: 00007ffe47cc7650 RDI: 0000000000000003
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.768655] RBP: 0000000000000030 R08: 0000000000000000 R09: 00007ffe47cc7730
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.768890] R10: fffffffffffffeb0 R11: 0000000000000293 R12: 00007fe9ccafec20
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.769165] R13: 0000000000000000 R14: 00007fe9ccafec78 R15: 0000000000000000
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.769402] Code: 89 50 10 31 c0 5b 5d c3 66 90 0f 1f 44 00 00 55 48 8b bf b8 00 00 00 48 89 e5 e8 eb 5a 24 c5 5d c3 66 0f 1f 84 00 00 00 00 00 55 <48> 8b 87 b8 00 00 00 48 89 e5 5d c3 0f 1f 00 0f 1f 44 00 00 55
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.770311] RIP: get_ccp_priv_state+0x1/0x10 [ccp] RSP: ffffb342806f3bd0
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.770571] CR2: 00000000000000b8
Jun 5 16:47:21 ubuntu1710 kernel: [ 92.770909] ---[ end trace ebe62ca4544907cc ]---
This should be reproducible by running the following script on the vagrant VM supplied in the eval-scripts
repo: https://github.com/kevindrosendahl/eval-scripts/blob/641c686c69ff224840a2613dec42eefa33bd8117/reproduction/crash/reproduce.sh
instead of returning -1, -2, -3, etc we should define a bunch of different error types and return that const instead (and have the print convert to a nice string error description)
Right now, libccp just exits upon any error; it might be nice for this to send error messages up to Portus and for Portus to handle these errors.
Reset the time properly.
A lot of the serialization code calls memcpy a number of times when it could just call it once, e.g. serialize.c:
memcpy(msg, buf, 2 * sizeof(u32));
buf += 2 * sizeof(u32);
memcpy(&msg->exprs, buf, msg->num_expressions * sizeof(struct ExpressionMsg));
buf += msg->num_expressions * sizeof(struct ExpressionMsg);
memcpy(&msg->instrs, buf, msg->num_instructions * sizeof(struct InstructionMsg));
buf += msg->num_expressions * sizeof(struct InstructionMsg);
and read_msg could just cast the buffer to an installexpressionmsg rather than allocating space for one on the stack and copying it over. then we're only limited by the size of the message (which is 32K for netlink)
(We need to do this before adding to pantheon.)
Add checks for..
And add corresponding unit tests.
Before performing operations, check if they would overflow or underflow the unsigned integer. If they would, treat them the same as a divide by zero.
not sure of the root cause at the moment but:
after a while, all datapath programs fail to be installed with error code -1, which means no more datapath program storage -- this happens even on a fresh run of the ccp process, which should mean an entirely new empty piece of memory within libccp
currently, sizeof(struct ccp_priv_state) = 1952
, which is dominated by assuming a maximum of 110 control registers and 110 report registers, which are each 8 bytes.
This seems like way more than necessary. We could make this smaller by default and configurable as a kernel module option if someone needs an especially large program.
Better yet, now that init_programs is only run once at the beginning, we could dynamically allocate the space for registers when programs are installed, that way we store exactly as much space for registers as we need.
Instead of the current PRINT macro logging, allow the datapath to control where logs are printed by optionally providing a printf style log() callback.
Jun 19 21:54:37 pd2 kernel: [ 434.821955] CPU: 7 PID: 2240 Comm: reno Tainted: G OEL 4.14.0+ #2
Jun 19 21:54:37 pd2 kernel: [ 434.821956] Hardware name: Dell Inc. PowerEdge R320/0R5KP9, BIOS 2.0.22 11/19/2013
Jun 19 21:54:37 pd2 kernel: [ 434.821957] Call Trace:
Jun 19 21:54:37 pd2 kernel: [ 434.821958] <IRQ>
Jun 19 21:54:37 pd2 kernel: [ 434.821963] dump_stack+0x63/0x8b
Jun 19 21:54:37 pd2 kernel: [ 434.821966] nmi_cpu_backtrace+0xca/0xd0
Jun 19 21:54:37 pd2 kernel: [ 434.821969] ? irq_force_complete_move+0x130/0x130
Jun 19 21:54:37 pd2 kernel: [ 434.821971] nmi_trigger_cpumask_backtrace+0xde/0x110
Jun 19 21:54:37 pd2 kernel: [ 434.821973] arch_trigger_cpumask_backtrace+0x19/0x20
Jun 19 21:54:37 pd2 kernel: [ 434.821975] rcu_dump_cpu_stacks+0xa3/0xd6
Jun 19 21:54:37 pd2 kernel: [ 434.821977] rcu_check_callbacks+0x758/0x8b0
Jun 19 21:54:37 pd2 kernel: [ 434.821979] ? account_system_index_time+0x63/0x70
Jun 19 21:54:37 pd2 kernel: [ 434.821981] ? tick_sched_do_timer+0x50/0x50
Jun 19 21:54:37 pd2 kernel: [ 434.821984] update_process_times+0x2f/0x60
Jun 19 21:54:37 pd2 kernel: [ 434.821985] tick_sched_handle+0x2a/0x60
Jun 19 21:54:37 pd2 kernel: [ 434.821987] tick_sched_timer+0x39/0x80
Jun 19 21:54:37 pd2 kernel: [ 434.821989] __hrtimer_run_queues+0xe5/0x210
Jun 19 21:54:37 pd2 kernel: [ 434.821992] hrtimer_interrupt+0xa3/0x1e0
Jun 19 21:54:37 pd2 kernel: [ 434.821994] smp_apic_timer_interrupt+0x5f/0x120
Jun 19 21:54:37 pd2 kernel: [ 434.821996] apic_timer_interrupt+0x93/0xa0
Jun 19 21:54:37 pd2 kernel: [ 434.821999] RIP: 0010:native_queued_spin_lock_slowpath+0x177/0x1a0
Jun 19 21:54:37 pd2 kernel: [ 434.822000] RSP: 0018:ffff8de96ebc3930 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff10
Jun 19 21:54:37 pd2 kernel: [ 434.822002] RAX: 0000000000000101 RBX: ffff8de9365e4fb0 RCX: 0000000000000001
Jun 19 21:54:37 pd2 kernel: [ 434.822003] RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff8de9365e4fb0
Jun 19 21:54:37 pd2 kernel: [ 434.822004] RBP: ffff8de96ebc3930 R08: 0000000000000101 R09: 000000000000002d
Jun 19 21:54:37 pd2 kernel: [ 434.822005] R10: 00000000000000f9 R11: 000000004453c857 R12: ffff8de962700000
Jun 19 21:54:37 pd2 kernel: [ 434.822006] R13: 000000000000002d R14: 000000007a782e3d R15: 0000000000001204
Jun 19 21:54:37 pd2 kernel: [ 434.822010] _raw_spin_lock+0x20/0x30
Jun 19 21:54:37 pd2 kernel: [ 434.822014] ccp_invoke+0x46/0x80 [ccp]
Jun 19 21:54:37 pd2 kernel: [ 434.822017] tcp_ccp_cong_control+0x1da/0x220 [ccp]
Jun 19 21:54:37 pd2 kernel: [ 434.822019] ? tcp_rearm_rto.part.60+0x5f/0xe0
Jun 19 21:54:37 pd2 kernel: [ 434.822020] tcp_ack+0x9e6/0x1160
Jun 19 21:54:37 pd2 kernel: [ 434.822023] tcp_rcv_established+0x146/0x560
Jun 19 21:54:37 pd2 kernel: [ 434.822025] tcp_v4_do_rcv+0x139/0x1d0
Jun 19 21:54:37 pd2 kernel: [ 434.822027] tcp_v4_rcv+0x9b7/0xac0
Jun 19 21:54:37 pd2 kernel: [ 434.822030] ip_local_deliver_finish+0x97/0x1d0
Jun 19 21:54:37 pd2 kernel: [ 434.822033] ip_local_deliver+0x6f/0xe0
Jun 19 21:54:37 pd2 kernel: [ 434.822035] ? tcp_v4_early_demux+0xad/0x150
Jun 19 21:54:37 pd2 kernel: [ 434.822036] ip_rcv_finish+0x118/0x400
Jun 19 21:54:37 pd2 kernel: [ 434.822038] ip_rcv+0x282/0x390
Jun 19 21:54:37 pd2 kernel: [ 434.822040] ? __build_skb+0x2b/0xf0
Jun 19 21:54:37 pd2 kernel: [ 434.822043] __netif_receive_skb_core+0x7f8/0xac0
Jun 19 21:54:37 pd2 kernel: [ 434.822044] ? __build_skb+0x2b/0xf0
Jun 19 21:54:37 pd2 kernel: [ 434.822046] ? tcp4_gro_receive+0x10c/0x1b0
Jun 19 21:54:37 pd2 kernel: [ 434.822049] __netif_receive_skb+0x18/0x60
Jun 19 21:54:37 pd2 kernel: [ 434.822050] ? __netif_receive_skb+0x18/0x60
Jun 19 21:54:37 pd2 kernel: [ 434.822052] netif_receive_skb_internal+0x37/0xe0
Jun 19 21:54:37 pd2 kernel: [ 434.822054] napi_gro_frags+0xee/0x220
Jun 19 21:54:37 pd2 kernel: [ 434.822058] mlx4_en_process_rx_cq+0xac7/0xc10 [mlx4_en]
Jun 19 21:54:37 pd2 kernel: [ 434.822060] ? __wake_up_common_lock+0x8e/0xc0
Jun 19 21:54:37 pd2 kernel: [ 434.822064] mlx4_en_poll_rx_cq+0x5f/0x100 [mlx4_en]
Jun 19 21:54:37 pd2 kernel: [ 434.822067] net_rx_action+0x248/0x380
Jun 19 21:54:37 pd2 kernel: [ 434.822069] __do_softirq+0x104/0x28f
Jun 19 21:54:37 pd2 kernel: [ 434.822071] irq_exit+0xb6/0xc0
Jun 19 21:54:37 pd2 kernel: [ 434.822073] do_IRQ+0x4f/0xd0
Jun 19 21:54:37 pd2 kernel: [ 434.822075] common_interrupt+0x93/0x93
Jun 19 21:54:37 pd2 kernel: [ 434.822076] </IRQ>
Jun 19 21:54:37 pd2 kernel: [ 434.822079] RIP: 0010:do_set_cwnd+0x0/0x30 [ccp]
Jun 19 21:54:37 pd2 kernel: [ 434.822080] RSP: 0018:ffff9eee46267bb0 EFLAGS: 00000206 ORIG_RAX: ffffffffffffff58
Jun 19 21:54:37 pd2 kernel: [ 434.822082] RAX: ffff8de912c77720 RBX: ffff8de9655e921c RCX: 0000000000000019
Jun 19 21:54:37 pd2 kernel: [ 434.822083] RDX: 00000000005e4b22 RSI: ffff8de962700000 RDI: ffff8de912c77720
Jun 19 21:54:37 pd2 kernel: [ 434.822084] RBP: ffff9eee46267bb8 R08: ffffffffb93ea580 R09: ffff8de9655e9200
Jun 19 21:54:37 pd2 kernel: [ 434.822085] R10: 0000000000000200 R11: ffff8de9655e9200 R12: ffff8de962700000
Jun 19 21:54:37 pd2 kernel: [ 434.822086] R13: ffff8de9365e0000 R14: 0000000000000004 R15: 0000000000000000
Jun 19 21:54:37 pd2 kernel: [ 434.822090] ? update_register+0x78/0x80 [ccp]
Jun 19 21:54:37 pd2 kernel: [ 434.822093] ccp_read_msg+0x13d/0x380 [ccp]
Jun 19 21:54:37 pd2 kernel: [ 434.822094] ? __netlink_lookup+0xd6/0x140
Jun 19 21:54:37 pd2 kernel: [ 434.822097] nl_recv+0x24/0x50 [ccp]
Jun 19 21:54:37 pd2 kernel: [ 434.822100] netlink_unicast+0x18c/0x220
Jun 19 21:54:37 pd2 kernel: [ 434.822102] netlink_sendmsg+0x2d2/0x3c0
Jun 19 21:54:37 pd2 kernel: [ 434.822105] sock_sendmsg+0x38/0x50
Jun 19 21:54:37 pd2 kernel: [ 434.822106] ___sys_sendmsg+0x2d7/0x2f0
Jun 19 21:54:37 pd2 kernel: [ 434.822110] ? wake_up_q+0x44/0x80
Jun 19 21:54:37 pd2 kernel: [ 434.822112] ? do_futex+0x927/0xb60
Jun 19 21:54:37 pd2 kernel: [ 434.822115] __sys_sendmsg+0x54/0x90
Jun 19 21:54:37 pd2 kernel: [ 434.822116] ? __sys_sendmsg+0x54/0x90
Jun 19 21:54:37 pd2 kernel: [ 434.822119] SyS_sendmsg+0x12/0x20
Jun 19 21:54:37 pd2 kernel: [ 434.822121] entry_SYSCALL_64_fastpath+0x1e/0xa9
Jun 19 21:54:37 pd2 kernel: [ 434.822122] RIP: 0033:0x7f33df086460
Jun 19 21:54:37 pd2 kernel: [ 434.822123] RSP: 002b:00007ffe64419e90 EFLAGS: 00000293 ORIG_RAX: 000000000000002e
Jun 19 21:54:37 pd2 kernel: [ 434.822125] RAX: ffffffffffffffda RBX: 00007ffe6441c240 RCX: 00007f33df086460
Jun 19 21:54:37 pd2 kernel: [ 434.822125] RDX: 0000000000000000 RSI: 00007ffe6441a120 RDI: 0000000000000003
Jun 19 21:54:37 pd2 kernel: [ 434.822126] RBP: 00007ffe6441a5f0 R08: 0000000000000000 R09: 0000000000000000
Jun 19 21:54:37 pd2 kernel: [ 434.822127] R10: 0000000000000000 R11: 0000000000000293 R12: 0000559cae958a90
Jun 19 21:54:37 pd2 kernel: [ 434.822128] R13: 00007ffe6441c3f0 R14: 00007ffe6441c288 R15: 00007ffe6441c238
Jun 19 21:55:01 pd2 CRON[2291]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Jun 19 21:55:03 pd2 kernel: [ 460.380323] watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [iperf:2243]
Jun 19 21:55:03 pd2 kernel: [ 460.388088] Modules linked in: ccp(OE) intel_rapl sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel dcdbas kvm ipmi_ssif irqbypass ipmi_si joydev input_leds ipmi_devintf mei_me acpi_power_meter ipmi_msghandler intel_cstate mei shpchp intel_rapl_perf acpi_pad lpc_ich mac_hid ib_iser rdma_cm iw_cm ib_cm ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_decompress zstd_compress xxhash raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mlx4_en hid_generic usbhid hid crct10dif_pclmul crc32_pclmul ghash_clmulni_intel mlx4_core pcbc tg3 aesni_intel aes_x86_64 crypto_simd glue_helper ptp ahci cryptd libahci devlink pps_core wmi
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.