Comments (20)
List of kernel modules needed for microk8s
nfnetlink
nf_tables
xt_comment
xt_mark
nf_defrag_ipv4
nf_defrag_ipv6
nf_conntrack
nf_nat
iptable_nat
iptable_mangle
iptable_filter
nf_reject_ipv4
ipt_REJECT
ip6_tables
ip6table_nat
ip6table_mangle
ip6table_filter
nf_reject_ipv6
ip6t_REJECT
from fenix.
Detected, somehow at some point I installed selinux-policy-default
, introducing this problem. I have removed it (fully, also with dpkg --purge
to solve it). And I have added to uEnv.txt apparmor=1
.
At least it is registered in this issue. I think we can close it :)
Thanks for your time ;)
from fenix.
Hello @rNoz
Thanks for your feedback, we will check this issue.
from fenix.
Hi @rNoz
Here is the list of modules needed for docker. You can create /etc/modules-load.d/docker.conf
with the following content. Docker should start working after reboot.
nfnetlink
nf_tables
nft_counter
nft_objref
nft_compat
nf_defrag_ipv4
nf_defrag_ipv6
nf_conntrack
nf_nat
nft_chain_nat
xt_MASQUERADE
xt_addrtype
xt_conntrack
xt_tcpudp
xt_nat
I am also checking on microk8s and k3s and will shortly revert back on the same.
from fenix.
Raised #300 to load the modules by default.
from fenix.
Thank you!
from fenix.
@viraniac Any idea why snapd does not work right now for Vim4 in the latest Ubuntu 24.04?
k@Khadas:~$ sudo apt update && sudo apt upgrade
# everything up to date
k@Khadas:~$ sudo snap remove microk8s
error: system does not fully support snapd: cannot mount squashfs image using
"squashfs": ----- mount: /tmp/syscheck-mountpoint-2162884218: wrong fs type,
bad option, bad superblock on /dev/loop0, missing codepage or helper program,
or other error.
dmesg(1) may have more information after failed mount system call.
-----
k@Khadas:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04 LTS
Release: 24.04
Codename: noble
k@Khadas:~$ uname -a
Linux Khadas 5.15.119 #1.6.9.1 SMP PREEMPT Thu Jun 20 09:22:43 CST 2024 aarch64 aarch64 aarch64 GNU/Linux
/boot/uEnv.txt:
boot_user_args=cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory systemd.unified_cgroup_hierarchy=0
/etc/modules-load.d/microk8s.conf:
nfnetlink
nf_tables
xt_comment
xt_mark
nf_defrag_ipv4
nf_defrag_ipv6
nf_conntrack
nf_nat
iptable_nat
iptable_mangle
iptable_filter
nf_reject_ipv4
ipt_REJECT
ip6_tables
ip6table_nat
ip6table_mangle
ip6table_filter
nf_reject_ipv6
ip6t_REJECT
k@Khadas:~$ dpkg -l | grep -i -E '(khadas|vim|linux|snap)' | grep -E '^ii\s*(khadas|linux|snap|libsnap)'
ii khadas-vim4-linux-5.4-dt-overlays 1.6.5~2337ee9 arm64 Linux DTB overlays.
ii libsnapd-glib-2-1:arm64 1.64-0ubuntu5 arm64 GLib snapd library
ii libsnappy1v5:arm64 1.1.10-1build1 arm64 fast compression/decompression library
ii linux-base 4.5ubuntu9 all Linux image base package
ii linux-board-package-noble-vim4 1.6.9 arm64 Fenix tweaks for Ubuntu-noble on VIM4
ii linux-dtb-amlogic-5.15 1.6.9.1 arm64 Linux DTB, version 5.15.119
ii linux-gpu-mali-wayland 1.4-r44p0-202311 arm64 Lib Mali Wayland for A311D2
ii linux-headers-amlogic-5.15 1.6.9.1 arm64 Linux kernel headers for 5.15.119 on arm64
ii linux-image-amlogic-5.15 1.6.9.1 arm64 Linux kernel, version 5.15.119
ii linux-libc-dev:arm64 6.8.0-38.38 arm64 Linux Kernel Headers for development
ii linux-sound-base 1.0.25+dfsg-0ubuntu7 all base package for ALSA and OSS sound systems
ii linux-u-boot-vim4-vendor 1.6.9-2019.01 arm64 U-boot 2019.01
ii snapd 2.63+24.04 arm64 Daemon and tooling that enable snap packages
No matter the command (remove, install) or application.
Writing here because the PR 300 was merged.
from fenix.
Writing here because the PR 300 was merged.
The changes in the #300 are merged, but was not part of last image release and should get included in the next release.
I see you are trying to remove microk8s. Did you had it installed before? Does snap list
show any installed snap packages? By default no snap packages are included in the image, and they get installed as needed when user runs the snap install command
from fenix.
@rNoz I am trying to replicate the issue. Will get back to you with my findings.
from fenix.
@rNoz I can't replicate your issue. Please find my output here - https://paste.armbian.com/paxixiquyo.bash
What I have tried:
- flashed latest Fenix 1.6.9 Ubuntu 24.04 gnome image using oowow
- connected to wifi and made sure all updates are installed i.e.
sudo apt-get update && sudo apt-get dist-upgrade && sudo reboot
- Downloaded the modules list from the fenix repo and rebooted again
wget -O- https://raw.githubusercontent.com/khadas/fenix/master/archives/filesystem/special/VIM-COMMON/etc/modules-load.d/docker.conf | sudo tee /etc/modules-load.d/docker.conf
wget -O- https://raw.githubusercontent.com/khadas/fenix/master/archives/filesystem/special/VIM-COMMON/etc/modules-load.d/microk8s.conf | sudo tee /etc/modules-load.d/microk8s.conf
- installed and configured microk8s
sudo snap install microk8s --channel=1.28/stable --classic
sudo usermod -aG microk8s khadas
newgrp microk8s
microk8s disable ha-cluster
microk8s enable dashboard
microk8s enable dns
microk8s enable hostpath-storage
- checked the status of the microk8s node
microk8s kubectl get all --all-namespaces
from fenix.
I have the same modules and everything up to date, but snap (not necessarily for microk8s) is not working due to the squashfs error.
k@Khadas:~$ sudo snap list
No snaps are installed yet. Try 'snap install hello-world'.
k@Khadas:~$ sudo snap install hello-world
error: system does not fully support snapd: cannot mount squashfs image using "squashfs": -----
mount: /tmp/syscheck-mountpoint-1357931385: wrong fs type, bad option, bad superblock on
/dev/loop0, missing codepage or helper program, or other error.
dmesg(1) may have more information after failed mount system call.
-----
journal
Jul 15 10:41:45 Khadas snapd[2199]: daemon.go:519: gracefully waiting for running hooks
Jul 15 10:41:45 Khadas snapd[2199]: daemon.go:521: done waiting for running hooks
Jul 15 10:41:48 Khadas snapd[2199]: overlord.go:515: Released state lock file
Jul 15 10:41:48 Khadas snapd[2199]: daemon stop requested to wait for socket activation
Jul 15 10:41:48 Khadas systemd[1]: snapd.service: Deactivated successfully.
Jul 15 10:42:36 Khadas systemd[1]: Starting snapd.service - Snap Daemon...
Jul 15 10:42:36 Khadas snapd[2276]: overlord.go:271: Acquiring state lock file
Jul 15 10:42:36 Khadas snapd[2276]: overlord.go:276: Acquired state lock file
Jul 15 10:42:36 Khadas snapd[2276]: daemon.go:247: started snapd/2.63+24.04 (series 16; classic; devmode) ubuntu/24.04 (arm64) linux/5.15.119.
Jul 15 10:42:36 Khadas snapd[2276]: main.go:125: system does not fully support snapd: cannot mount squashfs image using "squashfs":
Jul 15 10:42:36 Khadas snapd[2276]: -----
Jul 15 10:42:36 Khadas snapd[2276]: mount: /tmp/syscheck-mountpoint-787852083: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
Jul 15 10:42:36 Khadas snapd[2276]: dmesg(1) may have more information after failed mount system call.
Jul 15 10:42:36 Khadas snapd[2276]: -----
Jul 15 10:42:36 Khadas snapd[2276]: daemon.go:340: adjusting startup timeout by 30s (pessimistic estimate of 30s plus 5s per snap)
Jul 15 10:42:36 Khadas snapd[2276]: backends.go:58: AppArmor status: apparmor not enabled
Jul 15 10:42:36 Khadas systemd[1]: Started snapd.service - Snap Daemon.
Jul 15 10:42:41 Khadas snapd[2276]: daemon.go:519: gracefully waiting for running hooks
Jul 15 10:42:41 Khadas snapd[2276]: daemon.go:521: done waiting for running hooks
Jul 15 10:42:44 Khadas snapd[2276]: overlord.go:515: Released state lock file
Jul 15 10:42:44 Khadas snapd[2276]: daemon stop requested to wait for socket activation
Jul 15 10:42:44 Khadas systemd[1]: snapd.service: Deactivated successfully.
dmesg
[ 43.091529] loop0: detected capacity change from 0 to 8
[ 43.092295] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev ?, type ?) errno=-22
[ 43.093260] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev loop0, type squashfs) errno=-22
[ 51.966584] loop0: detected capacity change from 0 to 8
[ 51.968003] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev ?, type ?) errno=-22
[ 51.968817] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev loop0, type squashfs) errno=-22
[ 86.892334] audit: type=1107 audit(1721040135.261:12): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:kernel_t:s0 msg='avc: denied { status } for auid=n/a uid=0 gid=0 path="/usr/lib/systemd/system/snapd.service" cmdline="" function="mac_selinux_filter" scontext=system_u:system_r:kernel_t:s0 tcontext=system_u:object_r:systemd_unit_t:s0 tclass=service permissive=1 exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?'
[ 108.174239] loop0: detected capacity change from 0 to 8
[ 108.174950] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev ?, type ?) errno=-22
[ 108.175399] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev loop0, type squashfs) errno=-22
Do I have to do anything with SELinux? I am not sure why snapd stopped working
from fenix.
Indeed, just by restarting snapd
, I get that error:
Jul 15 10:58:14 Khadas systemd[1]: snapd.service: Deactivated successfully.
Jul 15 10:58:17 Khadas systemd[1]: Starting snapd.service - Snap Daemon...
Jul 15 10:58:17 Khadas snapd[5958]: overlord.go:271: Acquiring state lock file
Jul 15 10:58:17 Khadas snapd[5958]: overlord.go:276: Acquired state lock file
Jul 15 10:58:17 Khadas snapd[5958]: daemon.go:247: started snapd/2.63+24.04 (series 16; classic; devmode) ubuntu/24.04 (arm64) linux/5.15.119.
Jul 15 10:58:17 Khadas snapd[5958]: main.go:125: system does not fully support snapd: cannot mount squashfs image using "squashfs":
Jul 15 10:58:17 Khadas snapd[5958]: -----
Jul 15 10:58:17 Khadas snapd[5958]: mount: /tmp/syscheck-mountpoint-1668328185: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
Jul 15 10:58:17 Khadas snapd[5958]: dmesg(1) may have more information after failed mount system call.
Jul 15 10:58:17 Khadas snapd[5958]: -----
Jul 15 10:58:17 Khadas snapd[5958]: daemon.go:340: adjusting startup timeout by 30s (pessimistic estimate of 30s plus 5s per snap)
Jul 15 10:58:17 Khadas snapd[5958]: backends.go:58: AppArmor status: apparmor not enabled
Jul 15 10:58:17 Khadas systemd[1]: Started snapd.service - Snap Daemon.
I have already tried installing sudo apt install libsquashfuse0 squashfuse fuse squash squashfs-tools
, but the same error.
More trials:
k@Khadas:~$ sudo apt install selinux-policy-default
# already installed
k@Khadas:~$ sestatus
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: default
Current mode: permissive
Mode from config file: permissive
Policy MLS status: enabled
Policy deny_unknown status: allowed
Memory protection checking: actual (secure)
Max kernel policy version: 33
k@Khadas:~$ sudo setenforce 0
Ok, Steps:
sudo apt remove snapd
reboot
sudo apt install snapd
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
gir1.2-gnomeautoar-0.1 gir1.2-gnomedesktop-3.0 gjs libcue2 libdee-1.0-4 libexempi8
libgexiv2-2 libgsf-1-114 libgsf-1-common libnautilus-extension4 libntfs-3g89t64
libportal-gtk4-1 libportal1 libtotem-plparser-common libtotem-plparser18
libtracker-sparql-3.0-0 libunity-protocol-private0
libunity-scopes-json-def-desktop libunity9 nautilus-data tracker tracker-extract
tracker-miner-fs xwayland
Use 'sudo apt autoremove' to remove them.
The following NEW packages will be installed:
snapd
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 25.2 MB of archives.
After this operation, 106 MB of additional disk space will be used.
Get:1 http://ports.ubuntu.com noble-updates/main arm64 snapd arm64 2.63+24.04 [25.2 MB]
Fetched 25.2 MB in 11s (2,327 kB/s)
Selecting previously unselected package snapd.
(Reading database ... 150033 files and directories currently installed.)
Preparing to unpack .../snapd_2.63+24.04_arm64.deb ...
Unpacking snapd (2.63+24.04) ...
Setting up snapd (2.63+24.04) ...
snapd.failure.service is a disabled or a static unit not running, not starting it.
snapd.snap-repair.service is a disabled or a static unit not running, not starting it.
Failed to restart snapd.mounts-pre.target: Operation refused, unit snapd.mounts-pre.target may be requested by dependency only (it is configured to refuse manual start/stop).
See system logs and 'systemctl status snapd.mounts-pre.target' for details.
Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 148.
Processing triggers for gnome-menus (3.36.0-1.1ubuntu3) ...
Processing triggers for man-db (2.12.0-4build2) ...
Processing triggers for dbus (1.14.10-4ubuntu4) ...
Processing triggers for desktop-file-utils (0.27-2build1) ...
dmesg:
[ 81.656526] audit: type=1400 audit(1721041718.739:12): avc: denied { transition } for pid=2131 comm="dpkg" path="/var/lib/dpkg/tmp.ci/preinst" dev="mmcblk0p5" ino=386038 scontext=system_u:system_r:kernel_t:s0 tcontext=system_u:system_r:dpkg_script_t:s0 tclass=process permissive=1
[ 81.656566] audit: type=1400 audit(1721041718.739:13): avc: denied { entrypoint } for pid=2131 comm="dpkg" path="/var/lib/dpkg/tmp.ci/preinst" dev="mmcblk0p5" ino=386038 scontext=system_u:system_r:dpkg_script_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
[ 89.009671] loop0: detected capacity change from 0 to 8
[ 89.010167] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev ?, type ?) errno=-22
[ 89.010487] SELinux: security_context_str_to_sid(system_u:object_r:snappy_snap_t:s0) failed for (dev loop0, type squashfs) errno=-22
[ 90.543760] audit: type=1400 audit(1721041727.627:14): avc: denied { entrypoint } for pid=2461 comm="dpkg" path="/var/lib/dpkg/info/man-db.postinst" dev="mmcblk0p5" ino=185094 scontext=system_u:system_r:dpkg_script_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=file permissive=1
[ 91.561867] audit: type=1107 audit(1721041728.643:15): pid=904 uid=101 auid=4294967295 ses=4294967295 subj=system_u:system_r:kernel_t:s0 msg='avc: denied { send_msg } for msgtype=method_call interface=org.freedesktop.DBus member=Hello dest=org.freedesktop.DBus spid=2464 scontext=system_u:system_r:dpkg_script_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclass=dbus permissive=1 exe="/usr/bin/dbus-daemon" sauid=101 hostname=? addr=? terminal=?'
from fenix.
Indeed, just by restarting
snapd
, I get that error:
Tried restarting snapd, I don't get any errors
khadas@Khadas:~$ sudo systemctl status
[sudo] password for khadas:
sudo: a password is required
khadas@Khadas:~$ sudo systemctl status snapd
[sudo] password for khadas:
● snapd.service - Snap Daemon
Loaded: loaded (/usr/lib/systemd/system/snapd.service; enabled; preset: enabled)
Active: active (running) since Mon 2024-07-15 11:07:30 UTC; 1min 11s ago
TriggeredBy: ● snapd.socket
Main PID: 613285 (snapd)
Tasks: 15 (limit: 7100)
Memory: 13.1M ()
CPU: 3.323s
CGroup: /system.slice/snapd.service
└─613285 /usr/lib/snapd/snapd
Jul 15 11:07:30 Khadas systemd[1]: Starting snapd.service - Snap Daemon...
Jul 15 11:07:30 Khadas snapd[613285]: overlord.go:271: Acquiring state lock file
Jul 15 11:07:30 Khadas snapd[613285]: overlord.go:276: Acquired state lock file
Jul 15 11:07:30 Khadas snapd[613285]: daemon.go:247: started snapd/2.63+24.04 (series 16; classic; devmode) ubuntu/24.04 (arm64) linux/5.15.119.
Jul 15 11:07:30 Khadas snapd[613285]: daemon.go:340: adjusting startup timeout by 45s (pessimistic estimate of 30s plus 5s per snap)
Jul 15 11:07:30 Khadas snapd[613285]: backends.go:58: AppArmor status: apparmor not enabled
Jul 15 11:07:30 Khadas systemd[1]: Started snapd.service - Snap Daemon.
from fenix.
@rNoz I am struggling to reproduce your issue. Have you made any other changes? Can you try once on a fresh installation?
from fenix.
I have no squashfs related errors. Mount command shows snaps mounted using squashfs as well.
khadas@Khadas:~$ mount | grep squash
/var/lib/snapd/snaps/snapd_21761.snap on /snap/snapd/21761 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/core20_2321.snap on /snap/core20/2321 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/microk8s_6893.snap on /snap/microk8s/6893 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/core22_1383.snap on /snap/core22/1383 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/bare_5.snap on /snap/bare/5 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/chromium_2899.snap on /snap/chromium/2899 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/cups_1059.snap on /snap/cups/1059 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/gtk-common-themes_1535.snap on /snap/gtk-common-themes/1535 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
/var/lib/snapd/snaps/gnome-42-2204_178.snap on /snap/gnome-42-2204/178 type squashfs (ro,nodev,relatime,errors=continue,x-gdu.hide,x-gvfs-hide)
from fenix.
I would like to solve this because I got this trouble in the past and a fresh install solved it, but with just a few weeks working with it, it appeared again. So, I would prefer to fix it, as it is not possible to always do a fresh install to fix it. In that case I won't use snapd, since it is unstable in khadas.
Any idea how can I go for it?
It is definitely related with SELinux and a kernel version.
# edit /etc/selinux/config
SELINUX=disabled
Then reboot.
Now it works. No errors. microk8s
is installed and works successfully.
So, any idea how we can make snapd working with the default policy for selinux (SELINUX=permissive) for khadas vim3/4?
from fenix.
So, any idea how we can make snapd working with the default policy for selinux (SELINUX=permissive) for khadas vim3/4?
AFAIK, there is no selinux policy installed by default and with permissive selinux should allow everything. I do find it weird that selinux is enabled as default security on ubuntu image. I only see libselinux installed on my system in the dpkg output. Have you installed any other selinux packages?
from fenix.
So, any idea how we can make snapd working with the default policy for selinux (SELINUX=permissive) for khadas vim3/4?
Either you can add the required policy using audit2allow. Or if selinux is not required add security=apparmor to your kernel commandline to switch to using apparmor by default
from fenix.
Btw, I needed to add the module xt_multiport
to make ingress
work with microk8s
.
from fenix.
Btw, I needed to add the module
xt_multiport
to makeingress
work withmicrok8s
.
Thanks for reporting. I will add that to the list.
from fenix.
Related Issues (20)
- Make fails with Error: cp: cannot stat 'bionic/arm64/VIM4/*': No such file or directory
- linux upstream
- RuntimeError: Bad magic number in .pyc file HOT 1
- u-boot OP-TEE error
- Building Ubuntu 22.04 for Edge 2 cannot find mpp-rockchip / rga-rockchip HOT 1
- VIM3 linux-mainline-6.0 build failure HOT 3
- fatal error: uapi/gpu/arm/midgard/mali_base_kernel.h: No such file or directory HOT 1
- aarch64-none-linux-gnu-gcc unable to compile float expressions
- VIM1 images for Ubuntu 22.04 and Debian 11 won't boot HOT 5
- Typo in the VIM1S chipset model HOT 1
- Personalizatiion of S905Y4
- usbmon kernel option not enabled on VIM4 Debian11 HOT 2
- Strange kernel-config menus bug
- VIM4 missing all OPTEE utlities on Debian11 HOT 1
- How to add Board fo S905X3 HOT 1
- Khadas Edge-V + TS050 + Linux does not work
- make linux for fenix sdk is error! HOT 8
- how to build for banana pi cm4? HOT 1
- Corrupt video stream for MIPI CSI2 (IMX415) when docker or podman is installed (Edge2) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fenix.