tkestack / galaxy Goto Github PK

View Code? Open in Web Editor NEW

108.0 14.0 39.0 34.32 MB

Providing high-performance network for Kubernetes

License: Other

Go 95.66% Shell 2.00% Dockerfile 0.43% Makefile 1.92%

cni-plugin kubernetes scheduler-plugin

galaxy's People

Contributors

Stargazers

Watchers

galaxy's Issues

quick start failed

k8s version:

# kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:09:25Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:00:47Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

galaxy version: v1.0.7
flannel version: v0.13.0

following the guide, just change the command to private-cloud .
The logs as follow:

I0113 17:00:58.195570   11205 flags.go:52] FLAG: --add-dir-header="false"
I0113 17:00:58.195915   11205 flags.go:52] FLAG: --alsologtostderr="false"
I0113 17:00:58.195923   11205 flags.go:52] FLAG: --bridge-nf-call-iptables="true"
I0113 17:00:58.195932   11205 flags.go:52] FLAG: --cni-paths="[/opt/cni/galaxy/bin]"
I0113 17:00:58.195943   11205 flags.go:52] FLAG: --flannel-allocated-ip-dir="/var/lib/cni/networks,/var/lib/cni/networks/galaxy-flannel"
I0113 17:00:58.195950   11205 flags.go:52] FLAG: --flannel-gc-interval="10s"
I0113 17:00:58.195955   11205 flags.go:52] FLAG: --gc-dirs="/var/lib/cni/flannel,/var/lib/cni/galaxy,/var/lib/cni/galaxy/port"
I0113 17:00:58.195962   11205 flags.go:52] FLAG: --hostname-override=""
I0113 17:00:58.195966   11205 flags.go:52] FLAG: --ip-forward="true"
I0113 17:00:58.195971   11205 flags.go:52] FLAG: --json-config-path="/etc/galaxy/galaxy.json"
I0113 17:00:58.195978   11205 flags.go:52] FLAG: --kubeconfig=""
I0113 17:00:58.195983   11205 flags.go:52] FLAG: --log-backtrace-at=":0"
I0113 17:00:58.195991   11205 flags.go:52] FLAG: --log-dir=""
I0113 17:00:58.195996   11205 flags.go:52] FLAG: --log-file=""
I0113 17:00:58.196001   11205 flags.go:52] FLAG: --log-file-max-size="1800"
I0113 17:00:58.196006   11205 flags.go:52] FLAG: --log-flush-frequency="5s"
I0113 17:00:58.196030   11205 flags.go:52] FLAG: --logtostderr="true"
I0113 17:00:58.196044   11205 flags.go:52] FLAG: --master=""
I0113 17:00:58.196051   11205 flags.go:52] FLAG: --network-conf-dir="/etc/cni/net.d/"
I0113 17:00:58.196056   11205 flags.go:52] FLAG: --network-policy="false"
I0113 17:00:58.196061   11205 flags.go:52] FLAG: --pprof="false"
I0113 17:00:58.196066   11205 flags.go:52] FLAG: --route-eni="false"
I0113 17:00:58.196071   11205 flags.go:52] FLAG: --skip-headers="false"
I0113 17:00:58.196076   11205 flags.go:52] FLAG: --skip-log-headers="false"
I0113 17:00:58.196081   11205 flags.go:52] FLAG: --stderrthreshold="2"
I0113 17:00:58.196085   11205 flags.go:52] FLAG: --v="3"
I0113 17:00:58.196090   11205 flags.go:52] FLAG: --version="false"
I0113 17:00:58.196095   11205 flags.go:52] FLAG: --vmodule=""
I0113 17:00:58.196259   11205 galaxy.go:77] Json Config: {
  "NetworkConf":[
    {"name":"tke-route-eni","type":"tke-route-eni","eni":"eth1","routeTable":1},
    {"name":"galaxy-flannel","type":"galaxy-flannel", "delegate":{"type":"galaxy-veth"},"subnetFile":"/run/flannel/subnet.env"},
    {"name":"galaxy-k8s-vlan","type":"galaxy-k8s-vlan", "device":"eth1", "default_bridge_name": "br0"},
    {"name":"galaxy-k8s-sriov","type": "galaxy-k8s-sriov", "device": "eth1", "vf_num": 10}
  ],
  "DefaultNetworks": ["galaxy-flannel"],
  "ENIIPNetwork": "galaxy-k8s-vlan"
}
I0113 17:00:58.198627   11205 iptables.go:218] Could not connect to D-Bus system bus: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory
W0113 17:00:58.198661   11205 client_config.go:541] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0113 17:00:58.198876   11205 galaxy.go:159] QPS: 1.000000e+03, Burst: 2000
I0113 17:00:58.200330   11205 galaxy.go:165] apiserver address https://172.31.0.1:443
I0113 17:00:58.258646   11205 portmapping.go:122] listening to tcp 10027
I0113 17:00:58.258666   11205 portmapping.go:138] Opened local port tcp:10027

And when creating a test pod, it failed. using the describe command:

  Warning  FailedCreatePodSandBox  20m                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "823ddaf206a4e7dc81a8c978d19ca02dc509cffadc05db016f73886eab88c55b" network for pod "test-metallb-dpl-7fb5cc5679-hnqkq": networkPlugin cni failed to set up pod "test-metallb-dpl-7fb5cc5679-hnqkq_default" network: missing network name:
  Normal   SandboxChanged          10m (x252 over 20m)  kubelet            Pod sandbox changed, it will be killed and re-created.
  Warning  FailedCreatePodSandBox  49s (x506 over 20m)  kubelet            (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "eef210c085fb8cde7397612098c735a57ea20122f5039a488744b32ead1a66f7" network for pod "test-metallb-dpl-7fb5cc5679-hnqkq": networkPlugin cni failed to set up pod "test-metallb-dpl-7fb5cc5679-hnqkq_default" network: missing network name:

Remove database ipam

Database ipam was implemented when k8s doesn't support CRD. Now that CRD ipam is stable enough, we can remove database ipam now.

change log level if bridge-nf-call-iptables module is not installed

Error set kernel args /proc/sys/net/bridge/bridge-nf-call-iptables: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory

galaxy not work as expected

When I test the networkpolicy function with galaxy，I encountered some problem. Have I used the wrong image or wrong way to test it. Pls do me a favor

Software version：
galaxy 1.0.2
k8s v1.14.6
docker 18.09.9

Actions:
[root@9 /install/policy]# kubectl -n mesh-demo get pods
NAME READY STATUS RESTARTS AGE
consumer-1-1-0-787888cb94-gzxvn 1/1 Running 0 14h
provider-1-1-0-bbbb9cd65-95wxc 1/1 Running 0 14s

[root@9 /install/policy]# cat provider_ingress.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: provider-network-policy
namespace: mesh-demo
spec:
podSelector:
matchLabels:
tcnp-service-runtime: provider
tcnp-service-version: 1.1.0
policyTypes:

Ingress
ingress:
from:
- namespaceSelector:
  matchLabels:
  a: b
  [root@9 /install/policy]# kubectl -n mesh-demo apply -f provider_ingress.yaml
  networkpolicy.networking.k8s.io/provider-network-policy created

Problem description:
When I use components described above to launch a tke cluster for business-purpose，according to the instroductions（https://github.com/tkestack/galaxy/blob/master/doc/network-policy.md）.Some problem just happen.

1 The galaxy image does not have ipset and its dependencies installed. When open the networkpolicy switch in launching command, it printed out the following error:

running command:
[root@9 ~]# kubectl -n kube-system exec -it galaxy-daemonset-
galaxy-daemonset-5qxps galaxy-daemonset-rbxll
[root@9 ~]# kubectl -n kube-system exec galaxy-daemonset-5qxps -- ps -ef|grep galaxy
root 6840 3279 0 03:43 ? 00:00:00 kubectl -n kube-system exec galaxy-daemonset-5qxps -- ps -ef
root 6841 3279 0 03:43 ? 00:00:00 grep --color=auto galaxy
root 25740 25709 0 Mar01 ? 00:00:00 /bin/sh -c cp -p /etc/cni/net.d/00-galaxy.conf /host/etc/cni/net.d/; cp -p /opt/cni/bin/* /host/opt/cni/bin/; /usr/bin/galaxy --network-policy --logtostderr=true --v=5
root 25836 25740 0 Mar01 ? 00:02:39 /usr/bin/galaxy --network-policy --logtostderr=true --v=5

Err info:
02 03:51:30.927516 2493 policy.go:1035] failed to add entry 172.20.1.218 to ipset GLX-ip-24U5H3NXPZITELP7: error adding entry 172.20.1.218, error: executable file not found in $PATH

2、When opening networkpolicy switch，galaxy pod watched k8s networkpolicies and fetched relative business pods which matching label selectors described in networkpolicy manifests and hostName of the K8S node which the galaxy pod located on， for matching purpose。Which way is supposed to filter the pods by the HostName。Below is the code（https://github.com/tkestack/galaxy/blob/v1.0.2/pkg/policy/policy.go）：
`func (p *PolicyManager) syncPods() {
...
if p.podCachedInformer.HasSynced() {
...
nodeHostName := k8s.GetHostname()
glog.V(4).Infof("find %d pods, nodeHostName %s", len(pods), nodeHostName)
for i := range pods {
if pods[i].Spec.NodeName != nodeHostName {
continue
}
wg.Add(1)
glog.V(4).Infof("starting goroutine to sync pod chain for %s_%s", pods[i].Name, pods[i].Namespace)
go syncPodChains(pods[i])
}
} else {
...
}

func GetHostname() string {
hostname := *flagHostnameOverride
if hostname == "" {
hostname = os.Getenv("MY_NODE_NAME")
if hostname == "" {
nodename, err := os.Hostname()
if err != nil {
glog.Fatalf("Couldn't determine hostname: %v", err)
}
hostname = nodename
}
}
return strings.ToLower(strings.TrimSpace(hostname))
}`

In my environment，pods[i].Spec.NodeName是IP like:
root@9 /install/policy]# kubectl -n mesh-demo get pods provider-1-1-0-bbbb9cd65-95wxc -o yaml |grep -i nodename nodeName: 1.2.3.4

but k8s.GetHostname() is: vm_host_4
then the iptables chain like GLX-INGRESS/GLX-EGRESS/GLX-POD-XXXX will not be create because of pods[i].Spec.NodeName != nodeHostName

3 、when I used workaround method to ensure “pods[i].Spec.NodeName == nodeHostName”，then the iptables chains and rules will be properly set . The networkpolicy did take effect. But when I delete the business pod or networkpolicy resource， it seemed like the iptables chains and iptable rules is still there，didn't be sync to delete

Write a warning log if two different binaries of the same cni type exists both in /opt/cni/bin and /opt/cni/galaxy/bin/

if two different binaries of the same cni type exists both in /opt/cni/bin and /opt/cni/galaxy/bin/, galaxy invokes the first one to set up network. This may cause unexpected results if the first binary is an very old version and has some bugs.

Supporting ipv6

Vlan CNI switch=ipvlan ipvlan_mode=l3, l3s do not send gratuitous arp

in code

if d.IpVlanMode == "l3" || d.IpVlanMode == "l3s" {
		_ = utils.SendGratuitousARP(d.Device, result.IP4.IP.IP.String(), "", d.GratuitousArpRequest)
		return nil
	}

we want to send gratuitous arp.but we in code

if d.MacVlanMode() || d.IPVlanMode() {
		return nil
	}
	if d.PureMode() {
		if err := d.initPureModeArgs(); err != nil {
			return err
		}
		return utils.EnableNonlocalBind()
	}

wo did not set EnableNonlocalBind for ipvlan l3 l3s mode. and we ignore the utils.SendGratuitousARP error result.

Upgrade to scheduler framework

Maybe it's better to separate scheduler plugin to a new repo other than galaxy.

Auto reloading of galaxy etc config

Fix ldflags of galaxy and ipam binary

Version command returns incorrect versions currently.

$ bin/galaxy --version
Kubernetes v0.0.0-master+$Format:%h$

Revisit release ip api

The api is not atomic, race conditions may occur.

Network bandwidth control

Add bandwidth-limiting of pods's ingress/egress traffic.

Replace privileged securityContext of galaxy with network admin capability

Build error on "./hack/build.sh"

Describe the bug
I just follow the guide https://github.com/tkestack/galaxy/blob/master/doc/building.md to build binaries, but on command ./hash/build.sh getting errors:

root@d49c9bf924fe:/go/src/tkestack.io/galaxy# ./hack/build.sh
Building galaxy
go build -o bin/galaxy tkestack.io/galaxy/cmd/galaxy
# tkestack.io/galaxy/pkg/utils/iptables
pkg/utils/iptables/iptables_unsupported.go:42:6: grabIptablesLocks redeclared in this block
        previous declaration at pkg/utils/iptables/iptables_linux.go:68:62
pkg/utils/iptables/iptables_unsupported.go:46:6: grabIptablesFileLock redeclared in this block
        previous declaration at pkg/utils/iptables/iptables_linux.go:115:39

Environment

OS: Linux VM_149_11_centos 3.10.107-1-tlinux2_kvm_guest-0049 #1 SMP Tue Jul 30 23:46:29 CST 2019 x86_64 x86_64 x86_64 GNU/Linux
golang: go version go1.12.4 linux/amd64

option `--flannel-allocated-ip-dir` should be set to `/var/lib/cni/networks/galaxy-flannel` by default

Clean nodeSubnet cache if floatingip configuration updated

Plugin searches node's subnet by floatingip configuration. If we changed routableSubnet of floatingip configuration, then node's subnet may change, so we'd better clean nodeSubnet cache if floatingip configuration updated.

Avoid allocating ips that are used by other cloud systems

Such as cvm, clb, cdb..

Separate ipvlan/macvlan/pure/bridge CNI plugin

Those plugins are currently mixing in
https://github.com/tkestack/galaxy/blob/master/cni/k8s-vlan/k8s_vlan.go which makes them hard to maintain or to add new feature.

It's better to deprecate the old one first and adding each of ipvlan/macvlan/pure/bridge as a new plugin. We can eventually remove the old one when we have all clusters upgraded at future.

Support for dynamic network config

This makes running galaxy peacefully with other CNI plugins.

Remove subnet from floatingip spec

Subnet is not used currently and can be quite huge on qcloud cluster.

Enable recreated ENI without restart

Fix random port mapping

Galaxy supports mapping container ports to random host ports (just like what docker run -P does).
We used to hack kubelet to send all container ports including those with hostport=0 to CNI plugin. We should stop hacking and in stead getting those ports from podspec.

This is where galaxy gets container ports from kubelet now https://github.com/tkestack/galaxy/blob/master/pkg/api/galaxy/galaxy.go#L121-L127

This is where kubelet forbids passing hostport=0 container ports to CNI plugin https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/dockershim/network/cni/cni.go#L414-L416

Expose important metrics

Such as total num of fips, allocated fip, filter duration, bind duration, cloud provider assign IP successful count, unassign IP successful count.

galaxy-k8s-vlan ipvlan mode not work

my test yaml:

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: common-nginx
  labels:
    app: common-nginx
spec:
  replicas: 6
  selector:
    matchLabels:
      app: common-nginx
  template:
    metadata:
      name: common-nginx
      labels:
        app: common-nginx
      annotations:
        k8s.v1.cni.cncf.io/networks: "galaxy-k8s-vlan"
    spec:
      containers:
      - name: nginx
        image: registry.tcnp.com/library/nginx
        resources:
          requests:
            tke.cloud.tencent.com/eni-ip: "1"
          limits:
            tke.cloud.tencent.com/eni-ip: "1"

my galaxy config

[root@localhost ~]# kubectl -n kube-system get cm galaxy-etc -o yaml
apiVersion: v1
data:
  galaxy.json: |
    {
      "NetworkConf":[
        {"name":"tke-route-eni","type":"tke-route-eni","eni":"eth1","routeTable":1},
        {"name":"galaxy-flannel","type":"galaxy-flannel", "delegate":{"type":"galaxy-veth"},"subnetFile":"/run/flannel/subnet.env"},
        {"name":"galaxy-k8s-vlan","type":"galaxy-k8s-vlan", "device":"eth0", "switch":"ipvlan"},
        {"name":"galaxy-k8s-sriov","type": "galaxy-k8s-sriov", "device": "eth0", "vf_num": 10}
      ],
      "DefaultNetworks": ["galaxy-flannel"],
      "ENIIPNetwork": "galaxy-k8s-vlan"
    }

my floatingip config

[root@localhost ~]# kubectl -n kube-system get cm floatingip-config  -o yaml
apiVersion: v1
data:
  floatingips: '[{"nodeSubnets":["192.168.104.0/24"],"ips":["192.168.104.130~192.168.104.180"],"subnet":"192.168.104.0/24","gateway":"192.168.104.254"}]'
kind: ConfigMap

pod already have ip in 192.168.104.0/24 cidr

[root@localhost ~]# kubectl get pod -o wide
NAME                 READY   STATUS    RESTARTS   AGE   IP                NODE              NOMINATED NODE   READINESS GATES
common-nginx-c7d8f   1/1     Running   0          13h   192.168.104.131   192.168.104.111   <none>           <none>
common-nginx-ftpcf   1/1     Running   0          13h   192.168.104.153   192.168.104.128   <none>           <none>
common-nginx-gk8ss   1/1     Running   0          13h   192.168.104.158   192.168.104.111   <none>           <none>
common-nginx-lwh2p   1/1     Running   0          13h   192.168.104.130   192.168.104.111   <none>           <none>
common-nginx-q8mq8   1/1     Running   0          13h   192.168.104.133   192.168.104.111   <none>           <none>
common-nginx-z85cj   1/1     Running   0          13h   192.168.104.142   192.168.104.111   <none>           <none>

my host node gateway is 192.168.104.254

[root@localhost ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.104.254 0.0.0.0         UG    0      0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.20.1.0      172.20.1.0      255.255.255.0   UG    0      0        0 flannel.1
172.20.2.0      172.20.2.0      255.255.255.0   UG    0      0        0 flannel.1
192.168.104.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0

use nsenter to enter pod common-nginx-q8mq8 namespace with ip 192.168.104.133, can't ping gateway 192.168.104.254 ok.

[root@localhost ~]# e common-nginx-q8mq8 default 
entering pod netns for default/common-nginx-q8mq8
nsenter -n --target 8733
[root@localhost ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.104.254 0.0.0.0         UG    0      0        0 eth0
192.168.104.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0
[root@localhost ~]# ping 192.168.104.254
PING 192.168.104.254 (192.168.104.254) 56(84) bytes of data.
^C
--- 192.168.104.254 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

[root@localhost ~]# ping -c 3 192.168.104.254
PING 192.168.104.254 (192.168.104.254) 56(84) bytes of data.


--- 192.168.104.254 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2000ms

OS version: CentOS Linux release 7.5.1804 (Core)
kernel version: 4.4.236-1.el7.elrepo.x86_64
k8s version: v1.18.3
galaxy version: v1.0.4
galaxy-ipam version: v1.0.4

Galaxy hangs

CPU 100%

galaxy-sdn hangs

One of galaxy threads calls sched_yield() forever

GDB

galaxy --version
go-version go1.14.1, git-commit ce883dce, build-time 2020-04-10T10:00:25Z

Seems due to golang bug golang/go#38023

Refactor multiple ips allocation

SecondIPAM is a bad design which is hard to extend to multiple ips and does not guarantee atomic allocation and releasing of multiple ips. Instead we should handle mutiple ips allocation all in a single IPAM.

The user interface should be carefully designed including how to specify the needs of multiple ips, each ip's allocation range and routes of multiple veth device.

release floating ips automatically if they are unused for a configured time

pod ip conflicted

kubernetes version: 1.16
galaxy version: v1.0.8
network type: galaxy-underlay-veth
The pods get IP, it will be conflicted Occasionally

vlan cni: move ip from vlan device to bridge

galaxy-veth creates veth but not work in arm64

Expected Behavior

pod can communicate with the host it exists.

Current Behavior

pod can't communiate with the host it exists.

# on the pod
[root@localhost data]# nsenter  -n -t 324701 ip nei
10.122.1.15 dev eth0 lladdr 26:b0:2e:9d:f5:bf STALE
169.254.1.1 dev eth0 lladdr 0e:ba:fd:2c:a4:4f PERMANENT
[root@localhost data]# nsenter  -n -t 324701 ip r
default via 169.254.1.1 dev eth0
169.254.1.1 dev eth0 scope link

# on the host
[root@localhost data]# ip r get 192.168.0.11
192.168.0.11 dev v-h6ea2d3176 src 10.122.1.15 uid 0
    cache

[root@localhost data]# ip a s v-h6ea2d3176
31: v-h6ea2d3176@if30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
    link/ether 26:b0:2e:9d:f5:bf brd ff:ff:ff:ff:ff:ff link-netnsid 2
    inet6 fe80::24b0:2eff:fe9d:f5bf/64 scope link
       valid_lft forever preferred_lft foreve

the pod's default gateway and the gateway's mac are added by galaxy-veth, but the problem is the gateway's mac should be the veth mac of host netns(v-h6ea2d3176).

Possible Solution

add delay(sleep) after https://github.com/tkestack/galaxy/blob/master/pkg/utils/utils.go#L243-L246
TODO.

Steps to Reproduce

kernel: 4.19.90-17.ky10.aarch64, arch: arm64
k8s 1.16.6, (tke install)
network: galaxy-sdn + galaxy-veth (0.4.0)
kubectl create pod

B: (iproute2 5.4.0)

ip monitor (foregroud)

ip l add veth-pba type veth peer name veth-pab
output:

 inet veth-pab forwarding on rp_filter strict mc_forwarding off proxy_neigh off ignore_routes_with_linkdown off
 inet6 veth-pab forwarding off mc_forwarding off proxy_neigh off ignore_routes_with_linkdown off
121: veth-pab@NONE: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default
  link/ether 62:37:44:12:b0:e5 brd ff:ff:ff:ff:ff:ff
 inet veth-pba forwarding on rp_filter strict mc_forwarding off proxy_neigh off ignore_routes_with_linkdown off
 inet6 veth-pba forwarding off mc_forwarding off proxy_neigh off ignore_routes_with_linkdown off
122: veth-pba@veth-pab: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN group default
   link/ether a2:ff:ed:39:22:4c brd ff:ff:ff:ff:ff:ff
121: veth-pab@veth-pba: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN group default
    link/ether 6a:2e:95:ff:9b:b0 brd ff:ff:ff:ff:ff:ff
122: veth-pba@veth-pab: <BROADCAST,MULTICAST,M-DOWN> mtu 1500 qdisc noop state DOWN group default
    link/ether e2:5c:00:33:cc:5e brd ff:ff:ff:ff:ff:ff

mac address flapping in the creating process

Context (Environment)

strace -f -s 100 ip l add veth-pba type veth peer name veth-pab

execve("/usr/sbin/ip", ["ip", "l", "add", "veth-p130", "type", "veth", "peer", "name", "veth-p013"], 0xffffd45c7580 /* 30 vars */) = 0
brk(NULL)                               = 0xaaae33200000
faccessat(AT_FDCWD, "/etc/ld.so.preload", R_OK) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/dev/cur_gl", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=106368, ...}) = 0
mmap(NULL, 106368, PROT_READ, MAP_PRIVATE, 3, 0) = 0xfffc48570000
close(3)                                = 0
openat(AT_FDCWD, "/lib64/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\260m\0\0\0\0\0\0@\0\0\0\0\0\0\0000\2\3\0\0\0\0\0\0\0\0\0@\08\0\10\0@\0\34\0\33\0\1\0\0\0\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20l\2\0\0\0\0\0\20l\2\0\0\0\0\0\0\0\1\0\0\0\0\0\1\0\0\0\6\0\0\0(\366\2\0\0\0\0\0(\366\3\0\0\0\0\0(\366\3\0\0\0\0\08\n\0\0\0\0\0\0`/\0\0\0\0\0\0\0\0\1\0\0\0\0\0\2\0\0\0\6\0\0\0\300\366\2\0\0\0\0\0\300\366\3\0\0\0\0\0\300\366\3\0\0\0\0\0000\2\0\0\0\0\0\0000\2\0\0\0\0\0\0\10\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0\0\2\0\0\0\0\0\0\0\2\0\0\0\0\0\0\0\2\0\0\0\0\0\0$\0\0\0\0\0\0\0$\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0\7\0\0\0\4\0\0\0(\366\2\0\0\0\0\0(\366\3\0\0\0\0\0(\366\3\0\0\0\0\0(\0\0\0\0\0\0\0\341\0\0\0\0\0\0\0\10\0\0\0\0\0\0\0P\345td\4\0\0\0\350\t\2\0\0\0\0\0\350\t\2\0\0\0\0\0\350\t\2\0\0\0\0\0D\v\0\0\0\0\0\0D\v\0\0\0\0\0\0\4\0\0\0\0\0\0\0Q\345td\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0R\345td\4\0\0\0(\366\2\0\0\0\0\0(\366\3\0\0\0\0\0(\366\3\0\0\0\0\0\330\t\0\0\0\0\0\0\330\t\0\0\0\0\0\0\1\0\0\0\0\0\0\0\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\352\365\353'\275\30\35\343\233\n\235\344c?\243\33\244J\313\313\0\0\0\0\305\0\0\0\222\0\0\0 \0\0\0\v\0\0\0\230 \4\201 L\0\3#\20\300\nT\210\357\322B\6h\10\24\26\0\6\20\4\4\0\0\311@\1P.\21\30\0&\th\30 \200*H\10\22\1\1,!\0\20\0\217\7\250\1@S\0\274\0\0\0R\20\236\4\1\5\24\33\21\33\214 \0\3C\10\1H\21\2\2\0\200j\17\304\1\2\210`\203\200\2*\200\202@\200\4\0\0C\30\4\0\20\200:\0!\246\10\5\0\206\0\4\0@\204\20\0@\n\0\200 $`0T\1@\304\0\20\0\240\24\20\234\10\200\0D\0\204\10@&\7\202\0\200\200\1\10\10\0\2\2\0\200\232\203\4\304\25\220(\2\200\0\10\0\201@\0\200\200\210@@\241\1\2\21\200@\24\200\4\221\313@ \10D\1\0\0\0\2F@\1\0A\200q\0@Eb\0\t\"\0\20@@Il\341\24\200<@!\0\0 \0\0\200\310\206\343@\200\23\0\3 @\2!\200T\2!\222\0\0\0\0\0\0\0", 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=198960, ...}) = 0
mmap(NULL, 271752, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffc48520000
mmap(0xfffc48550000, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x20000) = 0xfffc48550000
close(3)                                = 0
openat(AT_FDCWD, "/dev/cur_gl", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/lib64/libelf.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\0200\0\0\0\0\0\0@\0\0\0\0\0\0\0\340\1\2\0\0\0\0\0\0\0\0\0@\08\0\10\0@\0\34\0\33\0\1\0\0\0\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\240\202\1\0\0\0\0\0\240\202\1\0\0\0\0\0\0\0\1\0\0\0\0\0\1\0\0\0\6\0\0\0\20\372\1\0\0\0\0\0\20\372\2\0\0\0\0\0\20\372\2\0\0\0\0\0\370\5\0\0\0\0\0\0\10\6\0\0\0\0\0\0\0\0\1\0\0\0\0\0\2\0\0\0\6\0\0\0\320\373\1\0\0\0\0\0\320\373\2\0\0\0\0\0\320\373\2\0\0\0\0\0000\2\0\0\0\0\0\0000\2\0\0\0\0\0\0\10\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0\0\2\0\0\0\0\0\0\0\2\0\0\0\0\0\0\0\2\0\0\0\0\0\0$\0\0\0\0\0\0\0$\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0\7\0\0\0\4\0\0\0\20\372\1\0\0\0\0\0\20\372\2\0\0\0\0\0\20\372\2\0\0\0\0\0\0\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0P\345td\4\0\0\0\20E\1\0\0\0\0\0\20E\1\0\0\0\0\0\20E\1\0\0\0\0\0\244\6\0\0\0\0\0\0\244\6\0\0\0\0\0\0\4\0\0\0\0\0\0\0Q\345td\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0R\345td\4\0\0\0\20\372\1\0\0\0\0\0\20\372\2\0\0\0\0\0\20\372\2\0\0\0\0\0\360\5\0\0\0\0\0\0\360\5\0\0\0\0\0\0\1\0\0\0\0\0\0\0\4\0\0\0\24\0\0\0\3\0\0\0GNU\0:f[/\356\360\f\361C\327\257&\222D\361:\275\351\22M\0\0\0\0a\0\0\0003\0\0\0\20\0\0\0\n\0\0\0C\10\2\0\20\3\10\300\24\0\10\372\207\204\0\10\0\200\201\6\0\0\0\10\200\4\0\0000 \1\240`\20\264S\r\230\30\20\22\0`!\202\210D\n\200\240\300\316\300\201\22\3\10 E\1\310\300\2\2\2\v\21\200A\24\6\n\10\0\0\0\30\202\0 \0\202\200\200\261\1 \0(\1\1T\22H\3642B\20\22\241\10B\210A\0\200\0\200A\200\10\207\4\200\0A\200P\0\1A  \0\215UH\0243\0\0\0004\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0005\0\0\0007\0\0\08\0\0\0;\0\0\0\0\0\0\0>\0\0\0\0\0\0\0?\0\0\0\0\0\0\0\0\0\0\0A\0\0\0D\0\0\0F\0\0\0G\0\0\0\0\0\0\0H\0\0\0J\0\0\0L\0\0\0\0\0\0\0M\0\0\0O\0\0\0\0\0\0\0Q\0\0\0R\0\0\0T\0\0\0V\0\0\0W\0\0\0Y\0\0\0Z\0\0\0", 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=133344, ...}) = 0
mmap(NULL, 196632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffc484e0000
mmap(0xfffc48500000, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x10000) = 0xfffc48500000
close(3)                                = 0
openat(AT_FDCWD, "/dev/cur_gl", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/lib64/libmnl.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\260\34\0\0\0\0\0\0@\0\0\0\0\0\0\0\330\1\1\0\0\0\0\0\0\0\0\0@\08\0\7\0@\0\33\0\32\0\1\0\0\0\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\204A\0\0\0\0\0\0\204A\0\0\0\0\0\0\0\0\1\0\0\0\0\0\1\0\0\0\6\0\0\0\340\373\0\0\0\0\0\0\340\373\1\0\0\0\0\0\340\373\1\0\0\0\0\0(\4\0\0\0\0\0\0000\4\0\0\0\0\0\0\0\0\1\0\0\0\0\0\2\0\0\0\6\0\0\0p\374\0\0\0\0\0\0p\374\1\0\0\0\0\0p\374\1\0\0\0\0\0 \2\0\0\0\0\0\0 \2\0\0\0\0\0\0\10\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0$\0\0\0\0\0\0\0$\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0P\345td\4\0\0\0\0205\0\0\0\0\0\0\0205\0\0\0\0\0\0\0205\0\0\0\0\0\0t\2\0\0\0\0\0\0t\2\0\0\0\0\0\0\4\0\0\0\0\0\0\0Q\345td\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0R\345td\4\0\0\0\340\373\0\0\0\0\0\0\340\373\1\0\0\0\0\0\340\373\1\0\0\0\0\0 \4\0\0\0\0\0\0 \4\0\0\0\0\0\0\1\0\0\0\0\0\0\0\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\205\257\t\351\211\274Mn\24P\261\324\327|q{\243\177\22\325\0\0\0\0C\0\0\0\33\0\0\0\10\0\0\0\t\0\0\0  fPH\360@@\20\200\16\2\0\300E\0001\220\245h\0F\300\20\30\0\200\0@\0B\30\220\20(@\220\0\1\201@\202@\f\30\0@\30\205\21P\0@b\20@\203\244G/\7*\315A\33\0\0\0\0\0\0\0\35\0\0\0\36\0\0\0 \0\0\0!\0\0\0\"\0\0\0\0\0\0\0&\0\0\0'\0\0\0)\0\0\0*\0\0\0\0\0\0\0\0\0\0\0,\0\0\0\0\0\0\0-\0\0\0/\0\0\0\0\0\0\0\0\0\0\0000\0\0\0001\0\0\0\0\0\0\0\0\0\0\0002\0\0\0\0\0\0\0005\0\0\08\0\0\09\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0;\0\0\0<\0\0\0>\0\0\0\0\0\0\0?\0\0\0A\0\0\0\0\0\0\0\0\0\0\0B\0\0\0C\0\0\0D\0\0\0E\0\0\0H\0\0\0I\0\0\0J\0\0\0K\0\0\0\0\0\0\0M\0\0\0Q\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0R\0\0\0S\0\0\0T\0\0\0\0\0\0\0U\0\0\0\0\0\0\0W\0\0\0[\0\0\0]\0\0\0`\0\0\0", 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=67736, ...}) = 0
mmap(NULL, 131088, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffc484b0000
mmap(0xfffc484c0000, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xfffc484c0000
close(3)                                = 0
openat(AT_FDCWD, "/dev/cur_gl", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/lib64/libcap.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0P\26\0\0\0\0\0\0@\0\0\0\0\0\0\0\360\3\1\0\0\0\0\0\0\0\0\0@\08\0\7\0@\0\32\0\31\0\1\0\0\0\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0PB\0\0\0\0\0\0PB\0\0\0\0\0\0\0\0\1\0\0\0\0\0\1\0\0\0\6\0\0\0\300\375\0\0\0\0\0\0\300\375\1\0\0\0\0\0\300\375\1\0\0\0\0\0p\4\0\0\0\0\0\0x\4\0\0\0\0\0\0\0\0\1\0\0\0\0\0\2\0\0\0\6\0\0\0\320\375\0\0\0\0\0\0\320\375\1\0\0\0\0\0\320\375\1\0\0\0\0\0\340\1\0\0\0\0\0\0\340\1\0\0\0\0\0\0\10\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0$\0\0\0\0\0\0\0$\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0P\345td\4\0\0\0\2748\0\0\0\0\0\0\2748\0\0\0\0\0\0\2748\0\0\0\0\0\0L\1\0\0\0\0\0\0L\1\0\0\0\0\0\0\4\0\0\0\0\0\0\0Q\345td\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0R\345td\4\0\0\0\300\375\0\0\0\0\0\0\300\375\1\0\0\0\0\0\300\375\1\0\0\0\0\0@\2\0\0\0\0\0\0@\2\0\0\0\0\0\0\1\0\0\0\0\0\0\0\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\300\350\371oWH\245B\4\305Z\3169\257\356\372p@3\221\0\0\0\0\21\0\0\0 \0\0\0\4\0\0\0\10\0\0\0@\10$\201\30\20@\204\230\246\0(\212\0\30\244\20\202\220\4\0\210\233\0\n\0H\340\204P\2\220 \0\0\0\0\0\0\0\0\0\0\0\"\0\0\0&\0\0\0'\0\0\0(\0\0\0*\0\0\0,\0\0\0001\0\0\0006\0\0\08\0\0\0\0\0\0\0:\0\0\0;\0\0\0=\0\0\0@\0\0\0z\177\352\33E3vg\216\332\241]\nzqP\210q\356\262[\207i\224\301\277<\316\37\344l\224\30\277[\27\261D=\\t\343\324\225{a,\20\334\347\250\231\266\257\203\372\360\342kP\226t\207c\25\6\365\262`\n\352\262\302.\334:\226\327kPB\312*\362M\247:\352NI\2032\337\374~\25,\366-\20#\330\250\231}\335\201c\354\26\v\210#\322\201c\252T\275\331\322^3\20\377\356\311x\261s\16\210\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\t\0(\24\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\25\0\370\0\2\0\0\0\0\0\0\0\0\0\0\0\0\0r\0\0\0\22\0\0\0\0\0\0\0\0\0\0\0", 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=68208, ...}) = 0
mmap(NULL, 131640, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffc48480000
mmap(0xfffc48490000, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xfffc48490000
close(3)                                = 0
openat(AT_FDCWD, "/dev/cur_gl", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\0\21\0\0\0\0\0\0@\0\0\0\0\0\0\0\330\1\1\0\0\0\0\0\0\0\0\0@\08\0\7\0@\0\34\0\33\0\1\0\0\0\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0@%\0\0\0\0\0\0@%\0\0\0\0\0\0\0\0\1\0\0\0\0\0\1\0\0\0\6\0\0\0\210\374\0\0\0\0\0\0\210\374\1\0\0\0\0\0\210\374\1\0\0\0\0\0\200\3\0\0\0\0\0\0\330\3\0\0\0\0\0\0\0\0\1\0\0\0\0\0\2\0\0\0\6\0\0\0\250\374\0\0\0\0\0\0\250\374\1\0\0\0\0\0\250\374\1\0\0\0\0\0000\2\0\0\0\0\0\0000\2\0\0\0\0\0\0\10\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0D\0\0\0\0\0\0\0D\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0P\345td\4\0\0\0\350\37\0\0\0\0\0\0\350\37\0\0\0\0\0\0\350\37\0\0\0\0\0\0\344\0\0\0\0\0\0\0\344\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0Q\345td\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0R\345td\4\0\0\0\210\374\0\0\0\0\0\0\210\374\1\0\0\0\0\0\210\374\1\0\0\0\0\0x\3\0\0\0\0\0\0x\3\0\0\0\0\0\0\1\0\0\0\0\0\0\0\4\0\0\0\24\0\0\0\3\0\0\0GNU\0]\375\236\"\352\256$\367\300\0\26+^\325\334\237\364P\3\213\4\0\0\0\20\0\0\0\1\0\0\0GNU\0\0\0\0\0\3\0\0\0\7\0\0\0\0\0\0\0\0\0\0\0?\0\0\0-\0\0\0\0\0\0\0\30\0\0\0\26\0\0\0%\0\0\0\0\0\0\0\0\0\0\0)\0\0\0\23\0\0\0\0\0\0\0!\0\0\0\"\0\0\0\0\0\0\0\10\0\0\0\37\0\0\0\27\0\0\0\36\0\0\0\t\0\0\0\0\0\0\0\7\0\0\0\0\0\0\0\22\0\0\0\0\0\0\0\35\0\0\0\32\0\0\0(\0\0\0\0\0\0\0$\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\31\0\0\0\0\0\0\0\24\0\0\0\0\0\0\0\f\0\0\0\0\0\0\0&\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\16\0\0\0\25\0\0\0'\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\4\0\0\0#\0\0\0\r\0\0\0\33\0\0\0\0\0\0\0\0\0\0\0,\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\n\0\0\0\0\0\0\0\v\0\0\0\17\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=67800, ...}) = 0
mmap(NULL, 131168, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffc48450000
mmap(0xfffc48460000, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xfffc48460000
close(3)                                = 0
openat(AT_FDCWD, "/dev/cur_gl", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0 A\2\0\0\0\0\0@\0\0\0\0\0\0\0\350L\27\0\0\0\0\0\0\0\0\0@\08\0\n\0@\0L\0K\0\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0000\2\0\0\0\0\0\0000\2\0\0\0\0\0\0\10\0\0\0\0\0\0\0\3\0\0\0\4\0\0\0\210\237\23\0\0\0\0\0\210\237\23\0\0\0\0\0\210\237\23\0\0\0\0\0\33\0\0\0\0\0\0\0\33\0\0\0\0\0\0\0\10\0\0\0\0\0\0\0\1\0\0\0\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0`a\26\0\0\0\0\0`a\26\0\0\0\0\0\0\0\1\0\0\0\0\0\1\0\0\0\6\0\0\0\10\327\26\0\0\0\0\0\10\327\27\0\0\0\0\0\10\327\27\0\0\0\0\08L\0\0\0\0\0\0\20\206\0\0\0\0\0\0\0\0\1\0\0\0\0\0\2\0\0\0\6\0\0\0\250\372\26\0\0\0\0\0\250\372\27\0\0\0\0\0\250\372\27\0\0\0\0\0\360\1\0\0\0\0\0\0\360\1\0\0\0\0\0\0\10\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0p\2\0\0\0\0\0\0p\2\0\0\0\0\0\0p\2\0\0\0\0\0\0D\0\0\0\0\0\0\0D\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0\7\0\0\0\4\0\0\0\10\327\26\0\0\0\0\0\10\327\27\0\0\0\0\0\10\327\27\0\0\0\0\0\20\0\0\0\0\0\0\0\220\0\0\0\0\0\0\0\10\0\0\0\0\0\0\0P\345td\4\0\0\0\244\237\23\0\0\0\0\0\244\237\23\0\0\0\0\0\244\237\23\0\0\0\0\0\314X\0\0\0\0\0\0\314X\0\0\0\0\0\0\4\0\0\0\0\0\0\0Q\345td\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0R\345td\4\0\0\0\10\327\26\0\0\0\0\0\10\327\27\0\0\0\0\0\10\327\27\0\0\0\0\0\370(\0\0\0\0\0\0\370(\0\0\0\0\0\0\1\0\0\0\0\0\0\0\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\334=\347%8\227WC\33\234\6}=\305-\250\344ct\222\4\0\0\0\20\0\0\0\1\0\0\0GNU\0\0\0\0\0\3\0\0\0\7\0\0\0\0\0\0\0\0\0\0\0\371\3\0\0\371\10\0\0\352\1\0\0(\3\0\0\273\10\0\0)\4\0\0\302\5\0\0\332\2\0\0\0\0\0\0\267\0\0\0\274\1\0\0\217\6\0\0\255\0\0\0@\4\0\0\"\1\0\0\204\4\0\0\235\1\0\0\6\7\0\0\346\1\0\0\352\3\0\0\246\7\0\0\22\6\0\0\214\7\0\0a\5\0\0\0\0\0\0h\5\0\0}\7\0\0\0\0\0\0\252\7\0\0\367\5\0\0\345\6\0\0\n\4\0\0\362\3\0\0\32\6\0\0", 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1531880, ...}) = 0
mmap(NULL, 1596696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffc482c0000
mmap(0xfffc48430000, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x160000) = 0xfffc48430000
close(3)                                = 0
openat(AT_FDCWD, "/dev/cur_gl", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/lib64/libsecurity.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\240\7\0\0\0\0\0\0@\0\0\0\0\0\0\0\310\1\1\0\0\0\0\0\0\0\0\0@\08\0\7\0@\0\31\0\30\0\1\0\0\0\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0X\f\0\0\0\0\0\0X\f\0\0\0\0\0\0\0\0\1\0\0\0\0\0\1\0\0\0\6\0\0\0X\375\0\0\0\0\0\0X\375\1\0\0\0\0\0X\375\1\0\0\0\0\0\260\2\0\0\0\0\0\0\270\2\0\0\0\0\0\0\0\0\1\0\0\0\0\0\2\0\0\0\6\0\0\0h\375\0\0\0\0\0\0h\375\1\0\0\0\0\0h\375\1\0\0\0\0\0\0\2\0\0\0\0\0\0\0\2\0\0\0\0\0\0\10\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0$\0\0\0\0\0\0\0$\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0P\345td\4\0\0\0\24\v\0\0\0\0\0\0\24\v\0\0\0\0\0\0\24\v\0\0\0\0\0\0<\0\0\0\0\0\0\0<\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0Q\345td\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0R\345td\4\0\0\0X\375\0\0\0\0\0\0X\375\1\0\0\0\0\0X\375\1\0\0\0\0\0\250\2\0\0\0\0\0\0\250\2\0\0\0\0\0\0\1\0\0\0\0\0\0\0\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\16\257\226]g\0345\6\322>\257\246\214\344G\253f\\Ve\0\0\0\0\2\0\0\0\20\0\0\0\1\0\0\0\6\0\0\0\0\0\0\0\0 \0\4\0\0\0\0\20\0\0\0\255^\333_\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\t\0\330\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\3\0\24\0\0\0\2\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0 \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0F\0\0\0\"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0t\0\0\0\21\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\232\0\0\0\22\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\243\0\0\0\22\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\1\0\0\0 \0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0U\0\0\0\21\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\314\0\0\0\22\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0{\0\0\0\22\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0m\0\0\0\22\0\0\0", 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=67592, ...}) = 0
mmap(NULL, 131088, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffc48290000
mmap(0xfffc482a0000, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xfffc482a0000
close(3)                                = 0
openat(AT_FDCWD, "/dev/cur_gl", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/lib64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\200#\0\0\0\0\0\0@\0\0\0\0\0\0\0000\3\10\0\0\0\0\0\0\0\0\0@\08\0\7\0@\0\32\0\31\0\1\0\0\0\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0(\272\7\0\0\0\0\0(\272\7\0\0\0\0\0\0\0\1\0\0\0\0\0\1\0\0\0\6\0\0\0\340\372\7\0\0\0\0\0\340\372\10\0\0\0\0\0\340\372\10\0\0\0\0\0\210\6\0\0\0\0\0\0\350\6\0\0\0\0\0\0\0\0\1\0\0\0\0\0\2\0\0\0\6\0\0\0\270\373\7\0\0\0\0\0\270\373\10\0\0\0\0\0\270\373\10\0\0\0\0\0\20\2\0\0\0\0\0\0\20\2\0\0\0\0\0\0\10\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0$\0\0\0\0\0\0\0$\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0P\345td\4\0\0\0Xh\7\0\0\0\0\0Xh\7\0\0\0\0\0Xh\7\0\0\0\0\0L\7\0\0\0\0\0\0L\7\0\0\0\0\0\0\4\0\0\0\0\0\0\0Q\345td\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0R\345td\4\0\0\0\340\372\7\0\0\0\0\0\340\372\10\0\0\0\0\0\340\372\10\0\0\0\0\0 \5\0\0\0\0\0\0 \5\0\0\0\0\0\0\1\0\0\0\0\0\0\0\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\331\\\211\253\213R\370q\301\2320\220\"\210(\v\2517_\334\0\0\0\0C\0\0\0\34\0\0\0\10\0\0\0\t\0\0\0#&`tELfQ\3D\6\0\245\244\25\252\2\2\0\3@\0\0\0\30\22\201\210\0\200H\0\"\0020\20@\2\30\0t\20AA(SB\201\0 \31\244\222\2@T\18\314\0217\4\1&\34\0\0\0\35\0\0\0\37\0\0\0!\0\0\0#\0\0\0\0\0\0\0\0\0\0\0%\0\0\0\0\0\0\0'\0\0\0)\0\0\0\0\0\0\0*\0\0\0\0\0\0\0+\0\0\0\0\0\0\0,\0\0\0-\0\0\0.\0\0\0/\0\0\0000\0\0\0001\0\0\0\0\0\0\0003\0\0\0005\0\0\0\0\0\0\0\0\0\0\0006\0\0\0\0\0\0\0007\0\0\08\0\0\09\0\0\0:\0\0\0<\0\0\0>\0\0\0?\0\0\0@\0\0\0B\0\0\0C\0\0\0E\0\0\0F\0\0\0G\0\0\0\0\0\0\0J\0\0\0\0\0\0\0M\0\0\0N\0\0\0\0\0\0\0Q\0\0\0R\0\0\0\0\0\0\0S\0\0\0T\0\0\0V\0\0\0W\0\0\0X\0\0\0Y\0\0\0Z\0\0\0\\\0\0\0^\0\0\0`\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0", 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=526768, ...}) = 0
mmap(NULL, 590280, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffc481f0000
mmap(0xfffc48270000, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x70000) = 0xfffc48270000
close(3)                                = 0
openat(AT_FDCWD, "/dev/cur_gl", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/lib64/libz.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\340$\0\0\0\0\0\0@\0\0\0\0\0\0\0\330\1\2\0\0\0\0\0\0\0\0\0@\08\0\7\0@\0\33\0\32\0\1\0\0\0\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0000\256\1\0\0\0\0\0000\256\1\0\0\0\0\0\0\0\1\0\0\0\0\0\1\0\0\0\6\0\0\0\260\372\1\0\0\0\0\0\260\372\2\0\0\0\0\0\260\372\2\0\0\0\0\0X\5\0\0\0\0\0\0`\5\0\0\0\0\0\0\0\0\1\0\0\0\0\0\2\0\0\0\6\0\0\0\20\374\1\0\0\0\0\0\20\374\2\0\0\0\0\0\20\374\2\0\0\0\0\0 \2\0\0\0\0\0\0 \2\0\0\0\0\0\0\10\0\0\0\0\0\0\0\4\0\0\0\4\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0\310\1\0\0\0\0\0\0$\0\0\0\0\0\0\0$\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0P\345td\4\0\0\0\340\206\1\0\0\0\0\0\340\206\1\0\0\0\0\0\340\206\1\0\0\0\0\0\304\3\0\0\0\0\0\0\304\3\0\0\0\0\0\0\4\0\0\0\0\0\0\0Q\345td\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0R\345td\4\0\0\0\260\372\1\0\0\0\0\0\260\372\2\0\0\0\0\0\260\372\2\0\0\0\0\0P\5\0\0\0\0\0\0P\5\0\0\0\0\0\0\1\0\0\0\0\0\0\0\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\271\263\36/[g\360'1\327<\234rDa\26b\0E\206\0\0\0\0a\0\0\0\31\0\0\0\20\0\0\0\n\0\0\0\0\"@b\f\3\2\0E\0\2A\10\4%\22\4\1\4\4\0\6\10\0\4\1\26\n\0\21\2\0\0A\0\224\204\204\10\10\0l\0\24 \260\204\200\0\2603@\t\360d\200\2B\206\0\0\0\4\20\0\1 \24\0\2\2\0\10\0#\n\2\10\7 \f\0\4\340\1\0 \0 \0H!\22\35\230\221\0/12\1\6\0\10\4\0\0\0\200!\2\n\200\2\0\10 G\0\1Q$\0\0\2\202\0\0\0\0\0\0\31\0\0\0\34\0\0\0\35\0\0\0\0\0\0\0\37\0\0\0\"\0\0\0$\0\0\0%\0\0\0)\0\0\0\0\0\0\0\0\0\0\0*\0\0\0,\0\0\0-\0\0\0\0\0\0\0\0\0\0\0.\0\0\0000\0\0\0004\0\0\0005\0\0\0\0\0\0\0007\0\0\0\0\0\0\0\0\0\0\0\0\0\0\08\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\09\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0:\0\0\0<\0\0\0\0\0\0\0\0\0\0\0>\0\0\0?\0\0\0@\0\0\0\0\0\0\0\0\0\0\0A\0\0\0\0\0\0\0", 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=133272, ...}) = 0
mmap(NULL, 196624, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffc481b0000
mmap(0xfffc481d0000, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x10000) = 0xfffc481d0000
close(3)                                = 0
openat(AT_FDCWD, "/dev/cur_gl", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
openat(AT_FDCWD, "/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\08t\0\0\0\0\0\0@\0\0\0\0\0\0\0\340\f\2\0\0\0\0\0\0\0\0\0@\08\0\t\0@\0\"\0!\0\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0\370\1\0\0\0\0\0\0\370\1\0\0\0\0\0\0\10\0\0\0\0\0\0\0\3\0\0\0\4\0\0\0Pe\1\0\0\0\0\0Pe\1\0\0\0\0\0Pe\1\0\0\0\0\0\33\0\0\0\0\0\0\0\33\0\0\0\0\0\0\0\10\0\0\0\0\0\0\0\1\0\0\0\5\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\370\240\1\0\0\0\0\0\370\240\1\0\0\0\0\0\0\0\1\0\0\0\0\0\1\0\0\0\6\0\0\0\340\370\1\0\0\0\0\0\340\370\2\0\0\0\0\0\340\370\2\0\0\0\0\0X\7\0\0\0\0\0\0xH\0\0\0\0\0\0\0\0\1\0\0\0\0\0\2\0\0\0\6\0\0\0\220\372\1\0\0\0\0\0\220\372\2\0\0\0\0\0\220\372\2\0\0\0\0\0000\2\0\0\0\0\0\0000\2\0\0\0\0\0\0\10\0\0\0\0\0\0\0\4\0\0\0\4\0\0\08\2\0\0\0\0\0\08\2\0\0\0\0\0\08\2\0\0\0\0\0\0D\0\0\0\0\0\0\0D\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0P\345td\4\0\0\0le\1\0\0\0\0\0le\1\0\0\0\0\0le\1\0\0\0\0\0\324\10\0\0\0\0\0\0\324\10\0\0\0\0\0\0\4\0\0\0\0\0\0\0Q\345td\6\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\0\0\0\0\0\0R\345td\4\0\0\0\340\370\1\0\0\0\0\0\340\370\2\0\0\0\0\0\340\370\2\0\0\0\0\0 \7\0\0\0\0\0\0 \7\0\0\0\0\0\0\1\0\0\0\0\0\0\0\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\3)\354\364\355\24c\244\324\375*\261\240%~\205\340\210\27G\4\0\0\0\20\0\0\0\1\0\0\0GNU\0\0\0\0\0\3\0\0\0\7\0\0\0\0\0\0\0\0\0\0\0\236\2\0\0d\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0z\0\0\0\220\0\0\0\230\0\0\0\0\0\0\0\0\0\0\0D\0\0\0\0\0\0\0O\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0[\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\333\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0P\1\0\0\0\0\0\0\0\0\0\0\22\0\0\0\0\0\0\0\0\0\0\0\365\0\0\0\0\0\0\0\326\0\0\0\0\0\0\0!\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0b\0\0\0/\1\0\0\0\0\0\0\22\1\0\0\316\0\0\0", 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=136544, ...}) = 0
mmap(NULL, 213336, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xfffc48170000
mmap(0xfffc48190000, 131072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x10000) = 0xfffc48190000
close(3)                                = 0
mprotect(0xfffc48430000, 65536, PROT_READ) = 0
mprotect(0xfffc48190000, 65536, PROT_READ) = 0
mprotect(0xfffc481d0000, 65536, PROT_READ) = 0
mprotect(0xfffc48270000, 65536, PROT_READ) = 0
mprotect(0xfffc482a0000, 65536, PROT_READ) = 0
mprotect(0xfffc48460000, 65536, PROT_READ) = 0
mprotect(0xfffc48490000, 65536, PROT_READ) = 0
mprotect(0xfffc484c0000, 65536, PROT_READ) = 0
mprotect(0xfffc48500000, 65536, PROT_READ) = 0
mprotect(0xfffc48550000, 65536, PROT_READ) = 0
mprotect(0xaaadfbff0000, 65536, PROT_READ) = 0
mprotect(0xfffc485e0000, 65536, PROT_READ) = 0
munmap(0xfffc48570000, 106368)          = 0
set_tid_address(0xfffc485f6060)         = 2614803
set_robust_list(0xfffc485f6070, 24)     = 0
rt_sigaction(SIGRTMIN, {sa_handler=0xfffc48176e98, sa_mask=[], sa_flags=SA_SIGINFO}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {sa_handler=0xfffc48176f68, sa_mask=[], sa_flags=SA_RESTART|SA_SIGINFO}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
prlimit64(0, RLIMIT_STACK, NULL, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
statfs("/sys/fs/selinux", 0xffffcc63fe40) = -1 ENOENT (没有那个文件或目录)
statfs("/selinux", 0xffffcc63fe40)      = -1 ENOENT (没有那个文件或目录)
brk(NULL)                               = 0xaaae33200000
brk(0xaaae33230000)                     = 0xaaae33230000
openat(AT_FDCWD, "/proc/filesystems", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tramfs\nnodev\tbdev\nnodev\tproc\nnodev\tcpuset\nnodev\tcgroup\nnodev\tcgroup2\nnodev\ttmpfs\nnodev\tdevtmpfs\nnodev\tconfigfs\nnodev\tdebugfs\nnodev\tsecurityfs\nnodev\tsockfs\nnodev\tdax\nnodev\tbpf\nnodev\tpipefs\nnodev\thugetlbfs\nnodev\tdevpts\n\text3\n\text2\n\text4\nnodev\tecryptfs\nnodev\tautofs\n\txfs\nnodev\tefivarfs\nnodev\tmqueue\nnodev\tpstore\n\tvfat\nnodev\trpc_pipefs\nnodev\toverlay\nnodev\tbinfmt_misc\n", 1024) = 394
read(3, "", 1024)                       = 0
close(3)                                = 0
faccessat(AT_FDCWD, "/etc/selinux/config", F_OK) = 0
getuid()                                = 0
socket(AF_NETLINK, SOCK_RAW|SOCK_CLOEXEC, NETLINK_ROUTE) = 3
setsockopt(3, SOL_SOCKET, SO_SNDBUF, [32768], 4) = 0
setsockopt(3, SOL_SOCKET, SO_RCVBUF, [1048576], 4) = 0
setsockopt(3, SOL_NETLINK, NETLINK_EXT_ACK, [1], 4) = 0
bind(3, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 0
getsockname(3, {sa_family=AF_NETLINK, nl_pid=2614803, nl_groups=00000000}, [12]) = 0
setsockopt(3, SOL_NETLINK, NETLINK_DUMP_STRICT_CHK, [1], 4) = -1 ENOPROTOOPT (不可用的协议)
sendto(3, {{len=32, type=RTM_NEWLINK, flags=NLM_F_REQUEST|NLM_F_ACK, seq=0, pid=0}, {ifi_family=AF_UNSPEC, ifi_type=ARPHRD_NETROM, ifi_index=0, ifi_flags=0, ifi_change=0}}, 32, 0, NULL, 0) = 32
recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=52, type=NLMSG_ERROR, flags=0, seq=0, pid=2614803}, {error=-ENODEV, msg={{len=32, type=RTM_NEWLINK, flags=NLM_F_REQUEST|NLM_F_ACK, seq=0, pid=0}, {ifi_family=AF_UNSPEC, ifi_type=ARPHRD_NETROM, ifi_index=0, ifi_flags=0, ifi_change=0}}}}, iov_len=16384}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 52
futex(0xfffc48470048, FUTEX_WAKE_PRIVATE, 2147483647) = 0
openat(AT_FDCWD, "/usr/lib/ip/link_veth.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (没有那个文件或目录)
sendmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=100, type=RTM_NEWLINK, flags=NLM_F_REQUEST|NLM_F_ACK|NLM_F_EXCL|NLM_F_CREATE, seq=1604388683, pid=0}, {ifi_family=AF_UNSPEC, ifi_type=ARPHRD_NETROM, ifi_index=0, ifi_flags=0, ifi_change=0}, [{{nla_len=14, nla_type=IFLA_IFNAME}, "veth-p130"}, {{nla_len=52, nla_type=IFLA_LINKINFO}, [{{nla_len=8, nla_type=IFLA_INFO_KIND}, "veth"...}, {{nla_len=40, nla_type=IFLA_INFO_DATA}, "\x24\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0e\x00\x03\x00\x76\x65\x74\x68\x2d\x70\x30\x31\x33\x00\x00\x00"}]}]}, iov_len=100}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 100
recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base=NULL, iov_len=0}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_TRUNC}, MSG_PEEK|MSG_TRUNC) = 36
recvmsg(3, {msg_name={sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, msg_namelen=12, msg_iov=[{iov_base={{len=36, type=NLMSG_ERROR, flags=NLM_F_CAPPED, seq=1604388683, pid=2614803}, {error=0, msg={len=100, type=RTM_NEWLINK, flags=NLM_F_REQUEST|NLM_F_ACK|NLM_F_EXCL|NLM_F_CREATE, seq=1604388683, pid=0}}}, iov_len=32768}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 36
exit_group(0)                           = ?
+++ exited with 0 +++

Detailed Description

Possible Implementation

Pod liveness and readiness gates failed with ipvlan l2 mode

galaxy-sdn network name missing

Support configuring multiple device of vlan cni plugin

Reserve an ip without restarting galaxy-ipam

Currently, if we want to reserve an ip, we need to update the floatingip-config configmap. But this is not possible if floatingip-config configmap is synced by a program automatically which may override the update.
We can also create a floatingip crd and restart galaxy-ipam to reserve an ip. To avoid restarting, we can watch floatingip objects and thus reserve the ip.

是否可以增加一个配置 sriov 和 macvlan 的教程？

目前在互联网上关于 galaxy 的资料不多，对初学者来说，使用门槛较高，
希望作者能够增加一个配置 sriov 和 macvlan 的教程，谢谢，感激不尽！

Upgrade cni version

We are currently using cni v0.2.0, it's time to upgrade it.

resync of veth and container routes

Network manager and restarting network can delete veth and container routes. We should resync of veth and container routes in a loop?

Fix mounting docker socket file

Docker socket file is currently bind mounting into galaxy image and if docker restarts, galaxy fails to connect to docker because of inode of docker.socket changes on host.

Nodes in multiple subnets sharing a Floatingip subnet

On qcloud VPC, nodes in a same zone could share a ENI subnet regardless of whether their subnet matches.

Add liveness and readiness gates for galaxy-ipam

Implementing preempt API

Galaxy-ipam currently doesn't register preempt API, If preemption happens kube-scheduler may make bad decision of nominated nodes for pods.

Support allocating floatingip to all pods

Currently, galaxy-ipam only supports tapp/statefulset/deployment/replicasets pods.
https://github.com/tkestack/galaxy/blob/master/pkg/ipam/schedulerplugin/util/utils.go#L133

make galaxy pods critical guaranteed scheduling

https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/

Support configuring routes if setup multiple networks for a pod

将release-policy设置为never时，删除POD 重建后，IP 并不能固定

之前我在同样的环境试过是能够固定 ip 的，请问下大佬，这个问题该如何定位？会不会是 etcd 的数据错乱导致？

实验步骤和环境信息见下图：

第一次删除（ip 从 251 变成 253）：

测试 demo：

第二次删除（ip 从 253 变成 200）：

我开始认为与s tke.cloud.tencent.com/eni-ip:“1” 有关，但我重试将该参数加上，最后的验证结果依然是一样的（如下图）：
（ip 从 202 变成 204）

环境信息如下（galaxy 的 image 我重新 build 过，主要是将 macvlan copy 进去）：
galaxy 是 1.0.8
k8s 是 1.19.11
docker 是19.03.12
all-in-one 单节点环境

macvlan配置如下：

[root@rhl7base yaml]# cat /etc/cni/net.d/20-macvlan.conf
{
"cniVersion": "0.2.0",
"type": "macvlan",
"name": "galaxy-k8s-macvlan",
"master": "eno16777736",
"mode": "bridge",
"ipam": {
"type": "host-local",
"ranges": [
[ {
"subnet": "172.16.49.0/24",
"rangeStart": "172.16.49.200",
"rangeEnd": "172.16.49.253",
"gateway": "172.16.49.2"
} ]
]
}
}

我试过使用 ipvlan，同样是以上效果。

[{
	"nodeSubnets": ["10.0.0.32/32", "10.0.0.48/32"],
	"ips": ["192.168.0.2~192.168.0.254"],
	"subnet": "192.168.0.0/24",
	"gateway": "192.168.0.1"
}, {
	"nodeSubnets": ["10.0.0.32/32", "10.0.0.48/32"],
	"ips": ["192.168.1.2~192.168.1.254"],
	"vlan": 3,
	"subnet": "192.168.1.0/24",
	"gateway": "192.168.1.1"
}]

I propose adding a FloatingIPPoolConfV2 to rewrite it into

[{
	"nodeSubnets": ["10.0.0.32/32", "10.0.0.48/32"],
	"podSubnets": [{
		"ips": ["192.168.0.2~192.168.0.254"],
		"subnet": "192.168.0.0/24",
		"gateway": "192.168.0.1"
	}, {
		"ips": ["192.168.1.2~192.168.1.254"],
		"vlan": 3,
		"subnet": "192.168.1.0/24",
		"gateway": "192.168.1.1"
	}]
}]

Distinguish scheduling fail reason of NoFIPLeft and NodeNotInPreservingSubnet

Add underlay network quick start doc

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.