tkestack / tke Goto Github PK
View Code? Open in Web Editor NEWNative Kubernetes container management platform supporting multi-tenant and multi-cluster
License: Other
Native Kubernetes container management platform supporting multi-tenant and multi-cluster
License: Other
What happened:
Use TKE local registry, then push a large image (> 2GB):
The push refers to repository
87b1b5a611f8: Layer already exists
68dd2761fd81: Pushing [==================================================>] 2.086GB/2.086GB
received unexpected HTTP status: 504 Gateway Timeout
Environment:
kubectl version
):What would you like to be added:
Once the csi-operator
is released, a new csi-operator
image needs to be integrated into TKE's release package pipeline.
Why is this needed:
Prerequisites for publication.
What would you like to be added:
When to install TKEStack, if If set no-HA mode, the tke-installer should block user to add multi master node
Why is this needed:
If set no-HA mode, there is no VIP assigned to the apiservers, other components will connect multi apiservers address in random, that will involve risks.
Describe the bug
when creating a new cluster, specify the target node’s username and password, if input non-root username and password, the installation will get error like below:
Note: User ubuntu is a valid user for the node, and it has sudo permission
Environment
狗日的
What happened:
I'm trying to import tkeclientset
as my development library.
What you expected to happen:
go mod vendor -v
without error
How to reproduce it (as minimally and precisely as possible):
go.mod
and main.go
running go mod vendor -v
module github.com/John-Lin/client
go 1.13
replace tkestack.io/tke => github.com/tkestack/tke v1.0.1
require tkestack.io/tke v1.0.1
package main
import (
tkeclientset "tkestack.io/tke/api/client/clientset/versioned"
)
func main() {
restConfig := &rest.Config{
Host: "https://127.0.0.1:9443",
BearerToken: "token",
Timeout: time.Second * 5,
TLSClientConfig: rest.TLSClientConfig{
Insecure: true,
},
}
client, err := tkeclientset.NewForConfig(restConfig)
if err != nil {
fmt.Printf("%v", err)
}
}
Anything else we need to know?:
Environment:
kubectl version
): None安装好集群后,如何再添加node节点
What happened:
After installation, the registry address in global master node hosts is refer to the installer node
Environment:
kubectl version
):What would you like to be added:
Support installer to listen on all network interface
Why is this needed:
In the user multi-NIC scenario, it is impossible to determine the specific network topology of the user
What would you like to be added:
Once the galaxy
is released, a new galaxy
image needs to be integrated into TKE's release package pipeline.
Why is this needed:
Prerequisites for publication.
Create a business cluster with the config 2048 services per cluster on console, finally get only 256 services per cluster. And the --service-cluster-ip-range is 192.168.255.0/24, actually only support 256 services.
Environment:
kubectl version
):Hello
What would you like to be added:
Could you plz upload your monitor.json and notify.json for running locally?
And Can you give me your contact information? Like wechat group
Why is this needed:
What would you like to be added:
Why is this needed:
What would you like to be added:
Why is this needed:
now I want to know some knowledge about TKE. Have wechat group ?
What would you like to be added:
Once the volume-decorator
is released, a new volume-decorator
image needs to be integrated into TKE's release package pipeline.
Why is this needed:
Prerequisites for publication.
tke-installer-v0.15.0 doesn't contain local-tcr and kubernetes images, so I need to download these images manually.
installer log:
v0.1: digest: sha256:cc153f2ed523952e85a73fafaff87d05fb14fb7e068ef330fb671274c4ed5998 size: 734
2019/12/04 08:08:38 upload registry.tke.com/library/configmap-reload:v0.1 to registry success[46/46]
2019/12/04 08:08:38 3.Push images to registry.tke.com/library [Success] [2.350667s]
docker images:
# docker images | grep kube
tkestack/kube-state-metrics v1.6.0 87970984231a 6 months ago 230MB
registry.tke.com/library/kube-state-metrics v1.6.0 87970984231a 6 months ago 230MB
# docker images | grep registry.tke.com | wc -l
46
# docker images | grep registry.tke.com | grep kube
registry.tke.com/library/kube-state-metrics v1.6.0 87970984231a 6 months ago 230MB
# kubectl create -f cluster2.yaml
error: SchemaError(io.tkestack.tke.pkg.monitor.services.rest.Response.data): Unknown primitive type: "interface{}"
cluster2.yaml
apiVersion: platform.tkestack.io/v1
kind: Cluster
metadata:
generateName: cls
spec:
apiServerExtraArgs:
service-node-port-range: 1-65535
displayName: cluster2
clusterCIDR: 192.168.0.0/16
dnsDomain: cluster.local
features:
ipvs: false
finalizers:
- cluster
networkDevice: eth1
properties:
maxClusterServiceNum: 2048
maxNodePodNum: 256
tenantID: default
type: Baremetal
version: 1.14.6
machines:
- ip: xxx
port: xxx
username: root
password: xxx
Environment:
kubectl version
):What would you like to be added:
Once the tapp
is released, a new tapp
image needs to be integrated into tke's release package pipeline. At the same time, you need to specify the cluster's tapp
resource forwarding in the platform api to ensure that the forwarded path matches the CRD API group and version of the tapp
.
Why is this needed:
Prerequisites for publication.
What would you like to be added:
Once the gpu-manager
is released, a new gpu-manager
image needs to be integrated into TKE's release package pipeline.
Why is this needed:
Prerequisites for publication.
What would you like to be added:
Why is this needed:
incremental compilation
when I use make release
to build the installer image, it will cost duplicative effort to generate the images.tar.gz
, even the file already exist.
incremental installation
each time when I start a new installation, tke-installer always load the image.tar.gz, no matter if i have done this before.
root 16070 16052 0 15:48 ? 00:00:00 /app/bin/tke-installer
root 16187 16070 3 15:49 ? 00:00:00 docker load -i images.tar.gz
root 16197 5766 12 15:49 ? 00:00:01 docker-untar / /var/lib/docker/tmp/docker-import-404992003
Describe the bug
Create an user in ‘access management’->’user management’, open a new browser and use the new created one to login,
Then it jump into an error page, endless, again and again, even close the page and reopen.
Environment
Describe the bug
Just use the tke-installer-x86_64-10fd8a9.run version to deploy tkestack, but the installation hang at ‘Install tke-monitor-api doing’
Check the pod status, it is pending on Insufficient CPU.
Hope the installer will check the node’s cpu&mem capacity before installing the master node
[root@ruyingzhe8 ~]# kubectl get pods -n tke
NAME READY STATUS RESTARTS AGE
influxdb-0 1/1 Running 0 100s
tke-auth-586bd7d8f5-slwf2 1/1 Running 0 2m53s
tke-auth-586bd7d8f5-ssnvx 1/1 Running 0 2m53s
tke-business-api-68bd76b66f-sjs8t 1/1 Running 0 2m3s
tke-business-api-68bd76b66f-zm9vv 1/1 Running 0 2m3s
tke-business-controller-547b5bf744-w9b2l 1/1 Running 0 105s
tke-business-controller-547b5bf744-zzkmk 1/1 Running 0 105s
tke-monitor-api-5664f5c876-qjtpq 1/1 Running 0 95s
tke-monitor-api-5664f5c876-zhrjb 0/1 Pending 0 95s
tke-platform-api-6dd5b47bd7-42jr5 1/1 Running 0 2m38s
tke-platform-api-6dd5b47bd7-qz4f2 1/1 Running 0 2m38s
tke-platform-controller-7b5755f47-mp6bs 1/1 Running 0 2m23s
tke-platform-controller-7b5755f47-wpbgf 1/1 Running 0 2m23s
tke-registry-api-74bdb659b9-5kp6p 1/1 Running 0 2m18s
[root@ruyingzhe8 ~]#
[root@ruyingzhe8 ~]#
[root@ruyingzhe8 ~]# kubectl describe pod -n tke tke-monitor-api-5664f5c876-zhrjb
Name: tke-monitor-api-5664f5c876-zhrjb
Namespace: tke
Priority: 0
PriorityClassName: <none>
Node: <none>
Labels: app=tke-monitor-api
pod-template-hash=5664f5c876
Annotations: <none>
Status: Pending
IP:
Controlled By: ReplicaSet/tke-monitor-api-5664f5c876
Containers:
tke-monitor-api:
Image: registry.tke.com/library/tke-monitor-api:10fd8a9
Port: 9455/TCP
Host Port: 0/TCP
Args:
-C=/app/conf/tke-monitor-api.toml
Limits:
cpu: 500m
memory: 1Gi
Requests:
cpu: 250m
memory: 256Mi
Liveness: http-get https://:9455/healthz delay=15s timeout=1s period=20s #success=1 #failure=3
Readiness: http-get https://:9455/healthz/ping delay=5s timeout=1s period=10s #success=1 #failure=3
Environment: <none>
Mounts:
/app/certs from certs-volume (rw)
/app/conf from tke-monitor-api-volume (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-s2vkn (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
certs-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: certs
Optional: false
tke-monitor-api-volume:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: tke-monitor-api
Optional: false
default-token-s2vkn:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-s2vkn
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 35s (x4 over 116s) default-scheduler 0/1 nodes are available: 1 Insufficient cpu.
Environment
Describe the bug
Just follow the guide in “organization resource”->”registry management”->”your registory”->”image upload guide”, the tag and pull commands runs into error:
the guide under “organization resource”->”access certificate” has the same problem.
# sudo docker tag nginx:latest https://default.registry.tke.com/myreg/nginx:latest
Error parsing reference: "https://default.registry.tke.com/myreg/nginx:latest" is not a valid repository/tag: invalid reference format
Environment
What happened:
2019/12/12 05:01:53 upload registry.tke.com/library/configmap-reload:v0.1 to registry success[48/48]
2019/12/12 05:01:53 3.Push images to registry.tke.com/library [Success] [274.427330s]
2019/12/12 05:01:53 4.Generate certificates for TKE components doing
2019/12/12 05:01:54 4.Generate certificates for TKE components [Success] [0.921570s]
2019/12/12 05:01:54 5.Prepare front proxy certificates doing
2019/12/12 05:01:55 5.Prepare front proxy certificates [Failed] [0.644614s] error create file error:/etc/kubernetes/admin.crt:file does not exist
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:
kubectl version
):root@VM-64-34-ubuntu:/home/ubuntu# uname -a
Linux VM-64-34-ubuntu 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
root@VM-64-34-ubuntu:/home/ubuntu# cat tke-installer-x86_64-v1.0.0.run.sha256
dfb68e96c5831a3cda5725ed2f5ad10136bf6bd244afac095b71dc9a32682e82 tke-installer-x86_64-v1.0.0.run
Describe the bug
When using the make release
command to build the installer, errors occurred like below.
===========> Pushing tke-platform-api c72b21f image to tkestack
The push refers to repository [docker.io/tkestack/tke-platform-api]
675b40083459: Preparing
83b390cc87eb: Preparing
6132d92de499: Preparing
77cae8ab23bf: Preparing
denied: requested access to the resource is denied
make[2]: *** [image.push.tke-platform-api] Error 1
make[2]: Leaving directory `/root/gopath/src/tkestack.io/tke'
make[1]: *** [push] Error 2
make[1]: Leaving directory `/root/gopath/src/tkestack.io/tke'
make: *** [release] Error 2
Since I don’t have the permission of tkestack repo in dockerhub, I can’t build the tke-installer image for now.
Hope tkestack support releasing image repos other than tkestack/*
, user can customize their own repository
Environment
What happened: install cmd error
error
brew service start etcd
true
brew services start etcd
Describe the bug
Just use the tke-installer-x86_64-10fd8a9.run version to deploy tkestack, and the installation complete successfully.
But cannot access the console link, raise HTTP 502 ERROR.
Just check the pod status, found the the-gateway pod is in creating status.
The installer should check all of the components status, before sending the install task success message.
Environment
What would you like to be added:
Why is this needed:
Describe the bug
I just run the tke-installer:v0.15.0
image and to install the first new master node for global cls.
After I click the accept button and wait 5min, the console look like the picture attached.
Login to the master node, there are few images on the node, just run:
[root@VM_252_13_centos docker]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
tkestack/kube-apiserver v1.14.6 0e422c9884cf 3 months ago 209MB
tkestack/kube-controller-manager v1.14.6 4bb274b1f2c3 3 months ago 157MB
tkestack/kube-scheduler v1.14.6 d27987bc993e 3 months ago 81.6MB
tkestack/etcd v3.3.12 28c771d7cfbf 9 months ago 40.6MB
tkestack/pause 3.1 da86e6ba6ca1 23 months ago 742kB
[root@VM_252_13_centos ~]# docker pull registry.tke.com/library/tke-auth:v0.15.0
Error response from daemon: Get https://registry.tke.com/v2/: dial tcp 10.0.252.2:443: connect: connection refused
[root@VM_252_13_centos ~]# cat /etc/docker/daemon.json
{
"debug": false,
"insecure-registries": [
"docker.io"
],
registry.tke.com
is the default registry configuration I set in the console.
Environment
What happened:
installer log:
2019/12/04 13:09:47 OnInitialize.EnsureKubeadmInitUploadConfigPhase [Failed] [0.113197s] reason: FailedProcess message: exec "kubeadm init phase upload-config all --config=/etc/kubernetes/kubeadm-config.yaml" failed:exit 1:stderr error execution phase upload-config/kubeadm: error uploading the kubeadm ClusterConfiguration: unable to create ConfigMap: Post https://10.19.0.152:6443/api/v1/namespaces/kube-system/configmaps: Forbidden :error %!s() retry: 10
How to reproduce it (as minimally and precisely as possible):
install Global to one node
Anything else we need to know?:
~/.kube/config apiserver address is 127.0.0.1:6443, and kubectl get cs is ok:
# kubectl get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health":"true"}
But if I changed the apiserver address to node ip 10.19.0.152, then it returned Forbidden:
# kubectl get cs
Unable to connect to the server: Forbidden
So, I think the certificate is error.
Environment:
kubectl version
):What would you like to be added:
Support different scenarios or cluster scale installation
Why is this needed:
Reduce resource constraints
go mod vendor
go: github.com/coreos/[email protected] requires
github.com/prometheus/[email protected]+incompatible: reading https://goproxy.io/github.com/prometheus/prometheus/@v/v2.9.2+incompatible.mod: 410 Gone
What happened:
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:
kubectl version
):installer log:
Loaded image: tkestack/log-collector:v1.1.0
Loaded image: tkestack/nvidia-device-plugin:1.0.0-beta4
Loaded image: tkestack/tke-notify-api:v0.15.0
Loaded image: tkestack/tke-platform-controller:v0.15.0
Loaded image: tkestack/tke-registry-api:v0.15.0
2019/12/03 09:22:31 1.Load images [Success] [57.028089s]
2019/12/03 09:22:31 2.Setup local registry doing
f4383bf275822cf6eb76df15e591be92b94cae557016c6741c37b3a86d05b0d4
2019/12/03 09:22:32 2.Setup local registry [Failed] [0.404418s] error open hosts: no such file or directory
cmd/tke-installer/app/installer/installer.go
line 1148: localHosts := hosts.LocalHosts{Host: server, File: "hosts"}
If specify ' File:"hosts" ', localHost.File would be 'hosts' rather than '/etc/hosts'
Could you please add Docker Root Dir, registry(already configure registry.tke.com, but don't work in docker daemon.json) and http_proxy configuration of the Global cluster on installer web page? Otherwise, after docker installed, its root dir is /var/lib/docker and its registry is docker.io and my network cannot connect to docker.io,
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 18.09.9
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 3.10.107-1-tlinux2-0046
Operating System: Tencent tlinux 2.2 (Final)
OSType: linux
Architecture: x86_64
CPUs: 80
Total Memory: 125.3GiB
Name: TENCENT64.site
ID: AVWD:B3RY:H7V6:2L3D:VUZO:PJ6M:E4B3:SEC2:5DQR:SRLJ:UM75:7KSN
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Registry Mirrors:
https://docker.io/
Live Restore Enabled: true
Product License: Community Engine
What would you like to be added:
Once the LBCF
is released, a new lb-controlling-framework
image needs to be integrated into TKE's release package pipeline. At the same time, you need to specify the cluster's:
resources forwarding in the platform api to ensure that the forwarded path matches the CRD API group and version of the LBCF
.
Why is this needed:
Prerequisites for publication.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.