kyma-project / cli Goto Github PK
View Code? Open in Web Editor NEWSimple set of commands to manage a Kyma installation
License: Apache License 2.0
Simple set of commands to manage a Kyma installation
License: Apache License 2.0
See changes in minikube.sh which should get adopted.
kyma-project/kyma#2893
Have integration tests which are using latest CLI revision (master) with latest revision of Kyma (master).
As the reason for failures are not clearly assignable to a repo, the test should not block any pull request and might be executed on a periodic base reporting to a slack channel.
It should do the same steps as kyma-project/kyma#4048 but should be based on gke
Description
It would be helpful to have an option like --branch
. This could mainly be used to install the current master version of Kyma.
Reasons
Currently, installing the master branch of Kyma is done by checking out the Kyma sources locally and triggering the installation. It is more convenient if the CLI takes care of checking out the branch and then triggers a local installation based on the the Kyma sources.
What happened
I've been trying to install the latest version:
$ kyma install
! You are using an unsupported kubectl version '1.14.0'. This may not work. It is recommended to use kubectl version '1.12.0'
✓ Requirements are fine
Installing Kyma in version '0.9.1'
✓ Tiller installed
✓ kyma-installer installed
✕ Configuring Helm
Error: open /Users/username/.helm/ca.pem: no such file or directory
open /Users/username/.helm/ca.pem: no such file or directory
I've had the same result with kubectl version 1.11.0, so I updated to a newer version.
The cluster was just provisioned before running the installer with the following output:
$ kyma provision minikube
! You are using an unsupported kubectl version '1.14.0'. This may not work. It is recommended to use kubectl version '1.12.0'
✓ Requirements are fine
Preparing minikube using domain 'kyma.local' and vm-driver 'hyperkit'
? Do you want to remove previous minikube cluster [y/N]: y
✓ Minikube status is fine
✓ Minikube config initialized
✓ Minukube up and running
Please enter your password if requested
Password:
✓ Adjustments finished
Minikube cluster is installed
host: Running
kubelet: Running
apiserver: Running
kubectl: Correctly Configured: pointing to minikube-vm at 192.168.64.21
Happy Minikube-ing! :)
Expected
I would expect the installer to ask me to generate the certificate or add a task kyma certgen
to put the certificates in the assumed location, that can be run once before kyma install
task.
Description
The source code for CLI should follow standards and common best practices for Go code:
Reasons
The current installation scripts have a special flag to indicate a kyma installation based on knative. That mode will preset the minikube memory and disk settings and will preset the installer configuration.
Goal: Support installation of kyma in knative mode
Kyma will be switching to xip DNS as default: kyma-project/kyma#2563
Description
Introduce Kyma test functionality based on the Kyma test suite and the octopus framework. A testing.sh
script should not be required anymore as you can run a suite with one command. And figure out the problem area with one command. That commands are easily understandable by just reading the in-line help.
Reason
Executing the kyma tests got improved a lot by having it based on Octopus. Still, it can be quite inconvenient to create the resources and figure out the status of a test execution in comparison to execute a simple command. From devX perspective, it should be a trivial task to execute the kyma test suite and investigate the problems.
Details
Overall goal is to remove the testing.sh
in kyma repo and instead use the CLI commands.
The proposal is to have following commands:
kyma test run <tests>
- creates new TestSuite
Parameters:
tests
- comma-separated list of names, selector.matchNames
in TestSuite, empty list selects all testsOptions:
-n/--name
- name of the test suite, generated by default--timeout
- suiteTimeout
in TestSuite--concurrency
- concurrency
in TestSuite-l/--labels
- comma-separated list of labels, selector.matchLabels
in TestSuite-c/--count
- number of times suite should be executed, count
in TestSuite--max-retries
- allowed number of retries per test, maxRetries
in TestSuite-w/--watch
- watch status of TestSuite until completion--delete
- remove TestSuite after successful completionkyma test delete <name> <name> ..
- deletes TestSuite and all its resources
Parameters:
name
- mandatory name of the test suite to delete, can be provided multiple timesOptions:
kyma test status <name> <name>
- prints nicely formatted status of TestSuite, prints command instructions to easily re-run a suite
Parameters:
name
- optional name of the test suite to describe, can be provided multiple timesOptions:
-w/--watch
- watch status of TestSuite until completion-o json|yaml
- prints raw testSuite resource in specified formatkyma test logs [<name>]
- print failing logs from test suite
Parameters:
name
- optional name of the test suite to retrieve logs from, default is latest started suiteOptions:
-f/--follow
- follow logs--test-status
- selects logs of pods in status "Running, Failed, Success"--definitions
- selects the pods related to the definitions in the test suite to show logs ofkyma test list
- print all created suites with their statuses
Parameters:
Options:
kyma test definitions
- print all available TestDefinitions
Parameters:
Options:
Example Usage
With the definition above testing.sh
would become something like this:
echo "----------------------------"
echo "- Testing Kyma..."
echo "----------------------------"
kyma test run --name kyma-suite --timeout 30m --watch
testCheckCore=$?
kyma test status kyma-suite
kyma test logs kyma-suite
kyma test clean kyma-suite
exit ${testCheckCore}
So there is a problem with minikube in windows. Although I actually deleted minikube and could provision a new minikube cluster the status of minikube shows that it's still installed and when I use the CLI to provision a new cluster it asks me if I want to delete the previous cluster. If I choose Yes it fails and if I choose No it exits so then I am stuck and can't use the tool. So I would Like to suggest that if I choose No the provisioning continues without the delete command
Description
The flag for the CPU setting in kyma provision minikube
is --cpu
. However, it should be --cpus
to be consistent with the namings in Minikube.
If there is an error during installation installer logs are printed to the console. As all overrides are written there they may contain sensitive data. This is security vulnerability and prevents kyma-cli from being used in production environment. There is a plan to add additional error log to Installation
CRD status. As soon as it is done we need to utilise it and remove printing installer logs.
Right now we have dependency on following external tools:
and soon to come:
There is growing problem we'll have to face one day, which is how do we approach versioning of those dependencies. Two basic approaches are:
None of those two solves problem of breaking changes in CRDs.
I suggest a mixed approach. There are three tools that are needed to provide core functionality: kyma CRDs, kubernetes and helm, which will soon get replaced by octopus.
Kubernetes is the only binary dependency we have. According to client-go it should be backwards compatible so we could bind kyma-cli to version supported by latest Kyma release. This should give us the ability to work with past Kyma releases as well as with future ones. Same applies to helm until we get rid of it.
We have also dependency on Kyma and Octopus CRDs. They are versioned together so we may treat them as one. The only option I see is to have a switch in the code which chooses appropriate "driver" depending on dependency version.
All the other dependencies will be served as binaries. If you want to run Kyma on minikube, you have to have minikube installed in appropriate version. If you want to install on GKE, you have to have gcloud installed, and so on. Alternatively GKE module may use go client for GKE, but I wouldn't force anything here.
Description
The CLI allows to install kyma on minikube with two easy commands. By doing that, it supports very technical options to customize the installation by specifying custom config.yaml and configuration overrides.
There are two things very typical customizations you require quite often:
As inspiration take a look at the kyma installation scripts, there is a hidden feature by specifying a components.env file (kyma-project/kyma#3995)
Reasons
The CLI should abstract any technical details and should assure that installing kyma will be an easy task keeping full flexibility.
Description
The installation got simplified, see kyma-project/kyma#3932
We need to adjust to it before first official release
Reasons
Most likely the installation procedure implemented in CLI will not work with Kyma 1.2 anmyore
Description
The "provision minikube" command adds entries to the hosts file by executing a sudo command. The logs in the CLI only says that you should enter your password, it is not explaining why. If I don't want to grant root access, I have no chance to skip it.
Goal
Description
The error messages returned by the CLI are rather brief and don't give much insight into what actually went wrong, even when in --verbose
mode.
For example, this istio error:
Using the is-installed.sh script of Kyma it was showing at some point that detail message:
Step error: Details: Helm install error: rpc error: code = Unknown desc = validation failed: [unable to recognize "": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3", unable to recognize "": no matches for kind "DestinationRule" in version "networking.istio.io/v1alpha3", unable to recognize "": no matches for kind "attributemanifest" in version "config.istio.io/v1alpha2"
This error description is available to the user but requires a lot of digging.
And it says that the installation failed but indeed it re-tries and may succeed later.
Reasons
Help users identify the exact errors they get in the installation process which in turn allows them to find solutions in Troubleshooting docs and ask the right questions on Slack.
Goal
Description
Following the instructions provided by cobra to enable completion for the kyma CLI commands does not work showing the following error:
/dev/fd/11:type:299: bad option: -t
Expected result
Completions work after running the command as explained by the output of kyma completion -h
Actual result
Error is displayed:
/dev/fd/11:type:299: bad option: -t
Steps to reproduce
. <(kyma completion)
, which should add the completions code to your bash session .Kyma 0.8 will be distributed with file which contains supported versions of required tools: kyma-project/kyma#2644. We could use this to dynamically check if requirements are met. Two checks should be in place:
Description
Current way of configuring the installation process is by providing yaml files with configmaps containing the configuration overrides. That yaml files will be passed with the --override option at the "kyma install" command.
When doing that, the override mechanism assumes that the configmap already exists. That often is not the case, think of configuration for cms component as an example.
Expected result
I can configure/override all documented configuration options
Actual result
Providing configuration for a component having no configmap defined in the basis configuration will fail
Steps to reproduce
Configure overrides for cms
Troubleshooting
Download the basis configuration and add the configmap here and then use the --config parameter
Description
Kyma tests no longer use helm test
to run, but octopus
instead. The deprecated test command and dependency to helm need to be removed from the CLI source code.
AC:
Current project name kymactl
suggests binary name (like kubectl
). However, the binary is named differently: kyma
. I suggest changing name of the project to kyma-cli
or simply cli
.
Description
Have a "health" command to quickly get an overview of the cluster (in regards to problems).
It should focus on retrieving general health information and the installer status. Using --watch you can follow the health in case of unhealthyness (wait till it gets healthy)
Details:
Reasons
Often you need a way to quickly check if all is running fine, like after installation you want to assure that all pods are running. With the health command it is easy to get that certainty and you can attach to it and wait till the condition is met.
Description
Minikube is released finally with in a stable version and we should support that. Currently the installation scripts and the CLI are warning you to use that version.
Goal
Official support Minikube 1.0.x with Kyma when installed via CLI
Details:
There are four places where the minikube version is referenced
In the epic of this ticket the docu and script gets deprecated, here we could keep the old version, but the CLI and prow should use the new version.
When installing kyma based on minikube you need to configure the kyma specific domains for domain resolution in your local hosts file and in the hosts file on minikube.
That is currently done by the CLI but it is not doing it with proper shell output encoding. So the configured domain could contain malicious shell commands getting executed without knowledge of the user.
PR kyma-project/kyma#2909 introduced new override that has to be set in local environment. We must support it.
Description
Error while running .\kyma.exe provision minikube --vm-driver virtualbox
...
Starting VM...
Getting VM IP address...
E0204 08:12:50.106726 182476 start.go:210] Error parsing version semver: Version string empty
F0204 08:12:50.115248 182476 start.go:244] Error getting cluster bootstrapper: Unknown bootstrapper:
localkube
' and error message 'exit status 1'
Failed executing minikube command 'minikube [start --memory 8192 --cpus 4 --extra-config=apiserver.Authorization.Mode=RBAC --extra-config=apiserver.GenericServerRunOptions.CorsAllowedOriginList='.*' --extra-config=controller-manager.ClusterSigningCertFile='/var/lib/local
kube/certs/ca.crt' --extra-config=controller-manager.ClusterSigningKeyFile='/var/lib/localkube/certs/ca.key' --extra-config=apiserver.admission-control='LimitRanger,ServiceAccount,DefaultStorageClass,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota' --ku
bernetes-version=v1.10.0 --vm-driver=virtualbox --disk-size=20g --feature-gates='MountPropagation=false' -b=localkube]' with output 'Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Getting VM IP address...
E0204 08:12:50.106726 182476 start.go:210] Error parsing version semver: Version string empty
F0204 08:12:50.115248 182476 start.go:244] Error getting cluster bootstrapper: Unknown bootstrapper: localkube
' and error message 'exit status 1'
PS C:\golang\work\src\github.com\kyma-incubator\kyma-cli\bin> minikube version
minikube version: v0.31.0
PS C:\golang\work\src\github.com\kyma-incubator\kyma-cli\bin>
-bootstrapper=localkube
must be replaced by --bootstrapper=kubeadm
.
Description
The uninstallation fails always in the "Delete tiller" phase:
Deleting tiller
Executed command:
kubectl -n kube-system delete all -l name=tiller
with output:
error: the server doesn't have a resource type "brokers"
and error:
exit status 1
X Deleting tiller
Error: Failed executing kubectl 'kubectl -n kube-system delete all -l name=tiller' command with output 'error: the server doesn't have a resource type "brokers"
' and error message 'exit status 1'
It seems that the uninstallation removes the CRDs of knative.dev api group but somehow the "kubectl delete all" command still relies on it, so complains that it cannot find the type "brokers.eventing.knative.dev" for example.
Repeating the delete command manually will then succeed, so maybe it is a timing issue that the CRDs are not fully unregistered yet.
Expected result
kyma uninstall command succeeds
Actual result
kyma uninstall command fails
Steps to reproduce
kyma provision minikube
kyma install
kyma uninstall
Troubleshooting
How about to add two types of installation:
The idea is, that by default kyma-cli
will install the lite version but one may provide a flag (--full
) to install monitoring subsystem as well.
Come up and implement a release process based on prow, see also the kyma release process:
https://github.com/kyma-project/test-infra/blob/master/prow/jobs/kyma/kyma-artifacts.yaml
https://github.com/kyma-project/test-infra/blob/master/prow/jobs/kyma/kyma-github-release.yaml
https://github.com/kyma-project/test-infra/blob/master/development/github-release.sh
https://github.com/kyma-project/test-infra/blob/master/prow/scripts/build-kyma-artifacts.sh
Goal:
Have an automated release process in place based on branches/tags assuring that the final binaries for a release are well-tested and published by CI infrastructure only.
There is upcoming feature in Kyma: kyma-project/kyma#1923
It will change how dependencies versions (minikube specifically) are verified by installation scripts:
Description
Improve documentation for install
and version
commands. Go with an in-line documentation for now (accessible via --help). If we see that it gets to messy, move it into proper md files and just reference from in-help.
It should contain:
Reason
People should understand the effect of a command and the parameters.
I was thinking about format of install command and came to conclusion that what we have now will lead to problems in future.
Why is that? kyma provision ...
requires some provider specific parameters, e.g. kyma provision gke
would require additional --domain
flag so it can create appropriate DNS records. On the other hand kyma install
would also require this parameter. This will lead us to duplication and potential mismatch in cluster and kyma configurations. Moreover, type of the cluster provider is important for installation. Some steps need to be enabled or disabled depending on provider (mostly minikube vs rest of the world). Some features are enabled or not. Some works in a different way.
In my opinion, there are two approaches to make it easier to understand and manage:
kyma
config. For example one would call kyma provision local --provider minikube --vm-driver hyperkit
and then reference this cluster in kyma install --cluster local
.kyma provision ...
followed by kyma install ...
there would be only one call, like those:
kyma install --provider minikube --minikube-vm-driver hyperkit
orkyma install --provider minikube --skip-provision
orkyma install --provider gke --gke-cluster-name my-cluster --domain my.cluster.kyma.org
I'm leaning towards option 2, as it's a lot simpler. @lilitgh @a-thaler @mszostok What do you think?
Kubernetes-sigs have a kubectl plugin manager called krew. https://github.com/kubernetes-sigs/krew
We can add kyma-cli to krew index repo, https://github.com/kubernetes-sigs/krew-index, so that people can install cli with the command kubectl krew install kyma
. It's a yaml file with our released binaries.
Description
A local Kyma installation comes with a local wildcard self-signed server.crt
certificate.
Mozilla Firefox uses its own certificate keychain. If you want to access the Console UI though Firefox, add the Kyma wildcard certificate to the certificate keychain of the browser. To access the Application Connector and connect an external solution to the local deployment of Kyma, you must add the certificate to the trusted certificate storage of your programming environment. Read this document to learn more.
Goal
Add this certificate to the OS trusted certificates as part of the minikube setup.
Be aware of the sudo problematic, see existing hosts file manipulation logic.
Remove an existing kyma certificate like we do for hosts file manipulation (to not spam the truststore)
Description
When I run kyma install --kubeconfig <config-file>
, the config file I'm passing to the command is just ignored.
Expected result
I'd expect the config I'm passing would be used.
Description
This is a follow-up for kyma-project/kyma#4045.
Generated changelog for releases should be enhanced using the changelog-generator.
*Hint: Check Custom release notes section here.
When done, please generate and add download urls for all released binaries
In windows /etc/hosts file can't read more than 9 hostnames per ip. That's why , when outputting the steps for windows users during installation, there should be several lines with the same ip and different hostnames. The related kyma installation code is here:
Description
Functions checkIfMinikubeIsInitialized and waitForMinikubeToBeUp in provision/minikube.go use flag --format
as is shown below.
statusText, err := internal.RunMinikubeCmdE([]string{"status", "-b=" + bootstrapper, "--format", "'{{.MinikubeStatus}}'"})
This originates the following error
Minikube is initialized and status is 'E0204 08:05:51.556475 180376 status.go:137] Error executing status template: template: status:1:3: executing "status" at <.MinikubeStatus>: cant evaluate field MinikubeStatus in type cmd.Status'
In order to solve this issue {{.MinikubeStatus}} must be replaced by {{.Host}} as is indicated in minikube status --help
Flags:
--format string Go template format string for the status output. The format for Go templates can be found here: https://golang.org/pkg/text/template/
For the list accessible variables for the template, see the struct values here:
https://godoc.org/k8s.io/minikube/cmd/minikube/cmd#Status (default "host: {{.Host}}\nkubelet: {{.Kubelet}}\napiserver: {{.ApiServer}}\nkubectl: {{.Kubeconfig}}")```
Description
In iTerm with ZSH running this command to install Kyma CLI on MacOS:
curl -Lo kyma.tar.gz https://github.com/kyma-project/cli/releases/download/$(curl -s https://api.github.com/repos/kyma-project/cli/releases/latest | grep tag_name | cut -d '"' -f 4)/kyma_Darwin_x86_64.tar.gz \
&& mkdir kyma-release && tar -C kyma-release -zxvf kyma.tar.gz && chmod +x kyma-release/kyma && sudo mv kyma-release/kyma /usr/local/bin \
&& rm -rf kyma-release kyma.tar.gz
Fails with this error:
zsh: parse error near `)'
When I run the same command in MacOS Terminal with bash, everything works as expected.
Expected result
Can install Kyma CLI using zsh.
Actual result
Can't install with zsh, can install with bash.
Steps to reproduce
Install iTerm, switch shell to zsh and run the command listed in the problem description.
Troubleshooting
I tried Terminal with bash and it worked.
At the moment the "kyma install" command prints the following after successful installation:
Kyma is installed in version 1.0.0
Kyma console: https://console.kyma.local
Kyma admin email: [email protected]
Kyma admin pwd: xxx
Happy Kyma-ing! :)
Please suppress the password log if:
This ticket tracks what needs to be done in order to fully migrate Kyma installation scripts to this CLI.
run.sh
)--skip-minikube-start
) See above--vm-driver
) Realized with --vm-driver
option--password
)--cr
) Might be realized by allowing to specify multiple --config yaml files which are getting merged together with the base yaml file.--knative
)--local
optioncreate-cr.sh
)replace-placeholder.sh
is used for)configure-comppnents.sh
)testing.sh
) There is a separate command to test kymawatch-pods.sh
)installation/resources
The default Kyma installation will configure dex with an initial generated admin password.
Instead of figuring out the password after installation and potentially changing it, there should be a way to configure the password as part of installation.
Proposal:
Introduce -p/--password
flag to install
command which configured the password for the installer.
Assure that the password gets printed nowhere, especially not at the summary section at the end of installation.
Kyma 0.8 will be released in a moment and the CLI should support it.
To install kyma you need to apply the installer CR as well as a set of predefined configMaps which you can customize. At the moment you can specify a custom yaml file containing the configMaps and the installer CR all in one file using the --config flag at install
Sometimes it is handy to replace just the installer CR or just one dedicated configMap. The rest of the configuration should be still based on the base yaml file provided by the release. To easily manage your changes it will be helpful to configure the different customizations individually, not all in one file.
Instead of replacing the full yaml config using the --config flag, support a override of the configmaps and the installer cr defined in the base yaml file
Before: kyma install --config customConfig.yaml
After: kyma install --config monitoringConfig.yaml --config installerCR.yaml --config eventBusConfig.yaml
Description
The Kyma Installer will be improved for the uninstallation action, see kyma-project/kyma#2942. Assure that "kyma uninstall" command will work with that changes
Reasons
Assure working software
Starting from minikube version 0.34.1 the "kyma provision minikube" command seems to fail like that:
✕ Adjusting minikube cluster
Error: Failed executing command ’minikube ssh -- ‘sudo sysctl -w fs.inotify.max_user_instances=524288’' with output ’bash: sudo sysctl -w fs.inotify.max_user_instances=524288: command not found
Executing the command manually is working without error message.
Review the language and consistency of all
Kubeconfig file may consists of several contexts. While using kubectl you may provide --context
flag if you don't want to use default one. Helm also supports it by providing --kube-context
flag. We should do the same.
Description
In order to improve development/maintainability/testability we should use the kubectl
library instead of the binary. Especially testing and response processing will be simplified.
AC:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.