remche / terraform-openstack-rke Goto Github PK
View Code? Open in Web Editor NEWTerraform Openstack RKE
License: Mozilla Public License 2.0
Terraform Openstack RKE
License: Mozilla Public License 2.0
It would be great to support Octavia. We should then provision a LB for master HA and take advantage of Octavia Kubernetes cloud provider support.
I keep getting:
Error: Failed to instantiate provider "rke" to obtain schema: Incompatible API version with plugin. Plugin version: 4, Client versions: [5]
What I've done:
$ tf providers
.
├── provider.openstack
└── module.rke
├── module.edge
│ ├── provider.null
│ └── provider.openstack (inherited)
├── module.keypair
│ └── provider.openstack (inherited)
├── module.master
│ ├── provider.null
│ └── provider.openstack (inherited)
├── module.network
│ └── provider.openstack (inherited)
├── module.rke
│ ├── provider.local
│ ├── provider.null
│ ├── provider.openstack
│ └── provider.rke
├── module.secgroup
│ └── provider.openstack (inherited)
└── module.worker
├── provider.null
└── provider.openstack (inherited)
$ tf version
tf version
Terraform v0.12.24
+ provider.local v1.4.0
+ provider.null v2.1.2
+ provider.openstack v1.26.0
+ provider.rke v1.1
+ provider.rke v1.4
add custom taints to nodes group.
When setting everything up, it fails
Docker logs says:
failed to run Kubelet: could not init cloud provider "openstack": Authentication failed
As per openshift/origin#19659 passwords with special characters must be provided with single quotes. Unfortunately I have a password with special characters and I can't do anything about it.
I looked into the code but found no place where it is defined.
Creating PVs fails, wehen installing in an openstack environment, where you have multiple AZs for compute (e.g. ams-a, ams-b, ams-c) but only one for storage (e.g. nova). Kubernetes tries to create them in the compute AZs, which openstack doesn't allow.
There is a solution in openshift/installer#2844 (comment)
Especially you need the "ignore_volume_az" option in cloud provider config.
When I set
use_ssh_agent = false
ssh_keypair_name = "thatcher" # existing key on openstack
The module fails with the error:
"rke" Failed initializing cluster err:Error while reading SSH key file: "file name too long", it also prints the full content of my RSA key. It is somehow trying to use the content of my SSH key as the filename.
setting the variable ssh_key_file doesn't seem to make a difference.
When I switched to using ssh-agent ssh-add, no arguments
this error went away.
Some infrastructure dont play well with kubernetes integration. Make the openstack_cloud_provider conditionnal.
I get an entry in the logs about the "openstack" cloud provider being deprecated.
Since terraform 1.0.0-rc3, downscale does not work anymore :
rancher/terraform-provider-rke#178
segmentio/terraform-docs
This might be a bug in the RKE provider, however since I am not using that directly I figured it makes sense to report it here (first). Using the following config:
data "openstack_images_image_v2" "ubuntu" {
name = "Ubuntu-18.04"
most_recent = true
}
resource "openstack_compute_keypair_v2" "keypair" {
name = "my-application-keypair-${var.environment}"
}
module "rke" {
cluster_name = "my-application-${var.environment}"
source = "remche/rke/openstack"
version = "0.5.4"
image_name = data.openstack_images_image_v2.ubuntu.name
public_net_name = "external"
master_flavor_name = "m1.medium"
worker_flavor_name = "m1.large"
os_auth_url = "https://myopenstackprovider.com:5000"
os_password = var.os_password
edge_count = 0
worker_count = 4
master_count = 1
use_ssh_agent = true
ssh_keypair_name = openstack_compute_keypair_v2.keypair.name
master_labels = { "node-role.kubernetes.io/master" = "true" }
edge_labels = { "node-role.kubernetes.io/edge" = "true" }
user_data_file = "cloud-init.yaml"
system_user = "ubuntu"
nodes_config_drive = true
deploy_traefik = true
deploy_nginx = false
}
When I increase worker_count
to 5 and do terraform apply -auto-approve
, it spins up a new instance on my Openstack provider, however the instance does not register as a node with the RKE cluster that is already running on the existing instances. This used to be the case when I still used 0.4.2
of this provider, but is no longer the case with 0.5.4
. I've tested on two separate existing clusters, both successfully create the new instance on Openstack but fail to recognize the new node. In both cases, the apply gets interrupted with:
time="2020-10-05T14:00:49+02:00" level=error msg="Failed to upgrade hosts: my-application-staging-worker-004 with error [Failed to verify healthcheck: Failed to check http://localhost:10248/healthz for service [kubelet] on host [192.168.42.42]: Get http://localhost:10248/healthz: Unable to access the service on localhost:10248. The service might be still starting up. Error: ssh: rejected: connect failed (Connection refused), log: F1005 12:00:45.096391 25275 server.go:274] failed to run Kubelet: could not init cloud provider \"openstack\": Authentication failed]"
Failed running cluster err:[workerPlane] Failed to upgrade Worker Plane: [Failed to verify healthcheck: Failed to check http://localhost:10248/healthz for service [kubelet] on host [192.168.42.42]: Get http://localhost:10248/healthz: Unable to access the service on localhost:10248. The service might be still starting up. Error: ssh: rejected: connect failed (Connection refused), log: F1005 12:00:45.096391 25275 server.go:274] failed to run Kubelet: could not init cloud provider "openstack": Authentication failed]
========================================
on .terraform/modules/rke/modules/rke/main.tf line 54, in resource "rke_cluster" "cluster":
54: resource "rke_cluster" "cluster" {
However, in both cases just retrying terraform apply -auto-approve
eventually results in Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
.
Terraform v0.13.2
+ provider registry.terraform.io/hashicorp/local v1.4.0
+ provider registry.terraform.io/hashicorp/null v2.1.2
+ provider registry.terraform.io/rancher/rke v1.1.2
+ provider registry.terraform.io/terraform-provider-openstack/openstack v1.32.0
+ provider registry.terraform.io/terraform-providers/openstack v1.32.0
I was trying to make this Terraform module to work on, but I got a little confused about the use_ssh_agent
. What exactly does it do?
The default is set to true, but that means you can't provide a private SSH key to access your newly provisioned nodes:
I was only getting timeouts with the default value:
Error: timeout - last error: Error connecting to bastion: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
When setting the use_ssh_agent
on false, I could get connection.
Hi, I'm trying to install RKE but I'm receiving some errors.
Openstack version: "Train"
terraform.tvars:
edge_count = 1
worker_count = 1
master_count = 1
master_labels = { "node-role.kubernetes.io/master" = "true" }
edge_labels = { "node-role.kubernetes.io/edge" = "true" }
public_net_name = "provider"
master_flavor_name = "a.large"
worker_flavor_name = "a.large"
edge_flavor_name = "a.large"
cluster_name = "rke"
ssh_keypair_name = "local"
nodes_net_cidr = "10.13.0.0/24"
dns_servers = ["8.8.8.8"]
dns_domain = "arkan.cloud."
use_ssh_agent = false
image_name = "ubuntu-18.04-minimal-cloudimg-amd64"
user_data_file = "rancher.yml"
acme_email = "[email protected]"
rancher.yml
#cloud-config
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYQykR9v8sGtHV0fl1Otm8N3nGFUYg8iO5IhODQO1zHtIAB8/px0JSu8g1EifbXzvx3GUUYDW2lNBbUYPOP1Os27M6lYz68qYKxMjLXraEHt9jNe7aUVIyNu2iGc3ZAgSrkRabw7P05ijLdH3A6MscQnc4tqLE92a3z/QGcANvtymJvpkvhuE3iUz92NxyR9AHaj9ejGbzJ2vu9kISVx2cUEvymW6x8zGh/agljsIXcp/KYUVr/MvpfVrdk7tUrLg3vAN5+273pWRGNTrtEdwhxBvSAYyU/p/C66G7ZHAILKj45rm0kpNqCNeQh2UiwmDbFEqQmtSZsEHKwRNCbdAx arkan@DESKTOP-8FQ42NL
packages:
- docker.io
# create the docker group
groups:
- docker
# Add default auto created user to docker group
system_info:
default_user:
groups: [docker]
Hello,
first of all let me thank you very much for sharing this awesome module!
I have a small issue when using the latest release of the module (0.6.0) and Terraform 0.14.2 when setting enable_loadbalancer
to true
.
The simplest way to demonstrate it is to start from the first example of the README, after sourcing my openrc file, terraform init
then terraform apply
.
The config is:
variable "os_auth_url" {}
variable "os_password" {}
module "rke" {
source = "remche/rke/openstack"
image_name = "ubuntu-20.04-docker-x86_64" #An actual image in my Openstack project
public_net_name = "public"
master_flavor_name = "m1.small"
worker_flavor_name = "m1.small"
os_auth_url = var.os_auth_url
os_password = var.os_password
enable_loadbalancer = true
}
Will result in the following output:
Error: Unsupported attribute
on .terraform/modules/rke/output.tf line 33, in output "loadbalancer_floating_ip":
33: value = var.enable_loadbalancer ? module.loadbalancer.floating_ip : ""
|----------------
| module.loadbalancer is tuple with 1 element
This value does not have any attributes.
I have done some experiment on my own and the culprit seems to be the fact that the module loadbalancer
is conditionally instantiated (through count = var.enable_loadbalancer ? 1 : 0
) and that confuses somehow Terraform that doesn't recognize the output within the module.
Unfortunately I wasn't able to devise a reasonable fix to the issue, otherwise I would have opened a PR myself.
Do you have any idea on how we can workaround this problem? (for the time being I'm using the module from a local clone where I've removed the output :D )
Thank you!
work in tf-13.0 branch :
0.5.x will break Terraform 0.12.x compatibility, 0.4.x branch will remain compatible
Make worker nodes an array to enable different worker type.
Wait for hashicorp/terraform#17519
Since recent terraform_quality_gate change, code-quality action fails.
dallinwright/terraform_quality_gate#5
we should be able to disable write of kubeconfig file on disk.
I tested the script, but unfortunately the install fails, because no docker engine is installed. I tried with
wait_for_commands = [
"sudo apt update",
"sudo apt upgrade -y",
"sudo curl https://releases.rancher.com/install-docker/19.03.sh | sh"
]
but without any success, because the worker nodes have no direct internet connection. Only private IP. How do I do that?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.