Giter VIP home page Giter VIP logo

gitpod-k3s-guide's Introduction

Running Gitpod in k3s

Archived: please see Gitpod Self-Hosted instead

Before starting the installation process, you need:

  • Target building resources.
    • Ubuntu 20.04/22.04 machine(s) with SSH credentials.
      • At least one, but also the script can also work for multiple nodes. The hostname of each node can be called node0, node1, etc.
      • All nodes have ports 22 (SSH), 80 (HTTP), 443 (HTTPS) and 6443 (Kubernetes) exposed. All nodes are better to be in the same vlan so they can communicate with each other.
      • Each node needs to have least 4 cores, 16GB RAM and 100GB storage.
    • A domain and some wildcard subdomains managed by Cloudflare (free), GCP, or Route53 see price. Please see the "DNS and TLS configured" section in the Gitpod docs for more information. These DNS services will have and manage free Let's Encrypt certificates for you. If you choose not to use these commercial DNS services, you will need to use self-signed certificates and manage them manually.
  • A .env file or environment variables with basic details about the environment.
    • We provide an example of such file here
  • Building environment. You can either:
    • Build on a local Linux machine - needs to install kubectl, Helm, K3sup. You may need to clean your ${HOME}/.kube directory if there was a previous gitpod-k3s entry.
    • Use Docker installed on your machine and Docker file is at .gitpod/gitpod.Dockerfile.
    • Even better, use a Gitpod workspace๐Ÿ˜€.
Example VM on GCP

Create GCP VM with Ubuntu 20.04 with 4 cores, 16GB of RAM, and 100GB of storage:

gcloud compute instances create gitpod-x509 \
  --image=ubuntu-2004-focal-v20220712 \
  --image-project=ubuntu-os-cloud \
  --machine-type=n2-standard-4 \
  --boot-disk-size=100GB \
  --tags k3s
# Created [https://www.googleapis.com/compute/v1/projects/adrien-self-hosted-testing-5k4/zones/us-west1-c/instances/gitpod-k3s].
# NAME         ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP     STATUS
# gitpod-k3s  us-west1-c  n2-standard-4               10.138.0.6   169.254.87.220  RUNNING

A firewall rule must be added to allow the current system to connect to the Kubernetes API. As we don't want to expose the Kubernetes API to the entire Internet this firewall rule allows the current host to connect to the k3s VM.

Note: If you're using a remote workspace (such as Gitpod) you'll need to include the public IP address the Gitpod instance as well as the public IP address of your local machine as the source ranges of this firewall rule.

gcloud compute firewall-rules create k3s \
  --source-ranges="$(curl -s ifconfig.me)/32" \
  --allow=tcp:6443,tcp:443,tcp:80 \
  --target-tags=k3s
gcloud compute config-ssh
# You should now be able to use ssh/scp with your instances.
# For example, try running:
#
# ssh gitpod-k3s.us-west1-c.adrien-self-hosted-testing-5k4

DNS and TLS

There are a number of options you may use for your DNS and TLS certificates:

  • Cloudflare - certificate verified via LetsEncrypt
  • A self-signed certificate - you will need to install your CA certificate (full instructions in KOTS dashboard)
  • None - you can do this manually

This has been tested on bare-metal Ubuntu and Multipass. Multi-node clusters are supported - it is assumed that all nodes are configured identically.

To start the installation, execute:

./setup.sh install

This process takes about 5 minutes. This will configure your k3s instance so it can accept a Gitpod installation.

As k3s tends to use the internal IP address, you will need to manually configure A records for:

  • $DOMAIN
  • *.$DOMAIN
  • *.ws.$DOMAIN

Upon completion, it will print the config for the resources created and instructions on what to do next.

Monitoring

You can optionally install a monitoring application to provide observatibility for you cluster.

Troubleshooting

  • Pods running out of resources

    This is a single-instance cluster. You will need to either add additional nodes or use a machine with greater resources. The seggested size is 4vCPUs and RAM in excess of 16GB. Disk size should also break a minimum of 100GB.

  • Some pods never start (Init state)

    kubectl get pods -l component=proxy
    NAME                     READY   STATUS    RESTARTS   AGE
    proxy-5998488f4c-t8vkh   0/1     Init 0/1  0          5m

    The most likely reason is that the DNS01 challenge has yet to resolve. If using MANAGED_DNS_PROVIDER, you will need to update your DNS records to the IP of your machine.

    Once the DNS record has been updated, you will need to delete all Cert Manager pods to retrigger the certificate request

    kubectl delete pods -n cert-manager --all

    After a few minutes, you should see the https-certificate become ready.

    kubectl get certificate
    NAME                        READY   SECRET                      AGE
    https-certificates          True    https-certificates          5m

Removing a node

Remove a node from the cluster by running:

./setup.sh delete-node <node name>

Warnings

  • If run on a control-plane node, this may have severe negative consequences for your cluster's long-term health.
  • This will only remove the node from the cluster. It does not uninstall k3s from the machine or delete the VM.

Destroy the resources

Remove k3s from your machine by running:

./setup.sh uninstall

If you created any cloud resources you can delete them with the following:

  • GCP

    GCP resource cleanup
    gcloud compute firewall-rules delete k3s --quiet
    gcloud compute instances delete gitpod-k3s --quiet

Retrieving credentials

Sometimes, you just want to get the credentials

./setup.sh credentials

Contributing

Contributions are always welcome. Please raise an issue first before raising a pull request.

Commit messages must adhere to the Conventional Commit format.

gitpod-k3s-guide's People

Contributors

adrienthebo avatar kernel-sanders avatar mrsimonemms avatar sethvoltz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

gitpod-k3s-guide's Issues

make the name of private key of cloudflare more clear.

In .env.example, it said to use

CLOUDFLARE_API_KEY

cloudflare does have a global api key, however, it won't work if I put in the settings.

The right way is to generate an api token with permissions as Zone.Zone, Zone.DNS and then put the token in the setting. No big issue here, just think if this can be more clear then it would be better.

https://github.com/MrSimonEmms/gitpod-k3s-guide/blob/main/.env.example#L23

linux-headers-5.4.0-90-generic missing

I have this problem:

kubectl logs ws-daemon-2nd7s -c shiftfs-module-loader

  • mkdir -p /lib/modules/5.4.0-90-generic
  • ln -s /usr/src_node/linux-headers-5.4.0-90-generic /lib/modules/5.4.0-90-generic/build
  • dkms install -m shiftfs -v 1.2 -k 5.4.0-90-generic --kernelsourcedir /usr/src_node/linux-headers-5.4.0-90-generic

Creating symlink /var/lib/dkms/shiftfs/1.2/source ->
/usr/src/shiftfs-1.2

DKMS: add completed.
Error! Your kernel headers for kernel 5.4.0-90-generic cannot be found.
Please install the linux-headers-5.4.0-90-generic package,
or use the --kernelsourcedir option to tell DKMS where it's located

Not able to finish script: /etc/rancher/k3s/k3s.yaml: permission denied

Hello,

I tried to install via this script. Firstly I used Ubuntu 22.04 and had this issue, then I tried Ubuntu 20.04 as that was a version I was able to use this script in the past.

No I am ending on:

WARN[0000] Unable to read /etc/rancher/k3s/k3s.yaml, please start server with --write-kubeconfig-mode to modify kube config permissions 
error: error loading config file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied

and have no idea, how to fix this. I event tried to google the issue, but no solution (like this ) did not work, but only poushed be to error:

error: no context exists with the name: "gitpod-k3s"

Here is my whole log with server IP redacted: https://paste.brcb.eu/jymuzuboti.sql

I tried everything I was able to find or debug by myself, but now I am at the end of my capabilities.

Thank you for your help!

Ending on Set the k3s config template

Hello,

i wanted to use this script to install Gitpods on fresh VM of ubuntu 22.04 server.

I created the .env file and installed all required dependecies (docker).

But I am ending on a run like this:

simonc@gitpod:~/gitpod-k3s-guide$ bash setup.sh install
Loading configuration from /home/simonc/gitpod-k3s-guide/.env.
Installing Gitpod to k3s cluster
Install k3s with k3sup
Set the k3s config template

Do you please have any idea why?

shiftfs-module-loader erroring

The shiftfs-module-loader init container is erroring and not sure of the root cause. It's probable that it's either a missing dependency. This has been tested in both Azure and GCP with their standard VMs running Ubuntu 20.04.

The nodes have had linux-headers-$(uname -r) and linux-headers-generic installed.

Terminal

Creating symlink /var/lib/dkms/shiftfs/1.2/source ->
                 /usr/src/shiftfs-1.2

DKMS: add completed.

Preparing kernel 5.11.0-1028-azure for module build:
(This is not compiling a kernel, just preparing kernel symbols)
Storing current .config to be restored when complete
Running Generic preparation routine
make mrproper....(bad exit status: 2)
/usr/sbin/dkms: line 1211: .config: Read-only file system
Warning: using /usr/src_node/linux-headers-5.11.0-1028-azure/.config
(I hope this is the correct config for this kernel)
make oldconfig...(bad exit status: 2)
make prepare-all...(bad exit status: 2)

Building module:
cleaning build area...
make -j2 KERNELRELEASE=5.11.0-1028-azure -C /usr/src_node/linux-headers-5.11.0-1028-azure M=/var/lib/dkms/shiftfs/1.2/build...(bad exit status: 2)
Error! Bad return status for module build on kernel: 5.11.0-1028-azure (x86_64)
Consult /var/lib/dkms/shiftfs/1.2/build/make.log for more information.

/var/lib/dkms/shiftfs/1.2/build/make.log

DKMS make.log for shiftfs-1.2 for kernel 5.11.0-1028-azure (x86_64)
Sun Feb 13 17:10:45 UTC 2022
make: Entering directory '/usr/src_node/linux-headers-5.11.0-1028-azure'
  CC [M]  /var/lib/dkms/shiftfs/1.2/build/shiftfs.o
/var/lib/dkms/shiftfs/1.2/build/shiftfs.c: In function 'shiftfs_fiemap':
/var/lib/dkms/shiftfs/1.2/build/shiftfs.c:727:13: error: dereferencing pointer to incomplete type 'struct fiemap_extent_info'
  if (fieinfo->fi_flags & FIEMAP_FLAG_SYNC)
             ^~
/var/lib/dkms/shiftfs/1.2/build/shiftfs.c:727:26: error: 'FIEMAP_FLAG_SYNC' undeclared (first use in this function); did you mean 'FS_XFLAG_SYNC'?
  if (fieinfo->fi_flags & FIEMAP_FLAG_SYNC)
                          ^~~~~~~~~~~~~~~~
                          FS_XFLAG_SYNC
/var/lib/dkms/shiftfs/1.2/build/shiftfs.c:727:26: note: each undeclared identifier is reported only once for each function it appears in
/var/lib/dkms/shiftfs/1.2/build/shiftfs.c: In function 'shiftfs_btrfs_ioctl_fd_restore':
/var/lib/dkms/shiftfs/1.2/build/shiftfs.c:1422:2: error: implicit declaration of function '__close_fd'; did you mean 'close_fd'? [-Werror=implicit-function-declaration]
  __close_fd(current->files, fd);
  ^~~~~~~~~~
  close_fd
cc1: some warnings being treated as errors
scripts/Makefile.build:288: recipe for target '/var/lib/dkms/shiftfs/1.2/build/shiftfs.o' failed
make[1]: *** [/var/lib/dkms/shiftfs/1.2/build/shiftfs.o] Error 1
Makefile:1849: recipe for target '/var/lib/dkms/shiftfs/1.2/build' failed
make: *** [/var/lib/dkms/shiftfs/1.2/build] Error 2
make: Leaving directory '/usr/src_node/linux-headers-5.11.0-1028-azure'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.