Kubernetes Manifest Files about go-random-chat HOT 14 CLOSED

Yes, any Redis cluster will work!
As for uploader, if you don't need the file uploading feature, you could just ignore it as well as S3 deployment.

If there is any question, feel free to let me know.

from go-random-chat.

That was indeed the problem! Got fixed once I exposed the ports 4000 on both chat and user.

Thanks again for your patience! I am not closing this thread as of now because I might have queries regarding minio (S3). If you would like it, then I will be more than happy to share my manifest files as a PR.

from go-random-chat.

Hi,

There are three main parts for deployment, stateless application, stateful storages and Traefik (reverse proxy). You should prepare all of them in order to have everything up and running.

1. Stateless application: including web, user, match, chat, and uploader. Their images are already pushed to Dockerhub, so you could deploy them by writing Kubernetes Deployment manifest files. However for each service, you have to change storage settings to your own one in the env.
2. Stateful storages: including Redis cluster, S3-compatible object storage, and optionally Prometheus + Jaeger for observability. You could either deploy them in Kubernetes cluster directly or use external services like Amazon Elasticache and S3.
3. Traefik (reverse proxy): if you deploy everything on Kubernetes, you don't really need it as manages Kubernetes already has its own ingress (depending on what cloud provider you choose). Kubernetes ingress is another big topic, and I'm not sure how much you know about it.

Simply convert docker-compose to K8s manifest will not work here as this project is a bit complicated.

from go-random-chat.

Thanks for your response!

yes I was able to figure out #1 (stateless applications).

#2: For redis, I am using a default redis cluster made by bitnami using helm charts: https://bitnami.com/stack/redis-cluster/helm
. Do you think this template redis cluster will work with your chat application? I am for now ignoring S3 because I believe that is being consumed only by the uploader tool (which I don’t need really). Is my understanding correct about S3 being used only for uploader tool?

#3: Yes I have been struggling a little with traefik, more specifically ingress routes and writing their kubernetes manifests somehow gives me errors. I will default to kubernetes ingress controller instead.

from go-random-chat.

Hi, I am trying to get the redis cluster to work with the stateless microservices like user, match, chat etc. I am using a standard helm chart of redis (https://bitnami.com/stack/redis-cluster/helm). However, I keep on getting the following error:

_level=fatal msg="dial tcp: lookup my-release-redis-master.redis.svc.cluster.local on 10.96.0.10:53: no such host"_
This is the deployment manifest file for the user that I am using.

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.26.1 (HEAD)
    traefik.enable: "true"
    traefik.http.routers.user-grpc.entrypoints: web
    traefik.http.routers.user-grpc.rule: Headers(`content-type`,`application/grpc`) && Headers(`service-id`, `user`)
    traefik.http.routers.user-grpc.service: user-grpc
    traefik.http.routers.user.entrypoints: web
    traefik.http.routers.user.rule: PathPrefix(`/api/user`)
    traefik.http.routers.user.service: user
    traefik.http.services.user-grpc.loadbalancer.server.port: "4000"
    traefik.http.services.user-grpc.loadbalancer.server.scheme: h2c
    traefik.http.services.user.loadbalancer.server.port: "80"
  creationTimestamp: null
  labels:
    io.kompose.service: user
  name: user
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: user
  strategy: {}
  template:
    metadata:
      annotations:
        kompose.cmd: kompose convert
        kompose.version: 1.26.1 (HEAD)
        traefik.enable: "true"
        traefik.http.routers.user-grpc.entrypoints: web
        traefik.http.routers.user-grpc.rule: Headers(`content-type`,`application/grpc`) && Headers(`service-id`, `user`)
        traefik.http.routers.user-grpc.service: user-grpc
        traefik.http.routers.user.entrypoints: web
        traefik.http.routers.user.rule: PathPrefix(`/api/user`)
        traefik.http.routers.user.service: user
        traefik.http.services.user-grpc.loadbalancer.server.port: "4000"
        traefik.http.services.user-grpc.loadbalancer.server.scheme: h2c
        traefik.http.services.user.loadbalancer.server.port: "80"
      creationTimestamp: null
      labels:
        io.kompose.service: user
    spec:
      containers:
        - args:
            - user
          env:
            - name: OBSERVABILITY_PROMETHEUS_PORT
              value: "8080"
            - name: OBSERVABILITY_TRACING_JAEGERURL
              value: http://jaeger:14268/api/traces
            - name: REDIS_ADDRS
              value: my-release-redis-master.redis.svc.cluster.local:6379
            - name: REDIS_PASSWORD # REDIS_PASSWORD is used by the app to authenticate with the redis host
              value: ${REDIS_PASSWORD}
            - name: REDIS_EXPIRATIONHOUR
              value: "24"
            - name: USER_GRPC_SERVER_PORT
              value: "4000"
            - name: USER_HTTP_SERVER_PORT
              value: "80"
          image: minghsu0107/random-chat-api:main
          name: user
          ports:
            - containerPort: 80
          resources: {}
      restartPolicy: Always
status: {}

from go-random-chat.

Did you deploy Redis cluster successfully? The error message seems like my-release-redis-master.redis.svc.cluster.local is not in the Kubernetes DNS.

from go-random-chat.

When ssh'ing into the redis master and running an info replication command I get the following:

127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:3
slave0:ip=redis-replicas-0.,port=6379,state=online,offset=13468,lag=1
slave1:ip=redis-replicas-1.,port=6379,state=online,offset=13468,lag=0
slave2:ip=redis-replicas-2.,port=6379,state=online,offset=13468,lag=0
master_failover_state:no-failover
master_replid:ca853f3f970074e31821637022d33e9e041f5547
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:13468
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:13468

Although I did skip the step that is prompted after the redis helm chart is deployed.

`
To connect to your Redis™ server:

1. Run a Redis™ pod that you can use as a client:

   kubectl run --namespace redis redis-client --restart='Never' --env REDIS_PASSWORD=$REDIS_PASSWORD --image docker.io/bitnami/redis:6.2.7-debian-10-r23 --command -- sleep infinity

   Use the following command to attach to the pod:

   kubectl exec --tty -i redis-client
   --namespace redis -- bash

2. Connect using the Redis™ CLI:
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-master
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-replicas

To connect to your database from outside the cluster execute the following commands:

kubectl port-forward --namespace redis svc/redis-master 6379:6379 &
REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379

`

Is this part important too?

from go-random-chat.

I think the redis nodes have started but they are not exposed. Do you think I should expose them using a NodePort or LoadBalancer? Although since all other services are hosted in the same cluster, so is it important?

from go-random-chat.

You should expose your Redis with Kubernetes Service. Since all applications connects to Redis internally, external exposure such as NodePort and LoadBalancer are not needed.

By the way, the application works only when connecting to Redis cluster:

But what you have deployed is Redis with master-slave mode:

So you may want to redeploy your Redis using https://github.com/bitnami/charts/tree/master/bitnami/redis-cluster instead.

from go-random-chat.

I did try redis-cluster too. Even that seems to give the same error: No such host found.

Are you suggesting that I should expose each and every redis pod deployment using the following command?

kubectl expose deployment redis-cluster-0
kubectl expose deployment redis-cluster-1
kubectl expose deployment redis-cluster-2

and so on?

from go-random-chat.

Hi,

I was able to get around the problem of redis-cluster connection to microservices. Turns out I was not using proper addressing to communicate. I was using <pod-name>.<random-service-name>.<namespace>:6379 and then I moved to use the right address which is: <pod-name>.<headless-service>.<namespace>:6379
However, now I have another problem of websockets. When I click on the match button, I get the following error on the browser:

WebSocket connection to 'ws://hello-world.info/api/match?uid=410921637035114503' failed:
More specifically ln 107 in assets/home.js

ws = new WebSocket(matchUrl)
On the server logs for the match deployment, I get this error:
time="2022-06-06T03:11:26Z" level=error msg="rpc error: code = Unavailable desc = last resolver error: produced zero addresses" type=http time="2022-06-06T03:11:26Z" level=error duration_ms=0.55 method=GET path="/api/match?uid=410967052673613831" referrer= status=500 trace_id= type=http

Any suggestions or hints on how to resolve this problem?

from go-random-chat.

match service will make grpc requests to user and chat in order to check whether user exists and create a channel. So what you are missing are grpc endpoints for user and chat service respectively. You should specify them in env MATCH_GRPC_CLIENT_USER_ENDPOINT and MATCH_GRPC_CLIENT_CHAT_ENDPOINT.

from go-random-chat.

Thanks for the prompt response! Yes that error seems resolved now. But now I am getting another timeout error on the server logs:

desc = \"transport: Error while dialing dial tcp 10.111.186.81:4000: i/o timeout\"" type=http time="2022-06-06T06:11:05Z" level=error duration_ms=2791.89 method=GET path="/api/match?uid=410984621723877381" referrer= status=500 trace_id= type=http

from go-random-chat.

Hi,
It seems like user or chat is not reachable from match. You can check whether grpc port 4000 is exposed for your pod.

from go-random-chat.