stazdx / otel-springboot-grafana-tools Goto Github PK

helm upgrade --install tempo grafana/tempo-distributed -n observability -f tempo.yaml
Release "tempo" has been upgraded. Happy Helming!
NAME: tempo
LAST DEPLOYED: Thu May 11 13:44:30 2023
NAMESPACE: observability
STATUS: deployed
REVISION: 3
TEST SUITE: None
NOTES:
***********************************************************************
 Welcome to Grafana Tempo
 Chart version: 1.3.0
 Tempo version: 2.1.1
***********************************************************************

Installed components:
* ingester
* distributor
* querier
* query-frontend
* compactor
* memcached

kubectl get po -n observability
NAME                                                        READY   STATUS              RESTARTS      AGE
promtail-85dn2                                              1/1     Running             0             22m
loki-loki-distributed-gateway-55df6fdb4f-fcrqd              1/1     Running             0             20m
loki-loki-distributed-query-frontend-67fddf75cc-p9tr8       1/1     Running             0             20m
loki-loki-distributed-distributor-6b45465f45-d47tz          1/1     Running             0             20m
loki-loki-distributed-querier-0                             1/1     Running             0             20m
loki-loki-distributed-ingester-0                            1/1     Running             0             20m
tempo-memcached-0                                           1/1     Running             0             17m
kube-prometheus-stack-grafana-57d7b4968c-xs26t              0/3     ContainerCreating   0             15m
kube-prometheus-stack-operator-599c5b6fff-2qh8h             1/1     Running             0             15m
kube-prometheus-stack-prometheus-node-exporter-rqf9v        1/1     Running             0             15m
kube-prometheus-stack-kube-state-metrics-65b455c7b6-jpnlj   1/1     Running             0             15m
alertmanager-kube-prometheus-stack-alertmanager-0           2/2     Running             0             15m
prometheus-kube-prometheus-stack-prometheus-0               2/2     Running             0             15m
tempo-ingester-2                                            0/1     CrashLoopBackOff    8 (55s ago)   17m
tempo-ingester-1                                            0/1     CrashLoopBackOff    8 (53s ago)   17m
tempo-ingester-0                                            0/1     CrashLoopBackOff    8 (41s ago)   17m
tempo-compactor-877495479-8l67l                             0/1     CrashLoopBackOff    8 (48s ago)   17m
tempo-querier-9747c4994-r2lvb                               0/1     CrashLoopBackOff    8 (45s ago)   17m
tempo-distributor-d79d4d4fb-j8ltk                           0/1     CrashLoopBackOff    8 (30s ago)   17m
tempo-query-frontend-bc468b9bc-n8dbh                        0/1     CrashLoopBackOff    8 (40s ago)   17m

kubectl  describe  pod  tempo-distributor-d79d4d4fb-j8ltk    -n observability
Name:             tempo-distributor-d79d4d4fb-j8ltk
Namespace:        observability
Priority:         0
Service Account:  tempo
Node:             vmdemo/192.168.64.2
Start Time:       Thu, 11 May 2023 13:28:21 +0800
Labels:           app.kubernetes.io/component=distributor
                  app.kubernetes.io/instance=tempo
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=tempo
                  app.kubernetes.io/part-of=memberlist
                  app.kubernetes.io/version=2.1.1
                  helm.sh/chart=tempo-distributed-1.3.0
                  pod-template-hash=d79d4d4fb
Annotations:      checksum/config: 6b25904b7a1aef8cdc4feb63bfff42e8bb5ebc9cfe094c8429a7f47be6fba8b2
Status:           Running
IP:               10.42.0.20
IPs:
  IP:           10.42.0.20
Controlled By:  ReplicaSet/tempo-distributor-d79d4d4fb
Containers:
  distributor:
    Container ID:  containerd://3c52ae1d6412601407a628b5a1db6605a0979abc079c9974c186dc53253a7a78
    Image:         docker.io/grafana/tempo:2.1.1
    Image ID:      docker.io/grafana/tempo@sha256:d9eb004dc35c0dfab8bb7ced02604079628977014ac180c0c98c034aa8b6a705
    Ports:         7946/TCP, 3100/TCP, 4318/TCP, 4317/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP, 0/TCP
    Args:
      -target=distributor
      -config.file=/conf/tempo.yaml
      -mem-ballast-size-mbs=1024
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Thu, 11 May 2023 13:44:59 +0800
      Finished:     Thu, 11 May 2023 13:45:00 +0800
    Ready:          False
    Restart Count:  8
    Readiness:      http-get http://:http-metrics/ready delay=30s timeout=1s period=10s #success=1 #failure=3
    Environment:    <none>
    Mounts:
      /conf from config (rw)
      /var/tempo from tempo-distributor-store (rw)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      tempo-config
    Optional:  false
  tempo-distributor-store:
    Type:                     EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:                   
    SizeLimit:                <unset>
QoS Class:                    BestEffort
Node-Selectors:               <none>
Tolerations:                  node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                              node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Topology Spread Constraints:  failure-domain.beta.kubernetes.io/zone:ScheduleAnyway when max skew 1 is exceeded for selector app.kubernetes.io/component=distributor,app.kubernetes.io/instance=tempo,app.kubernetes.io/name=tempo
Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  18m                   default-scheduler  Successfully assigned observability/tempo-distributor-d79d4d4fb-j8ltk to vmdemo
  Normal   Pulling    18m                   kubelet            Pulling image "docker.io/grafana/tempo:2.1.1"
  Normal   Pulled     18m                   kubelet            Successfully pulled image "docker.io/grafana/tempo:2.1.1" in 23.69951422s (23.69951926s including waiting)
  Normal   Created    17m (x4 over 18m)     kubelet            Created container distributor
  Normal   Started    17m (x4 over 18m)     kubelet            Started container distributor
  Normal   Pulled     16m (x4 over 18m)     kubelet            Container image "docker.io/grafana/tempo:2.1.1" already present on machine
  Warning  BackOff    3m27s (x75 over 18m)  kubelet            Back-off restarting failed container distributor in pod tempo-distributor-d79d4d4fb-j8ltk_observability(38d90696-5c17-4330-9595-d1e5e4b76b20)