Giter VIP home page Giter VIP logo

Comments (11)

kvrhdn avatar kvrhdn commented on August 17, 2024

This is a Tempo issue, I'll migrate it to grafana/tempo

from tempo.

kvrhdn avatar kvrhdn commented on August 17, 2024

As Mario mentioned in slack, your backend endpoint is most likely invalid. Can you share your config?

from tempo.

raghav20241 avatar raghav20241 commented on August 17, 2024 
# -- Overrides the chart's name
nameOverride: ""

# -- Overrides the chart's computed fullname
fullnameOverride: ""

# -- Define the amount of instances
replicas: 1

# -- Number of old history to retain to allow rollback (If not set, default Kubernetes value is set to 10)
# revisionHistoryLimit: 1

# -- Annotations for the StatefulSet
annotations: {}

tempo:
  readinessProbe:
    httpGet:
      path: /ready
      port: http-traces
    initialDelaySeconds: 30
    timeoutSeconds: 1
  image:
    # -- The Docker registry
    registry: ages/docker.io
    # -- Docker image repository  
    repository: grafana/tempo
    # -- Overrides the image tag whose default is the chart's appVersion 
    tag: "2.5.0"
    pullPolicy: IfNotPresent

  ## Optionally specify an array of imagePullSecrets.
  ## Secrets must be manually created in the namespace.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
  ##
  # pullSecrets:
  #   - myRegistryKeySecretName

  updateStrategy: RollingUpdate
  resources:
    requests:
      cpu: 1000m
      memory: 4Gi
    limits:
      cpu: 2000m
      memory: 6Gi

  memBallastSizeMbs: 1024
  multitenancyEnabled: false
  # -- If true, Tempo will report anonymous usage data about the shape of a deployment to Grafana Labs
  reportingEnabled: true
  metricsGenerator:
    # -- If true, enables Tempo's metrics generator (https://grafana.com/docs/tempo/next/metrics-generator/)
    enabled: false
    remoteWriteUrl: "http://prometheus.monitoring:9090/api/v1/write"
  # -- Configuration options for the ingester
  ingester: {}
  # -- Configuration options for the querier
  querier: {}
  # -- Configuration options for the query-fronted
  queryFrontend: {}
  retention: 24h
  # Global overrides
  global_overrides:
    per_tenant_override_config: /conf/overrides.yaml
  overrides: {}

  # Tempo server configuration
  # Refers to https://grafana.com/docs/tempo/latest/configuration/#server
  server:
    # -- HTTP server listen port
    http_listen_port: 3100
  storage:
    trace:
      # tempo storage backend
      # refer https://grafana.com/docs/tempo/latest/configuration/
      ## Use s3 for example
       backend: s3
      # store traces in s3
       s3:
         bucket: tempo-stg     # store traces in this bucket
         region: eu-central-1
         access_key: ---------------
         secret_key: -----------------------
                            
      #   endpoint: s3.dualstack.us-east-2.amazonaws.com  # api endpoint
      #   access_key: ...                                 # optional. access key when using static credentials.
      #   secret_key: ...                                 # optional. secret key when using static credentials.
      #   insecure: false                                 # optional. enable if endpoint is http
      #backend: local
      #local:
        #path: /var/tempo/traces
      #wal:
       # path: /var/tempo/wal
  # this configuration will listen on all ports and protocols that tempo is capable of.
  # the receives all come from the OpenTelemetry collector.  more configuration information can
  # be found there: https://github.com/open-telemetry/opentelemetry-collector/tree/master/receiver
  receivers:
    jaeger:
      protocols:
        grpc:
          endpoint: 0.0.0.0:14250
        thrift_binary:
          endpoint: 0.0.0.0:6832
        thrift_compact:
          endpoint: 0.0.0.0:6831
        thrift_http:
          endpoint: 0.0.0.0:14268
    opencensus:
    otlp:
      protocols:
        grpc:
          endpoint: "0.0.0.0:4317"
        http:
          endpoint: "0.0.0.0:4318"
  securityContext: {}
    # allowPrivilegeEscalation: false
    #  capabilities:
    #    drop:
    #    - ALL
    # readOnlyRootFilesystem: true
  ## Additional container arguments
  extraArgs: {}
  # -- Environment variables to add
  extraEnv: []
  # -- Environment variables from secrets or configmaps to add to the ingester pods
  extraEnvFrom: []
  # -- Volume mounts to add
  extraVolumeMounts: []
  # - name: extra-volume
  #   mountPath: /mnt/volume
  #   readOnly: true
  #   existingClaim: volume-claim

# -- Tempo configuration file contents
# @default -- Dynamically generated tempo configmap
config: |
    multitenancy_enabled: {{ .Values.tempo.multitenancyEnabled }}
    usage_report:
      reporting_enabled: {{ .Values.tempo.reportingEnabled }}
    compactor:
      compaction:
        block_retention: {{ .Values.tempo.retention }}
    distributor:
      receivers:
        {{- toYaml .Values.tempo.receivers | nindent 8 }}
    ingester:
      {{- toYaml .Values.tempo.ingester | nindent 6 }}
    server:
      {{- toYaml .Values.tempo.server | nindent 6 }}
    storage:
      {{- toYaml .Values.tempo.storage | nindent 6 }}
    querier:
      {{- toYaml .Values.tempo.querier | nindent 6 }}
    query_frontend:
      {{- toYaml .Values.tempo.queryFrontend | nindent 6 }}
    overrides:
      {{- toYaml .Values.tempo.global_overrides | nindent 6 }}
      {{- if .Values.tempo.metricsGenerator.enabled }}
          metrics_generator_processors:
          - 'service-graphs'
          - 'span-metrics'
    metrics_generator:
          storage:
            path: "/tmp/tempo"
            remote_write:
              - url: {{ .Values.tempo.metricsGenerator.remoteWriteUrl }}
      {{- end }}

tempoQuery:
  repository: grafana/tempo-query
  tag: null
  pullPolicy: IfNotPresent
  ## Optionally specify an array of imagePullSecrets.
  ## Secrets must be manually created in the namespace.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
  ##
  # pullSecrets:
  #   - myRegistryKeySecretName

  # -- if False the tempo-query container is not deployed
  enabled: false

  service:
    port: 16686

  ingress:
    enabled: false
    # For Kubernetes >= 1.18 you should specify the ingress-controller via the field ingressClassName
    # See https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#specifying-the-class-of-an-ingress
    # ingressClassName: nginx
    # Values can be templated
    annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
    labels: {}
    path: /

    # pathType is only for k8s >= 1.1=
    pathType: Prefix

    hosts:
      - query.tempo.example.com
    ## Extra paths to prepend to every host configuration. This is useful when working with annotation based services.
    extraPaths: []
    # - path: /*
    #   backend:
    #     serviceName: ssl-redirect
    #     servicePort: use-annotation
    ## Or for k8s > 1.19
    # - path: /*
    #   pathType: Prefix
    #   backend:
    #     service:
    #       name: ssl-redirect
    #       port:
    #         name: use-annotation


    tls: []
    #  - secretName: tempo-query-tls
    #    hosts:
    #      - query.tempo.example.com

  resources: {}
  #  requests:
  #    cpu: 1000m
  #    memory: 4Gi
  #  limits:
  #    cpu: 2000m
  #    memory: 6Gi

  ## Additional container arguments
  extraArgs: {}
  # -- Environment variables to add
  extraEnv: []
  # -- Volume mounts to add
  extraVolumeMounts: []
  # - name: extra-volume
  #   mountPath: /mnt/volume
  #   readOnly: true
  #   existingClaim: volume-claim
  securityContext: {}
    # allowPrivilegeEscalation: false
    #  capabilities:
    #    drop:
    #    - ALL
    # readOnlyRootFilesystem: false # fails if true, do not enable

# -- securityContext for container
securityContext: {}
  # runAsUser: 65532
  # runAsGroup: 65532
  # fsGroup: 65532
  # runAsNonRoot: true

serviceAccount:
  # -- Specifies whether a ServiceAccount should be created
  create: true
  # -- The name of the ServiceAccount to use.
  # If not set and create is true, a name is generated using the fullname template
  name: null
  # -- Image pull secrets for the service account
  imagePullSecrets: []
  # -- Annotations for the service account
  annotations: {}
  # -- Labels for the service account
  labels: {}
  automountServiceAccountToken: true

service:
  type: ClusterIP
  annotations: {}
  labels: {}

serviceMonitor:
  enabled: false
  interval: ""
  additionalLabels: {}
  annotations: {}
  # scrapeTimeout: 10s

persistence:
  enabled: false
  # storageClassName: local-path
  accessModes:
    - ReadWriteOnce
  size: 10Gi

# -- Pod Annotations
podAnnotations: {}

# -- Pod (extra) Labels
podLabels: {}

# Apply extra labels to common labels.
extraLabels: {}

# -- Volumes to add
extraVolumes: []

# -- Node labels for pod assignment. See: https://kubernetes.io/docs/user-guide/node-selection/
nodeSelector: {}

# -- Tolerations for pod assignment. See: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
tolerations: []

# -- Affinity for pod assignment. See: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
affinity: {}

# -- The name of the PriorityClass
priorityClassName: null

from tempo.

mapno avatar mapno commented on August 17, 2024

You don't have an endpoint defined in your S3 configuration. Notice that it's commented out.

#   endpoint: s3.dualstack.us-east-2.amazonaws.com  # api endpoint

from tempo.

raghav20241 avatar raghav20241 commented on August 17, 2024

I have added in values.yaml stil getting same issue can you share any values.yaml with helm for example
policy attached to tempo user for s3 backend
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "TempoPermissions",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket",
"s3:DeleteObject",
"s3:GetObjectTagging",
"s3:PutObjectTagging"
],
"Resource": [
"arn:aws:s3:::tempo-stg/*",
"arn:aws:s3:::tempo-stg"
]
}
]

from tempo.

kvrhdn avatar kvrhdn commented on August 17, 2024

Can you share the final tempo configmap? Specifically the storage section. It should look something like this generally

storage:
  trace:
    backend: s3
    block:
      version: vParquet4
    s3:
      bucket: <bucket name>
      endpoint: s3.dualstack.us-east-2.amazonaws.com
    wal:
      path: /var/tempo/wal
      version: vParquet4

from tempo.

raghav20241 avatar raghav20241 commented on August 17, 2024 
# -- Overrides the chart's name
nameOverride: ""

# -- Overrides the chart's computed fullname
fullnameOverride: ""

# -- Define the amount of instances
replicas: 1

# -- Number of ol      accesssecret_key: history to retain to allow rollback (If not set, default Kubernetes value is set to 10)
# revisionHistoryLimit: 1

# -- Annotations for the StatefulSet
annotations: {}

tempo:
  repository: grafana/tempo
  tag: ""
  pullPolicy: IfNotPresent
  ## Optionally specify an array of imagePullSecrets.
  ## Secrets must be manually created in the namespace.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
  ##
  # pullSecrets:
  #   - myRegistryKeySecretName

  updateStrategy: RollingUpdate
  resources:
    requests:
      cpu: 1000m
      memory: 4Gi
    limits:
      cpu: 2000m
      memory: 6Gi

  memBallastSizeMbs: 1024
  multitenancyEnabled: false
  # -- If true, Tempo will report anonymous usage data about the shape of a deployment to Grafana Labs
  reportingEnabled: true
  metricsGenerator:
    # -- If true, enables Tempo's metrics generator (https://grafana.com/docs/tempo/next/metrics-generator/)
    enabled: false
    remoteWriteUrl: "http://prometheus.monitoring:9090/api/v1/write"
  # -- Configuration options for the ingester
  ingester: {}
  # -- Configuration options for the querier
  querier: {}
  # -- Configuration options for the query-fronted
  queryFrontend: {}
  retention: 24h
  # Global overrides
  global_overrides:
    per_tenant_override_config: /conf/overrides.yaml
  overrides: {}

  # Tempo server configuration
  # Refers to https://grafana.com/docs/tempo/latest/configuration/#server
  server:
    # -- HTTP server listen port
    http_listen_port: 3100
  storage:
    trace:
      backend: s3
      block:
        version: vParquet4
    s3:
      bucket: tempo-stg
      region: eu-central-1
      endpoint: s3.dualstack.eu-central-1.amazonaws.com
      access_key: ------
      secret_key: --------

    wal:
      path: /var/tempo/wal
      version: vParquet4
      #storage:
      #trace:
      # tempo storage backend
      # refer https://grafana.com/docs/tempo/latest/configuration/
      ## Use s3 for example
      # backend: s3
      # store traces in s3
      # s3:
      #   bucket: <your s3 bucket>                        # store traces in this bucket
      #   endpoint: s3.dualstack.us-east-2.amazonaws.com  # api endpoint
      #   access_key: ...                                 # optional. access key when using static credentials.
      #   secret_key: ...                                 # optional. secret key when using static credentials.
      #   insecure: false                                 # optional. enable if endpoint is http
      #backend: local
      #local:
      # path: /var/tempo/traces
      #wal:
      # path: /var/tempo/wal
  # this configuration will listen on all ports and protocols that tempo is capable of.
  # the receives all come from the OpenTelemetry collector.  more configuration information can
  # be found there: https://github.com/open-telemetry/opentelemetry-collector/tree/master/receiver
  receivers:
    jaeger:
      protocols:
        grpc:
          endpoint: 0.0.0.0:14250
        thrift_binary:
          endpoint: 0.0.0.0:6832
        thrift_compact:
          endpoint: 0.0.0.0:6831
        thrift_http:
          endpoint: 0.0.0.0:14268
    opencensus:
    otlp:
      protocols:
        grpc:
          endpoint: "0.0.0.0:4317"
        http:
          endpoint: "0.0.0.0:4318"
  securityContext: {}
    # allowPrivilegeEscalation: false
    #  capabilities:
    #    drop:
    #    - ALL
    # readOnlyRootFilesystem: true
  ## Additional container arguments
  extraArgs: {}
  # -- Environment variables to add
  extraEnv: []
  # -- Environment variables from secrets or configmaps to add to the ingester pods
  extraEnvFrom: []
  # -- Volume mounts to add
  extraVolumeMounts: []
  # - name: extra-volume
  #   mountPath: /mnt/volume
  #   readOnly: true
  #   existingClaim: volume-claim

# -- Tempo configuration file contents
# @default -- Dynamically generated tempo configmap
config: |
    multitenancy_enabled: {{ .Values.tempo.multitenancyEnabled }}
    usage_report:
      reporting_enabled: {{ .Values.tempo.reportingEnabled }}
    compactor:
      compaction:
        block_retention: {{ .Values.tempo.retention }}
    distributor:
      receivers:
        {{- toYaml .Values.tempo.receivers | nindent 8 }}
    ingester:
      {{- toYaml .Values.tempo.ingester | nindent 6 }}
    server:
      {{- toYaml .Values.tempo.server | nindent 6 }}
    storage:
      {{- toYaml .Values.tempo.storage | nindent 6 }}
    querier:
      {{- toYaml .Values.tempo.querier | nindent 6 }}
    query_frontend:
      {{- toYaml .Values.tempo.queryFrontend | nindent 6 }}
    overrides:
      {{- toYaml .Values.tempo.global_overrides | nindent 6 }}
      {{- if .Values.tempo.metricsGenerator.enabled }}
          metrics_generator_processors:
          - 'service-graphs'
          - 'span-metrics'
    metrics_generator:
          storage:
            path: "/tmp/tempo"
            remote_write:
              - url: {{ .Values.tempo.metricsGenerator.remoteWriteUrl }}
      {{- end }}

tempoQuery:
  repository: grafana/tempo-query
  tag: null
  pullPolicy: IfNotPresent
  ## Optionally specify an array of imagePullSecrets.
  ## Secrets must be manually created in the namespace.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
  ##
  # pullSecrets:
  #   - myRegistryKeySecretName

  # -- if False the tempo-query container is not deployed
  enabled: false

  service:
    port: 16686

  ingress:
    enabled: false
    # For Kubernetes >= 1.18 you should specify the ingress-controller via the field ingressClassName
    # See https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#specifying-the-class-of-an-ingress
    # ingressClassName: nginx
    # Values can be templated
    annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
    labels: {}
    path: /

    # pathType is only for k8s >= 1.1=
    pathType: Prefix

    hosts:
      - query.tempo.example.com
    ## Extra paths to prepend to every host configuration. This is useful when working with annotation based services.
    extraPaths: []
    # - path: /*
    #   backend:
    #     serviceName: ssl-redirect
    #     servicePort: use-annotation
    ## Or for k8s > 1.19
    # - path: /*
    #   pathType: Prefix
    #   backend:
    #     service:
    #       name: ssl-redirect
    #       port:
    #         name: use-annotation


    tls: []
    #  - secretName: tempo-query-tls
    #    hosts:
    #      - query.tempo.example.com

  resources: {}
  #  requests:
  #    cpu: 1000m
  #    memory: 4Gi
  #  limits:
  #    cpu: 2000m
  #    memory: 6Gi

  ## Additional container arguments
  extraArgs: {}
  # -- Environment variables to add
  extraEnv: []
  # -- Volume mounts to add
  extraVolumeMounts: []
  # - name: extra-volume
  #   mountPath: /mnt/volume
  #   readOnly: true
  #   existingClaim: volume-claim
  securityContext: {}
    # allowPrivilegeEscalation: false
    #  capabilities:
    #    drop:
    #    - ALL
    # readOnlyRootFilesystem: false # fails if true, do not enable

# -- securityContext for container
securityContext: {}
  # runAsUser: 65532
  # runAsGroup: 65532
  # fsGroup: 65532
  # runAsNonRoot: true

serviceAccount:
  # -- Specifies whether a ServiceAccount should be created
  create: true
  # -- The name of the ServiceAccount to use.
  # If not set and create is true, a name is generated using the fullname template
  name: null
  # -- Image pull secrets for the service account
  imagePullSecrets: []
  # -- Annotations for the service account
  annotations: {}
  # -- Labels for the service account
  labels: {}
  automountServiceAccountToken: true

service:
  type: ClusterIP
  annotations: {}
  labels: {}

serviceMonitor:
  enabled: false
  interval: ""
  additionalLabels: {}
  annotations: {}
  # scrapeTimeout: 10s

persistence:
  enabled: false
  # storageClassName: local-path
  accessModes:
    - ReadWriteOnce
  size: 10Gi

# -- Pod Annotations
podAnnotations: {}

# -- Pod (extra) Labels
podLabels: {}

# Apply extra labels to common labels.
extraLabels: {}

# -- Volumes to add
extraVolumes: []

# -- Node labels for pod assignment. See: https://kubernetes.io/docs/user-guide/node-selection/
nodeSelector: {}

# -- Tolerations for pod assignment. See: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
tolerations: []

# -- Affinity for pod assignment. See: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
affinity: {}

# -- The name of the PriorityClass
priorityClassName: null

from tempo.

raghav20241 avatar raghav20241 commented on August 17, 2024

failed parsing config: failed to parse configFile /conf/tempo.yaml: yaml: unmarshal errors:
line 30: field s3 not found in type storage.Config
line 44: field wal not found in type storage.Config

from tempo.

raghav20241 avatar raghav20241 commented on August 17, 2024

above shared is values.yaml and below is config map taken from artifact hub default

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ template "tempo.name" . }}
  namespace: {{ .Release.Namespace }}
  labels:
    {{- include "tempo.labels" . | nindent 4 }}
data:
  overrides.yaml: |
    overrides:
      {{- toYaml .Values.tempo.overrides | nindent 6 }}
  tempo.yaml: |
    {{- tpl .Values.config . | nindent 4 }}

from tempo.

raghav20241 avatar raghav20241 commented on August 17, 2024

hi any changes to be made for files

from tempo.

mapno avatar mapno commented on August 17, 2024

I believe the s3 and wal blocks need one more indentation. They're in storage.trace, not storage.

from tempo.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.