jaegertracing / jaeger-idl Goto Github PK

Requirement

In protobuf 3.20.0, we can use protoc to generate pyi for python's type hint. This has been pulled into a released gRPC version(>= 1.48.2, as metioned here: grpc/grpc#29650 (comment))

Note that the current version of grpc is 1.35.0, so in the next upgrade, if the version is met, hopefully the relevant builds will be added.

Problem

pyi will provide python users with a great programming experience and enable type-checking tools.

Proposal

No response

Open questions

No response

Hello All,

I do know how to retrieve the trace from jaeger using TraceID with help of curl command. Can anyone suggest here how to do the same using gRPc. It will greatly help in my research.

Hey,

I'm trying to generate all needed python files to use the gRPC api inside my own python project. I've seen that @lonewolf3739 has fixed some generation problems (jaegertracing/jaeger-client-csharp#197, https://githubmemory.com/repo/jaegertracing/docker-protobuf/issues).
After all I'm struggling on how to do it at all.

I've generated a model_pb2.py and a query_pb2.py file. As soon as I tried to use them in my code it throws an ModuleNotFoundError: No module named 'gogoproto'
Next I've generated the dependency code. gogo_pb2.py is now located in the same location as the files above.
I'm still getting the ModuleNotFoundError

Could someone explain me how to structure the files correctly?

According to the sampling.thrift spec, the maxTracesPerSecond field should be an short value. The Java implementation uses a double value in it's ratelimiting_sampling.json and also in it's implementation.

I assume the thrift file is out of date and double is the correct type?

Describe the bug
The go bindings have been broken for a while because of upstream changes in open-telemetry/opentracing - they recently merged a PR that will fix golang bindings built from their .proto files

To Reproduce
Steps to reproduce the behavior:

1. make proto
2. oh no the golang proto files don't build

Expected behavior
The proto files to build

Screenshots
nope

Version (please complete the following information):

• OS: linux
• Jaeger version:
• Deployment:

What troubleshooting steps did you try?

Additional context
the cause was the upstream proto files depending on generated go that didn't exist anywhere publicly, they now import the correct repository

The opentracing-proto submodule needs to be bumped to (at least) open-telemetry/opentelemetry-proto@0e254b5

I avoided just making a PR to do this because I'm unsure of any other changes that might affect this repository

What happened?

The golang package of model.proto should be api_v2. There is even a comment that states this...

Steps to reproduce

1. Generate golang protobuf
2. Notice it has a package of model.

Expected behavior

The golang protobuf should compile without errors.

Relevant log output

No response

Screenshot

No response

Additional context

No response

Jaeger backend version

No response

SDK

No response

Pipeline

No response

Stogage backend

No response

Operating system

No response

Deployment model

No response

Deployment configs

No response

tracetest.thrift declares a recursive member in its struct. I have found the generated code fails to compile in C++. It seems Thrift does not support any sort of recursive types as far as I can tell. What is the best way to fix/address this?

SamplingStrategyResponse can contain one of 3 types of sampling strategies, but the SamplingStrategyType enum only defines 2 values.

The strategyType field is not used by the Go client: https://github.com/jaegertracing/jaeger-client-go/blob/master/sampler_remote.go

Need to check other langs.

The main blocker is we need to find a way to automatically remove gogo annotations from .proto files, OR prepare a build tool-chain (e.g. a Docker image) where all gogo plugins are installed and it can be used to generate code for other languages.

Related issues:

• jaegertracing/jaeger#1484 "add protobuf model to jaeger-idl"
• jaegertracing/jaeger#1213 "ignore gogo annotations"

A trace is a collection of multiple spans. What does it mean to pass maxDuration or minDuration to the /api/traces endpoint? The documentation says:

minDuration: Ex. 100000 (for 100ms). Only return traces whose Span.duration is greater than or equal to minDuration microseconds.
maxDuration: Only return traces whose Span.duration is less than or equal to maxDuration microseconds. Only valid with minDuration.

Must all spans in a trace be larger than minDuration, if set? Must only one span in a trace fit within the min and max duration parameters? It's not really obvious what the intention is here.

Thrift span def doesn't have process field and Batch makes it mandatory. Proto span def has process as mandatory span field and for Batch it is nullable. Is this desired? Why should there be process field for proto span definition since it remains constant for all the telemetry within same process?

Hi everyone, as already mentioned on the jaeger slack channel on tuesday we have issues fetching traces via gRPC when using the latest version of the docker all-in-one image. When using an older image version, we used 1.18, it works perfectly fine. Here is our code we used to test against the images:

import com.google.protobuf.ByteString
import com.google.protobuf.Timestamp
import io.grpc.ManagedChannelBuilder
import io.jaegertracing.api_v2.Query
import io.jaegertracing.api_v2.QueryServiceGrpc
import io.vertx.core.json.JsonArray
import io.vertx.core.json.JsonObject
import io.vertx.core.logging.LoggerFactory
import io.vertx.ext.unit.TestContext
import io.vertx.ext.unit.junit.VertxUnitRunner
import org.apache.commons.codec.binary.Hex
import org.junit.BeforeClass
import org.junit.Test
import org.junit.runner.RunWith
import java.time.Instant
import java.time.LocalDateTime
import java.time.ZoneId
import java.time.format.DateTimeFormatter

@RunWith(VertxUnitRunner::class)
class JaegerGrpcTest {

companion object {

    lateinit var queryService: QueryServiceGrpc.QueryServiceBlockingStub

    private val jaegerIp = "127.0.0.1"
    private val jaegerPort = "16686"
    private val logger = LoggerFactory.getLogger(JaegerGrpcTest::class.java)

    @BeforeClass
    @JvmStatic
    fun setup(testContext: TestContext) {

        queryService = createJaegerQueryService()
    }


    private fun createJaegerQueryService(): QueryServiceGrpc.QueryServiceBlockingStub {
        logger.info("creating grpc query service with host $jaegerIp")
        val channel = ManagedChannelBuilder.forTarget("$jaegerIp:$jaegerPort").usePlaintext().build()
        return QueryServiceGrpc.newBlockingStub(channel)
    }

}

@Test
fun testFetchTraceStatistics() {
    val startService = "test-service"
    val startTime = 1616667377L 
    val traceQuery = buildTraceQuery(startService, startTime)
    val traceRequest = buildFindTraceRequest(traceQuery)
    val traces = queryService.findTraces(traceRequest)
    if (traces.hasNext()){
        logger.info("found traces for provided service and start time")
    } else {
        logger.info("no traces found for the provided look back time")
    }
}

private fun buildTraceQuery(tracingStartService: String?, startTime: Long): Query.TraceQueryParameters {
    val endTime = System.currentTimeMillis() / 1000
    logger.info("building trace query with start $startTime and end $endTime using service $tracingStartService")
    return Query.TraceQueryParameters.newBuilder()
        .setStartTimeMin(Timestamp.getDefaultInstance().toBuilder().setSeconds(startTime))
        .setStartTimeMax(Timestamp.getDefaultInstance().toBuilder().setSeconds(endTime))
        .setServiceName(tracingStartService)
        .build()
}

private fun buildFindTraceRequest(traceQuery: Query.TraceQueryParameters): Query.FindTracesRequest {
    logger.info("building trace request for fetching traces from jaeger")
    return Query.FindTracesRequest.newBuilder().setQuery(traceQuery).build()
}

}

defaultUpperBoundTracesPerSecond was going to be used to throttle the number of traces that were generated but we've moved onto using an actual throttler. This variable is no longer necessary

hey guys, help me please !!! I meet the same question in thrift. when I try to make it, it shows error like this:make[3]: Entering directory /home/worker/thrift/thrift/compiler/cpp/src' /bin/sh ../../../ylwrap thrift/thrifty.yy y.tab.c thrift/thrifty.cc y.tab.h echo thrift/thrifty.cc | sed -e s/cc$/hh/ -e s/cpp$/hpp/ -e s/cxx$/hxx/ -e s/c++$/h++/ -e s/c$/h/` y.output thrift/thrifty.output -- byacc -d
byacc: e - line 1 of "/home/worker/thrift/thrift/compiler/cpp/src/thrift/thrifty.yy", syntax error
%code requires {
^

do you have some mind

What happened?

I followed the guide to clone the repo and run make proto on my local desktop, then I am seeing this error

$ make proto
mkdir -p proto-gen-go \
		proto-gen-java \
		proto-gen-python \
		proto-gen-js \
		proto-gen-cpp \
		proto-gen-csharp
docker run --rm -u 501 -v "/Users/haoyuxu/workspace/jaeger-idl:/Users/haoyuxu/workspace/jaeger-idl" -w /Users/haoyuxu/workspace/jaeger-idl jaegertracing/protobuf:0.5.0 --proto_path=/Users/haoyuxu/workspace/jaeger-idl -Iproto/api_v2 -Iproto -I/usr/include/github.com/gogo/protobuf -Iopentelemetry-proto --gogo_out=plugins=grpc,Mgoogle/protobuf/descriptor.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/empty.proto=github.com/gogo/protobuf/types,Mgoogle/api/annotations.proto=github.com/gogo/googleapis/google/api,Mmodel.proto=github.com/jaegertracing/jaeger/model:/Users/haoyuxu/workspace/jaeger-idl/proto-gen-go --java_out=proto-gen-java --python_out=proto-gen-python --js_out=proto-gen-js --cpp_out=proto-gen-cpp --csharp_out=base_namespace:proto-gen-csharp \
		proto/api_v2/model.proto
docker run --rm -u 501 -v "/Users/haoyuxu/workspace/jaeger-idl:/Users/haoyuxu/workspace/jaeger-idl" -w /Users/haoyuxu/workspace/jaeger-idl jaegertracing/protobuf:0.5.0 --proto_path=/Users/haoyuxu/workspace/jaeger-idl -Iproto/api_v2 -Iproto -I/usr/include/github.com/gogo/protobuf -Iopentelemetry-proto --gogo_out=plugins=grpc,Mgoogle/protobuf/descriptor.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/empty.proto=github.com/gogo/protobuf/types,Mgoogle/api/annotations.proto=github.com/gogo/googleapis/google/api,Mmodel.proto=github.com/jaegertracing/jaeger/model:/Users/haoyuxu/workspace/jaeger-idl/proto-gen-go --java_out=proto-gen-java --python_out=proto-gen-python --js_out=proto-gen-js --cpp_out=proto-gen-cpp --csharp_out=base_namespace:proto-gen-csharp --grpc-java_out=proto-gen-java --grpc-python_out=proto-gen-python --grpc-js_out=proto-gen-js --grpc-cpp_out=proto-gen-cpp --grpc-csharp_out=proto-gen-csharp \
		proto/api_v2/query.proto \
		proto/api_v2/collector.proto \
		proto/api_v2/sampling.proto
--grpc-csharp_out: protoc-gen-grpc-csharp: Plugin killed by signal 11.
make: *** [proto-api-v2] Error 1

Steps to reproduce

1. Clone the repo
2. Run make proto

Expected behavior

All the gRPC files should be generated.

Relevant log output

$ make proto
mkdir -p proto-gen-go \
		proto-gen-java \
		proto-gen-python \
		proto-gen-js \
		proto-gen-cpp \
		proto-gen-csharp
docker run --rm -u 501 -v "/Users/haoyuxu/workspace/jaeger-idl:/Users/haoyuxu/workspace/jaeger-idl" -w /Users/haoyuxu/workspace/jaeger-idl jaegertracing/protobuf:0.5.0 --proto_path=/Users/haoyuxu/workspace/jaeger-idl -Iproto/api_v2 -Iproto -I/usr/include/github.com/gogo/protobuf -Iopentelemetry-proto --gogo_out=plugins=grpc,Mgoogle/protobuf/descriptor.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/empty.proto=github.com/gogo/protobuf/types,Mgoogle/api/annotations.proto=github.com/gogo/googleapis/google/api,Mmodel.proto=github.com/jaegertracing/jaeger/model:/Users/haoyuxu/workspace/jaeger-idl/proto-gen-go --java_out=proto-gen-java --python_out=proto-gen-python --js_out=proto-gen-js --cpp_out=proto-gen-cpp --csharp_out=base_namespace:proto-gen-csharp \
		proto/api_v2/model.proto
docker run --rm -u 501 -v "/Users/haoyuxu/workspace/jaeger-idl:/Users/haoyuxu/workspace/jaeger-idl" -w /Users/haoyuxu/workspace/jaeger-idl jaegertracing/protobuf:0.5.0 --proto_path=/Users/haoyuxu/workspace/jaeger-idl -Iproto/api_v2 -Iproto -I/usr/include/github.com/gogo/protobuf -Iopentelemetry-proto --gogo_out=plugins=grpc,Mgoogle/protobuf/descriptor.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/empty.proto=github.com/gogo/protobuf/types,Mgoogle/api/annotations.proto=github.com/gogo/googleapis/google/api,Mmodel.proto=github.com/jaegertracing/jaeger/model:/Users/haoyuxu/workspace/jaeger-idl/proto-gen-go --java_out=proto-gen-java --python_out=proto-gen-python --js_out=proto-gen-js --cpp_out=proto-gen-cpp --csharp_out=base_namespace:proto-gen-csharp --grpc-java_out=proto-gen-java --grpc-python_out=proto-gen-python --grpc-js_out=proto-gen-js --grpc-cpp_out=proto-gen-cpp --grpc-csharp_out=proto-gen-csharp \
		proto/api_v2/query.proto \
		proto/api_v2/collector.proto \
		proto/api_v2/sampling.proto
--grpc-csharp_out: protoc-gen-grpc-csharp: Plugin killed by signal 11.
make: *** [proto-api-v2] Error 1

Requirement - what kind of business use case are you trying to solve?

We need to get proto-gen-csharp usable. As of now, proto-gen-csharp can not be used because google/api/annotations.proto, gogoproto/gogo.proto and protoc-gen-swagger/options/annotations.proto are not created but necessary for reflection (which can't be removed).

Problem - what in Jaeger blocks you from solving the requirement?

gogoproto/goto.proto can't be generated in C# since it's using proto2 syntax which only got added with Protocol Buffers v3.11.0 which was available starting gRPC v1.27.0. This docker currently targets 1.26.0.

This is blocked by jaegertracing/docker-protobuf#14.

Proposal - what do you suggest to solve the problem or improve the existing situation?

Update the Makefile to generate google/api/annotations.proto, gogoproto/gogo.proto and protoc-gen-swagger/options/annotations.proto.

Any open questions to address

I will create the pull request myself.

Describe the bug
Cloning the repo and running make proto in the main directory leads to the following output:

mkdir -p proto-gen-go \
        proto-gen-java \
        proto-gen-python \
        proto-gen-js \
        proto-gen-cpp \
        proto-gen-csharp
docker run --rm -u 1000 -v "/home/fex/repos/jaeger-idl:/home/fex/repos/jaeger-idl" -w /home/fex/repos/jaeger-idl jaegertracing/protobuf:0.3.1 --proto_path=/home/fex/repos/jaeger-idl -Iproto/api_v2 -Iproto -I/usr/include/github.com/gogo/protobuf -Iopentelemetry-proto --gogo_out=plugins=grpc,Mgoogle/protobuf/descriptor.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/empty.proto=github.com/gogo/protobuf/types,Mgoogle/api/annotations.proto=github.com/gogo/googleapis/google/api,Mmodel.proto=github.com/jaegertracing/jaeger/model:/home/fex/repos/jaeger-idl/proto-gen-go --java_out=proto-gen-java --python_out=proto-gen-python --js_out=proto-gen-js --cpp_out=proto-gen-cpp --csharp_out=base_namespace:proto-gen-csharp \
        proto/api_v2/model.proto
docker run --rm -u 1000 -v "/home/fex/repos/jaeger-idl:/home/fex/repos/jaeger-idl" -w /home/fex/repos/jaeger-idl jaegertracing/protobuf:0.3.1 --proto_path=/home/fex/repos/jaeger-idl -Iproto/api_v2 -Iproto -I/usr/include/github.com/gogo/protobuf -Iopentelemetry-proto --gogo_out=plugins=grpc,Mgoogle/protobuf/descriptor.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/empty.proto=github.com/gogo/protobuf/types,Mgoogle/api/annotations.proto=github.com/gogo/googleapis/google/api,Mmodel.proto=github.com/jaegertracing/jaeger/model:/home/fex/repos/jaeger-idl/proto-gen-go --java_out=proto-gen-java --python_out=proto-gen-python --js_out=proto-gen-js --cpp_out=proto-gen-cpp --csharp_out=base_namespace:proto-gen-csharp --grpc-java_out=proto-gen-java --grpc-python_out=proto-gen-python --grpc-js_out=proto-gen-js --grpc-cpp_out=proto-gen-cpp --grpc-csharp_out=proto-gen-csharp \
        proto/api_v2/query.proto \
        proto/api_v2/collector.proto \
        proto/api_v2/sampling.proto
# API v3
docker run --rm -u 1000 -v "/home/fex/repos/jaeger-idl:/home/fex/repos/jaeger-idl" -w /home/fex/repos/jaeger-idl jaegertracing/protobuf:0.3.1 --proto_path=/home/fex/repos/jaeger-idl -Iproto/api_v2 -Iproto -I/usr/include/github.com/gogo/protobuf -Iopentelemetry-proto --gogo_out=plugins=grpc,Mgoogle/protobuf/descriptor.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/empty.proto=github.com/gogo/protobuf/types,Mgoogle/api/annotations.proto=github.com/gogo/googleapis/google/api,Mmodel.proto=github.com/jaegertracing/jaeger/model:/home/fex/repos/jaeger-idl/proto-gen-go --java_out=proto-gen-java --python_out=proto-gen-python --js_out=proto-gen-js --cpp_out=proto-gen-cpp --csharp_out=base_namespace:proto-gen-csharp --grpc-java_out=proto-gen-java --grpc-python_out=proto-gen-python --grpc-js_out=proto-gen-js --grpc-cpp_out=proto-gen-cpp --grpc-csharp_out=proto-gen-csharp \
        proto/api_v3/query_service.proto
opentelemetry/proto/trace/v1/trace.proto: File not found.
api_v3/query_service.proto:19:1: Import "opentelemetry/proto/trace/v1/trace.proto" was not found or had errors.
api_v3/query_service.proto:40:12: "opentelemetry.proto.trace.v1.ResourceSpans" is not defined.
make: *** [Makefile:100: proto] Error 1

(most relevant are probably the last 4 lines:
opentelemetry/proto/trace/v1/trace.proto: File not found.
api_v3/query_service.proto:19:1: Import "opentelemetry/proto/trace/v1/trace.proto" was not found or had errors.
api_v3/query_service.proto:40:12: "opentelemetry.proto.trace.v1.ResourceSpans" is not defined.
make: *** [Makefile: 100: proto] Error 1

Nonetheless I copied the resulting files in jaeger-idl/proto-gen-python into my Python project directory and imported the query-files with import query_pb2, query_pb2_grpc - to then run into ModuleNotFoundError: No module named 'gogoproto'

So - is this a bug or am I holding it wrong?
And on a side note - does anyone know a working example querying Jaeger via Python? Preferred via gRPC?