startreedata / pinot-client-go Goto Github PK

Problem

No trace info can be placed on the backend service for tracking

Expect

The trace field is added to better allow the backend to debug problems

Should follow python client lib to create a CI including all the unit test and integration test by launching a pinot quickstart in docker, then query.

https://github.com/python-pinot-dbapi/pinot-dbapi

Currently, the Go version is 1.13 which is almost three years old and also there is a vulnerability mentioned in the old version.

Can we please add support for PreparedStatement in the go client as we have for the Java client?

https://docs.pinot.apache.org/users/clients/java#query-methods

Could a new 0.2.0 release be created given the last one was in 2022?

It would be great to support passing in sql params for parameterized queries in the execute sql function

Problem

• Currently, we are using the samuel/go-zookeeper Go package; however, the package has been archived from mid-2021, and the forked repo is also dormant. It would be better to use the active zookeeper package.

Proposed Solution
Here are some of the other zookeeper packages

• https://pkg.go.dev/github.com/prometheus/prometheus
• https://pkg.go.dev/github.com/hashicorp/vault/physical/zookeeper

This issue is to finalize the library and using the new one.

Field alignment is a standard simple trick recommended in Go-based apps for better memory usage and fast processing.
I found that the structs defined in the package aren't following the field alignment. Here is the output of the lint

$go vet -vettool=$(which fieldalignment) ./...
# github.com/startreedata/pinot-client-go/pinot
pinot/config.go:6:19: struct with 56 pointer bytes could be 32
pinot/config.go:25:22: struct with 32 pointer bytes could be 24
pinot/config.go:33:23: struct with 32 pointer bytes could be 16
pinot/controllerBasedBrokerSelector.go:28:30: struct with 96 pointer bytes could be 56
pinot/controllerResponse.go:8:16: struct with 32 pointer bytes could be 24
pinot/dynamicBrokerSelector.go:21:28: struct with 96 pointer bytes could be 64
pinot/dynamicBrokerSelector.go:30:19: struct with 40 pointer bytes could be 32
pinot/response.go:6:21: struct with 72 pointer bytes could be 56
pinot/response.go:54:16: struct with 16 pointer bytes could be 8
pinot/controllerBasedBrokerSelector_test.go:16:28: struct with 24 pointer bytes could be 16

This issue tracks to change the defined structs with aligned fields and add support in the lint for future checks.

I will work on this issue. Thanks

cc @xiangfu0

The client should have an option flag to be enabled to query v2 endpoint.

Issue type: feature request

Pinot version: 0.8.0

Description:

We are using the Golang module for connecting with Pinot in our internal applications. So far, the JSON-based communication does not pose a bottleneck for smaller result sets (order of magnitude: 10^3).

However, for moderate result sets (order of magnitude: 10^4-10^5), the JSON-based communication starts to be inefficient.

To mitigate this, it would be convenient to implement a gRPC communication in this module, since Pinot has gRPC communication implemented internally and few other systems (i.e. PrestoDB) use this for fast fetching of the result sets.

Currently, golangci-lint run commands fail for a few linters, so we disabled them. Here is the list

  disable:
    - godox
    - ineffassign
    - errcheck
    - staticcheck
    - gosimple
    - stylecheck
    - wsl
    - revive

This issue will work on resolving all the lining errors and enabling them. I will work on it. Thanks

I would like to bring up an issue regarding the use of an internal logger within the library. It's generally advised that libraries should avoid directly writing logs. This practice can lead to several challenges, such as cluttered log outputs and potential conflicts with the application's own logging strategy.

https://github.com/search?q=repo%3Astartreedata%2Fpinot-client-go+log.Error+NOT+path%3Aexamples%2F+NOT+path%3AREADME.md&type=code

Thank you for your work on this project, and I hope this suggestion contributes positively to its development.