The problem is that there is no namespace_packages defined in setup(...) call in setup.py.
It is easy to fix.

How to reproduse:
install style-analyzer: pip3 install style-analyzer and run

>>> import lookout.core
>>> import lookout.style
>>> import lookout.sdk
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'lookout.sdk'

but lookout-sdk, as well as lookout-sdk-ml, are installed as dependencies.

Analyzers using sdk should retry to connect to data server if the connection is broken.

Ref: src-d/style-analyzer#509

Related to src-d/lookout#339

The idea is to hide bblfsh dependencies from a user of sdk.
Current proposal is to create helper for client initializer. But maybe we can do better.
One of the pain points is that bblfsh client of user can be incompatible with lookout.
And the same problem exists with filtering.

This project contains _tools/install-lookout-latest.sh, and both https://github.com/src-d/lookout-gometalint-analyzer and https://github.com/src-d/lookout-sonarcheck-analyzer have their own clone. Since this is a helper for Travis CI, it would make sense to have a unique common place for this. Probably it could live in this repo, and clone/wget it where needed.

We need to download the specific lookout-sdk binary which works with our client code for tests.
Our current approach is to take it from Gopkg.lock in src-d/lookout which works beautifully and reliably.
This is how it looks like: https://github.com/src-d/lookout-sdk-ml/blob/master/lookout/core/api/version.py

Since we want to eventually switch from artisanal homebrew to this project, I request the same feature here.

Since the secure connection is not a priority, we could add the grpc insecure connection option to the sdk helpers.

As can be seen here: https://travis-ci.org/src-d/lookout-sdk/jobs/481415428#L368

It looks like the travis xenial image has changed the default python version from 3.6 to 3.7. I think we can't set a python version for a job that already has language: go; go: '1.10'. So we either install python 3.6, or generate the code with 3.7.

We have introduced new higher level go sdk package: #88

It makes sense to move custom code we wrote before from pb package (which contains auto generated code) to the new one. But we need to review first what we have there. Maybe not everything should be moved.

We also need to move /pb to /go/pb

To follow our conventions we should test the latest two stable major versions of Go: https://github.com/src-d/guide/blob/master/engineering/conventions-go.md#supported-go-versions

I did a quick test and the travis matrix expansion does not work with the jobs/include scripts

Each job included in jobs.include inherits the first value of the array that defines a matrix dimension.
https://docs.travis-ci.com/user/build-stages#build-stages-and-build-matrix-expansion

Lookout contains two gRPC interceptors in the grpchelper package, see NewServer and DialContext in helper.go.

The Ctxlog interceptor requires that the analyzers also have these interceptors to pass around the log fields like this: lookoutd client -> analyzer server -> analyzer client -> lookoutd server.

If we move the Ctxtlog interceptor to this repo it probably makes sense to move also the Log interceptor, to have uniform logs for lookoutd and all the analyzers.

I'm not sure about extracting the util/ctxlog package to this repo. Maybe we just want to have methods to set and get map[string]interface{} in the context, using context.WithValue(), ctx.Value().

Ideally the python code should have the same functionality to get the log fields from the gRPC metadata.

https://github.com/src-d/lookout/blob/master/docs/architecture.md

Looking at the README, I realised that at some part you're pointed to gopkg.in/src-d/lookout-sdk.v0/pb, but the badge with the godoc reference points to github.com/src-d/lookout-sdk. They're equivalent, but they should be the consistent.

It would be great to have an ability to load UASTs from disk instead of parsing files every time. This is critical for ML benchmarks and quality evaluations, where we can analyze 1,000 predefined repositories at the same revisions. Thus we would have more stable timings, more stable experience (some driver may crash, e.g. cpp, or parse differently, e.g. js) and also run faster.

In my specific case I'm using a python env built using conda and not venv. When I ran make protogen it actually installed grpcio_tools=1.13.0 in my global env as pip3 was pointing to the global pip. Given that inside an environment pip points to the "correct" pip2 or pip3 depending on the python version of the environment I'd suggest to replace pip3 with pip.

If we're concerned about the python version I'd add a check that tests it before generating the python files.

The quickstart for python mentions this files but it doesn't exist.

It seems safe to delete this line since the pip install lookout-sdk dependency already mentioned above in the quickstart section seems to be enough.

Dependants may need to remove some log fields before being intercepted by the interceptors, we should provide a way to do this. Context here.

We need to update travis to test on 1.11 and 1.12.

I have fixed install-lookout script and now I see the errors but CI is green:
https://travis-ci.org/src-d/lookout-sdk/jobs/457551518

There are fields that need to be documented

I stumbled on 2 problems with current scripts:

1. sed command is incompatible with sed installed by default on macos
2. pip3 install --user doesn't work in virtualenv:

Can not perform a '--user' install. User site-packages are not visible in this virtualenv.

This is an umbrella issue to keep track of the upgrade to 0.6.1 in lookout and our analyzers.

• https://github.com/src-d/lookout-test-fixtures/
• lookout
  • src-d/lookout#598
  • Tag a new release
• https://github.com/src-d/lookout-gometalint-analyzer
  • src-d/lookout-gometalint-analyzer#31
  • Tag a new release
• https://github.com/src-d/lookout-sonarcheck-analyzer
  • src-d/lookout-sonarcheck-analyzer#25
  • Tag a new release

Since sdk is a lookout dependency, I wonder if it could be tested deeper here.
Currently, we only test the example analyzers with lookout-sdk binary (lookout-tool) what might be not enough.

We could try, for example, building lookout-sdk binary (lookout-tool) to ensure the sdk didn't break lookout.

install-lookout-latest.sh is not using GITHUB_TOKEN. As seen for example here:
https://travis-ci.org/src-d/lookout-sdk/jobs/445019187#L774

Which is a good thing, because if it worked as intended we would be writing the token to the build logs as part of the curl command, and these are public. The token is shown as [secure], docs.

The problem is that the encrypted env var is not available to PRs made from forks.

Right now, before the release one have to manually updated&commit a new version number for python library in setup.py → setup.py
It would be nice to get it from the git tag instead.

Many analyzers can work only with some specific language or set of languages.
But receive all of them (including UAST parsing on lookoutd side and transferring huge payload).

how about moving Tostruct and Tovalue function to gogo/protobuf repo? This is very helpful