When I start my application, the serving load one big model fail. The error message is clear, but I don't know how to resolve it in the tensorflow_serving.
I tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:149] Aspiring 1 versions for servable common1 [libprotobuf ERROR external/tf/google/protobuf/src/google/protobuf/io/coded_stream.cc:207] A protocol message was rejected because it was too big (more than 67108864 bytes). To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.

Hi,

I tried to install serving of tensorflow. My numpy is compiled with openblas and is installed at /usr/lib/python2.7/site-packages/numpy (I used Linux Mint 17.3). While I am installing tensorflow_serving, the error happens, saying cannot find numpy.

It seems that bazel only seeks numpy at /usr/lib/python2.7/dist-packages, NOT /usr/lib/python2.7/site-packages/numpy.

I linked the /usr/lib/python2.7/site-packages/numpy to /usr/lib/python2.7/dist-packages, and bazel can find numpy, but it cannot find libopenblas.so.0, which is installed at /usr/local. But bazel throw an error saying that it cannot find the .so file.

Is it a bug to fix or should I do something else?

Thanks

===ERROR1: Cannot find numpy==========
dnn01@DL02 ~/serving $ bazel test tensorflow_serving/...
INFO: Found 90 targets and 37 test targets...
ERROR: /home/dnn01/serving/tensorflow_serving/session_bundle/example/BUILD:34:1: Executing genrule //tensorflow_serving/session_bundle/example:half_plus_two failed: bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped).
/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles:/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/:/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf:/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/six_archive
['/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/tensorflow_serving/session_bundle/example', '/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles', '/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf', '/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/six_archive', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']
Traceback (most recent call last):
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/tensorflow_serving/session_bundle/example/export_half_plus_two.py", line 31, in
import tensorflow as tf
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/init.py", line 23, in
from tensorflow.python import *
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/python/init.py", line 49, in
from tensorflow import contrib
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/contrib/init.py", line 23, in
from tensorflow.contrib import layers
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/contrib/layers/init.py", line 67, in
from tensorflow.contrib.layers.python.framework.tensor_util import *
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/contrib/layers/python/framework/tensor_util.py", line 20, in
from tensorflow.python.framework import dtypes
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/python/framework/dtypes.py", line 24, in
import numpy as np
ImportError: No module named numpy

===ERROR2: Cannot find libopenblas==========
ERROR: /home/dnn01/serving/tensorflow_serving/session_bundle/example/BUILD:34:1: Executing genrule //tensorflow_serving/session_bundle/example:half_plus_two failed: bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped).
/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles:/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/:/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf:/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/six_archive
['/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/tensorflow_serving/session_bundle/example', '/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles', '/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf', '/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/six_archive', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']
Traceback (most recent call last):
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/tensorflow_serving/session_bundle/example/export_half_plus_two.py", line 31, in
import tensorflow as tf
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/init.py", line 23, in
from tensorflow.python import *
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/python/init.py", line 49, in
from tensorflow import contrib
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/contrib/init.py", line 23, in
from tensorflow.contrib import layers
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/contrib/layers/init.py", line 67, in
from tensorflow.contrib.layers.python.framework.tensor_util import *
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/contrib/layers/python/framework/tensor_util.py", line 20, in
from tensorflow.python.framework import dtypes
File "/home/dnn01/.cache/bazel/_bazel_dnn01/681ee4690332907d7b23cc5c101faf32/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/python/framework/dtypes.py", line 24, in
import numpy as np
File "/usr/lib/python2.7/dist-packages/numpy/init.py", line 180, in
from . import add_newdocs
File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in
from numpy.lib import add_newdoc
File "/usr/lib/python2.7/dist-packages/numpy/lib/init.py", line 8, in
from .type_check import *
File "/usr/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, in
import numpy.core.numeric as _nx
File "/usr/lib/python2.7/dist-packages/numpy/core/init.py", line 14, in
from . import multiarray
ImportError: libopenblas.so.0: cannot open shared object file: No such file or directory

It looks like the examples for TensorFlow Serving rely on c code only for running the server. Things like querying the server and exporting models can be done in Python. However, it also seems like you're obliged to write the python code then compile the damn thing with Bazel (what is it with the documentation's obsessive love for Bazel?) into executables, making rapid iteration difficult (at least for me).

Is there a way to use the pure python parts of TF Serving interactively? I.e., actually run from tensorflow_serving.session_bundle import exporter in an interpreter? With regular TF, you can build a pip repo and install it that way, but this doesn't appear to be an option for TF Serving. Are there plans to implement one?

Hi,

Environment is Mac, numpy installed with apt-get command. What version of numpy does it require?

ERROR: /home/xxxx/tensorflow2/serving/tensorflow_serving/session_bundle/e xample/BUILD:34:1: Executing genrule //tensorflow_serving/session_bundle/ex ample:half_plus_two failed: bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped).
2546 RuntimeError: module compiled against API version a but this version of num py is 9
2547 Traceback (most recent call last):
2548 File "/home/yhwang/.cache/bazel/_bazel_yhwang/85b1cab135cf9be23a6843da5c4 16dee/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/ export_half_plus_two.runfiles/tensorflow_serving/session_bundle/example/exp ort_half_plus_two.py", line 31, in
2549 import tensorflow as tf
2550 File "/home/yhwang/.cache/bazel/_bazel_yhwang/85b1cab135cf9be23a6843da5c4 16dee/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/ export_half_plus_two.runfiles/external/tf/tensorflow/init.py", line 23, in
2551 from tensorflow.python import *

Would you like to replace any double quotes by angle brackets around file names for include statements?

I'm using Bazel 0.2.2, CUDA 7.5, cuDNN v5 on Ubuntu 14.04.
After configuring TensorFlow (@ ef7f54f) I'm able to build an example with GPU support:

$ cd tensorflow
$ ./configure
$ bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer

However I'm not able to build TF serving examples with CUDA support. I use the inception_export.py as an example but the real model I want to export uses NCHW and some CPU Ops only support NHWC.

$ cd ..
$ bazel build -c opt --config=cuda //tensorflow_serving/example:inception_export
ERROR: /home/parviz/.cache/bazel/_bazel_parviz/3889ffe662ea382c49c2732eaedd13b5/external/tf/tensorflow/core/kernels/BUILD:1304:1: undeclared inclusion(s) in rule '@tf//tensorflow/core/kernels:scatter_op_gpu':
this rule is missing dependency declarations for the following files included by 'external/tf/tensorflow/core/kernels/scatter_op_gpu.cu.cc':
  '/usr/local/cuda-7.5/include/cuda_runtime.h'
  '/usr/local/cuda-7.5/include/host_config.h'
  '/usr/local/cuda-7.5/include/builtin_types.h'
...
  '/usr/local/cuda-7.5/include/curand_lognormal.h'
  '/usr/local/cuda-7.5/include/curand_poisson.h'
  '/usr/local/cuda-7.5/include/curand_discrete2.h'.
nvcc warning : option '--relaxed-constexpr' has been deprecated and replaced by option '--expt-relaxed-constexpr'.
nvcc warning : option '--relaxed-constexpr' has been deprecated and replaced by option '--expt-relaxed-constexpr'.
Target //tensorflow_serving/example:inception_export failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 18.663s, Critical Path: 18.36s

What is the best approach to serve models with CUDA support? Should I convert to NHWC or is there a way to get CUDA support working?

Hi guys can anyone provide docs for installing on Centos, also would this be the preferable set up to use with SyntaxNet and be able to query it via an API? TIA

I noticed that the Inception example does not yet support batching, which would offer a considerable speedup (and serve as a helpful tutorial on how to implement batching in TF Serving in general!). I'm very willing to help implement this, but I'm confused as to what barriers are in place to its implementation; why wasn't it implemented in the first place? It seems like a relatively trivial change.

The server file refers to the export python script "for details" but this appears only to say:

# TODO(b/27776734): Add batching support.

I'm not sure what the issue is here though. IIRC, the model doesn't care if the inputs are variable-sized in the first dimension. If it does, would it be helpful or possible to add in "dummy" images that will stand in for available, but unused, slots in the batch? Presumably this would negatively affect performance when the requests are coming in at some low number per batch-job sent to the GPU, but this performance cost would be bounded by the fact that subsequent batch-jobs sent to the GPU would consist of more true images per batch as requests from clients piled up.

I have some queries around some real world, production deployments of the system.

• Is there a way to package the C++, python and proto files together in some way via Bazel? Some sort of an "uberjar" type container which contains a manifest? At this moment, installing grpc, grpcio, bazel, compiling everything via it, having a bunch of pre-built libs et al is quite time consuming, and there's no clear way to distribute a TF Serving system. Ideally, I'd take this packaged "thing" and use upstart or daemontools to daemonize it. Is there a better way?
• Incidentally, the compiled version of the dependencies are almost 1.5G with tf taking up a lot of it. Is there way to make this footprint "thinner"?

3.5M    ./tensorflow_serving/session_bundle
3.5M    ./tensorflow_serving
8.0K    ./_solib_local/_U@tf_S_Sthird_Uparty_Sgpus_Scuda_Ccudnn___Uexternal_Stf_Sthird_Uparty_Sgpus_Scuda_Slib64
8.0K    ./_solib_local/_U@tf_S_Sthird_Uparty_Sgpus_Scuda_Ccudart___Uexternal_Stf_Sthird_Uparty_Sgpus_Scuda_Slib64
8.0K    ./_solib_local/_U@tf_S_Sthird_Uparty_Sgpus_Scuda_Ccufft___Uexternal_Stf_Sthird_Uparty_Sgpus_Scuda_Slib64
8.0K    ./_solib_local/_U@tf_S_Sthird_Uparty_Sgpus_Scuda_Ccublas___Uexternal_Stf_Sthird_Uparty_Sgpus_Scuda_Slib64
36K ./_solib_local
1.2M    ./external/jpeg_archive
396K    ./external/zlib_archive
716K    ./external/png_archive
17M ./external/grpc
5.7M    ./external/re2
1.2G    ./external/tf
8.0M    ./external/boringssl_git
1.3G    ./external
216M    ./mycode/inference_server
216M    ./mycode
1.5G    .

• Are there any best practices out there for statusz/healthz style interfaces for this server? I realize this might be more of a grpc related question, but getting some insights/examples from people who've deployed TF Serving in production would be useful.

Lastly, the example folders contain a bunch of pb2.py files, for which the tutorials don't seem to talk about - for instance, installing grpc, and protobuf3 c++/python, and how to use protoc to compile them into a service definition. Would be good to have for those who aren't familiar with grpc.

The inception_export.py has set wrong path for loading the checkpoint.

edward@edward-All-Series:~/git/serving$ bazel-bin/tensorflow_serving/example/inception_export --checkpoint_dir=inception-v3 --export_dir=inception-export
Traceback (most recent call last):
File "/home/edward/git/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 143, in
tf.app.run()
File "/home/edward/git/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/external/tf/tensorflow/python/platform/app.py", line 30, in run
sys.exit(main(sys.argv))
File "/home/edward/git/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 139, in main
export()
File "/home/edward/git/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 117, in export
saver.restore(sess, full_path)
File "/home/edward/git/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/external/tf/tensorflow/python/training/saver.py", line 1088, in restore
raise ValueError("Restore called with invalid save path %s" % save_path)
ValueError: Restore called with invalid save path inception-v3/inception-v3/model.ckpt-157585

I see the TensorFlow Serving dockerfile does not go with installed CUDA toolkits as well as Cudnn. Must I install the toolkit inside the container to let TF Serving using Cuda hardware, or based on nvidia-docker ?

Hi All-

I'm not sure if this is an issue with gRPC and so also belongs on the issue page over there or the way TensorFlow serving interacts with it, but I figure it will likely come up with others so I'm putting it here. Keep in mind I'm a complete gRPC neophyte and find its documentation a bit baffling.

It appears that if a (dynamic) stub is created as an attribute of a class, then the python script using an instance of that class will never halt. This may be related to issues gRPC issues 4986 and 5640. While it may not be an issue for the demo scripts presented in the tutorials, I don't think it's terribly outlandish to create a class you can import to manage all the stub / channel stuff, and so I think this may come up again for others.

Anyone have any thoughts? Creating a method like this:

def shutdown(self):
        del self.channel
        del self.stub

does not seem to fix it, unfortunately.

Hi there,

I got error when build example. My gcc is gcc (Ubuntu 4.8.4-2ubuntu1~14.04.1) 4.8.4, and
Here is the error information. Do u have a docker version for serving? Thanks!
(cd /home/xxxx/.cache/bazel/bazel_xxxx/85b1cab135cf9be23a6843da5c416dee/serving &&
exec env -
PATH=/home/xxxx/tensorflow2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/bin/libsvm/:/usr/bin/liblinear:/home/xxxx/bin
/usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -iquote external/tf -iquote bazel-out/local_linux-fastbuild/genfiles/external/tf -iquote external/bazel_tools -iquote bazel-out/local_linux-fastbuild/genfiles/external/bazel_tools -iquote external/jpeg_archive -iquote bazel-out/local_linux-fastbuild/genfiles/external/jpeg_archive -iquote external/png_archive -iquote bazel-out/local_linux-fastbuild/genfiles/external/png_archive -iquote external/re2 -iquote bazel-out/local_linux-fastbuild/genfiles/external/re2 -iquote external/eigen_archive -iquote bazel-out/local_linux-fastbuild/genfiles/external/eigen_archive -isystem external/tf/google/protobuf/src -isystem bazel-out/local_linux-fastbuild/genfiles/external/tf/google/protobuf/src -isystem external/bazel_tools/tools/cpp/gcc3 -isystem external/jpeg_archive/jpeg-9a -isystem bazel-out/local_linux-fastbuild/genfiles/external/jpeg_archive/jpeg-9a -isystem external/png_archive/libpng-1.2.53 -isystem bazel-out/local_linux-fastbuild/genfiles/external/png_archive/libpng-1.2.53 -isystem external/re2 -isystem bazel-out/local_linux-fastbuild/genfiles/external/re2 -isystem external/tf/third_party/eigen3 -isystem bazel-out/local_linux-fastbuild/genfiles/external/tf/third_party/eigen3 -isystem external/eigen_archive/eigen-eigen-ed4c9730b545 -isystem bazel-out/local_linux-fastbuild/genfiles/external/eigen_archive/eigen-eigen-ed4c9730b545 -fno-exceptions -DEIGEN_AVOID_STL_ARRAY -pthread -no-canonical-prefixes -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE_="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' '-frandom-seed=bazel-out/local_linux-fastbuild/bin/external/tf/tensorflow/core/kernels/_objs/cwise_op/external/tf/tensorflow/core/kernels/cwise_op_not_equal_to.pic.o' -MD -MF bazel-out/local_linux-fastbuild/bin/external/tf/tensorflow/core/kernels/_objs/cwise_op/external/tf/tensorflow/core/kernels/cwise_op_not_equal_to.pic.d -fPIC -c external/tf/tensorflow/core/kernels/cwise_op_not_equal_to.cc -o bazel-out/local_linux-fastbuild/bin/external/tf/tensorflow/core/kernels/_objs/cwise_op/external/tf/tensorflow/core/kernels/cwise_op_not_equal_to.pic.o).
gcc: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See file:///usr/share/doc/gcc-4.8/README.Bugs for instructions.
Target //tensorflow_serving/example:mnist_inference failed to build

Is there a way to specify multiple tensors into the Classification signature?

Eg, today you can do,

signature = exporter.classification_signature(input_tensor=model.input,
                                                scores_tensor=model.output)

but not

signature = exporter.classification_signature(input_tensor=[model.input, my_other_tensor],
                                                scores_tensor=model.output)

I did see there's a generic_signature function which can be used like

named_tensor_bindings = {"logical_input_A": v0,
                         "logical_input_B": v1}
signatures = {
    "generic": exporter.generic_signature(named_tensor_bindings)}

But this seems to be useful only when a single input is used for multiple outputs. I realize that in a classification system, only one tensor is used - but what if I wanted to have two tensors that are eventually concatenated using a graph model?

Is there an example/pointers to a case where,

• A model graph and its weights are exported separately? For instance, one graph but multiple datasets and thus, multiple weight files.
• Would it be possible to load this model once, and initialize multiple versions of it using different weights for the Classify() to run?

It's depressing when I finally find TensorFlow Serving can't support python 3 now.

In fact, I'm trying to build TensorFlow Serving under python 3.4 because I have been working on tensorflow 0.7.1 for python 3.4 for a while. And I don't find any statement that TensorfFlow Serving can only run on python 2.x.

I used the methods in #21 to make "tensorflow_serving/example/mnist_export.py" and "tensorflow_serving/example/mnist_client.py" support python 3.4.

Then the mnist_inferenece can be built and run on python 3.4 successfuly:

$ bazel-bin/tensorflow_serving/example/mnist_inference --port=9000 /tmp/mnist_model/00000001
I tensorflow_serving/session_bundle/session_bundle.cc:109] Attempting to load a SessionBundle from: /tmp/mnist_model/00000001
I tensorflow_serving/session_bundle/session_bundle.cc:86] Running restore op for SessionBundle
I tensorflow_serving/session_bundle/session_bundle.cc:157] Done loading SessionBundle
I tensorflow_serving/example/mnist_inference.cc:163] Running...

The mnist_client can also be built successfuly.

But when I run "bazel-bin/tensorflow_serving/example/mnist_client --num_tests=1000 --server=localhost:9000", I got "ImportError: No module named 'grpc'".

Unfortunately, gRPC can't support python 3 now. You can find the reason at Python3.4 support #282.

So is that mean TensorFlow Serving can't support python 3 until gRPC can support python 3?

Is there any plan to let TensorFlow Serving support thrift? thriftpy has been able to support python2.6+, python3.3+, pypy and pypy3.

Could someone elaborate on how the classes tensor gets used when a model is exported?

Lets say I have a list of labels that I can convert into a tensor of type text.

labels = ["label1", "label2"]
labels_tensor = tf.constant(labels)
..
..

signature = exporter.classification_signature(input_tensor=input,
                                                classes_tensor=labels_tensor,
                                                scores_tensor=output)

Now - when using this in a C++ inference server, you use them via session->run() or RunClassification, like so,

const tensorflow::Status run_status =
      RunClassification(signature, batched_input, bundle->session.get(),
                        &batched_classes, &batched_scores);

• What does batched_classes contain? Numerical identifiers or text?
• If it only contains numerical identifiers, why do we even needs this? Isn't the shape of the batched_scores allowing us to select the index of the right class by running an argmax? The problem with this approach is that the client also needs to be aware of the model details (eg labels), which I'd rather avoid.

Main question - how do I return a text label rather than a scores tensor for a given request?

Unless I'm missing something in the advanced serving tutorial.

It's there in commit 3985fca#diff-ca07f0e4bbb8e03072059aea8b5ce227, but seems to have moved since then.

/cc @fangweili

My Numpy version is 1.10.4 py27_1... and I have successfully build Tensorflow from source with CUDA 7.5 and tested it...

However when building Tensorflow_serving by ...
$ bazel build --local_resources 3072,0.5,1.0 tensorflow_serving/...

I get the error : (last traceback included only)

ERROR: /home/irtza/serving/tensorflow_serving/session_bundle/example/BUILD:34:1: Executing genrule //tensorflow_serving/session_bundle/example:half_plus_two failed: bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped).
RuntimeError: module compiled against API version a but this version of numpy is 9
Traceback (most recent call last):
====skipping to last =======
File "/home/irtza/.cache/bazel/_bazel_irtza/307be6be5aced7ceb1786e9bf1c7f317/serving/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/external/tf/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
ImportError: numpy.core.multiarray failed to import

When I check numpy import from python shell I can import numpy.core.multiarray ..

following is the output of testing tensorflow_serving which also Failed.

$ bazel test tensorflow_serving/...
INFO: Waiting for response from Bazel server (pid 2482)...
ERROR: package contains errors: tensorflow/bazel-tensorflow/third_party/gpus/cuda.
ERROR: error loading package 'tensorflow/bazel-tensorflow/third_party/gpus/cuda': Extension file not found. Unable to load package for '//tensorflow:tensorflow.bzl': BUILD file not found on package path.
INFO: Elapsed time: 5.636s
ERROR: Couldn't start the build. Unable to run tests.

@kirilg Can you please help me understand the problem?
Any help by anyone would be greatly appreciated.

The demo in basic tutorial does work followed https://tensorflow.github.io/serving/setup or https://tensorflow.github.io/serving/docker . But it indicated "No module named tensorflow_serving.session_bundle " in Python 2.7.10 and didn't do in the case of typing "import tensorflow as tf" . I think this problem is related with some lib path of tensorflow serving . Any suggestion is appreciated , Thank you .

I'm trying to use Tensorflow serving with Apache Spark. I have one grpc async client per spark worker node. The pseudo code of my client call is as follow :

class MNISTTFServingClient private (channel: ManagedChannel, stub: MnistServiceGrpc.MnistServiceStub) {


  def shutdown(): Unit = {
    channel.shutdown.awaitTermination(30, TimeUnit.SECONDS)
  }


  def classify(images: Iterator[Array[Float]]): Iterator[Seq[Float]] = {
    val requests = images.map(i => MnistRequest(imageData = i.toSeq))
    val futureResponse = Future.traverse(requests)(stub.classify)
    val response = Await.result(futureResponse, 30 seconds)

    response.map(_.value)
  }

}


val classProbabilities = images.mapPartitions { imgIt =>
      val client = workerStore.get[MNISTTFServingClient]("client")
      client.classify(imgIt)
 }

It take a whole spark data partitions (the number of element withing each partion may be quite large)
and set them as MNistRequest to Tensorflow serving. Inside each worker, we may also have several threads that send request to the server. It works when the data size is small. But when I run it on a whole mnist test set I got the following error :

io.grpc.StatusRuntimeException: UNAVAILABLE: This task would start a fresh batch, but all batch threads are busy, so at present there is no processing capacity available for this task

Any ideas to fix this ?

I'm unable to get tensorflow-serving to find my GPU. I can checkout and build Tensorflow from source, and get it to run on the GPU. However, using the same configuration but building with bazel build tensorflow_serving/... does not work. I can also successfully build tensor-flow serving without gpu support, and when running with log_device_placement=True, no devices are found and all operations run on '/cpu:0'.

Issue 1: Passing --config=cuda to bazel build tensorflow_serving/... does not seem to have any effect. I'm not too familiar with bazel, so I'm guessing that this is an issue where configuration options don't propagate to dependencies. What is the correct way to pass this option when building tensorflow-serving?

Issue 2: To attempt to manually set --config=cuda, I modified the function if_cuda in tensorflow/tensorflow/tensorflow.bzl in the following way to (hopefully) always build with cuda:

# ORIGINAL
def if_cuda(a, b=[]):
    return select({
        "//third_party/gpus/cuda:cuda_crosstool_condition": a,
        "//conditions:default": b,
    })

# MODIFIED
def if_cuda(a, b=[]):
    return a

With this modification, bazel build -c opt tensorflow_serving/... fails with this error

ERROR: /data/sda1/.cache/external/tf/third_party/gpus/cuda/BUILD:163:1: declared output 'external/tf/third_party/gpus/cuda/cuda.config' was not created by genrule. This is probably because the genrule actually didn't create this output, or because the output was a directory and the genrule was run remotely (note that only the contents of declared file outputs are copied from genrules run remotely).
ERROR: /data/sda1/.cache/external/tf/third_party/gpus/cuda/BUILD:163:1: not all outputs were created.
INFO: Elapsed time: 38.194s, Critical Path: 0.18s

GPU: Tesla K20c
OS: Ubuntu 14.04
cuda: 7.5
cudnn: 4.0.7

The documentation for TF Serving mentions the *.proto files, but does not do much in the way of explaining what they are or how they may be modified. For instance, I'm trying to do a regression task. I tried to modify the *.proto file to do this but it was not clear how.

Further, the creation of *_pb2.py files is never mentioned, but is apparently a necessary step if you wish to change anything. I did this by:

protoc --proto_path=serving/tensorflow_serving --python_out=serving/tensorflow_serving/custom_net serving/tensorflow_serving/custom_net/cutom_net_inference.proto

but the custom_net_inference_pb2.py file it output did not have the same functions as inception_inference_pb2.py; namely, it stopped after protoc_insertion_point.

Furthermore, the line:

service InceptionService { // Classifies an JPEG image into classes. rpc Classify(InceptionRequest) returns (InceptionResponse); }

is Classify a function? Namely the function in inception_inference.cc? Is it necessary to modify these names if the net performs a regression? My understanding is that the signatures themselves should change but the function names are irrelevant, so custom_net_inference.cc has much the same function names but just uses tensorflow::serving::RegressionSignature signature;

I'm following directions here:
https://tensorflow.github.io/serving/setup#prerequisites

Everything works great except for:
bazel build tensorflow_serving/...

I get the error:
ERROR: /home/ubuntu/.cache/bazel/_bazel_ubuntu/external/tf/google/protobuf/BUILD:520:1: no such package 'util/python': BUILD file not found on package path and referenced by '@tf//google/protobuf:internal/_api_implementation.so'.

Hi,
the existing examples all consist of model training step which use TensorFlow, but more generally, we want to use TF-serving to export pre-trained models (.pb files for tensorflow), so can you provide one such example ?
Furthermore, how to export models trained by other platform like caffe/MxNet ?

Thanks a lot ~

I have been staring at this problem for a few days now and cant figure it out. I have tried a few clean installs and nothing seems to work. The current setup is as follows:
gcc/g++: V 4.9
bazel: .22
CUDA Toolkit: V7.5
cuDnn: V4
TensorFlow: GPU version/0.8 (requires CUDA TOOLKIT 7.4 & cuDnn V4)
PATH contains: /usr/local/cuda-7.5/bin
LD_LIBRARY_PATH contains: /usr/local/cuda-7.5/lib64

I must not have everything setup right yet, maybe missing a dependency? I can compile the "serving" directory with bazel without the CUDA flag, but as soon as I add the flag:
bazel build -c opt --config=cuda --verbose_failures tensorflow_serving/...

I get the attached error (after a lot of compiling). I am not sure where to go after this as I have tried many combinations. I did compile the CUDA 7.5 sample files and ran the deviceQuery which passed.

Thanks for the help.

error.txt

Currently, all docs point to simply building things within the tf serving directory. One issue with that is that there's not much information on how to position BUILD and .bzl files in a custom project, in say, a separate code repo than the TF serving's own git repo.

Specifically,

• How do I include tensorflow_serving/servables/* into my own code without specifying visibility public in the BUILD files?
• Is there way to include tensorflow_serving as a prebuilt binary shared lib in the deps in a bazel BUILD file?

ERROR:
ERROR: /home/seetea/tensorflow-serving-latest/tensorflow_serving/session_bundle/example/BUILD:34:1: Executing genrule //tensorflow_serving/session_bundle/example:half_plus_two failed: bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1. File "/home/seetea/.cache/bazel/_bazel_seetea/8d38ebe87742a96afdc093845beacfa5/tensorflow-serving-latest/bazel-out/host/bin/tensorflow_serving/session_bundle/example/export_half_plus_two.runfiles/tensorflow_serving/session_bundle/example/export_half_plus_two.py", line 66 print "copying asset files to: %s" % export_path ^ SyntaxError: Missing parentheses in call to 'print'

Environment: CentOS 7 + Python 3.4

Description: When I use "bazel build tensorflow_serving/..." to build the tensorflow_serving source codes under python 3.4, I met the above error.

Solution:
I made the following modifications to make the codes work for both python 2.x and python 3.x:

1. In "serving/tensorflow_serving/session_bundle/example/export_half_plus_two.py", add
   from __future__ import print_function
   and change
   print "copying asset files to: %s" % export_path
   to
   print ("copying asset files to: %s" % export_path)
2. In "serving/tensorflow_serving/session_bundle/exporter.py",

import six
.....
#for name, tensor in name_tensor_map.iteritems():
for name, tensor in six.iteritems(name_tensor_map):
......
#for signature_name, signature in named_graph_signatures.iteritems():
for signature_name, signature in six.iteritems(named_graph_signatures):
.......

Build failed, anyone can help?
bazel build tensorflow_serving/...
WARNING: Sandboxed execution is not supported on your system and thus hermeticity of actions cannot be guaranteed. See http://bazel.io/docs/bazel-user-manual.html#sandboxing for more information. You can turn off this warning via --ignore_unsupported_sandboxing.
INFO: Found 168 targets...
ERROR: /home/gezi/other/serving/tensorflow_serving/session_bundle/example/BUILD:34:1: Executing genrule //tensorflow_serving/session_bundle/example:half_plus_two failed: bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 245.
INFO: Elapsed time: 1.545s, Critical Path: 0.92s

I'm trying to build tensorfolw_serving on OSX El Capitan and keep getting an error. I installed Bazel through homebrew.

Here is the error:
$ bazel build tensorflow_serving/...
Extracting Bazel installation...
............
ERROR: /private/var/tmp/_bazel_robertgrzesik/fb4fbbbabd0582fe8f5abe6ab13dbfd9/external/inception_model/inception/slim/BUILD:93:1: no such package 'inception': BUILD file not found on package path and referenced by '@inception_model//inception/slim:slim'.
ERROR: Loading failed; build aborted.
INFO: Elapsed time: 45.554s

I also tried giving it the package path on my system and got a different error:
$ bazel build --package_path /Users/robertgrzesik/anaconda/lib/python2.7/site-packages tensorflow_serving/...
ERROR: no targets found beneath 'tensorflow_serving'.
INFO: Elapsed time: 0.131s

different result between evaluate code version and tensorflow serving, anyone know?
and how to debug?

I built the Tensorflow Serving package on an Ubuntu AWS Machine and it passed all of the tests. However, when I import the package, I receive the following error:

ubuntu@ip-172-31-5-123:~/serving/tensorflow_serving/example$ python mnist_export.py
Traceback (most recent call last):
  File "mnist_export.py", line 32, in <module>
    from tensorflow_serving.example import mnist_input_data
ImportError: No module named tensorflow_serving.example

When I run

bazel test tensorflow_serving/...

I shows me that all of the tests have passed.

//tensorflow_serving/util:any_ptr_test                          (cached) PASSED in 0.0s
//tensorflow_serving/util:cleanup_test                          (cached) PASSED in 0.1s
//tensorflow_serving/util:event_bus_test                        (cached) PASSED in 0.1s
//tensorflow_serving/util:fast_read_dynamic_ptr_benchmark       (cached) PASSED in 28.0s
//tensorflow_serving/util:fast_read_dynamic_ptr_test            (cached) PASSED in 0.1s
//tensorflow_serving/util:inline_executor_test                  (cached) PASSED in 0.0s
//tensorflow_serving/util:observer_test                         (cached) PASSED in 0.2s
//tensorflow_serving/util:optional_test                         (cached) PASSED in 0.1s
//tensorflow_serving/util:periodic_function_test                (cached) PASSED in 0.5s
//tensorflow_serving/util:threadpool_executor_test              (cached) PASSED in 0.1s
//tensorflow_serving/util:unique_ptr_with_deps_test             (cached) PASSED in 0.0s

Executed 0 out of 44 tests: 44 tests pass.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

In any case, would love any insights you may have.

I installed and configured Tensorflow Serving successfully . I ran the test command as well and everything was successful .

Now I'm trying to use the MNIST tutorial and when I put
from tensorflow_serving.session_bundle import exporter

I get following error ,

ImportError: No module named tensorflow_serving.session_bundle

I wonder what the problem is , my operating system is CentOs and I use the latest version of Tensoflow and Tensorflow serving without GPU support .

As mentioned in https://tensorflow.github.io/serving/docker, it doesn't work because there isn't one url linkage for tensorflow serving like b.gcr.io/tensorflow/tensorflow for tensorflow . Please tell me Anything I misunderstand or misinformation . Thanks alot .

The mnist example shows how to use the model with one input tensor x and one output tensor y.
While I have two input, tensor x and a tf.placeholder named dropout_prob.
If I don't specify the dropout_prob value, it will produce error output log like

W external/tf/tensorflow/core/common_runtime/executor.cc:1157] 0x7fcf1c0023c0 Compute status: Invalid argument: You must feed a value for placeholder tensor 'dropout_prob' with dtype float
     [[Node: dropout_prob = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

Would you please show some example with multi input tensors or multi output tensors?

Well, unfortunately, seems here we can not visit googlesource site , is there a workaround so I can bazel build tensorflow serving? I'm using tensorflow right now. Thanks

no such package '@boringssl_git//': Error cloning repository: https://boringssl.googlesource.com/boringssl: cannot open git-upload-pack and referenced by '//external:libssl'.

I'm trying to implement some (extremely) rudimentary load balancing by having multiple machines serve up the same model. As mentioned in #10, submitting lots of jobs to the server will case the StreamingBatchScheduler to return:

io.grpc.StatusRuntimeException: UNAVAILABLE: This task would start a fresh batch, but all batch threads are busy, so at present there is no processing capacity available for this task

While normally resubmitting the job wouldn't be difficult, the way my system is architected makes this sort of hard to do without blocking. It looks a bit like this:

client <---> predictor <---> server, where predictor is actually akin to inception_client.py. However, predictor returns the result future to client, which then figures out which callbacks to attach depending on why it requested this particular prediction. Since predictor has a list of available servers it can submit to, it would be great if there was some way for predictor to work like:

1. Get job
2. Submit job to server n
3. Check if job was added / not rejected, and
   • if so, return result future to client.
   • otherwise go to (2) with server n+1

Edits:
#10 does contain three suggestions for handling this, although I don't think they apply here. (1) Even with arbitrary optimization of the serving parameters, we will still hit this issue, I think. (2) I don't want the request to block until it's been submitted; I want it to refuse the request so that client can try the next server in the list. (3) Would hide the load from the client.

Migrate to 0.13 to fix a known issue in gRPC 0.12.

ERROR: /home/ubuntu/serving/tensorflow_serving/batching/test_util/BUILD:38:1: Linking of rule '//tensorflow_serving/batching/test_util:puppet_batch_scheduler_test' failed: gcc failed: error executing command
(cd /home/ubuntu/.cache/bazel/_bazel_ubuntu/8bd6e58495e54c8cdf1fb8b1ed15e742/serving &&
exec env -
/usr/bin/gcc -o bazel-out/local_linux-fastbuild/bin/tensorflow_serving/batching/test_util/puppet_batch_scheduler_test '-Wl,-rpath,$ORIGIN/../../../_solib_local/' -Lbazel-out/local_linux-fastbuild/bin/_solib_local -pthread -B/usr/bin/ -Wl,-z,relro,-z,now -no-canonical-prefixes -pass-exit-codes '-Wl,--build-id=md5' '-Wl,--hash-style=gnu' -Wl,-S -Wl,@bazel-out/local_linux-fastbuild/bin/tensorflow_serving/batching/test_util/puppet_batch_scheduler_test-2.params): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.

Per viksit@: "the compiled version of the dependencies are almost 1.5G with tf taking up a lot of it."

Options:

1. Add a shared library. Example in TensorFlow.
2. Look at compilation options, e.g. -c opt.

What would be the best way to load and serve multiple models into the system?

Looking at the code that exists today, I couldn't see anything directly usable. If there's nothing that does this today, what would be a good set of things to look at to implement something like the following,

Some thoughts,

• Lets assume each model resided in a directory called /mnt/modelid/0000001
• The current FileSystemStoragePathSource only looks under a given path for the aspired version. So ideally, there are some methods somewhere that can instead help look for modelid rather than within it.
• The output of this should be a hashmap of servableId type objects, and I ought to be able to do a Get() from such an object to get the right modelId, and then run classify() on it.

Lastly - do you guys have use cases where there are multiple models spread out across multiple machines, and using some sort of a DHT implementation to access them?

In the section "Generic Signatures (custom or advanced usage)" of "serving/tensorflow_serving/session_bundle/README.md", "named_graph_signatures=signatures" is missing the "s" in the parameter name. The example causes an error due to the incorrect parameter name.

signatures = {
"regression": exporter.regression_signature(input_tensor=v0,
output_tensor=v1),
"generic": exporter.generic_signature(named_tensor_bindings)}
export = exporter.Exporter(saver)
export.init(sess.graph.as_graph_def(),
named_graph_signatures=signatures)

In the python world, the tf.reshape API works well to do this. But whats a good example in the C++ API?

The use case is that my graph needs a tensor of shape (1, x, y), and my input is coming in as (1, x* y). I'd like to transform the latter into the former, but can't find any examples to go on (yet).

I am working on a project and have use the tf.py_func() to create my own OP in python. I can compile the training script and properly train the model. I get the exported model and can see it in TensorBoard.

The problem is after I compile the "inference.cc". I can compile the inference.cc file and have used it to load other models to test in production. When I try to load the model with the tf.py_func() in it, I get the following error:

I tensorflow_serving/session_bundle/session_bundle.cc:130] Attempting to load a SessionBundle from: /tmp/export/00000001
E tensorflow_serving/ProductLoadModel/inference.cc:198] Fail to load tensorflow export: Op type not registered 'PyFunc'

Is there something I need to do to export the py_func() for TensorFlow Serving? Is there a missing import?

using tensorflow::serving::ClassificationSignature;
using tensorflow::serving::NT8Request;
using tensorflow::serving::ResponseToNT8;
using tensorflow::serving::NT8Service;
using tensorflow::serving::SessionBundle;
using tensorflow::string;
using tensorflow::Tensor;
using tensorflow::TensorShape;

Thanks

I would appreciate it If you provide the Google Inception example compatible with TensorFlow serving like MNIST example.

Recently you have introduced a very nice transfer learning example for image retraining. Please provide some informations about exporting the retrained inception model with TensorFlow Serving Exporter module. It would be very helpful if you make a full working example like MNIST.

@kirilg I would greatly appreciate it if you could provide us a tutorial on how to send an image to a tensorflow serving server and get a response with the results.

Regards

I'm trying to figure out how I can set tensorflow serving up as a distributed system, in which multiple models can be served simultaneously, and each model can be served by multiple machines.

I found this instruction about how tensorflow can be used as a distributed system, but it requires the client application to explicitly list all the servers involved. I wonder if there is something like a unified "frontend" to coordinate all the models and machines (if it is supported to run tensorflow serving as a distributed system). For example, a user query can be routed to the "frontend" machine with some specifications about what model it needs to run, and the "frontend" machine can figure out where the backend model is deployed and route the query to the machine.

Since tensorflow serving is a service-oriented framework, I'm guessing there probably already exists some support to scale it up to multiple server nodes.

Thanks!

After installing https://github.com/bazelbuild/bazel/releases/download/0.2.2b/bazel-0.2.2b-installer-linux-x86_64.sh the output of bazel build tensorflow_serving/... is

ERROR: /home/ubuntu/serving/WORKSPACE:18:1: Traceback (most recent call last):
    File "/home/ubuntu/serving/WORKSPACE", line 18
        check_version("0.2.0")
    File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/8bd6e58495e54c8cdf1fb8b1ed15e742/external/tf/tensorflow/tensorflow.bzl", line 22, in check_version
        _parse_bazel_version(native.bazel_version)
    File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/8bd6e58495e54c8cdf1fb8b1ed15e742/external/tf/tensorflow/tensorflow.bzl", line 15, in _parse_bazel_version
        int(number)
invalid literal for int(): "2b".
ERROR: Error evaluating WORKSPACE file.
ERROR: package contains errors: tensorflow_serving.
ERROR: no such package 'external': Package 'external' contains errors.
INFO: Elapsed time: 0.800s

Not a big deal, but I guess you can't assume they use numbers in their version strings...