Comments (7)
try -DCMAKE_PREFIX_PATH=/path/to/protobuf
from dali.
Hi,
We rely on FindProtobuf.cmake which is used by find_package.
Internally it relies on find_program, find_library, find_path (full code https://github.com/NVIDIA/DALI/blob/master/cmake/modules/FindProtobuf.cmake). They search patches until the first hit in the following order:
- Search paths specified in cmake-specific cache variables
- Search paths specified in cmake-specific environment variables
- Search the paths specified by the HINTS option
- Search the standard system environment variables - in our case PATH, LIB and INCLUDE
- Search cmake variables defined in the Platform files for the current system
- Search the paths specified by the PATHS option or in the short-hand version of the command
I recommend trying to set env variables: PATH, LIB and INCLUDE to values that point to lib, include and protoc executable.
from dali.
If that doesn't work for you please reopen this issue.
from dali.
@JanuszL
I tried to import my own compiled protobuf instead of the system being done, but it was not successful. Because I am on the server, I don't have permission, but I have to use my own compiled protobuf. Is there any good way? Just like NVJPEG, it can be set well with NVJPEG_ROOT_DIR.
cmake -DCMAKE_INSTALL_PREFIX=/home/guangyuan/lib/dali - DNVJPEG_ROOT_DIR=/home/guangyuan/lib/cuda-linux64-nvjpeg -DJPEG_TURBO_ROOT_DIR=/home/guangyuan/lib/libjpeg_turbo_1.5.9 -DProtobuf_INCLUDE_DIRS=/home/guangyuan/lib/protobuf_3.5.1/include -DProtobuf_LIBRARIES=/home/guangyuan/lib/protobuf_3.5.1/lib -DProtobuf_PROTOC_LIBRARIES=/home/guangyuan/lib/protobuf_3.5.1/lib -DProtobuf_PROTOC_EXECUTABLE=/home/guangyuan/lib/protobuf_3.5.1/bin/protoc ..
-- The CXX compiler identification is GNU 4.8.5
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- DALI version: 0.3.0
-- The C compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found PythonInterp: /home/guangyuan/anaconda3/envs/tensorflow/bin/python (found version "3.6.6")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Found Git: /usr/bin/git (found version "1.8.3.1")
-- git Version: v0.2.0-24297324-dirty
-- Version: 0.2.0
-- Performing Test HAVE_CXX_FLAG_STD_CXX11
-- Performing Test HAVE_CXX_FLAG_STD_CXX11 - Success
-- Performing Test HAVE_CXX_FLAG_WALL
-- Performing Test HAVE_CXX_FLAG_WALL - Success
-- Performing Test HAVE_CXX_FLAG_WEXTRA
-- Performing Test HAVE_CXX_FLAG_WEXTRA - Success
-- Performing Test HAVE_CXX_FLAG_WSHADOW
-- Performing Test HAVE_CXX_FLAG_WSHADOW - Success
-- Performing Test HAVE_CXX_FLAG_WERROR
-- Performing Test HAVE_CXX_FLAG_WERROR - Success
-- Performing Test HAVE_CXX_FLAG_PEDANTIC
-- Performing Test HAVE_CXX_FLAG_PEDANTIC - Success
-- Performing Test HAVE_CXX_FLAG_PEDANTIC_ERRORS
-- Performing Test HAVE_CXX_FLAG_PEDANTIC_ERRORS - Success
-- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32
-- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32 - Failed
-- Performing Test HAVE_CXX_FLAG_WFLOAT_EQUAL
-- Performing Test HAVE_CXX_FLAG_WFLOAT_EQUAL - Success
-- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING
-- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING - Success
-- Performing Test HAVE_CXX_FLAG_WZERO_AS_NULL_POINTER_CONSTANT
-- Performing Test HAVE_CXX_FLAG_WZERO_AS_NULL_POINTER_CONSTANT - Success
-- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING
-- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING - Success
-- Performing Test HAVE_CXX_FLAG_WD654
-- Performing Test HAVE_CXX_FLAG_WD654 - Failed
-- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY
-- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY - Failed
-- Performing Test HAVE_CXX_FLAG_COVERAGE
-- Performing Test HAVE_CXX_FLAG_COVERAGE - Success
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX -- compiled but failed to run
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK -- success
-- Found CUDA: /usr/local/cuda (found suitable version "9.0", minimum required is "8.0")
-- Found NVJPEG: /home/guangyuan/lib/cuda-linux64-nvjpeg/include (found suitable version "9.0", minimum required is "9.0")
-- JpegTurbo_INCLUDE_DIR (missing: VERSION_VAR)
-- Found OpenCV: /home/guangyuan/lib/opencv-3.10/include/opencv;/home/guangyuan/lib/opencv-3.10/include (found suitable version "3.1.0", minimum required is "2.0")
-- Found PythonLibs: /home/guangyuan/anaconda3/envs/tensorflow/lib/libpython3.6m.so
-- pybind11 v2.3.dev0
CMake Warning at cmake/modules/FindProtobuf.cmake:433 (message):
Protobuf compiler version 3.5.1 doesn't match library version 2.5.0
Call Stack (most recent call first):
cmake/Dependencies.cmake:132 (find_package)
CMakeLists.txt:43 (include)
-- Found Protobuf: /home/guangyuan/lib/protobuf_3.5.1/lib;-pthread (found suitable version "2.5.0", minimum required is "2.0")
-- TensorFlow TFRecord file format support is not available with Protobuf 2
-- Exclude libs 'libcudart_static.a:libnvjpeg_static.a:libnppc_static.a:libnppicom_static.a:libnppicc_static.a:libnppig_static.a:libculibos.a:libturbojpeg.a:libopencv_core.a:libopencv_imgproc.a:libopencv_highgui.a:libopencv_imgcodecs.a'
-- Adding dependencies to dali: '/usr/local/cuda/lib64/libcudart_static.a;-pthread;dl;/usr/lib64/librt.so;/home/guangyuan/lib/cuda-linux64-nvjpeg/lib64/libnvjpeg_static.a;/usr/local/cuda/lib64/libnppc_static.a;/usr/local/cuda/lib64/libnppicom_static.a;/usr/local/cuda/lib64/libnppicc_static.a;/usr/local/cuda/lib64/libnppig_static.a;/usr/local/cuda/lib64/libculibos.a;/home/guangyuan/lib/libjpeg_turbo_1.5.9/lib64/libturbojpeg.so;opencv_core;opencv_imgproc;opencv_imgcodecs'
-- Adding dependencies to dali_test.bin: 'dali'
-- Adding dependencies to dali_benchmark.bin: 'dali'
-- Adding dependencies to dali_tf: 'dali'
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- LTO enabled
-- Adding dependencies to backend_impl: 'dali'
-- Adding dependencies to backend_impl: 'dali_tf'
-- Configuring done
-- Generating done
-- Build files have been written to: /home/guangyuan/dali/build
(tensorflow) [guangyuan@172-16-30-205 build]$ make -j8
[ 1%] Running C++ protocol buffer compiler on proto/dali.proto
Scanning dependencies of target gtest
[ 2%] Running C++ protocol buffer compiler on proto/caffe.proto
[ 2%] Running C++ protocol buffer compiler on proto/caffe2.proto
Scanning dependencies of target benchmark
Scanning dependencies of target libdali.map
make[2]: Circular dali/CMakeFiles/libdali.map <- libdali.map dependency dropped.
[ 2%] Built target libdali.map
Scanning dependencies of target CAFFE_PROTO
Scanning dependencies of target CAFFE2_PROTO
[ 3%] Building CXX object dali/pipeline/operators/reader/parser/CMakeFiles/CAFFE2_PROTO.dir/caffe2.pb.cc.o
Scanning dependencies of target DALI_PROTO
[ 3%] Building CXX object dali/pipeline/operators/reader/parser/CMakeFiles/CAFFE_PROTO.dir/caffe.pb.cc.o
[ 4%] Building CXX object dali/pipeline/CMakeFiles/DALI_PROTO.dir/dali.pb.cc.o
[ 5%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/benchmark.cc.o
[ 6%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/colorprint.cc.o
[ 6%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/commandlineflags.cc.o
[ 7%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/benchmark_register.cc.o
In file included from /home/guangyuan/dali/build/dali/pipeline/dali.pb.cc:4:0:
/home/guangyuan/dali/build/dali/pipeline/dali.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
#error This file was generated by a newer version of protoc which is
^
/home/guangyuan/dali/build/dali/pipeline/dali.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
#error incompatible with your Protocol Buffer headers. Please update
^
/home/guangyuan/dali/build/dali/pipeline/dali.pb.h:14:2: error: #error your headers.
#error your headers.
^
In file included from /home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe2.pb.cc:4:0:
/home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe2.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
#error This file was generated by a newer version of protoc which is
^
/home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe2.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
#error incompatible with your Protocol Buffer headers. Please update
^
/home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe2.pb.h:14:2: error: #error your headers.
#error your headers.
^
In file included from /home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe.pb.cc:4:0:
/home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
#error This file was generated by a newer version of protoc which is
^
/home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
#error incompatible with your Protocol Buffer headers. Please update
^
/home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe.pb.h:14:2: error: #error your headers.
#error your headers.
^
[ 7%] Building CXX object third_party/googletest/googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o
In file included from /home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe.pb.cc:4:0:
/home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe.pb.h:23:35: fatal error: google/protobuf/arena.h: No such file or directory
#include <google/protobuf/arena.h>
^
In file included from /home/guangyuan/dali/build/dali/pipeline/dali.pb.cc:4:0:
/home/guangyuan/dali/build/dali/pipeline/dali.pb.h:23:35: fatal error: google/protobuf/arena.h: No such file or directory
#include <google/protobuf/arena.h>
^
In file included from /home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe2.pb.cc:4:0:
/home/guangyuan/dali/build/dali/pipeline/operators/reader/parser/caffe2.pb.h:23:35: fatal error: google/protobuf/arena.h: No such file or directory
#include <google/protobuf/arena.h>
^
compilation terminated.
compilation terminated.
compilation terminated.
[ 8%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/complexity.cc.o
make[2]: *** [dali/pipeline/operators/reader/parser/CMakeFiles/CAFFE_PROTO.dir/caffe.pb.cc.o] Error 1
make[1]: *** [dali/pipeline/operators/reader/parser/CMakeFiles/CAFFE_PROTO.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 9%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/console_reporter.cc.o
[ 10%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/counter.cc.o
make[2]: *** [dali/pipeline/CMakeFiles/DALI_PROTO.dir/dali.pb.cc.o] Error 1
make[1]: *** [dali/pipeline/CMakeFiles/DALI_PROTO.dir/all] Error 2
[ 10%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/csv_reporter.cc.o
make[2]: *** [dali/pipeline/operators/reader/parser/CMakeFiles/CAFFE2_PROTO.dir/caffe2.pb.cc.o] Error 1
make[1]: *** [dali/pipeline/operators/reader/parser/CMakeFiles/CAFFE2_PROTO.dir/all] Error 2
[ 11%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/json_reporter.cc.o
[ 12%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/reporter.cc.o
[ 13%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/sleep.cc.o
[ 13%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/statistics.cc.o
[ 14%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/string_util.cc.o
[ 15%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/sysinfo.cc.o
[ 16%] Building CXX object third_party/benchmark/src/CMakeFiles/benchmark.dir/timers.cc.o
[ 16%] Linking CXX static library libbenchmark.a
[ 16%] Built target benchmark
[ 17%] Linking CXX static library libgtest.a
[ 17%] Built target gtest
make: *** [all] Error 2
from dali.
Hi,
It is not that easy. For nvJpeg we are using our own custom way of localizing it in the system. For protobuf we are relying on CMake provided scripts.
You can try to google it - what I have found with a quick search is https://stackoverflow.com/questions/50239220/cmake-find-protobuf-compiled-from-source, maybe you can try to investigate this idea and make it work somehow.
from dali.
try -DCMAKE_PREFIX_PATH=/path/to/protobuf
Thank you.
from dali.
try -DCMAKE_PREFIX_PATH=/path/to/protobuf
Thank you
from dali.
Related Issues (20)
- Runtime error using Numpy readers: Unknown Numpy type c8 HOT 4
- Install DALI from source got error "file format not recognized" and "syntax error" HOT 3
- Why Decoder consumes 24 CPU cores even if using "mixed" device HOT 4
- Grab one sample from pipeline for shaping and reset pipeline HOT 6
- Can I use DALI only for data augmentation? HOT 2
- Checkpoint support for readers HOT 7
- How to use Dali tool for image sharpening operation HOT 4
- Ran of GPU memory when using Imagenet but not COCO-Stuff 2017 HOT 20
- how to do a image zoom? HOT 9
- Unrecognized image format HOT 1
- Inference Model without converting TensorGPU to TensorCPU HOT 2
- Stack a batch in one batch of this shape HOT 4
- How to get center crop HOT 6
- GitHub Roadmap 2024 HOT 4
- Why is the val_loss curve trained through Dali data loading method oscillating? HOT 2
- NumPy decoder HOT 8
- Using JPEG hardware decoder with DALI on A100 GPU
- Extracting properties from a list of DataNodes HOT 5
- A100 hardware decoder HOT 1
- Extract motion vectors HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dali.