Giter VIP home page Giter VIP logo

Comments (12)

linxGnu avatar linxGnu commented on May 21, 2024

@svenger87 could you please describe more how you install rocksdb:

  • Which version of rocksdb?
  • Shared or Static?
  • How you link grocksdb to shared/static rocksdb lib?
  • Which command did you invoke to build?
  • Could you please show me full error logs?

There is no builtin-static lib for arm64 at the moment. So, please make sure that you install rocksdb before building grocksdb.

from grocksdb.

svenger87 avatar svenger87 commented on May 21, 2024

Thanks for your answer so far.

It already fails for the go get.
I compiled rocksdb from source.

Package: librocksdb
Status: install ok installed
Priority: optional
Section: misc
Installed-Size: 5390
Maintainer: None <[email protected]>
Architecture: arm64
Version: 6.5.3-deb3-1

dpkg -s librocksdb-dev
Package: librocksdb-dev
Status: install ok installed
Priority: optional
Section: misc
Installed-Size: 804
Maintainer: None <[email protected]>
Architecture: all
Source: librocksdb
Version: 6.5.3-deb3-1
Depends: librocksdb (= 6.5.3-deb3-1)

go get -u github.com/linxGnu/grocksdb
go: downloading github.com/linxGnu/grocksdb v1.6.32
# github.com/linxGnu/grocksdb
/root/go/pkg/mod/github.com/linx!gnu/[email protected]/backup.go:44:13: could not determine kind of name for C.rocksdb_backup_engine_open_opts
/root/go/pkg/mod/github.com/linx!gnu/[email protected]/backup.go:136:2: could not determine kind of name for C.rocksdb_backup_engine_restore_db_from_backup```

from grocksdb.

linxGnu avatar linxGnu commented on May 21, 2024

@svenger87 as your rocksdb version (6.5.3), I think you should use this version of grocksdb: https://github.com/linxGnu/grocksdb/releases/tag/v1.6.11

Please try:

go get github.com/linxGnu/grocksdb@1.6.11

from grocksdb.

svenger87 avatar svenger87 commented on May 21, 2024

@linxGnu i recompiled to 6.16.3 and get the same.
dpkg -s librocksdb-dev
Package: librocksdb-dev
Status: install ok installed
Priority: optional
Section: misc
Installed-Size: 804
Maintainer: None [email protected]
Architecture: all
Source: librocksdb
Version: 6.16.3-deb3-1
Depends: librocksdb (= 6.16.3-deb3-1)
Description: RocksDB Key-Value database (development files)

from grocksdb.

linxGnu avatar linxGnu commented on May 21, 2024

@svenger87

Could you please try this:

CGO_CFLAGS="-I/path/to/rocksdb/include" \
CGO_LDFLAGS="-L/path/to/rocksdb -lrocksdb -lstdc++ -lm -lz -lsnappy -llz4 -lzstd" \
go get github.com/linxGnu/[email protected]

In this way, please replace /path/to/rocksdb with your shared lib. Normally, it's /usr/lib64 or /usr/local/lib

from grocksdb.

linxGnu avatar linxGnu commented on May 21, 2024

Just one more thing, as I see your upgraded to rocksdb 6.16.3. I think you could use latest version of grocksdb. I hope you could build it fluently.

Please let me know if I could help

from grocksdb.

svenger87 avatar svenger87 commented on May 21, 2024

1.6.11 is working. But as soon as i try to fetch a more recent version i get the errors above.
1.6.32 is needed for a depencency. So i can´t really go with a lower version.

This really only happens on arm64. amd64 is building fine.

from grocksdb.

linxGnu avatar linxGnu commented on May 21, 2024

Yeah, as I remember: from rocksdb 1.1x, there are breaking changes. That's why you should use proper version of grocksdb for rocksdb 1.1x.

I am sorry, I don't have arm64 device here to help you. So, currently:

  • 1.6.11 works for you with your rocksdb 6.5.3?
  • 1.6.32 not work for rocksdb 6.16.3?

Could you please confirm your current situation. I see the mix of issue reporting. First: rocksdb 6.5.3, then rocksdb 6.16.3.

from grocksdb.

svenger87 avatar svenger87 commented on May 21, 2024

1.6.11 is working for both. With 6.5.3 and 6.16.3.
1.6.32 and lower versions throw the errors during go get.

from grocksdb.

linxGnu avatar linxGnu commented on May 21, 2024

@svenger87 I could go get pretty well with my current ubuntu 20.04 ARM

OS:

uname -a
Linux ubuntuarm 5.4.0-67-generic #75-Ubuntu SMP Fri Feb 19 18:03:59 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

Go env:

GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/home/gnu/.cache/go-build"
GOENV="/home/gnu/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/gnu/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/gnu/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_arm64"
GOVCS=""
GOVERSION="go1.16.2"
GCCGO="/usr/bin/gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/gnu/go/src/github.com/linh/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build613075368=/tmp/go-build -gno-record-gcc-switches"

Build rocksdb with this script:

#!/bin/bash
DIRECTORY="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"

INSTALL_PREFIX=/usr/local

export CFLAGS='-fPIC -O3 -pipe -funroll-loops' 
export CXXFLAGS='-fPIC -O3 -pipe -funroll-loops'

BUILD_PATH=/tmp/build
mkdir -p $BUILD_PATH

CMAKE_REQUIRED_PARAMS="-DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_INSTALL_PREFIX=${INSTALL_PREFIX}"

zlib_version="1.2.11"
cd $BUILD_PATH && wget https://github.com/madler/zlib/archive/v${zlib_version}.tar.gz && tar xzf v${zlib_version}.tar.gz && cd zlib-${zlib_version} && \
    ./configure --prefix=$INSTALL_PREFIX --static && make -j16 install && \
    cd $BUILD_PATH && rm -rf *

snappy_version="1.1.8"
cd $BUILD_PATH && wget https://github.com/google/snappy/archive/${snappy_version}.tar.gz && tar xzf ${snappy_version}.tar.gz && cd snappy-${snappy_version} && \
    mkdir -p build_place && cd build_place && cmake $CMAKE_REQUIRED_PARAMS -DSNAPPY_BUILD_TESTS=OFF .. && make install/strip -j16 && \
    cd $BUILD_PATH && rm -rf *

lz4_version="1.9.3"
cd $BUILD_PATH && wget https://github.com/lz4/lz4/archive/v${lz4_version}.tar.gz && tar xzf v${lz4_version}.tar.gz && cd lz4-${lz4_version}/build/cmake && \
    cmake $CMAKE_REQUIRED_PARAMS -DLZ4_BUILD_LEGACY_LZ4C=OFF -DBUILD_SHARED_LIBS=OFF -DLZ4_POSITION_INDEPENDENT_LIB=ON && make -j16 install && \
    cd $BUILD_PATH && rm -rf *

zstd_version="1.4.9"
cd $BUILD_PATH && wget https://github.com/facebook/zstd/archive/v${zstd_version}.tar.gz && tar xzf v${zstd_version}.tar.gz && \
    cd zstd-${zstd_version}/build/cmake && mkdir -p build_place && cd build_place && \
    cmake -DZSTD_BUILD_PROGRAMS=OFF -DZSTD_BUILD_CONTRIB=OFF -DZSTD_BUILD_STATIC=ON -DZSTD_BUILD_SHARED=OFF -DZSTD_BUILD_TESTS=OFF \
    -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DZSTD_ZLIB_SUPPORT=ON -DZSTD_LZMA_SUPPORT=OFF -DCMAKE_BUILD_TYPE=Release .. && make -j$(nproc) install && \
    cd $BUILD_PATH && rm -rf * && ldconfig

rocksdb_version="6.16.3"
cd $BUILD_PATH && wget https://github.com/facebook/rocksdb/archive/v${rocksdb_version}.tar.gz && tar xzf v${rocksdb_version}.tar.gz && cd rocksdb-${rocksdb_version}/ && \
    mkdir -p build_place && cd build_place && cmake -DCMAKE_BUILD_TYPE=Release $CMAKE_REQUIRED_PARAMS -DCMAKE_PREFIX_PATH=$INSTALL_PREFIX -DWITH_TESTS=OFF -DWITH_GFLAGS=OFF \
    -DWITH_BENCHMARK_TOOLS=OFF -DWITH_TOOLS=OFF -DWITH_MD_LIBRARY=OFF -DWITH_RUNTIME_DEBUG=OFF -DROCKSDB_BUILD_SHARED=OFF -DWITH_SNAPPY=ON -DWITH_LZ4=ON -DWITH_ZLIB=ON \
    -DWITH_ZSTD=ON -DWITH_BZ2=OFF -WITH_GFLAGS=OFF .. && make -j16 install/strip && \
    cd $BUILD_PATH && rm -rf *

Noted:

  • Above script build all dependencies within /usr/local
  • Above script build static version of rocksdb. You could build share libs as well.

Then

go get -u github.com/linxGnu/grocksdb
# no problem

I think you could give it a try.

And, grocksdb 1.6.32 is built up on rocksdb 6.16.3. It DOES NOT work for rocksdb 6.5.3

from grocksdb.

svenger87 avatar svenger87 commented on May 21, 2024

Okay. go get is working now. No i have tons of other errors compiling the binary i want to build. 🛠️

Thanks for your help so far. Much appreciated.

from grocksdb.

svenger87 avatar svenger87 commented on May 21, 2024

I was missing gccgo. Fixed.

from grocksdb.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.