jcsilva / docker-kaldi-android Goto Github PK

Is there some code or instructions on how to call the online2-wav-nnet3-latgen-faster and build your own apk that uses the compiled android kaldi binaries?

https://github.com/ahmetaa/kaldi-jni
https://github.com/keenresearch/KeenASR-Android-PoC
https://github.com/Kidsense/Vocally

Hi jcsilva,
Thanks for your awesome project.

I find the Kaldi makefile had been changed to OPENFST_VERSION ?= 1.6.2(added char ?) after 16 Aug. This make compile-kaldi.sh run error.

The line 6 in compile-kaldi.sh should be modified to OPENFST_VERSION *.= *\K(.*)$

I managed to build kaldi for android (ndk r16b, api 21, arm) using your dockerfile and also crosscompiling on my linux_x86_64 machine following your blog post. However I get runtime errors when I start onlinedecoding. This happens only when compiling with "-O2" or "-O3". With "-O0" or "-O1" everything works like a charm. The behavior is the same with the docker-build and locally cross-compiled version.

This is the error:
ERROR: FstHeader::Read: Bad FST header: HCLG.fst
ERROR ([5.4.153~1-3f4f4]:ReadFstKaldiGeneric():kaldi-fst-io.cc:53) Reading FST: error reading FST header from HCLG.fst

Digging into the problem I found that kaldi::Input input stream is corrupted: (again, works with -O0/O1, so the fst-file should be alright)

kaldi::Input ki("HCLG.fst");
ki.Stream().good() // false
ki.Stream().eof() // false
ki.Stream().fail() // true
ki.Stream().bad() // true

Any Idea what might be the problem?

Can you have compiled version of 'kaldi' for Android.....?

你好，请问一下生成动态库需要怎么修改呀，看了别人修改的步骤，还是不太理解

We ran into a very strange iostream bug in Kaldi with its early ReadKaldiObject stuff. Only on Arm64, same code compiled with your docker on ArmV7 all good.

sudo docker run -v /home/haidar/kaldi:/opt/kaldi docker-kaldi-android:latest
Unable to find image 'docker-kaldi-android:latest' locally
docker: Error response from daemon: repository docker-kaldi-android not found: does not exist or no pull access.
See 'docker run --help'.

Hi @jcsilva

I've pulled your docker image and ran it by the following command:

docker run --entrypoint='' -it -v /root/git/kaldi-android:/opt/kaldi jcsilva/docker-kaldi-android:latest /bin/bash

This helped me make cub in Kaldi's tools folder, since Kaldi had a weird issue with --use-cuda=no
option passed to configure file (it still tried to include cude):

cd /opt/kaldi/src/tools && make cub

After that, I've run compile-kaldi.sh . It was working for a while until this issue pops up:

arm-linux-androideabi-clang++ -Wl,--no-warn-mismatch -pie matrix-lib-test.o kaldi-matrix.a ../base/kaldi-base.a /opt/kaldi/tools/openfst/lib/libfst.a /opt/OpenBLAS/install/lib/libopenblas.a /opt/OpenBLAS/install/lib/libclapack.a /opt/OpenBLAS/install/lib/liblapack.a /opt/OpenBLAS/install/lib/libblas.a /opt/OpenBLAS/install/lib/libf2c.a -lm -ldl -o matrix-lib-test
Running matrix-lib-test ... 1s... FAIL matrix-lib-test
Running sparse-matrix-test ... 0s... FAIL sparse-matrix-test
make[1]: *** [test] Error 1
make[1]: Leaving directory /opt/kaldi/src/matrix
make: *** [all] Error 2

I'm looking forward to a hint, @jcsilva :) Might it be a bug with arm-linux-androideabi-clang++?

Thank you!

/tmp/my-android-toolchain/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot find -lm_hard
clang50++: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [align-equal] Error 1
make[2]: *** Waiting for unfinished jobs....
arm-linux-androideabi-clang++ -std=c++11 -I.. -I/opt/kaldi/tools/openfst/include -Wno-sign-compare -Wall -Wno-sign-compare -Wno-unused-local-typedefs -Wno-deprecated-declarations -Winit-self -Wno-mismatched-tags -DKALDI_DOUBLEPRECISION=0 -DHAVE_CXXABI_H -DHAVE_OPENBLAS -DANDROID_BUILD -I/opt/OpenBLAS/install/include -I/tmp/my-android-toolchain/sysroot/usr/include -ftree-vectorize -mfloat-abi=hard -mfpu=neon -mhard-float -D_NDK_MATH_NO_SOFTFP=1 -pthread -O3 -DNDEBUG -c -o gmm-global-est-lvtln-trans.o gmm-global-est-lvtln-trans.cc
In file included from gmm-global-est-lvtln-trans.cc:27:

/tmp/my-android-toolchain/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot find -lm_hard
/tmp/my-android-toolchain/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot find -lm_hard

Hi,

I was wondering if you have ever tried building for IOS (obviously not in Docker!).
Should it be similar in principle?

Thanks

Env:
1 . Ubuntu 16.04 64

Step:
Follow the given step and mount docker container to /opt/kaldi.
docker run -v /home/user/mtk54090/kaldi/docker-kaldi-android/kaldi:/opt/kaldi jcsilva/docker-kaldi-android:latest
meet following error:
make[1]: Leaving directory `/opt/kaldi/tools/openfst-1.6.7'
ln: failed to create symbolic link 'openfst/openfst-1.6.7': File exists
./opt/compile-kaldi.sh: line 21: ./configure: Permission denied
sed: can't read kaldi.mk: No such file or directory
Makefile:30: kaldi.mk: No such file or directory
/bin/bash: ./configure: Permission denied
kaldi.mk does not exist; you have to run ./configure
make: *** [kaldi.mk] Error 1
Makefile:30: kaldi.mk: No such file or directory
/bin/bash: ./configure: Permission denied
kaldi.mk does not exist; you have to run ./configure
make: *** [kaldi.mk] Error 1
Makefile:30: kaldi.mk: No such file or directory
/bin/bash: ./configure: Permission denied
kaldi.mk does not exist; you have to run ./configure
make: *** [kaldi.mk] Error 1

I have try add sudo in the prefix of command, seems not work.
Any suggestion for this error?
-------------------------------------------------------------- log ------------------------------------

libtool: install: /usr/bin/install -c .libs/farcompilestrings /opt/kaldi/tools/openfst-1.6.7/bin/farcompilestrings
libtool: install: /usr/bin/install -c .libs/farcreate /opt/kaldi/tools/openfst-1.6.7/bin/farcreate
libtool: install: /usr/bin/install -c .libs/farequal /opt/kaldi/tools/openfst-1.6.7/bin/farequal
libtool: install: /usr/bin/install -c .libs/farextract /opt/kaldi/tools/openfst-1.6.7/bin/farextract
libtool: install: /usr/bin/install -c .libs/farinfo /opt/kaldi/tools/openfst-1.6.7/bin/farinfo
libtool: install: /usr/bin/install -c .libs/farisomorphic /opt/kaldi/tools/openfst-1.6.7/bin/farisomorphic
libtool: install: /usr/bin/install -c .libs/farprintstrings /opt/kaldi/tools/openfst-1.6.7/bin/farprintstrings
make[4]: Nothing to be done for install-data-am'. make[4]: Leaving directory /opt/kaldi/tools/openfst-1.6.7/src/extensions/far'
make[3]: Leaving directory /opt/kaldi/tools/openfst-1.6.7/src/extensions/far' Making install in ngram make[3]: Entering directory /opt/kaldi/tools/openfst-1.6.7/src/extensions/ngram'
make[4]: Entering directory /opt/kaldi/tools/openfst-1.6.7/src/extensions/ngram' /bin/mkdir -p '/opt/kaldi/tools/openfst-1.6.7/lib' /bin/bash ../../../libtool --mode=install /usr/bin/install -c libfstngram.la '/opt/kaldi/tools/openfst-1.6.7/lib' libtool: install: warning: relinking libfstngram.la'
libtool: install: (cd /opt/kaldi/tools/openfst-1.6.7/src/extensions/ngram; /bin/bash /opt/kaldi/tools/openfst-1.6.7/libtool --tag CXX --mode=relink clang++ -std=c++11 -version-info 10:0:0 -o libfstngram.la -rpath /opt/kaldi/tools/openfst-1.6.7/lib bitmap-index.lo ngram-fst.lo nthbit.lo ../../lib/libfst.la -lm -ldl -ldl )
libtool: relink: clang++ -fPIC -DPIC -shared -nostdlib /tmp/my-android-toolchain/bin/../sysroot/usr/lib/../lib/crtbegin_so.o .libs/bitmap-index.o .libs/ngram-fst.o .libs/nthbit.o -Wl,-rpath -Wl,/opt/kaldi/tools/openfst-1.6.7/lib -L/opt/kaldi/tools/openfst-1.6.7/lib -lfst -L/tmp/my-android-toolchain/lib64/clang/5.0.300080/lib/linux/arm -L/tmp/my-android-toolchain/bin/../lib/gcc/arm-linux-androideabi/4.9.x/armv7-a -L/tmp/my-android-toolchain/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/lib/../lib/armv7-a -L/tmp/my-android-toolchain/bin/../sysroot/usr/lib/../lib -L/tmp/my-android-toolchain/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/lib/armv7-a -L/tmp/my-android-toolchain/bin/../sysroot/usr/lib -lstdc++ -lm -lc -lgcc -ldl /tmp/my-android-toolchain/bin/../sysroot/usr/lib/../lib/crtend_so.o -Wl,-soname -Wl,libfstngram.so.10 -o .libs/libfstngram.so.10.0.0
libtool: install: /usr/bin/install -c .libs/libfstngram.so.10.0.0T /opt/kaldi/tools/openfst-1.6.7/lib/libfstngram.so.10.0.0
libtool: install: (cd /opt/kaldi/tools/openfst-1.6.7/lib && { ln -s -f libfstngram.so.10.0.0 libfstngram.so.10 || { rm -f libfstngram.so.10 && ln -s libfstngram.so.10.0.0 libfstngram.so.10; }; })
libtool: install: (cd /opt/kaldi/tools/openfst-1.6.7/lib && { ln -s -f libfstngram.so.10.0.0 libfstngram.so || { rm -f libfstngram.so && ln -s libfstngram.so.10.0.0 libfstngram.so; }; })
libtool: install: /usr/bin/install -c .libs/libfstngram.lai /opt/kaldi/tools/openfst-1.6.7/lib/libfstngram.la
libtool: install: /usr/bin/install -c .libs/libfstngram.a /opt/kaldi/tools/openfst-1.6.7/lib/libfstngram.a
libtool: install: chmod 644 /opt/kaldi/tools/openfst-1.6.7/lib/libfstngram.a
libtool: install: arm-linux-androideabi-ranlib /opt/kaldi/tools/openfst-1.6.7/lib/libfstngram.a
libtool: finish: PATH="/tmp/my-android-toolchain/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /opt/kaldi/tools/openfst-1.6.7/lib

Libraries have been installed in:
/opt/kaldi/tools/openfst-1.6.7/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:

• add LIBDIR to the `LD_LIBRARY_PATH' environment variable
  during execution
• add LIBDIR to the `LD_RUN_PATH' environment variable
  during linking
• use the `-Wl,-rpath -Wl,LIBDIR' linker flag
• have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.

/bin/mkdir -p '/opt/kaldi/tools/openfst-1.6.7/lib/fst'
/bin/bash ../../../libtool --mode=install /usr/bin/install -c ngram-fst.la '/opt/kaldi/tools/openfst-1.6.7/lib/fst'
libtool: install: /usr/bin/install -c .libs/ngram-fst.so.0.0.0 /opt/kaldi/tools/openfst-1.6.7/lib/fst/ngram-fst.so.0.0.0
libtool: install: (cd /opt/kaldi/tools/openfst-1.6.7/lib/fst && { ln -s -f ngram-fst.so.0.0.0 ngram-fst.so.0 || { rm -f ngram-fst.so.0 && ln -s ngram-fst.so.0.0.0 ngram-fst.so.0; }; })
libtool: install: (cd /opt/kaldi/tools/openfst-1.6.7/lib/fst && { ln -s -f ngram-fst.so.0.0.0 ngram-fst.so || { rm -f ngram-fst.so && ln -s ngram-fst.so.0.0.0 ngram-fst.so; }; })
libtool: install: /usr/bin/install -c .libs/ngram-fst.lai /opt/kaldi/tools/openfst-1.6.7/lib/fst/ngram-fst.la
libtool: install: /usr/bin/install -c .libs/ngram-fst.a /opt/kaldi/tools/openfst-1.6.7/lib/fst/ngram-fst.a
libtool: install: chmod 644 /opt/kaldi/tools/openfst-1.6.7/lib/fst/ngram-fst.a
libtool: install: arm-linux-androideabi-ranlib /opt/kaldi/tools/openfst-1.6.7/lib/fst/ngram-fst.a
libtool: finish: PATH="/tmp/my-android-toolchain/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /opt/kaldi/tools/openfst-1.6.7/lib/fst

Libraries have been installed in:
/opt/kaldi/tools/openfst-1.6.7/lib/fst

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:

• add LIBDIR to the `LD_LIBRARY_PATH' environment variable
  during execution
• add LIBDIR to the `LD_RUN_PATH' environment variable
  during linking
• use the `-Wl,-rpath -Wl,LIBDIR' linker flag
• have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.

make[4]: Leaving directory /opt/kaldi/tools/openfst-1.6.7/src/extensions/ngram' make[3]: Leaving directory /opt/kaldi/tools/openfst-1.6.7/src/extensions/ngram'
make[3]: Entering directory /opt/kaldi/tools/openfst-1.6.7/src/extensions' make[4]: Entering directory /opt/kaldi/tools/openfst-1.6.7/src/extensions'
make[4]: Nothing to be done for install-exec-am'. make[4]: Nothing to be done for install-data-am'.
make[4]: Leaving directory /opt/kaldi/tools/openfst-1.6.7/src/extensions' make[3]: Leaving directory /opt/kaldi/tools/openfst-1.6.7/src/extensions'
make[2]: Leaving directory /opt/kaldi/tools/openfst-1.6.7/src/extensions' make[2]: Entering directory /opt/kaldi/tools/openfst-1.6.7/src'
make[3]: Entering directory /opt/kaldi/tools/openfst-1.6.7/src' make[3]: Nothing to be done for install-exec-am'.
make[3]: Nothing to be done for install-data-am'. make[3]: Leaving directory /opt/kaldi/tools/openfst-1.6.7/src'
make[2]: Leaving directory /opt/kaldi/tools/openfst-1.6.7/src' make[1]: Leaving directory /opt/kaldi/tools/openfst-1.6.7/src'
make[1]: Entering directory /opt/kaldi/tools/openfst-1.6.7' make[2]: Entering directory /opt/kaldi/tools/openfst-1.6.7'
make[2]: Nothing to be done for install-exec-am'. make[2]: Nothing to be done for install-data-am'.
make[2]: Leaving directory /opt/kaldi/tools/openfst-1.6.7' make[1]: Leaving directory /opt/kaldi/tools/openfst-1.6.7'
ln: failed to create symbolic link 'openfst/openfst-1.6.7': File exists
./opt/compile-kaldi.sh: line 21: ./configure: Permission denied
sed: can't read kaldi.mk: No such file or directory
Makefile:30: kaldi.mk: No such file or directory
/bin/bash: ./configure: Permission denied
kaldi.mk does not exist; you have to run ./configure
make: *** [kaldi.mk] Error 1
Makefile:30: kaldi.mk: No such file or directory
/bin/bash: ./configure: Permission denied
kaldi.mk does not exist; you have to run ./configure
make: *** [kaldi.mk] Error 1
Makefile:30: kaldi.mk: No such file or directory
/bin/bash: ./configure: Permission denied
kaldi.mk does not exist; you have to run ./configure
make: *** [kaldi.mk] Error 1

Hi,

I am trying to get online decoding to work on android.
For the start I was following: http://kaldi-asr.org/doc/online_decoding.html
and executed src/online2bin/online2-wav-nnet2-latgen-faster which was build using the
approach from your readme with the jcsilva/docker-kaldi-android:latest docker pulled from Docker Hub.

src/online2bin/online2-wav-nnet2-latgen-faster --do-endpointing=false \
    --online=false \
    --config=nnet_a_gpu_online/conf/online_nnet2_decoding.conf \
    --max-active=7000 --beam=15.0 --lattice-beam=6.0 \
    --acoustic-scale=0.1 --word-symbol-table=graph/words.txt \
   nnet_a_gpu_online/final.mdl graph/HCLG.fst "ark:echo utterance-id1 utterance-id1|" "scp:echo utterance-id1 ENG_M.wav|" \
   ark:/dev/null

This all works and executes fine on android (adb shell on a P6). However, when I build your docker image from the repo, and use this to build kaldi, I get a runtime error executing the example:

LOG (orig[5.1.74~1391-c68a]:void kaldi::IvectorExtractor::ComputeDerivedVars()():ivector-extractor.cc:183) Computing derived variables for iVector extractor
WARNING (orig[5.1.74~1391-c68a]:void kaldi::TpMatrix<double>::Cholesky(const SpMatrix<Real> &) [Real = double]():tp-matrix.cc:110) Cholesky decomposition failed. Maybe matrix is not positive definite. Throwing error 
Cholesky decomposition failed

Digging into it, I found that the docker image from Docker Hub is probably not build from latest master, but from the build date (06-28) I tried commits from 06-27 and 06-28 which both fail the same way.
When analyzing the build executables with readelf -a -W and looking at their diffs (build with Docker-Hub-image and self-built-image-from-this-repo-latest) There are only two differences:

These are in the repo-build executable

_ZNKSt6__ndk16vectorIPN3fst11VectorStateINS1_6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS6_EEEENS7_ISA_EEE17__annotate_deleteEv
_ZNKSt6__ndk16vectorIPN3fst11VectorStateINS1_6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS6_EEEENS7_ISA_EEE14__annotate_newEj
_ZN3fst11VectorStateINS_6ArcTplINS_16LatticeWeightTplIdEEEENSt6__ndk19allocatorIS4_EEEnwEjPNS6_IS8_EE
_ZN3fst11VectorStateINS_6ArcTplINS_16LatticeWeightTplIdEEEENSt6__ndk19allocatorIS4_EEEC2ERKS7_
_ZNSt6__ndk16vectorIPN3fst11VectorStateINS1_6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS6_EEEENS7_ISA_EEE24__RAII_IncreaseAnnotatorC2ERKSC_j
_ZNSt6__ndk16vectorIPN3fst11VectorStateINS1_6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS6_EEEENS7_ISA_EEE21__push_back_slow_pathISA_EEvOT_
_ZNKSt6__ndk16vectorIPN3fst11VectorStateINS1_6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS6_EEEENS7_ISA_EEE8max_sizeEv
_ZN3fst11VectorStateINS_6ArcTplINS_16LatticeWeightTplIdEEEENSt6__ndk19allocatorIS4_EEE11ReserveArcsEj
_ZNSt6__ndk16vectorIN3fst6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS5_EEE7reserveEj
_ZNSt6__ndk114__split_bufferIN3fst6ArcTplINS1_16LatticeWeightTplIdEEEERNS_9allocatorIS5_EEEC2EjjS8_
_ZNSt6__ndk16vectorIN3fst6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS5_EEE26__swap_out_circular_bufferERNS_14__split_bufferIS5_RS7_EE
_ZNSt6__ndk114__split_bufferIN3fst6ArcTplINS1_16LatticeWeightTplIdEEEERNS_9allocatorIS5_EEED2Ev
_ZNKSt6__ndk16vectorIN3fst6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS5_EEE17__annotate_deleteEv
_ZNKSt6__ndk16vectorIN3fst6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS5_EEE14__annotate_newEj
_ZN3fst6ArcTplINS_16LatticeWeightTplIdEEEC2ERKS3_
_ZNSt6__ndk16vectorIN3fst6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS5_EEE24__RAII_IncreaseAnnotatorC2ERKS8_j
_ZNSt6__ndk16vectorIN3fst6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS5_EEE21__push_back_slow_pathIRKS5_EEvOT_
_ZNKSt6__ndk16vectorIN3fst6ArcTplINS1_16LatticeWeightTplIdEEEENS_9allocatorIS5_EEE8max_sizeEv
_ZN3fst16ImplToMutableFstINS_8internal13VectorFstImplINS_11VectorStateINS_6ArcTplINS_16LatticeWeightTplIdEEEENSt6__ndk19allocatorIS7_EEEEEENS_10MutableFstIS7_EEEC2ENS8_10shared_ptrISC_EE
_ZN3fst9VectorFstINS_6ArcTplINS_16LatticeWeightTplIdEEEENS_11VectorStateIS4_NSt6__ndk19allocatorIS4_EEEEED2Ev
_ZNK3fst9ImplToFstINS_8internal13VectorFstImplINS_11VectorStateINS_6ArcTplINS_16LatticeWeightTplIdEEEENSt6__ndk19allocatorIS7_EEEEEENS_10MutableFstIS7_EEE5StartEv
_ZNK3fst9VectorFstINS_6ArcTpl

while the docker-hub-version only has this instead:

_ZNKSt6__ndk16vectorIPN3fst11VectorStateINS1

also these are not in the repo-build-one:

00768b30  00004a16 R_ARM_JUMP_SLOT        00000000   clock_gettime@LIBC
00768b34  00004716 R_ARM_JUMP_SLOT        00000000   getrlimit@LIBC
00768b38  00004816 R_ARM_JUMP_SLOT        00000000   raise@LIBC
00768b3c  00018516 R_ARM_JUMP_SLOT        00000000   omp_in_parallel

I don't know how this could relate to the error I am getting, but maybe it helps someone to help me :)

Note: everything was executed with kaldi source commit: 99880f7 which was suggested in your guide.