hi, @tianzhi0549
you really did a great work. i am so appreciated with it.
for OCR, we need character segmentation given a detected text line.
so can you provide your code for character segmentation? or give some advice.
thank you.

I am trying to build caffe from your repository and I am not able to get past make test. I am getting the following error while running "make test".

.build_release/src/caffe/test/test_neuron_layer.o:(.data.rel.ro._ZTVN5caffe8LogLayerIdEE[_ZTVN5caffe8LogLayerIdEE]+0x28): undefined reference to caffe::LogLayer::LayerSetUp(std::vector<caffe::Blob, std::allocator<caffe::Blob> > const&, std::vector<caffe::Blob, std::allocator<caffe::Blob> > const&)'
.build_release/src/caffe/test/test_neuron_layer.o:(.data.rel.ro._ZTVN5caffe8LogLayerIdEE[_ZTVN5caffe8LogLayerIdEE]+0x98): undefined reference to caffe::LogLayer<double>::Forward_cpu(std::vector<caffe::Blob<double>*, std::allocator<caffe::Blob<double>*> > const&, std::vector<caffe::Blob<double>*, std::allocator<caffe::Blob<double>*> > const&)' .build_release/src/caffe/test/test_neuron_layer.o:(.data.rel.ro._ZTVN5caffe8LogLayerIdEE[_ZTVN5caffe8LogLayerIdEE]+0xa0): undefined reference to caffe::LogLayer::Forward_gpu(std::vector<caffe::Blob, std::allocator<caffe::Blob> > const&, std::vector<caffe::Blob, std::allocator<caffe::Blob> > const&)'
.build_release/src/caffe/test/test_neuron_layer.o:(.data.rel.ro._ZTVN5caffe8LogLayerIdEE[_ZTVN5caffe8LogLayerIdEE]+0xa8): undefined reference to caffe::LogLayer<double>::Backward_cpu(std::vector<caffe::Blob<double>*, std::allocator<caffe::Blob<double>*> > const&, std::vector<bool, std::allocator<bool> > const&, std::vector<caffe::Blob<double>*, std::allocator<caffe::Blob<double>*> > const&)' .build_release/src/caffe/test/test_neuron_layer.o:(.data.rel.ro._ZTVN5caffe8LogLayerIdEE[_ZTVN5caffe8LogLayerIdEE]+0xb0): undefined reference to caffe::LogLayer::Backward_gpu(std::vector<caffe::Blob, std::allocator<caffe::Blob> > const&, std::vector<bool, std::allocator > const&, std::vector<caffe::Blob, std::allocator<caffe::Blob> > const&)'
.build_release/src/caffe/test/test_neuron_layer.o:(.data.rel.ro._ZTVN5caffe8LogLayerIfEE[_ZTVN5caffe8LogLayerIfEE]+0x28): undefined reference to caffe::LogLayer<float>::LayerSetUp(std::vector<caffe::Blob<float>*, std::allocator<caffe::Blob<float>*> > const&, std::vector<caffe::Blob<float>*, std::allocator<caffe::Blob<float>*> > const&)' .build_release/src/caffe/test/test_neuron_layer.o:(.data.rel.ro._ZTVN5caffe8LogLayerIfEE[_ZTVN5caffe8LogLayerIfEE]+0x98): undefined reference to caffe::LogLayer::Forward_cpu(std::vector<caffe::Blob, std::allocator<caffe::Blob> > const&, std::vector<caffe::Blob, std::allocator<caffe::Blob> > const&)'
.build_release/src/caffe/test/test_neuron_layer.o:(.data.rel.ro._ZTVN5caffe8LogLayerIfEE[_ZTVN5caffe8LogLayerIfEE]+0xa0): undefined reference to caffe::LogLayer<float>::Forward_gpu(std::vector<caffe::Blob<float>*, std::allocator<caffe::Blob<float>*> > const&, std::vector<caffe::Blob<float>*, std::allocator<caffe::Blob<float>*> > const&)' .build_release/src/caffe/test/test_neuron_layer.o:(.data.rel.ro._ZTVN5caffe8LogLayerIfEE[_ZTVN5caffe8LogLayerIfEE]+0xa8): undefined reference to caffe::LogLayer::Backward_cpu(std::vector<caffe::Blob, std::allocator<caffe::Blob> > const&, std::vector<bool, std::allocator > const&, std::vector<caffe::Blob, std::allocator<caffe::Blob> > const&)'
.build_release/src/caffe/test/test_neuron_layer.o:(.data.rel.ro._ZTVN5caffe8LogLayerIfEE[_ZTVN5caffe8LogLayerIfEE]+0xb0): undefined reference to caffe::LogLayer<float>::Backward_gpu(std::vector<caffe::Blob<float>*, std::allocator<caffe::Blob<float>*> > const&, std::vector<bool, std::allocator<bool> > const&, std::vector<caffe::Blob<float>*, std::allocator<caffe::Blob<float>*> > const&)' collect2: error: ld returned 1 exit status make: *** [.build_release/test/test_neuron_layer.testbin] Error 1 make: *** Waiting for unfinished jobs....
Make all works without any trouble.
Any help would be appreciated.

I am compiling this on Ubuntu 14.04 without cudnn and to be exact on the NVIDIA's DevBox.

Thanks,
G

I0519 23:10:08.041847 18721 net.cpp:849] Ignoring source layer rpn_labels_rpn-data_0_split
I0519 23:10:08.041852 18721 net.cpp:849] Ignoring source layer rpn_loss_cls
I0519 23:10:08.041872 18721 net.cpp:849] Ignoring source layer rpn_loss_bbox
I0519 23:10:08.041877 18721 net.cpp:849] Ignoring source layer accuracy

Image: img_2.jpg
Segmentation fault (core dumped)

Is it possible to share c++ version of your all code?

Thank you

hi.
I compile it with CPU, but in the fourth step, I compile some cython files in the root directory, this is a warning of make.
it is warning:

cython src/utils/cpu_nms.pyx
gcc -shared -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing
-I/usr/include/python2.7 -o src/utils/cpu_nms.so src/utils/cpu_nms.c
In file included from /usr/include/python2.7/numpy/ndarraytypes.h:1777:0,
from /usr/include/python2.7/numpy/ndarrayobject.h:18,
from /usr/include/python2.7/numpy/arrayobject.h:4,
from src/utils/cpu_nms.c:435:
/usr/include/python2.7/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#warning "Using deprecated NumPy API, disable it by "
^
rm -rf src/utils/cpu_nms.c

I hope to get your help. thanks.

In file included from ./include/caffe/common.hpp:19:0,
from src/caffe/layers/more_sparse_im2col_layer.cpp:3:
./include/caffe/util/device_alternate.hpp:15:36: error: no ‘void caffe::MoreSparseIm2colLayer::Forward_gpu(const std::vector<caffe::Blob>&, const std::vector<caffe::Blob>&)’ member function declared in class ‘caffe::MoreSparseIm2colLayer’
const vector<Blob>& top) { NO_GPU; }
^
src/caffe/layers/more_sparse_im2col_layer.cpp:89:1: note: in expansion of macro ‘STUB_GPU’
STUB_GPU(MoreSparseIm2colLayer);
^
./include/caffe/util/device_alternate.hpp:19:39: error: no ‘void caffe::MoreSparseIm2colLayer::Backward_gpu(const std::vector<caffe::Blob>&, const std::vector&, const std::vector<caffe::Blob>&)’ member function declared in class ‘caffe::MoreSparseIm2colLayer’
const vector<Blob>& bottom) { NO_GPU; }
^
src/caffe/layers/more_sparse_im2col_layer.cpp:89:1: note: in expansion of macro ‘STUB_GPU’
STUB_GPU(MoreSparseIm2colLayer);
^
Makefile:518: recipe for target '.build_release/src/caffe/layers/more_sparse_im2col_layer.o' failed
make: *** [.build_release/src/caffe/layers/more_sparse_im2col_layer.o] Error 1

Any help would be appreciated.
I am compiling this on Ubuntu 16.04 without cudnn

Thanks so much

In my implementation suppose I input an image with dims as (350, 350, 3) and after the last fully convolutional layer which outputs (N, Spacial Height, Spacial Width, 2, K anchors). In my case the output turns out to be (N, 10, 10, 2, 10). My question how do you map this on the image because for each text proposal in the output it turns out to be of 35X35 pixels in the input image simply dividing (350/10). I dont understand how 16px width is worked out in input image because the coordinates are predicted for the each spacial location which is 35X35 and the horizontal location is fixed to 0 relative to the position of the text proposal. I have been implementing this reading your paper and it would be really great of you if you teach this little doubt of mine so I can compare the performance. Also a really nice research.

@tianzhi0549
I followed the instruction mentioned to build caffe. make -j4, make pycaffe and make test ; all work fine.However make runtest fails. I get the following error on make runtest:
[ 0%] Built target proto
[ 65%] Built target caffe
[ 65%] Built target gtest
[ 65%] Building CXX object src/caffe/test/CMakeFiles/test.testbin.dir/test_more_sparse_im2col_layer.cpp.o
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In member function ‘virtual void caffe::MoreSparseIm2colLayerTest_TestSetup_Test<gtest_TypeParam_>::TestBody()’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:47:3: error: ‘MoreSparseIm2colLayer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:47:30: error: expected primary-expression before ‘>’ token
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:47:49: error: there are no arguments to ‘layer’ that depend on a template parameter, so a declaration of ‘layer’ must be available [-fpermissive]
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:47:49: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:48:3: error: ‘layer’ was not declared in this scope
layer.SetUp(this->blob_bottom_vec_, this->blob_top_vec_);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In member function ‘virtual void caffe::MoreSparseIm2colLayerTest_TestGradient_Test<gtest_TypeParam_>::TestBody()’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:62:3: error: ‘MoreSparseIm2colLayer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:62:30: error: expected primary-expression before ‘>’ token
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:62:49: error: there are no arguments to ‘layer’ that depend on a template parameter, so a declaration of ‘layer’ must be available [-fpermissive]
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:64:36: error: ‘layer’ was not declared in this scope
checker.CheckGradientExhaustive(&layer, this->blob_bottom_vec_,
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In member function ‘virtual void caffe::MoreSparseIm2colLayerTest_TestRectGradient_Test<gtest_TypeParam_>::TestBody()’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:77:3: error: ‘MoreSparseIm2colLayer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:77:30: error: expected primary-expression before ‘>’ token
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:77:49: error: there are no arguments to ‘layer’ that depend on a template parameter, so a declaration of ‘layer’ must be available [-fpermissive]
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:79:36: error: ‘layer’ was not declared in this scope
checker.CheckGradientExhaustive(&layer, this->blob_bottom_vec_,
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestRectGradient_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::GPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:77:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestGradient_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::GPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:62:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestSetup_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::GPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:47:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestRectGradient_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::GPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:77:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestGradient_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::GPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:62:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestSetup_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::GPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:47:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestRectGradient_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::CPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:77:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestGradient_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::CPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:62:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestSetup_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::CPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:47:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestRectGradient_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::CPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:77:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestGradient_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::CPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:62:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp: In instantiation of ‘void caffe::MoreSparseIm2colLayerTest_TestSetup_Test<gtest_TypeParam_>::TestBody() [with gtest_TypeParam_ = caffe::CPUDevice]’:
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:83:1: required from here
/home/env114/Desktop/Imges/deeplearning/CTPN/caffe/src/caffe/test/test_more_sparse_im2col_layer.cpp:47:49: error: ‘layer’ was not declared in this scope
MoreSparseIm2colLayer layer(layer_param);
^
make[3]: *** [src/caffe/test/CMakeFiles/test.testbin.dir/test_more_sparse_im2col_layer.cpp.o] Error 1
make[2]: *** [src/caffe/test/CMakeFiles/test.testbin.dir/all] Error 2
make[1]: *** [src/caffe/test/CMakeFiles/runtest.dir/rule] Error 2
make: *** [runtest] Error 2

Please help me with the above issue. Thanks in advance.

Operating system:Ubuntu 14.04
Compiler:gcc version 4.8.4
CUDA version: 7.0
CUDNN version :3.0.8
Python : 2.4.7

Hello,I tried to implement CTPN training code on the framework of py-faster-rcnn(VGG16 E2E),but the model size is lager than yours(540M,your model is about 78M).I doubt whether I did the right thing which I discard RCNN compared to Faster-rcnn （VGG16 end to end）for training?
Looking forward to your reply!Thank you!

hi, @tianzhi0549 , Thanks for your perfect job. I have a few questions to ask.
1、This demo can be transplanted the Windows version and compile?
2、If you can, how much work there is?
3、If you can directly modify it under the official caffe, caffe version is which?
Hope to get your response. Thanks.

Hi Tianzhi,

Thanks for your perfect job. I have a few questions to ask.

Do you support to compute the offset mentioned in section 3.3 "Side-refinement" of your paper in your released model?

Do you do regression only to the anchors just close to the left/right side of the GT boxes instead of all the anchors?

By the way your web demo has better output than your released model. Do you have any improvements which can be released in public?

Hope to get your response. Thanks.

Hi, tianzhi,
I tried to implement CTPN training code on the framework of py-faster-rcnn (by RBG), but the results were different from yours (of course worse) .

1. Loss function. Did you revise the loss function (eg: SmoothL1Loss) of training code ?
2. vertical proposals heights in a textline. A complete textline constitue of several vertical anchors in sequnce, and the heights of them vary slightly in your implementation, however the heights in my implementation vary enormously. Sometime, the proposal fit tightly to the boundry of single character, if the heights of characters in a textline differs greatly, heights of proposals differs too. So I want to ask the question : how did you make the heights and y coordinate of proposal sequence uniformed ? Via lstm? Or, other kind of change in python layer? If the answer is lstm , does that mean lstm not working in my implementation ? FTPN (CTPN with No RNN) seems have the same problem.

First of all thanks for sharing this project.
I have tried with the project on the images of documents, it detect every text except for those word having letter 2 or less than 2.

Is there any option in the cfg.py which i could change in order to detect those too?

I could successfully build, but I am not able to run the demo however, I have used cuda 7.5 and cudnn version 3 as mentioned, but I am getting this error while running the demo .
`
Image: img_1.jpg

F1218 15:10:22.093291 6307 cudnn_conv_layer.cu:85] Check failed: status == CUDNN_STATUS_SUCCESS (9 vs. 0) CUDNN_STATUS_NOT_SUPPORTED
*** Check failure stack trace: ***
Aborted (core dumped)
`
I tried resolving it from the issue here but I am not able to follow it much. Thanks a lot in advance .

Is there any way to use this without a GPU?

Hey Can't reach your website. http://textdet.com/
502 Bad Gateway
Please check

Hi,
I have seen the demo of the repository, and I love it. I want to use it in my project. I would like to know whether it is possible at all to predict the text from the boxe? It will be awsome if the project detects the text and predict what it detects.
Kindly, share your view.

Hi:
I have a question about side-refinement.Traing time,you regress the anchor's relative offset to bounding.Testing time,what do you do ? We can get the text proposal according the defalut anchor,
then ,add the offset (compute with current anchor) to text proposal ,yes or no? Maybe ,the side-refinement only be used on the training time,to get better text proposal.On the test time,nothing should be done.

hi,
i am so appreciated with your great work. i modify the Makefile.config file like this:

Makefile.config start

USE_CUDNN := 1
CUDA_DIR := /usr/local/cuda
CUDA_ARCH := -gencode arch=compute_20,code=sm_20
-gencode arch=compute_20,code=sm_21
-gencode arch=compute_30,code=sm_30
-gencode arch=compute_35,code=sm_35
-gencode arch=compute_50,code=sm_50
-gencode arch=compute_52,code=sm_52
-gencode arch=compute_60,code=sm_60
-gencode arch=compute_61,code=sm_61
-gencode arch=compute_61,code=compute_61
BLAS := open
ANACONDA_HOME := /home/tengbaoqiang/tbq_virtual_evn/anaconda2
PYTHON_INCLUDE := $(ANACONDA_HOME)/include
$(ANACONDA_HOME)/include/python2.7
$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include
PYTHON_LIB := $(ANACONDA_HOME)/lib
WITH_PYTHON_LAYER := 1

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

BUILD_DIR := build
DISTRIBUTE_DIR := distribute
TEST_GPUID := 0
Q ?= @

Makefile.config end

and make -j && make pycaffe it comes many errors such:

error start

collect2: error: ld returned 1 exit status
make: *** [.build_release/examples/siamese/convert_mnist_siamese_data.bin] Error 1

error end

but i can make successfully in my other caffe-master folder. then i replace your caffe folder with my caffe-master, and rename it with caffe. but when i run the code 'python tools/demo.py', it comes the

error end

WARNING: Logging before InitGoogleLogging() is written to STDERR
E0315 10:53:09.159488 16021 common.cpp:114] Cannot create Cublas handle. Cublas won't be available.
E0315 10:53:09.597815 16021 common.cpp:121] Cannot create Curand generator. Curand won't be available.
W0315 10:53:09.599166 16021 _caffe.cpp:135] DEPRECATION WARNING - deprecated use of Python interface
W0315 10:53:09.599189 16021 _caffe.cpp:136] Use this instead (with the named "weights" parameter):
W0315 10:53:09.599197 16021 _caffe.cpp:138] Net('models/deploy.prototxt', 1, weights='models/ctpn_trained_model.caffemodel')
[libprotobuf ERROR google/protobuf/text_format.cc:298] Error parsing text-format caffe.NetParameter: 387:19: Message type "caffe.LayerParameter" has no field named "transpose_param".
F0315 10:53:09.603824 16021 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: models/deploy.prototxt
*** Check failure stack trace: ***
Aborted (core dumped)

error end

i am sure i can import caffe successfully in python console. i am so confused which leads to the error. i hope to get your help. thanks

Dear tianzhi,
I tried to compile the caffe with the cudnn5, cuda7.5, but i get the errors below:
`**/data/zhengjunjun/deeplearning/CTPN/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/data/zhengjunjun/deeplearning/CTPN/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

/data/zhengjunjun/deeplearning/CTPN/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/data/zhengjunjun/deeplearning/CTPN/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

/data/zhengjunjun/deeplearning/CTPN/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/data/zhengjunjun/deeplearning/CTPN/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

/data/zhengjunjun/deeplearning/CTPN/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/data/zhengjunjun/deeplearning/CTPN/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

/data/zhengjunjun/deeplearning/CTPN/caffe/include/caffe/util/cudnn.hpp(123): error: argument of type "int" is incompatible with parameter of type "cudnnNanPropagation_t"

/data/zhengjunjun/deeplearning/CTPN/caffe/include/caffe/util/cudnn.hpp(123): error: too few arguments in function call

2 errors detected in the compilation of "/tmp/tmpxft_0000c829_00000000-5_reduction_layer.cpp4.ii".
CMake Error at cuda_compile_generated_reduction_layer.cu.o.cmake:264 (message):
Error generating file
/data/zhengjunjun/deeplearning/CTPN/caffe/build/src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_reduction_layer.cu.o
make[2]: *** [src/caffe/CMakeFiles/cuda_compile.dir/layers/./cuda_compile_generated_reduction_layer.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
2 errors detected in the compilation of "/tmp/tmpxft_0000c811_00000000-5_embed_layer.cpp4.ii".**`

Do you have some ideas for these errors？
thank you!

@tianzhi0549 hi can you tell me what the difference between lstm layer and the one from blvc/caffe? thanks.

Hi, when I want to try the online demo textdet.com, I get a 502 bad gateway. Can you help to fix the server.

hi,
I am so appreciated with your great job. I have read your code and paper carefully.
I find that your paper have three outputs :2k vertical coordinates, 2k scores and k side-refinement
, but the output of the code have only 2 outputs: boudingboxes((19000,4)) and scores((19000,)) .

1)boundingboxes((19000,4)) == 2k vertical coordinates
scores((19000,)) == 2k scores. right? then where is the third output?

or 2)boundingboxes((19000,4)) == 2k vertical coordinates + side-refinement
scores((19000,)) == 2k scores. if this is right , scores should be k dimension, side-refinement is horizontal coordinates with 2k dimension.

Looking forward your reply, thanks

Is it possible to list the major changes to this version of caffe you use? I want know the potential issues/conflict while merging it with the newer version of caffe.

My GPU has to run with CUDA 8.0, which is not compatible this version of caffe.

Hi @tianzhi0549 , thanks for your test code and model! I want to train CTPN on my own dataset, but when I prepare annotation xml files, I don't know what each object should be. Is it just like a text proposal(fixed width:16px, precompute its coordinate from a whole word bounding box) or a whole word bounding box(like ICDAR 2013 annotations)? Thanks.

Hello, I want to use the text area in the project to detect, need your deploy.prototxt and ctpn_trained_model.caffemodel, how can i contact you?

Can you provide some examples for training or fine-tuning ? and some details of inputs?

thanks.

typeerror: no to_python(by-value) converter found for c++ type: class caffe::layerparameter.
I am getting this error while I transplant it to the windows。

Is there any script available to train CTPN on custom dataset?

If you will do that, can you give some ideas ( in detail )

I have see https://github.com/bgshih/crnn, but the result is unsatisfied.

When I install caffe,it shows like :
ld: framework not found vecLib
clang: error: linker command failed with exit code 1 (use -v to see invocation)

then I do like this :BVLC/caffe#4637
but it shows like :Undefined symbols for architecture x86_64:
"_H5LTfind_dataset", referenced from:

my os is Macos 10.12.5 Sierra,can anybody help me to solve it?

Hello,I want to train custom data, can I just implement it by using this caffe version? Thank you!

I am trying to run the demo.py file and getting the following error.
Is there something wrong that I am doing. I have the BVLC version of caffe also installed in a different location.

This is the error I am getting while trying to run demo.py

WARNING: Logging before InitGoogleLogging() is written to STDERR
W0104 15:27:11.308679 30453 _caffe.cpp:154] DEPRECATION WARNING - deprecated use of Python interface
W0104 15:27:11.308717 30453 _caffe.cpp:155] Use this instead (with the named "weights" parameter):
W0104 15:27:11.308722 30453 _caffe.cpp:157] Net('models/deploy.prototxt', 1, weights='models/ctpn_trained_model.caffemodel')
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 387:19: Message type "caffe.LayerParameter" has no field named "transpose_param".
F0104 15:27:11.310125 30453 upgrade_proto.cpp:79] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: models/deploy.prototxt
*** Check failure stack trace: ***
Aborted (core dumped)

hi @tianzhi0549 , i have compiled the caffe/pycaffe successfully on windows, and also the cython nms except change the line 25 from
cdef np.ndarray[np.int_t, ndim=1] order = scores.argsort()[::-1]
to
cdef np.ndarray[np.intp_t, ndim=1] order = scores.argsort()[::-1]
to avoid the error"ValueError: Buffer dtype mismatch, expected 'int_t' but got 'long'`"

after the modification, the program run successfully, but the result is not correct, "Number of the detected text lines: 0", no text detected!

could you please give me some hints on it, thank you.

my pc/environment:
os: windows 7 64bit
ide: visual studio 2013
caffe: based on your caffe, but modified on some files to avoid compiling errors on windows, i am sure that there is no problem with modification, as i only modified the io.cpp signalhandler.cpp, etc, which is not related with the layers, and the modification is similar when i ported the ssd/caffe into windows, there is no problem.
python: anaconda2/python2.7.13 64bit.

Hi, there
I saw the result of Challenge "Focused Scene Text", CTPN is great .
now I wanna do a full evaluation of CTPN on my own dataset,
is there some related script for training that could be shared?
thx!

Hi! @tianzhi0549. Could you share more information about how to organize the label for the side-refinement? Thank you in advance!

Dear Tianzhi:
I tried you demo, and obtained an exactly same reuslt on ICDAR 2013 Challenge 2 as you submited . It works perpectly ! BTW, OpenCV 3, CUDA 7.5 is compatible for this project.
Now I am trying to test the performance on ICDAR 2015 Challenge 4, which is constitute of many tilt and perspective texts, but the boudingbox returned by your method is a rectangle of whole textline, instead of separated words represented by 8 coordinates.
Did you submited the rectangle (4 coordinates) of whole textline in Challenge 4 as you did in Challenge 2 ? If not , what kind of adjustment is applied ? The publication did not mentioned any stuff about tilt and perpsective texts , so I got a little confused.

Best Regards

Hello, I have built this demo on my computer. But I found that the result on demo and another on website are some different. Could you tell me whether the algorithms between them may be a little different? Thanks a lot!

WARNING: Logging before InitGoogleLogging() is written to STDERR
W0626 13:00:38.930151 31448 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface
W0626 13:00:38.930178 31448 _caffe.cpp:123] Use this instead (with the named "weights" parameter):
W0626 13:00:38.930186 31448 _caffe.cpp:125] Net('/home/jsj/CTPN/models/deploy.prototxt', 1, weights='/home/jsj/CTPN/models/ctpn_trained_model.caffemodel')
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 387:19: Message type "caffe.LayerParameter" has no field named "transpose_param".
F0626 13:00:38.931586 31448 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /home/jsj/CTPN/models/deploy.prototxt
*** Check failure stack trace: ***
已放弃 (核心已转储) .
I need your help .Thanks in advence . Can you tell me which caffe that you used?,the caffe that i used is from "<module 'caffe' from '/home/jsj/caffe/python/caffe/init.pyc'>
@tianzhi0549

the follow is the demo.py that I adjusted :

import sys
caffe_root = '/home/jsj/CTPN/caffe/python/caffe' # this file is expected to be in {caffe_root}/examples
sys.path.insert(0,caffe_root)
from cfg import Config as cfg
from other import draw_boxes, resize_im, CaffeModel
import cv2, os, caffe
from detectors import TextProposalDetector, TextDetector
import os.path as osp
from timer import Timer
print caffe
DEMO_IMAGE_DIR="/home/jsj/CTPN/demo_images/"
NET_DEF_FILE="/home/jsj/CTPN/models/deploy.prototxt"
MODEL_FILE="/home/jsj/CTPN/models/ctpn_trained_model.caffemodel"

#if len(sys.argv)>1 and sys.argv[1]=="--no-gpu":

caffe.set_mode_cpu()

#else:

caffe.set_mode_cpu()

caffe.set_device(cfg.TEST_CPU_ID)

caffe.set_mode_cpu()

initialize the detectors

text_proposals_detector=TextProposalDetector(CaffeModel(NET_DEF_FILE, MODEL_FILE))
text_detector=TextDetector(text_proposals_detector)

demo_imnames=os.listdir(DEMO_IMAGE_DIR)
timer=Timer()

for im_name in demo_imnames:
print "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
print "Image: %s"%im_name

im_file=osp.join(DEMO_IMAGE_DIR, im_name)
im=cv2.imread(im_file)

timer.tic()

im, f=resize_im(im, cfg.SCALE, cfg.MAX_SCALE)
text_lines=text_detector.detect(im)

print "Number of the detected text lines: %s"%len(text_lines)
print "Time: %f"%timer.toc()

im_with_text_lines=draw_boxes(im, text_lines, caption=im_name, wait=False)

print "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
print "Thank you for trying our demo. Press any key to exit..."
cv2.waitKey(0)

Hi there,
recently I‘m going to do some tests on text detection，in natural image
I googled a lot and found this great work !
I found that most detections are failure on vertical text after tried some images .
how to train this network and is your dataset available for downloading and training ?
thanks !
Best regards,

Undefined symbols for architecture x86_64:
"_PyBuffer_Release", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyCFunction_NewEx", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PyCFunction_Type", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyCode_New", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___Pyx_AddTraceback in cpu_nms-2726be.o
"_PyDict_GetItem", referenced from:
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyDict_New", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyDict_Next", referenced from:
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
"_PyDict_SetItem", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyDict_Size", referenced from:
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
"_PyErr_Format", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___Pyx_ImportType in cpu_nms-2726be.o
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___Pyx_BufFmt_CheckString in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
...
"_PyErr_NormalizeException", referenced from:
___Pyx_Raise in cpu_nms-2726be.o
"_PyErr_Occurred", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
___Pyx_PyInt_As_int in cpu_nms-2726be.o
"_PyErr_SetString", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___Pyx_BufFmt_CheckString in cpu_nms-2726be.o
___Pyx_Raise in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
...
"_PyErr_WarnEx", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___Pyx_ImportType in cpu_nms-2726be.o
"_PyExc_BaseException", referenced from:
___Pyx_Raise in cpu_nms-2726be.o
"_PyExc_ImportError", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PyExc_IndexError", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyExc_NameError", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyExc_OverflowError", referenced from:
___Pyx_PyInt_As_int in cpu_nms-2726be.o
"_PyExc_SystemError", referenced from:
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyExc_TypeError", referenced from:
___Pyx_ImportType in cpu_nms-2726be.o
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___Pyx_Raise in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
___Pyx_PyInt_As_int in cpu_nms-2726be.o
...
"_PyExc_ValueError", referenced from:
___Pyx_ImportType in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___Pyx_BufFmt_CheckString in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
___Pyx_BufFmt_RaiseExpected in cpu_nms-2726be.o
___Pyx_BufFmt_ProcessTypeChunk in cpu_nms-2726be.o
"_PyExc_ZeroDivisionError", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyFloat_FromDouble", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyFloat_Type", referenced from:
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyFrame_New", referenced from:
___Pyx_AddTraceback in cpu_nms-2726be.o
"_PyImport_AddModule", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PyImport_Import", referenced from:
___Pyx_ImportType in cpu_nms-2726be.o
"_PyInt_FromLong", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyInt_Type", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyList_Append", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyList_New", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyLong_AsLong", referenced from:
___Pyx_PyInt_As_int in cpu_nms-2726be.o
"_PyLong_Type", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyMem_Malloc", referenced from:
___Pyx_AddTraceback in cpu_nms-2726be.o
"_PyMem_Realloc", referenced from:
___Pyx_AddTraceback in cpu_nms-2726be.o
"_PyMethod_Type", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyModule_GetDict", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PyNumber_Add", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyNumber_Int", referenced from:
___Pyx_PyInt_As_int in cpu_nms-2726be.o
"_PyNumber_Long", referenced from:
___Pyx_PyInt_As_int in cpu_nms-2726be.o
"_PyNumber_Multiply", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyNumber_Subtract", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyOS_snprintf", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___Pyx_ImportType in cpu_nms-2726be.o
"_PyObject_Call", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyObject_CallFunctionObjArgs", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PyObject_GetAttr", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___Pyx_ImportType in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
"_PyObject_GetBuffer", referenced from:
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
"_PyObject_GetItem", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyObject_IsTrue", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyObject_RichCompare", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyObject_SetAttrString", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PySlice_New", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PyString_AsString", referenced from:
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
"_PyString_FromFormat", referenced from:
___Pyx_AddTraceback in cpu_nms-2726be.o
"_PyString_FromString", referenced from:
___Pyx_ImportType in cpu_nms-2726be.o
___Pyx_AddTraceback in cpu_nms-2726be.o
"_PyString_FromStringAndSize", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PyString_InternFromString", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PyString_Type", referenced from:
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
"_PyTraceBack_Here", referenced from:
___Pyx_AddTraceback in cpu_nms-2726be.o
"_PyTuple_New", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyTuple_Pack", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PyTuple_Type", referenced from:
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyType_IsSubtype", referenced from:
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___Pyx_Raise in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyUnicodeUCS2_Compare", referenced from:
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
"_PyUnicodeUCS2_DecodeUTF8", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_PyUnicodeUCS2_Format", referenced from:
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"_PyUnicodeUCS2_FromStringAndSize", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_Py_GetVersion", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"_Py_InitModule4_64", referenced from:
_initcpu_nms in cpu_nms-2726be.o
"__PyString_Eq", referenced from:
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
"__PyThreadState_Current", referenced from:
___Pyx_AddTraceback in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___Pyx_Raise in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"__Py_CheckRecursionLimit", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"__Py_CheckRecursiveCall", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"__Py_NoneStruct", referenced from:
_initcpu_nms in cpu_nms-2726be.o
___pyx_pw_5utils_7cpu_nms_1cpu_nms in cpu_nms-2726be.o
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___Pyx_GetBufferAndValidate in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"__Py_TrueStruct", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
"__Py_ZeroStruct", referenced from:
___pyx_pf_5utils_7cpu_nms_cpu_nms in cpu_nms-2726be.o
___pyx_f_5numpy__util_dtypestring in cpu_nms-2726be.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [all] Error 1

at first thank you tianzhi0549 for your great work
I try to run this code but i cant ,and it seems that there are a lot of people who could run it so i need help please
if any one can write the steps in clearer way or make video with the steps i will be very grateful for him
and for example i am very confusing should i install caffe or the caffe in the CTPN folder is enough and another a lot of questions so if any one could help me with the steps i will be really grateful

@tianzhi0549 I figured out that the performance of CTPN on slanted/multi-oriented text is not very good. I want to tackle this by introducing the Spatial Transformer Networks just after the Convolution Layers and train the entire network again. Three questions:

1. Do we have to give image and it's bounding boxes ground truth values only to CTPN for training it or we need to give something more to the network?
2. Is there any good implementation of Spatial Transformer Network that will work with this version of caffe?
3. Could you please explain more about how can I compute the 3 losses mentioned in the paper?

Hi,

I'm currently successful on demo with --no-gpu. The result was so great. But it was slow, so I deployed it in AWS p2.xlarge with 1 GPU. My problem is that if I use CUDA 7 + CuDNN 3.0 as you used, I can't connect it with other works such as text recognition, which I often build in tensorflow because tensorlow can't run in CuDNN 3.0.

So do you have any version of this code in new CUDA+CuDNN version? Or do you have any version of this in other platforms: tensorflow, keras,...? If not, I will port it into other platforms, so are you ready to help me if I get problems?

Anyway, thanks for your work, excellent!

Hi @tianzhi0549 ,

First of all, excellent work! You have mentioned in your paper that side-refinement improves the performance on SWT and multi-lingual datasets by 2%. Would you be including this step in your code anytime soon or is it to remain excluded?

Thanks,
Deshana

Can you explain how to setup the website