Hi,
I tried to reproduce the reported errors on the chess dataset using the pre-trained models.
However, the performance does not match with the reported performance in your paper.

First, I created an lmdb dataset using the script caffe-posenet/posenet/scripts/create_posenet_lmdb_dataset.py. For the training set I used the dataset chess_train.txt and for the test set I used the dataset chess_val.txt from your 7Scenes_PoseNet_TrainVal folder. I downloaded the datasets from https://www.microsoft.com/en-us/research/project/rgb-d-dataset-7-scenes/.

Then, I created the dataset mean file using caffe-posenet/build/tools/compute_image_mean.

For testing I used your script caffe-posenet/posenet/scripts/test_posenet.py with your model (train_chess.prototxt, I changed the source and mean file to the appropriate directories) and your pre-trained weights (weights_chess.caffemodel) and set the iter to 2000 (as this is the size of the test set).

For testing on the chess dataset, I got:
Median error 0.435215592384 m and 10.7102198545 degrees.

I would appreciate if you could give me some advises about possible mistakes or anything that I might missed.

Best,
Hanna

Hi,

It seems that you use different split with Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images for Heads.

As described in Heads' TrainSplit.txt and TestSplit.txt,

• TrainSet: sequence2
• TestSet: sequence1

However, it seems that you use a contrasting split with it according to the given model and heads_train/val.txt.

Best regards,

Wu JIan

Hi,

What modifications do I need to use the given layers in order to handle gray scale image dataset?

Thanks in advance.

Regards

@alexgkendall

in directory "/home/ahmad/Desktop/caffe-posenet/posenet/scripts"
I type "python create_posenet_lmdb_dataset.py"

with Change lines 1, 11 & 12 to the appropriate directories

i got this error.

Regards

_Traceback (most recent call last):
File "create_posenet_lmdb_dataset.py", line 7, in
import caffe
File "/home/ahmad/Desktop/caffe-posenet/python/caffe/init.py", line 1, in
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver
File "/home/ahmad/Desktop/caffe-posenet/python/caffe/pycaffe.py", line 13, in
from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver,
ImportError: No module named caffe

Sorry for bothering.
I want to use PoseNet in my robot indoors , but when I collect 1000 images， I don't know how to set the arguments in the solver_posenet.prototxt.

My prototxt like this :

net: "./posenet/models/train.prototxt"
test_initialization: false
test_iter: 250    
test_interval: 20
base_lr: 0.001
lr_policy: "step"
gamma: 0.1
stepsize: 250
display: 20
max_iter: 30000
solver_type: ADAGRAD
weight_decay: 0.005
snapshot: 2000
snapshot_prefix: "./posenet/models/training"
solver_mode: GPU

but when I begin to train ,
I1212 12:48:09.793629 8576 solver.cpp:237] Iteration 0, loss = 584.759
the loss begin at 584.759 and always stay at 125. Does it mean PoseNet train successfully ?
Would you like to share the solver_posenet.prototxt to me?
I would appreciate if you give me some advises about possible mistakes that I might make.

Hi. Could you please share the original train_val.prototxt for GoogleNet that was modified to create PoseNet?

The difference in layer labels (between Caffe's GoogleNet prototxt and PoseNet prototxt) make it very difficult to compare GoogleNet with PoseNet to see what additions/modifications were made.

For example, in PoseNet prototxt there's a layer "icp1_reduction1" and the corresponding layer seems to be "inception_3a/1x1" in GoogleNet prototxt that's provided with Caffe.

Respected Sir,
According to your instructions I have successfully installed Caffe from your repository.
When I Run this command "Python create_posenet_lmdb_dataset.py" It shiows like this.

After that When i use this command "python test_posenet.py --model train_kingscollege.prototxt --weights weights_kingscollege.caffemodel --iter 100" I got error of some mean file missings as shown.

I0128 10:10:24.872309 20176 layer_factory.hpp:77] Creating layer data
I0128 10:10:24.872444 20176 net.cpp:106] Creating Layer data
I0128 10:10:24.872455 20176 net.cpp:411] data -> data
I0128 10:10:24.872473 20176 net.cpp:411] data -> label
I0128 10:10:24.872486 20176 data_transformer.cpp:25] Loading mean file from: ../imagemean.binaryproto
F0128 10:10:24.872498 20184 db_lmdb.hpp:14] Check failed: mdb_status == 0 (2 vs. 0) No such file or directory
*** Check failure stack trace: ***
F0128 10:10:24.872499 20176 io.cpp:54] Check failed: fd != -1 (-1 vs. -1) File not found: ../imagemean.binaryproto
*** Check failure stack trace: ***
Aborted (core dumped)

Kindly help me. I ll be thankful to you.

I recently came across posenet and tried to experiment with it. I modified the original architecture of posenet and now my input layer size is 1x100x224x224x3 where 1 is the batch size and 100 are the frames. Actually I have implemented lstm version of posenet. Now I have tried to run training on my own data, first on gtx750 and then on quadro k1200 4 GB gpu with tensorflow backend for keras(i am using posenet,written in keras) and also tried theano backend but everytime I get "out of memory" error. I want to ask what hardware specs your computer had on which you ran your training?
About Data: I have around 6000 png images for training

I have make a build file in caffe-posenet.
Then cd build and cmake ..
but error occurs.
Caffe has been successfully built before with no cuda and I set caffe_option in CMakeLists of caffe-posenet project "CPU_ONLY" like "caffe_option(CPU_ONLY "Build Caffe without CUDA support" OFF) # TODO: rename to USE_CUDA".

The question is why I still get this cuda error message. Please if anyone could help.

-- Install path : /home/godu/Documents/caffe-posenet/build/install

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_cublas_device_LIBRARY (ADVANCED)
linked by target "caffe" in directory /home/godu/Documents/caffe-posenet/src/caffe

-- Configuring incomplete, errors occurred!
See also "/home/godu/Documents/caffe-posenet/build/CMakeFiles/CMakeOutput.log".
See also "/home/godu/Documents/caffe-posenet/build/CMakeFiles/CMakeError.log".

the dataset I used is kingscollege, but these errors below occured.
I0720 17:26:07.043119 1349 net.cpp:84] Creating Layer slice_label
I0720 17:26:07.043123 1349 net.cpp:406] slice_label <- label
I0720 17:26:07.043129 1349 net.cpp:380] slice_label -> label_xyz
I0720 17:26:07.043138 1349 net.cpp:380] slice_label -> label_wpqr
F0720 17:26:07.043145 1349 slice_layer.cpp:33] Check failed: slice_axis_ < num_axes (1 vs. 1) slice_dim out of range.

Hello,
When I import caffe, it indicates:
File "/home/breeze/caffe-posenet/python/caffe/pycaffe.py", line 13, in
from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver,
ModuleNotFoundError: No module named 'caffe._caffe'

how to solve this?

Hello,

I just want to predict an image and get its localization results rather than 300 images of prediction error in the two test python file, what should I do?

Anyone has a script?I would appreciate your help.Thank you.

Hi,this is Ying Nie,
I have some problems about the caffe-posenet-master you offer.What's the different between the standard version of Caffe and the posenet version of Caffe you offer?And why would it caused to load the label incorrectly?Could you please to tell me what changes have you made to the standard version of Caffe?I'm looking forward for your reply.
Best regards,
Ying Nie

Hi @alexgkendall,

Firstly, great work!

I am trying to reproduce your results (from scratch), I've just had a quick look at the prototxt files and the names of the layers are different from the bvlc_googlenet prototxt. It is my understanding that with caffe you can transfer weights to another model if the layers have the same name.

How do you "finetune" a bvlc_googlenet model using your prototxt files?

Thanks

I can't seem to find the 'meanfile' file that is required in posenet/scripts/test_bayesian_posenet.py. Is that something left for us to compute or you would be so kind to share yours? :)

Is this 1*7 vector has the same meaning as what we can get from Colmap? i.e, X Y Z = TX TY TZ, W P Q R = QW QX QY QZ?

Could you tell me the camera intrinsic of Cambridge landmark?

I have caffe installed on my Macbook air already and now I tried to install caffe-posenet. However, I get this error:
CXX/LD -o .build_release/tools/caffe.bin clang: warning: argument unused during compilation: '-pthread' Undefined symbols for architecture x86_64: "caffe::Net<float>::Net(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, caffe::Phase, caffe::Net<float> const*)", referenced from: test() in caffe.o time() in caffe.o "caffe::Layer<float>::Lock()", referenced from: caffe::Layer<float>::Forward(std::__1::vector<caffe::Blob<float>*, std::__1::allocator<caffe::Blob<float>*> > const&, std::__1::vector<caffe::Blob<float>*, std::__1::allocator<caffe::Blob<float>*> > const&) in caffe.o "caffe::Layer<float>::Unlock()", referenced from: caffe::Layer<float>::Forward(std::__1::vector<caffe::Blob<float>*, std::__1::allocator<caffe::Blob<float>*> > const&, std::__1::vector<caffe::Blob<float>*, std::__1::allocator<caffe::Blob<float>*> > const&) in caffe.o "caffe::P2PSync<float>::run(std::__1::vector<int, std::__1::allocator<int> > const&)", referenced from: train() in caffe.o "caffe::P2PSync<float>::P2PSync(boost::shared_ptr<caffe::Solver<float> >, caffe::P2PSync<float>*, caffe::SolverParameter const&)", referenced from: train() in caffe.o "caffe::P2PSync<float>::~P2PSync()", referenced from: train() in caffe.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [.build_release/tools/caffe.bin] Error 1
Do you have any suggestion how I can solve it? Or do you know any way to get a trained posenet model weight in .npy format so that I don't have to execute the model ? Thanks!

I could not find a way to adapt CNNSaliencyMap (Deep Inside Convolutional Networks) to give the outputs on PoseNet. The main problem is the multiple outputs, and there is not much information available on how to perform a backward pass using multiple outputs. Any suggestion will be highly appreciated.

Hi, this is Jaehyun Lim

First of all, thank you for sharing your work :)

I tried to run the posenets with the uploaded pretrained models by using the provided caffe version of yours.

However, the performance (median error) of the posenets with following settings and the pretrained models does not match with the reported performance in your paper.

• non-bayesian
• single center crop

I got

• 7.98 m and 4.36 deg (median error) for KingsCollege
• 3.45 m and 6.61 deg (median error) for ShopFacade

I further tried followings;

• converted images to lmdb with different interpolation settings, both bilinear and bicubic, since it was sometimes matter on some dataset; however, the results aren't changed much
  • 8.12 m and 4.30 deg (median error) for KingsCollege
  • 3.16 m and 6.28 deg (median error) for ShopFacade
• ran test with bayesian setting by using test_bayesian_posnet.py and adding DROPOUT layer at somewhere icp9 according to the provided test_bayesian_posenet.prototxt; however, it also provided almost the same results.
  • 8.12 m and 4.31 (median error) for KingsCollege

I would appreciate if you give me some advises about possible mistakes that I might make.

Sincerely,

Jaehyun

I try to use google map to select a position(nearly same with the 8 images in demo) and take a screen cut. But when I upload such image to the website, it can't return any result. If I take a screen cut on the demo images, and then submit it to the website, it works fine.

Why this demo can't response to the images from google map?

'draw_net.py' can not draw the 'train_posenet.prototxt' net . the result is a blank image.

Hi, I am Ruihao, I try to test the posenet, but encounter the problem as below.

I run "python ./posenet/scripts/test_posenet.py --model posenet/models/train_posenet.prototxt --weights posenet/models/weights_kingscollege.caffemodel --iter 346" as introduced in your readme.txt.

I0225 00:34:30.682186 13179 layer_factory.hpp:77] Creating layer data
I0225 00:34:30.682914 13179 net.cpp:106] Creating Layer data
I0225 00:34:30.682925 13179 net.cpp:411] data -> data
I0225 00:34:30.682956 13179 net.cpp:411] data -> label
I0225 00:34:30.682967 13179 data_transformer.cpp:25] Loading mean file from: posenet/kings_mean.binaryproto
I0225 00:34:30.683506 13182 db_lmdb.cpp:38] Opened lmdb posenet/posenet_kings_test_lmdb/
I0225 00:34:30.690235 13179 data_layer.cpp:41] output data size: 1,3,224,224
I0225 00:34:30.691720 13179 net.cpp:150] Setting up data
I0225 00:34:30.691746 13179 net.cpp:157] Top shape: 1 3 224 224 (150528)
I0225 00:34:30.691752 13179 net.cpp:157] Top shape: 1 (1)
I0225 00:34:30.691756 13179 net.cpp:165] Memory required for data: 602116
I0225 00:34:30.691762 13179 layer_factory.hpp:77] Creating layer slice_label
I0225 00:34:30.691776 13179 net.cpp:106] Creating Layer slice_label
I0225 00:34:30.691781 13179 net.cpp:454] slice_label <- label
I0225 00:34:30.691788 13179 net.cpp:411] slice_label -> label_xyz
I0225 00:34:30.691797 13179 net.cpp:411] slice_label -> label_wpqr
F0225 00:34:30.691809 13179 slice_layer.cpp:33] Check failed: slice_axis_ < num_axes (1 vs. 1) slice_dim out of range.
*** Check failure stack trace: ***
Aborted (core dumped)

Could you tell me what should I do with it ? Thank you very much!

Kind regards

I have trained the network with 800 epoch using gray images but the prediction is incorrect and the average error is 2,086 m in an interior area of ​​3.5 meters. My question is whether it works with gray images or should I make any changes?

Regards,

Goto this link for step by step installation of this code.

https://github.com/DRAhmadFaraz/PoseNet-Caffe.git

Hi, I have some issue concerning the python wrapper in caffe-windows. For some reason, I should transplant the code and run in windows. After some toss, the script "create_posenet_lmdb_dataset.py" and "test_posenet.py" can run pass. Since "KingsCollege_test_lmdb" and "KingsCollege_train_lmdb" are created and the output of "test_posenet.py" are as follow:
...
Iteration: 344 Error XYZ (m): 0.759703 Error Q (degrees): 2.12587290039
Iteration: 345 Error XYZ (m): 0.957241 Error Q (degrees): 4.29497876379
Iteration: 345 Error XYZ (m): 0.957241 Error Q (degrees): 4.29497876379
Median error 1.91579926014 m and 5.39524091714 degrees.
Success!
( the code is run on a 8GB GPU: NVIDIA Quadro K5200. )
Seems it work. However, I try to train PoseNet with KingsCollege dataset, the loss value become invalid after "Iteration 20":
I0920 17:24:59.279114 7664 net.cpp:283] Network initialization done.
I0920 17:24:59.280113 7664 solver.cpp:60] Solver scaffolding done.
I0920 17:25:02.902320 7664 solver.cpp:288] Solving GoogLeNet
I0920 17:25:02.902320 7664 solver.cpp:289] Learning Rate Policy: step
I0920 17:25:05.076445 7664 solver.cpp:237] Iteration 0, loss = 16.29
I0920 17:25:05.076445 7664 solver.cpp:253] Train net output #0: loss1/loss1_wpqr = 0.00497414 (* 150 = 0.746121 loss)
I0920 17:25:05.076445 7664 solver.cpp:253] Train net output #1: loss1/loss1_xyz = 17.2635 (* 0.3 = 5.17906 loss)
I0920 17:25:05.076445 7664 solver.cpp:253] Train net output #2: loss2/loss2_wpqr = 0.00427971 (* 150 = 0.641956 loss)
I0920 17:25:05.076445 7664 solver.cpp:253] Train net output #3: loss2/loss2_xyz = 14.6876 (* 0.3 = 4.40629 loss)
I0920 17:25:05.076445 7664 solver.cpp:253] Train net output #4: loss3/loss3_wpqr = 0.00250141 (* 500 = 1.25071 loss)
I0920 17:25:05.076445 7664 solver.cpp:253] Train net output #5: loss3/loss3_xyz = 4.06588 (* 1 = 4.06588 loss)
I0920 17:25:05.076445 7664 sgd_solver.cpp:106] Iteration 0, lr = 0.001
I0920 17:25:33.759085 7664 solver.cpp:237] Iteration 20, loss = 1.#QNAN
I0920 17:25:33.759085 7664 solver.cpp:253] Train net output #0: loss1/loss1_wpqr = 1.#QNAN (* 150 = 1.#QNAN loss)
I0920 17:25:33.759085 7664 solver.cpp:253] Train net output #1: loss1/loss1_xyz = 1.#QNAN (* 0.3 = 1.#QNAN loss)
I0920 17:25:33.759085 7664 solver.cpp:253] Train net output #2: loss2/loss2_wpqr = 1.#QNAN (* 150 = 1.#QNAN loss)
I0920 17:25:33.759085 7664 solver.cpp:253] Train net output #3: loss2/loss2_xyz = 1.#QNAN (* 0.3 = 1.#QNAN loss)
I0920 17:25:33.759085 7664 solver.cpp:253] Train net output #4: loss3/loss3_wpqr = 1.#QNAN (* 500 = 1.#QNAN loss)
I0920 17:25:33.759085 7664 solver.cpp:253] Train net output #5: loss3/loss3_xyz = 1.#QNAN (* 1 = 1.#QNAN loss)

Here is my code:
import caffe
caffe.set_device(0)
caffe.set_mode_gpu()
solver = caffe.SGDSolver(r"../models/solver_posenet.prototxt")
solver.net.copy_from(r"../models/weights_KingsCollege.caffemodel")
solver.solve()

Do I miss some important things when I move the code to windows or I should adjust some parameters before training? My code is based on the caffe version https://github.com/microsoft/caffe

Hi, Alex,
I try to use VisualSFM to reconstruct the model of the scene with the "KingsCollege" pictures. But the result is not good, especially the "focal length" in ".nvm" file, and it ranges from 800 to about 7000, not 1668 as your "reconstruction.nvm" shows.
Did you use special configuration with VisualSFM?
Thanks for your time.

error msg -
" data_transformer.cpp:64] Check failed: datum_width == data_mean_.width() (455 vs. 256) "

installed caffe successfully, while running "test_posenet.py" above mentioned error message displayed.

plz help me to solve this issue...

please find the attached screenshot of error msg

Hi, I met some problems.
In the source code of 'test_posenet.py'. When I run the 'python test_posenet.py --model your_model.prototxt --weights your_weights.caffemodel --iter size_of_dataset', it shows AttributeError: module 'caffe' has no attribute 'Net'.
The original code is:
net = caffe.Net(args.model,
args.weights,
caffe.TEST)

Could anyone can tell me how to solve this problem?
Thank you so much for your help.

The link to the model and training weights is broken. Can someone please update it?

Hi,

I'm a little new to Caffe. (So sorry for the newbie questions)

How can I train a new dataset using your posenet architecture?
I wish to benchmark PoseNet against several other localization systems, but this is a little confusing for me.
(I can easily assume that using weights from another dataset will lead to terrible results).

Can you provide a quick guide of how you created your weights and models?

Thanks for your time.

Hi, this is Jaehyun Lim

I have a problems in learning rate scheduling to reproduce the results on the paper using sgd (as in the paper) (see https://github.com/lim0606/caffe-posenet-googlenet).

As far as I understood from the paper, it seems like that the model is trained until 100 epoch (or about to 100 epoch), i.e about 16 iterations for King's College dataset; however, it was very short for converge in my experience with the learning rate scheduling.

Therefore, I referred the maximum iterations for the datasets based on the adagrad in this repo, i.e. 30000, and it (fortunately) worked with King's College dataset. I got the results good enough compared to the record on the paper (actually better, see https://github.com/lim0606/caffe-posenet-googlenet).

However, I couldn't reproduce the results on the other datasets.

I would appreciate if you let me know some advises that I might miss or misunderstood from your paper.

Best regards,

Jaehyun

Hi, this is Fang
I have a question about the median translation and rotation error stated in your paper and calculated at your code. In your test_posenet_protext, you set batch size to 40, but at your test_posenet.py, your compute median error use network output, but the compute procedure involves np.linalg.norm and np.sum , but never set axis property, so I'd like to know if you compute median error with batch size 32 or 1? as different batch size will generate different result. @alexgkendall

There seems to be an error in the Trinity Great Court training data file. Downlaoded here.
"dataset_train.txt" line 1470 says:

seq5/frame00297.png 3117382476.410000 -137672612.290000 -1240777531.790000 0.308042 -0.508641 0.232939 0.769503

The position components are not correct.

Hi,
the loss term in the paper is:

but i can't find where the q normalization happens in the code, can someone please point it out?

I want to ask one important factor from you, Hope you will help me out.

I have trained successfully this Pose-NET approach successfully on my own dataset,
now for further calculations, I want to extract the "Predicted Poses" for every each corresponding image.

Predicted poses includes [ 7 X 1 ] matrix having 3 values of Translation and 4 values of Rotation quaternion

I have checked the code posenet/scripts/test_posenet.py and exclude the predicted poses but for these poses are calculated for 1 iteration but I need predicted poses for every corresponding images.

Can you please help me to sort it out.?
I will be thankful to you.

this work is really interesting.
would you mind sharing the predicted results on the test sequences of 7Scenes and Cambridge datasets?
I am not that failiar with caffe.
thank you very much.

Hi @alexgkendall

I've been looking through the train_posenet.prototxt and I can't see where the normalization of the predicted quaternion (from cls3_fc_pose_wpqr) happens. I see that the output of cls3_fc_pose_wpqr and the label go into the Euclidean loss (with weight beta), but in the paper equation 2 shows that the output quaternion is made a unit vector.

Am I missing something here?

Thanks,
Aly

Hi! Do you happen to have a deploy.prototxt built for Posenet yet?

./create_posenet_mean.sh
Computing image mean...
F0107 14:47:22.450881 10521 db_lmdb.hpp:14] Check failed: mdb_status == 0 (2 vs. 0) No such file or directory
*** Check failure stack trace: ***
@ 0x7fb2cf50a5cd google::LogMessage::Fail()
@ 0x7fb2cf50c433 google::LogMessage::SendToLog()
@ 0x7fb2cf50a15b google::LogMessage::Flush()
@ 0x7fb2cf50ce1e google::LogMessageFatal::~LogMessageFatal()
@ 0x7fb2cf8e5d02 caffe::db::LMDB::Open()
@ 0x40214b main
@ 0x7fb2cdd43830 __libc_start_main
@ 0x402b89 _start
@ (nil) (unknown)
Aborted (core dumped)