Slow prediction problem about omikuji HOT 3 CLOSED

Are you using the Python API for prediction? In that case you can try calling densify_weights with lower density number (e.g. 0.01) to reduce prediction time at the cost of more memory usage.

from omikuji.

Are you using the Python API for prediction? In that case you can try calling densify_weights with lower density number (e.g. 0.01) to reduce prediction time at the cost of more memory usage.

No, I'm using the CLI app. I've tried to lower the density number to 0.0001(which used 50GB memory, BTW), but this problem still exists. I think the major problem is CPU is not working most of the time, which is really weird. I guess some procedure of prediction is too slow so that CPU has to keep waiting.

Does the prediction part work well on your machine? If yours works fine, I guess it's my own problem.

By the way, the data I used to train/predict is really dense(dimension reduced from 100K to 200). I do not know if it may cause the problem.
Thanks for your help!

PS:
This is the result of my prediction process.

587084 / 587084 [=======================================================] 100.00 % 143.64/s
2020-07-03 09:37:16,940 INFO [omikuji::model::eval] Done testing on 587084 examples; it took 4087.13s

I implemented this code on the origin version of WikiLSHTC-325K, the slow prediction still exists!

from omikuji.

Hmm I couldn't reproduce the low CPU usage. Not sure if the problem is specific to Google cloud machines. Can you try manually setting --n_threads and see if that helps? By default it should detect the number of CPUs automatically, but I'm wondering if there's something non-standard with Google cloud machines.

from omikuji.