partial Update: elasticsearch/solr vs vespa about vespa HOT 8 CLOSED

200M a day is about 2k per second. That should work fine for any kind of field even on a single node.

from vespa.

Just the fields that you want to update. That is why we call it a partial update. Numeric fields like byte, int, float etc is faster then string. Agree with @bratseth, 200M updates a day should be no match for even a single core machine.

from vespa.

Vespa supports partial updates of existing indexed documents, fastest is for fields defined with 'attribute' and of type numeric. See http://docs.vespa.ai/documentation/reference/document-json-update-format.html for update json syntax.

from vespa.

vespa’s partial update just reindex the updated fields ? es will reindex all the fields

from vespa.

@zhuxiang1981 solr has in-place-updates but with some caveats (non indexed etc) https://lucene.apache.org/solr/guide/6_6/updating-parts-of-documents.html#UpdatingPartsofDocuments-In-PlaceUpdates

from vespa.

Any further questions on this topic @zhuxiang1981 ? Thanks

from vespa.

We recently saw that 16k updates/sec were successful in one of our experiments with a cluster having 3 nodes, although all were integer updates. It's a good enough for now. We want to achieve 100k/sec updates which we would horizontally scale and achieve. Though we found that update throughput got very low (4k/sec) after we simultaneously ran benchmarking and hit the system with lots of queries. Any suggestions ?

from vespa.

from vespa.