Make intra tasks in IndexingChain.flush parallel execute. about lucene HOT 3 CLOSED

@jpountz Please take a look when you get a chance.

from lucene.

Lucene already has a model for indexing/flushing concurrency that consists of indexing documents from multiple threads. I guess that the idea that you are suggesting could make sense when the indexing rate is low, so it can't use all resources, and you'd like to use these unused resources to decrease flushing latency. But if your indexing rate is low, flushing shouldn't be slow unless you very rarely reopen? So I'm not sure if this would fix an actual problem.

For merging, it's a bit more compelling as things like force-merging were completely single-threaded before the changes that you listed, despite potentially taking a very long time.

from lucene.

I think you are right @jpountz . Since indexing already use almost all resources in many cases, maybe it is less worth to add an executor to make intra tasks parallel execute for indexing.

I will close this issue.

from lucene.