This package can be used to index a huge amount of nodes in Elasticsearch indexes. This package use the Flowpack JobQueue packages to handle the indexing asynchronously.
Previously the Beanstalk queue package was installed by default, this is no longer the case.
You need to install the correct Queue package based on your needs.
Available packages:
Please check the package documentation for specific configurations.
The default configuration uses Beanstalkd, but you need to install it manually:
composer require flowpack/jobqueue-beanstalkd
Check the Settings.yaml to adapt based on the Queue package, you need to adapt the className:
Flowpack:
JobQueue:
Common:
queues:
'Flowpack.ElasticSearch.ContentRepositoryQueueIndexer':
className: 'Flowpack\JobQueue\Beanstalkd\Queue\BeanstalkdQueue'
'Flowpack.ElasticSearch.ContentRepositoryQueueIndexer.Live':
className: 'Flowpack\JobQueue\Beanstalkd\Queue\BeanstalkdQueue'
flow nodeindexqueue:build --workspace live
You can use this CLI command to process indexing job:
flow nodeindexqueue:work --queue batch
You can disable async live indexing by editing Settings.yaml:
Flowpack:
ElasticSearch:
ContentRepositoryQueueIndexer:
enableLiveAsyncIndexing: false
You can use this CLI command to process indexing job:
flow nodeindexqueue:work --queue live
You can use tools like supervisord to manage long runing process. Bellow you can
found a basic configuration:
[supervisord]
[supervisorctl]
[program:elasticsearch_batch_indexing]
command=php flow nodeindexqueue:work --queue batch
stdout_logfile=AUTO
stderr_logfile=AUTO
numprocs=4
process_name=elasticsearch_batch_indexing_%(process_num)02d
environment=FLOW_CONTEXT="Production"
autostart=true
autorestart=true
stopsignal=QUIT
[program:elasticsearch_live_indexing]
command=php flow nodeindexqueue:work --queue live
stdout_logfile=AUTO
stderr_logfile=AUTO
numprocs=4
process_name=elasticsearch_live_indexing_%(process_num)02d
environment=FLOW_CONTEXT="Production"
autostart=true
autorestart=true
stopsignal=QUIT
Development sponsored by ttree ltd - neos solution provider.
We try our best to craft this package with a lots of love, we are open to sponsoring, support request, ... just contact us.
Licensed under MIT, see LICENSE
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent