Application for parsing catalog items from search.wb.ru

Contains:

1. Console command for parsing 1000 items per iteration and saving into ClickHouse.
2. User-Form and Table View for visualizing data from ClickHouse.

• Docker
• Make
• PHP v8.1
• CakePHP v4.x
• ClickHouse v22.x

1. Clone this repository:

    git clone https://github.com/shirokovnv/wb_parser_example.git && cd wb_parser_example

2. Rename auth.json.example to auth.json and put your github oauth token there (needs for pulling eggheads/cakephp-clickhouse library from VCS repo)
3. Rename ./config/app_local.example.php to ./config/app_local.php
4. Run make build to build docker image
5. Run make install to install php dependencies
6. Run make up to run application containers

Visit http://localhost/

Run make shell to enter inside php-app container

1. Create Table:

bin/cake click_house_init

2. Drop Table:

bin/cake click_house_drop

3. Parse products:

bin/cake parse_products "QUERY STRING"

1. Create tmp dir in project root folder
2. Run make test for unit-testing.

Tests use mockery/mockery and fakerphp/faker packages as dependencies.

Run make help to see available commands.

Run make down for shutting down application containers.

MIT. Please see the license file for more information.