This in the version 6 of Karp backend, for the legacy version (v5).
This project uses poetry and MariaDB.
-
Run
make install
ormake install-dev
for a develop-install -
Install MariaDB and create a database
-
Setup environment variables (can be placed in a
.env
file in the root and then ?poetry run
sets those):export DB_DATABASE=<name of database> export DB_USER=<database user> export DB_PASSWORD=<user's password> export DB_HOST=localhost export AUTH_JWT_PUBKEY_PATH=/path/to/pubkey
-
Activate the virtual environment by running:
poetry shell
-
Run
karp-cli db up
to initialize database -
Run
make serve
ormake serve-w-reload
to start development serveror
poetry shell
and thenuvicorn asgi:app
-
To setup Elasticsearch, download Elasticsearch 6.x and start it
-
Install elasticsearch python libs for the right version
- If you use Elasticsearch 6.x, run
source <VENV_NAME>/bin/activate
andpip install -e .[elasticsearch6]
- If you use Elasticsearch 6.x, run
-
Add environment variables
export ES_ENABLED=true
export ELASTICSEARCH_HOST=localhost:9200
export SEARCH_CONTEXT=es6_search_service
poetry shell
and then:karp-cli entry-repo create karp/tests/data/config/places.json
karp-cli resource create karp/tests/data/config/places.json
karp-cli entries add places tests/data/places.jsonl
- Do the same for
municipalities
karp-cli resource publish places
karp-cli resource publish municipalities
** TODO: review this **
Can be used to have less downtime, because sometimes the preprocessing may
be faster on another machine than the machine that talks to Elasticsearch.
Do create
and import
on both machines, with the same data. Use
machine 1 to preprocess and use result on machine 2.
-
Create resource and import data as usual.
-
Run
karp-cli preprocess --resource_id places --version 2 --filename places_preprocessed
places_preprocessed
will contain a pickled dataset containing everything that is needed -
Run
karp-cli publish_preprocessed --resource_id places --version 2 --data places_preprocessed
-
Alternatively run
karp-cli reindex_preprocessed --resource_id places --data places_preprocessed
, if the resource was already published.
- Poetry >= 3.10
- FastAPI
- SQLAlchemy
- Typer
- Elasticsearch
- Elasticsearch DSL
- MariaDB
- Elasticsearch
Version can be bumped with bump2version
.
Usage:
- Increase patch number
a.b.X => a.b.(X+1)
:make bumpversion
orbumpversion patch
- Increase minor number
a.X.c => a.(X+1).0
:make bumpversion-minor
orbumpversion minor
- Increase major number
X.b.c => (X+1).0.0
:make bumpversion-major
orbumpversion major
- To custom version
a.b.c => X.Y.Z
:bumpversion --new-version X.Y.Z
bumpversion
is configured in .bumpversion.cfg
.
The version is changed in the following files: