Dockerized Kedro to run Spark.
- Define
ARG NB_UID
with your user id in./dockerfile/Dockerfile
. - Define
OWNER
variable in./build.sh
. - Run this script.
- Define
OWNER
andGIT_DIR
variable (with the path of the Kedro project) in./run.sh
. - Run this script.
The Kedro project will be mounted at /home/kedro/work
folder.
To add packages, add them to './context/requirements.txt' (pip requirements format) and build a new image.
-
Inside the running container on your kedro proyect path, run:
kedro viz
- You can use screen to run this command in a multiplexed terminal.
-
Open http://127.0.0.1:4141 in your preferred browser.
-
Parameters
--host=0.0.0.0 --no-browser
are not needed anymore.