Giter VIP home page Giter VIP logo

kedro-viz's Introduction

Kedro-Viz


Kedro-Viz Pipeline Visualisation

Data Science Pipelines. Beautifully Designed
Live Demo: https://demo.kedro.org/


CircleCI Documentation Python Version PyPI version Downloads npm version License code style: prettier Slack Organisation

Introduction

Kedro-Viz is an interactive development tool for building data science pipelines with Kedro. Kedro-Viz also allows users to view and compare different runs in the Kedro project.

Features

  • ✨ Complete visualisation of a Kedro project and its pipelines
  • 🎨 Supports light & dark themes out of the box
  • 🚀 Scales to big pipelines with hundreds of nodes
  • 🔎 Highly interactive, filterable and searchable
  • 🔬 Focus mode for modular pipeline visualisation
  • 📊 Rich metadata side panel to display parameters, plots, etc.
  • 📊 Supports all types of Plotly charts
  • ♻️ Autoreload on code change
  • 🧪 Supports tracking and comparing runs in a Kedro project
  • 🎩 Many more to come

Installation

There are two ways you can use Kedro-Viz:

  • As a Kedro plugin (the most common way).

    To install Kedro-Viz as a Kedro plugin:

    pip install kedro-viz
  • As a standalone React component (for embedding Kedro-Viz in your web application).

    To install the standalone React component:

    npm install @quantumblack/kedro-viz

Usage

Compatibility with Kedro

Ensure your Kedro-Viz and Kedro versions are compatible by referencing the following table:

Kedro-Viz version Kedro version
>=4.7 >=0.17.5
>=3.8.0, <4.7 >=0.16.6, <0.17.5
<3.8.0 <0.16.6

CLI Usage

To launch Kedro-Viz from the command line as a Kedro plugin, use the following command from the root folder of your Kedro project:

kedro viz

A browser tab opens automatically to serve the visualisation at http://127.0.0.1:4141/.

Kedro-Viz also supports the following additional arguments on the command line:

Usage: kedro viz [OPTIONS]

  Visualise a Kedro pipeline using Kedro-Viz.

Options:
  --host TEXT               Host that viz will listen to. Defaults to
                            localhost.

  --port INTEGER            TCP port that viz will listen to. Defaults to
                            4141.

  --browser / --no-browser  Whether to open viz interface in the default
                            browser or not. Browser will only be opened if
                            host is localhost. Defaults to True.

  --load-file FILE          Path to load the pipeline JSON file
  --save-file FILE          Path to save the pipeline JSON file
  --pipeline TEXT           Name of the registered pipeline to visualise. If not
                            set, the default pipeline is visualised

  -e, --env TEXT            Kedro configuration environment. If not specified,
                            catalog config in `local` will be used

  --autoreload              Autoreload viz server when a Python or YAML file change in
                            the Kedro project

  --params TEXT             Specify extra parameters that you want to pass to
                            the context initializer. Items must be separated
                            by comma, keys - by colon, example:
                            param1:value1,param2:value2. Each parameter is
                            split by the first comma, so parameter values are
                            allowed to contain colons, parameter keys are not.
                            To pass a nested dictionary as parameter, separate
                            keys by '.', example: param_group.param1:value1.

  -h, --help                Show this message and exit.

Experiment Tracking usage

To enable experiment tracking in Kedro-Viz, you need to add the Kedro-Viz SQLiteStore to your Kedro project.

This can be done by adding the below code to settings.py in the src folder of your Kedro project.

from kedro_viz.integrations.kedro.sqlite_store import SQLiteStore
from pathlib import Path
SESSION_STORE_CLASS = SQLiteStore
SESSION_STORE_ARGS = {"path": str(Path(__file__).parents[2] / "data")}

Once the above set-up is complete, tracking datasets can be used to track relevant data for Kedro runs. More information on how to use tracking datasets can be found in the experiment tracking documentation

Notes:

  • Experiment Tracking is only available for Kedro-Viz >= 4.0.2 and Kedro >= 0.17.5
  • Prior to Kedro 0.17.6, when using tracking datasets, you will have to explicitly mark the datasets as versioned for it to show up properly in Kedro-Viz experiment tracking tab. From Kedro >= 0.17.6, this is done automatically:
train_evaluation.r2_score_linear_regression:
  type: tracking.MetricsDataSet
  filepath: ${base_location}/09_tracking/linear_score.json
  versioned: true

Standalone React component usage

To use Kedro-Viz as a standalone React component, import the component and supply a data JSON as prop:

import KedroViz from '@quantumblack/kedro-viz';

const MyApp = () => (
  <div style={{ height: '100vh' }}>
    <KedroViz data={json} />
  </div>
);

The JSON can be obtained by running:

kedro viz --save-file=filename.json

We also recommend wrapping the Kedro-Viz component with a parent HTML/JSX element that has a specified height (as seen in the above example) in order for Kedro-Viz to be styled properly.

Our documentation contains additional examples on how to visualise with Kedro-Viz.

Feature Flags

Kedro-Viz uses features flags to roll out some experimental features. The following flags are currently in use:

Flag Description
sizewarning From release v3.9.1. Show a warning before rendering very large graphs (default true)
expandAllPipelines From release v4.3.2. Expand all modular pipelines on first load (default false)

To enable or disable a flag, click on the settings icon in the toolbar and toggle the flag on/off.

Kedro-Viz also logs a message in your browser's developer console to show the available flags and their values as currently set on your machine.

Maintainers

Kedro-Viz is maintained by the Kedro team and a number of contributors from across the world.

Contribution

If you want to contribute to Kedro-Viz, please check out our contributing guide.

License

Kedro-Viz is licensed under the Apache 2.0 License.

Citation

If you're an academic, Kedro-Viz can also help you, for example, as a tool to visualise how your publication's pipeline is structured. Find our citation reference on Zenodo.

kedro-viz's People

Contributors

richardwestenra avatar dependabot[bot] avatar tynandebold avatar rashidakanchwala avatar limdauto avatar studioswong avatar snyk-bot avatar bru5 avatar antonymilne avatar 921kiyo avatar huongg avatar cvetankanechevska avatar ravi-kumar-pilla avatar merelcht avatar deepyaman avatar tolomea avatar dmitriideriabinqb avatar jmholzer avatar mzjp2 avatar noklam avatar yetudada avatar jitu5 avatar andrii-ivaniuk avatar ahdrameraliqb avatar astrojuanlu avatar nakhan98 avatar waylonwalker avatar vladimir-mck avatar ottis avatar nikotome avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.