Giter VIP home page Giter VIP logo

apache / incubator-devlake-playground Goto Github PK

View Code? Open in Web Editor NEW
5.0 18.0 1.0 576 KB

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

Home Page: https://devlake.apache.org/

License: Apache License 2.0

Jupyter Notebook 14.17% Python 85.83%
dashboard-friendly data data-analysis data-engineering data-integration data-transfers devops domain-layer dora etl

incubator-devlake-playground's Introduction

DevLake Jupyter Playground

DevLake offers an abundance of data for exploration. This playground contains a basic set-up to interact with the data using Jupyter Notebooks and Pandas.

How to play

Prerequisites

Usage

  1. Have a local clone of this repository.
  2. Run poetry install in the root directory.
  3. Either:
    • navigate to the notebooks directory and run the jupyter server poetry run jupyter notebook
    • navigate to one of the notebook files (.ipynb) in the notebooks directory from your IDE directly
  4. Make sure the notebook uses the virtual environment created by poetry.
  5. Configure your database URL in the notebook code.
  6. Run the notebook.
  7. Start exploring the data in your own notebooks!

Create your own Jupyter Notebook

A good starting point for creating a new notebook is template.ipynb. It contains the basic steps you need to go from query to output.

To define a query, use the Domain Layer Schema to get an overview of the available tables and fields.

Use Pandas api to organize, transform, and analyze the query results.

Predefined notebooks and utilities

A notebook might offer a valuable perspective on the data not available within the capabilities of a Grafana dashboard. In this case, it's worthwhile to contribute this notebook to the community as a predefined notebook, e.g., process_analysis.ipynb (it depends on graphviz for its visualization).

The same goes for utility methods with, for example, predefined Pandas data transformations offering an interesting view on the data.

Contributing

Please check the contributing guidelines.

incubator-devlake-playground's People

Contributors

d4x1 avatar dependabot[bot] avatar lenntt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

xebiaquality

incubator-devlake-playground's Issues

Add some addtional documents and data

  1. I tried to run it locally, but there is no guide telling me how to run it. And I use PyCharm, it's convenient to run notebook, but I think there shoule be a topic about "How to run it" in readme(Or add an script to run the jupyter server).
  2. After opening the notebook process_analysis.ipynb, I update db configuration and try to run all cells, and I get a plot like this:
image I don't know what happens and weather I am right. So I think some basic demos are necessary. Or you can write some code in notebook, help to load some data, and show the result.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.