Giter VIP home page Giter VIP logo

rohankumardubey / thirdeye Goto Github PK

View Code? Open in Web Editor NEW

This project forked from startreedata/thirdeye

0.0 0.0 0.0 106.69 MB

ThirdEye is an integrated tool for realtime monitoring of time series and interactive root-cause analysis.

Home Page: https://dev.startree.ai/docs/startree-enterprise-edition/startree-thirdeye/

License: Other

Shell 0.18% JavaScript 1.40% Python 0.17% Java 44.32% TypeScript 50.35% HTML 0.02% FreeMarker 3.36% Dockerfile 0.03% SCSS 0.11% Mustache 0.06%

thirdeye's Introduction


SlackQuickStartBlogDocumentation

What is ThirdEye?

ThirdEye is an integrated tool for realtime monitoring of time series and interactive root-cause analysis. It enables anyone inside an organization to collaborate on effective identification and analysis of deviations in business and system metrics. ThirdEye supports the entire workflow from anomaly detection, over root-cause analysis, to issue resolution and post-mortem reporting.

Watch the introduction video Watch ThirdEye introduction video

... or read the summary

What is it for? (key features)

Online monitoring and analysis of business and system metrics from multiple data sources. ThirdEye comes batteries included for both detection and analysis use cases. It aims to minimize the Mean-Time-To-Detection (MTTD) and Mean-Time-To-Recovery (MTTR) of production issues. ThirdEye improves its detection and analysis performance over time from incremental user feedback.

Detection

  • Detection toolkit based on business rules and exponential smoothing
    • Realtime monitoring of high-dimensional time series
    • Native support for seasonality and permanent change points in time series
    • Email alerts with 1-click feedback for automated tuning of detection algorithms

Root-Cause Analysis

  • Collaborative root-cause analysis dashboards
    • Interactive slice-and-dice of data, correlation analysis, and event identification
    • Reporting and archiving tools for anomalies and analyses
    • Knowledge graph construction over time from user feedback

Integration

  • Connectors for continuous time series data from Pinot, Presto, MySQL and CSV
    • Connectors for discrete event data sources, such as holidays from Google calendar
    • Plugin support for detection and analysis components

What it isn't? (limitations)

ThirdEye maintains a dedicated meta-data store to capture data sources, anomalies, and relationships between entities but does not store raw time series data. It relies on systems such as Pinot, Presto, MySQL, RocksDB, and Kafka to obtain both realtime and historic time series data.

ThirdEye does not replace your issue tracker - it integrates with it. ThirdEye supports collaboration but focuses on the data-integration aspect of anomaly detection and root-cause analysis. After all, your organization probably already has a well-oiled issue resolution process that we don't want to disrupt.

ThirdEye is not a generic dashboard builder toolkit. ThirdEye attempts to bring overview data from different sources into one single place on-demand. In-depth data about events, such as A/B experiments and deployments, should be kept in their respective systems. ThirdEye can link to these directly.

Documentation

For a complete description of ThirdEye's features, see ThirdEye documentation.

Features

Alert simulation Automatic root cause analysis
Alert evaluation with ThirdEye Top contributors in ThirdEye
Interactive slice-and-dice heatmap Correlated events analysis
Heatmap in ThirdEye Events in Thirdeye

Build

Requirements

  • tested on Linux and Mac OS
  • Java 11
  • MySQL 8.0
  • the UI requires internal npm packages. Make sure you can access them. See thirdeye-ui prerequisites

Database setup

If you have MySQL 8.0 installed, run scripts/db-setup.sh. This script uses the root user to create the database and tables.

Alternatively, you can use docker to launch mysql. Simply execute scripts/mysql-docker-start.sh

Build with Maven

ThirdEye is a maven project and uses standard maven commands.

# Build ThirdEye from source
./mvnw -T 1C install

# To skip tests during build
./mvnw -T 1C install -DskipTests

# To Skip Integration tests
./mvnw -T 1C install -pl '!thirdeye-integration-tests'

To build the UI, see thirdeye-ui.

Running ThirdEye from Distribution

ThirdEye builds a tarball and creates an installed dir post build.
ThirdEye has 3 main components all of which start from a single launcher

  • Coordinator: This is the API server which exposes a swagger endpoint that will be used in this guide
  • Scheduler: This is the component that runs the cron jobs and automated pipelines
  • Worker: This is the component that does all the hard work: running detection tasks and generating anomalies.
# cd to the distribution dir
cd thirdeye-distribution/target/thirdeye-distribution-*-SNAPSHOT-dist/thirdeye-distribution-*-SNAPSHOT

# Run the coordinator
# To run a scheduler, enable scheduler.enabled: true inside the configuration
# To run a worker, enable taskDriver.enabled: true inside the configuration

# To load plugins, export the plugin directory
export THIRDEYE_PLUGINS_DIR="${PWD}/plugins"

# Run the server
bin/thirdeye.sh server 

Docker

Once a distribution is ready, you can package it into a docker container using the command below.

# Build server image
docker build -t thirdeye:latest .

# Build UI image
cd thirdeye-ui/ && docker build -t thirdeye-ui:latest . && cd ..

Start ThirdEye with Docker

You can modify server config in config/server.yaml and start the server:

docker run --name  thirdeye-server -p 8080:8080 -p 8090:8090 -d \
  -v ./config/server.yaml:/home/thirdeye/thirdeye/config/server.yaml \
  thirdeye:latest server

Next you should launch the UI:

docker run --name  thirdeye-ui -p 8085:8085 -d thirdeye-ui:latest

You can access the UI on 127.0.0.1:8085.

Developer Guide

See DEVELOPMENT.md

License FAQ

Please refer to the License FAQ section at https://www.startree.ai/legal

thirdeye's People

Contributors

suvodeep-pyne avatar gitstevenpham avatar cyrilou242 avatar apucher avatar brandtg avatar thirdeye-ci avatar shounakmk219 avatar joshigaurava avatar akshayrai avatar npawar avatar jihaozh avatar harleyjj avatar justyves avatar puneetjaiswal avatar chirag-madlani avatar cyy0714 avatar kishoreg avatar ttbach avatar startree-platform-ci avatar alcatraz627 avatar cecilynie avatar vincentchenjl avatar jacksonargo avatar xiangfu0 avatar kant777 avatar daniellavoie avatar bl44 avatar jiateoh avatar jieyingchen avatar tangdian avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.