Giter VIP home page Giter VIP logo

frappe / insights Goto Github PK

View Code? Open in Web Editor NEW
334.0 24.0 147.0 24.9 MB

Open source analytics / business intelligence tool (BI)

Home Page: https://frappe.io/products/insights

License: GNU Affero General Public License v3.0

HTML 0.13% JavaScript 15.31% Vue 49.85% CSS 0.71% Python 31.39% Shell 0.09% TypeScript 2.53%
data-analytics data-visualization erpnext frappe-framework analytics business-intelligence frappe

insights's Introduction

Frappe Insights logo

Frappe Insights

Simple. Crafted. Powerful. Data Analysis.

GitHub issues GitHub license GitHub release (latest by date) codecov unittests

Hero Image

Introduction

Frappe Insights is an open-source project designed to simplify data analysis and reporting from databases. Built upon the Frappe Framework, Frappe Insights aims to provide an intuitive interface that enables users to effortlessly generate complex reports and gain valuable insights from their data.

Motivation

The inspiration behind Frappe Insights was the need to simplify data analysis for the apps built upon Frappe Framework. While the framework provides robust tools for managing structured data using DocTypes, analyzing and visualizing that data often requires developer expertise. Frappe Insights aims to bridge this gap by offering a user-friendly solution that empowers non-technical users to perform data analysis tasks without relying on developers.

Key Features

Query Builder: Frappe Insights offers a user-friendly query builder interface that allows users to create database queries without the need for extensive SQL knowledge. The interface provides a step-by-step approach for building queries, empowering users to easily select tables, apply filters, perform calculations, and more.

Visualizations and Dashboards: Once a query is created, Frappe Insights enables users to visualize the results using a variety of charts and graphs. Users can easily create custom dashboards by combining multiple visualizations to gain a comprehensive view of their data.

Connectivity: Frappe Insights currently supports integration with MySQL databases, allowing users to connect to their existing databases and work directly with their data. More database integrations are planned for the future.

Under the Hood

Frappe Framework: Frappe Insights is built upon the Frappe Framework, a full-stack web application framework written in Python and Javascript. The framework provides a robust foundation for building web applications, including a database abstraction layer, user authentication, and a REST API.

Frappe UI: Frappe Insights uses the Frappe UI, a Vue-based UI library, to provide a modern and responsive user interface. The Frappe UI library provides a variety of components that can be used to build single-page applications on top of the Frappe Framework.

SQLAlchemy: Frappe Insights uses SQLAlchemy, a Python SQL toolkit and Object Relational Mapper, to interact with the database. SQLAlchemy provides a powerful and flexible interface for working with databases, allowing Frappe Insights to support multiple database integrations.

eCharts: Frappe Insights uses eCharts, a powerful open-source charting and visualization library, to render charts and graphs. eCharts provides a variety of chart types and customization options, allowing Frappe Insights to provide a rich and interactive data visualization experience.

Pandas: Frappe Insights uses Pandas, a Python data analysis library, to perform data analysis tasks. Pandas provides a variety of data structures and functions for working with structured data, allowing Frappe Insights to provide powerful data analysis capabilities.

Installation

Docker

You need Docker, docker-compose and git setup on your machine. Refer Docker documentation. After that, run the following commands:

git clone https://github.com/frappe/insights
cd insights/docker
docker-compose up

Wait for sometime until the setup script creates a site. After that you can Open the URL http://insights.localhost:8000/insights in your browser, you should see the app running should show up. You can login with the default credentials Administrator and admin.

Local

To setup the repository locally follow the steps mentioned below:

  1. Install bench and setup a frappe-bench directory by following the Installation Steps
  2. Start the server by running bench start
  3. In a separate terminal window, create a new site by running bench new-site insights.test
  4. Map your site to localhost with the command bench --site insights.test add-to-hosts
  5. Get the Insights app. Run bench get-app https://github.com/frappe/insights
  6. Run bench --site insights.test install-app insights.
  7. Now open the URL http://insights.test:8000/insights in your browser, you should see the app running

Contributions and Community

There are many ways you can contribute even if you don't code:

  1. You can start by giving a star to this repository!
  2. If you find any issues, even if it is a typo, you can raise an issue to inform us.
  3. You can join our telegram group and share your thoughts.

License

GNU Affero General Public License v3.0

insights's People

Contributors

0xsirsaif avatar ankush avatar arunmathaisk avatar blaggacao avatar crossxcell99 avatar dj12djdjs avatar gavindsouza avatar marination avatar michellealva avatar mohsinalimat avatar nagariahussain avatar nextchamp-saqib avatar semantic-release-bot avatar shantanoo avatar vjfalk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

insights's Issues

Alerts

Consider a query - "Revenue FY21-22"
A user should be able to create an alert such that once the result of this query changes, an alert (e.g., email) should be sent to the configured user.

The way this can be implemented is,

  • Select a query to set an alert on
  • Select interval at which the alert should trigger, eg. 5 mins (will check if total sales have changed every 5 mins)
  • The selected query will be ran every 5 mins to check if the value in the result has changed, if changed then send a email notification

Dynamic filters on queries

To do exploratory analysis like report views or dashboards we need dynamic filters.

E.g. "Customer", "site", "timespan"

These can be global to dashboard or local to a single query... lots of decisions to be taken. Leaving up to you ๐Ÿ˜ฌ

Need to delete dashboard twice

If I wnt to delete a dashboard, it doesn't delete the first time. The first time, all the graphs disappear, but the dashboard stays. so I need to re-delete it.

[Release] Checklist for 0.5.0-beta release

Checklist

Major ones

  • Website
  • SaaS Sign-ups
  • Demo Video
  • Update Readme
  • Documentation - https://frappeinsights.frappe.cloud/docs
  • Adding Users / Sign-up flow
    • Forward to framework login page for signups
  • Onboarding
  • Home Page
    • Added a Get Started page
  • Adding a Data Source (?)
  • Improve Dashboard Layout Editor

Minor ones

  • Add Format field in Dimension Picker for Date columns
  • Add empty states for all lists
  • Color Picker for Charts (?)
  • Remove unimplemented visualization types
  • Increase default size of newly added visualization
  • Expression function help only appears when autocompleted
  • Refreshing dashboard doesn't refresh table
  • Loading states for dashboard cards

Unable to add query to dashboard from query view.

https://screenrec.com/share/vLXGc8BSt0

Traceback (most recent call last):
  File "apps/frappe/frappe/app.py", line 58, in application
    response = frappe.api.handle()
  File "apps/frappe/frappe/api.py", line 54, in handle
    return frappe.handler.handle()
  File "apps/frappe/frappe/handler.py", line 45, in handle
    data = execute_cmd(cmd)
  File "apps/frappe/frappe/handler.py", line 83, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
  File "apps/frappe/frappe/__init__.py", line 1584, in call
    return fn(*args, **newargs)
  File "apps/frappe/frappe/handler.py", line 310, in run_doc_method
    response = doc.run_method(method, **args)
  File "apps/frappe/frappe/model/document.py", line 925, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "apps/frappe/frappe/model/document.py", line 1269, in composer
    return composed(self, method, *args, **kwargs)
  File "apps/frappe/frappe/model/document.py", line 1251, in runner
    add_to_return_value(self, fn(self, *args, **kwargs))
  File "apps/frappe/frappe/model/document.py", line 922, in fn
    return method_object(*args, **kwargs)
  File "apps/insights/insights/insights/doctype/insights_query_chart/insights_query_chart.py", line 25, in add_to_dashboard
    dashboard_doc.add_chart(self.name, layout)  # saves the dashboard
AttributeError: 'InsightsDashboard' object has no attribute 'add_chart'

Cards drill down

If double clicking the cards can open all relevant transactions then the information from such cards is actionable. For example, if there were a card "# issues that filed SLA", double clicking that should open all relevant issues. This information is actionable.

Else these are numbers that are good to look from a health check pov. only However it does not provide the user with drill down to take an action on items that need attention.

Unable to add charts to Dashboard

I have created Dashboard. I have created 3 queries. I am able to add 1st query to the Dashboard. For other 2 queries I will not get Dashboard name.
Even if I add directly I get loading screen.
Screenshot (653)
Screenshot (652)

VersionNotFound Error

Im using frappe version-13

on bench get-app or bench remove-app the below error occurs

  File "lib/python3.9/site-packages/bench/bench.py", line 262, in sync
    self.update_apps_states(
  File "lib/python3.9/site-packages/bench/bench.py", line 207, in update_apps_states
    "version": get_current_version(app, self.bench.name),
  File "lib/python3.9/site-packages/bench/utils/app.py", line 283, in get_current_version
    current_version = get_version_from_string(f.read())
  File "lib/python3.9/site-packages/bench/utils/app.py", line 120, in get_version_from_string
    raise VersionNotFound(f"{contents} is not a valid version")
bench.exceptions.VersionNotFound:  is not a valid version

Temporary table error

Traceback (most recent call last):
  File "apps/frappe/frappe/app.py", line 58, in application
    response = frappe.api.handle()
  File "apps/frappe/frappe/api.py", line 54, in handle
    return frappe.handler.handle()
  File "apps/frappe/frappe/handler.py", line 45, in handle
    data = execute_cmd(cmd)
  File "apps/frappe/frappe/handler.py", line 83, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
  File "apps/frappe/frappe/__init__.py", line 1597, in call
    return fn(*args, **newargs)
  File "apps/frappe/frappe/handler.py", line 304, in run_doc_method
    response = doc.run_method(method)
  File "apps/frappe/frappe/model/document.py", line 925, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "apps/frappe/frappe/model/document.py", line 1269, in composer
    return composed(self, method, *args, **kwargs)
  File "apps/frappe/frappe/model/document.py", line 1251, in runner
    add_to_return_value(self, fn(self, *args, **kwargs))
  File "apps/frappe/frappe/model/document.py", line 922, in fn
    return method_object(*args, **kwargs)
  File "apps/insights/insights/insights/doctype/insights_table/insights_table.py", line 26, in get_preview
    return data_source.get_table_preview(self.table)
  File "apps/insights/insights/insights/doctype/insights_data_source/insights_data_source.py", line 108, in get_table_preview
    return self.db.get_table_preview(table, limit)
  File "apps/insights/insights/insights/doctype/insights_data_source/sources/query_store.py", line 131, in get_table_preview
    self.create_temporary_table(table)
  File "apps/insights/insights/insights/doctype/insights_data_source/sources/query_store.py", line 112, in create_temporary_table
    self._execute(insert_records, values=result)
  File "apps/insights/insights/insights/doctype/insights_data_source/sources/query_store.py", line 127, in _execute
    frappe.log_error(f"Error fetching data from QueryStore: {e}")
  File "apps/frappe/frappe/__init__.py", line 2237, in log_error
    return error_log.insert(ignore_permissions=True)
  File "apps/frappe/frappe/model/document.py", line 256, in insert
    self._validate()
  File "apps/frappe/frappe/model/document.py", line 525, in _validate
    self._validate_length()
  File "apps/frappe/frappe/model/base_document.py", line 907, in _validate_length
    self.throw_length_exceeded_error(df, max_length, value)
  File "apps/frappe/frappe/model/base_document.py", line 941, in throw_length_exceeded_error
    frappe.throw(
  File "apps/frappe/frappe/__init__.py", line 522, in throw
    msgprint(
  File "apps/frappe/frappe/__init__.py", line 490, in msgprint
    _raise_exception()
  File "apps/frappe/frappe/__init__.py", line 442, in _raise_exception
    raise raise_exception(msg)
frappe.exceptions.CharacterLengthExceededError: Error Log 355: 'Title' (Error fetching data from QueryStore: (1292, "Incorrect datetime value: 'Oct 22' for column `_02cd5c47d5cfc415`.`QRY-0154`.`creation` at row 1")) will get truncated, as max characters allowed is 140

Issue in connecting Database

I am getting issue in connecting an erpnext site database.

ERPNext v14 beta5
Frappe V14 beta 4
while adding database, I am getting below error

Traceback (most recent call last):
  File "apps/frappe/frappe/app.py", line 67, in application
    response = frappe.api.handle()
  File "apps/frappe/frappe/api.py", line 54, in handle
    return frappe.handler.handle()
  File "apps/frappe/frappe/handler.py", line 39, in handle
    data = execute_cmd(cmd)
  File "apps/frappe/frappe/handler.py", line 77, in execute_cmd
    return frappe.call(method, **frappe.form_dict)
  File "apps/frappe/frappe/__init__.py", line 1502, in call
    return fn(*args, **newargs)
  File "apps/insights/insights/api/onboarding.py", line 41, in add_database
    data_source.save()
  File "apps/frappe/frappe/model/document.py", line 288, in save
    return self._save(*args, **kwargs)
  File "apps/frappe/frappe/model/document.py", line 310, in _save
    return self.insert()
  File "apps/frappe/frappe/model/document.py", line 268, in insert
    self.run_post_save_methods()
  File "apps/frappe/frappe/model/document.py", line 1071, in run_post_save_methods
    self.run_method("on_update")
  File "apps/frappe/frappe/model/document.py", line 912, in run_method
    out = Document.hook(fn)(self, *args, **kwargs)
  File "apps/frappe/frappe/model/document.py", line 1251, in composer
    return composed(self, method, *args, **kwargs)
  File "apps/frappe/frappe/model/document.py", line 1233, in runner
    add_to_return_value(self, fn(self, *args, **kwargs))
  File "apps/frappe/frappe/model/document.py", line 909, in fn
    return method_object(*args, **kwargs)
  File "apps/insights/insights/insights/doctype/data_source/data_source.py", line 26, in on_update
    self.import_tables()
  File "apps/insights/insights/insights/doctype/data_source/data_source.py", line 133, in import_tables
    table_links = self.get_foreign_key_constraints()
  File "apps/insights/insights/insights/doctype/data_source/data_source.py", line 213, in get_foreign_key_constraints
    return self.build_frappe_constraints()
  File "apps/insights/insights/insights/doctype/data_source/data_source.py", line 232, in build_frappe_constraints
    standard_links = self.execute_query(query, as_dict=1)
  File "apps/insights/insights/insights/doctype/data_source/data_source.py", line 88, in execute_query
    result = db.sql(query, **kwargs)
  File "apps/frappe/frappe/database/database.py", line 175, in sql
    self._cursor.execute(query)
  File "env/lib/python3.8/site-packages/pymysql/cursors.py", line 148, in execute
    result = self._query(query)
  File "env/lib/python3.8/site-packages/pymysql/cursors.py", line 310, in _query
    conn.query(q)
  File "env/lib/python3.8/site-packages/pymysql/connections.py", line 548, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "env/lib/python3.8/site-packages/pymysql/connections.py", line 775, in _read_query_result
    result.read()
  File "env/lib/python3.8/site-packages/pymysql/connections.py", line 1156, in read
    first_packet = self.connection._read_packet()
  File "env/lib/python3.8/site-packages/pymysql/connections.py", line 725, in _read_packet
    packet.raise_for_error()
  File "env/lib/python3.8/site-packages/pymysql/protocol.py", line 221, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "env/lib/python3.8/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1146, "Table 'frappe.tabDocField' doesn't exist")

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.