Giter VIP home page Giter VIP logo

jira-database-etl's Introduction

JIRA-to-Database Import

Python Pandas Requests SQLAlchemy PyMySQL GitHub Last Commit

JIRA SQL ETL

Extracts issues from a JIRA instance via the JIRA REST API, transforms the data, and loads data to a database.

To derive epic-based information from tickets, the script creates an JiraEpic table as well as a JiraIssue table. The JiraIssue table is joined with the former table to easily perform analysis on aggregated epic data.

Accompanying tutorial can be found here: https://hackersandslackers.com/jira-to-sql-etl/

Installation

Installation via requirements.txt:

$ git clone https://github.com/toddbirchard/jira-database-etl.git
$ cd jira-database-etl
$ python3 -m venv myenv
$ source myenv/bin/activate
$ pip3 install -r requirements.txt
$ flask run

Installation via Pipenv:

$ git clone https://github.com/toddbirchard/jira-database-etl.git
$ cd jira-database-etl
$ pipenv shell
$ pipenv update
$ flask run

Configuration

The following environment variables are needed to run this script:

  • SQLALCHEMY_DATABASE_URI: A URI for the database intended to store these tables (ie: mysql+pymysql://[USER]:[PASSWORD]@d[DATABASE_HOST]:[PORT]/[DATABASE_NAME])
  • SQLALCHEMY_EPIC_TABLE: Name of database table to store epics.
  • SQLALCHEMY_JIRA_TABLE: Name of database table to store JIRA issues.
  • JIRA_ENDPOINT: Your JIRA Cloud API endpoint for JQL searching (such as https://mydomain.atlassian.net/rest/api/3/search)
  • JIRA_USERNAME: Your JIRA username.
  • JIRA_API_KEY: An API key associated with the JIRA user.
  • JIRA_ISSUES_JQL: JQL to get JIRA issues.
  • JIRA_ISSUES_FIELDS: Specific fields to retrieve from the JIRA query.
  • JIRA_EPICS_JQL: JQL to get JIRA epics.
  • JIRA_EPICS_FIELDS: Specific fields to retrieve from the epics query.

Troubleshooting

Make sure your database contains tables named JiraEpic and JiraIssue prior to running this script (columns/schema don't matter, these will be overridden).


This project and all publically-visible repositories are free of charge. If you've found this project to be helpful, a small donation would be greatly appreciated to keep us in business. All proceeds go towards coffee, and all coffee goes towards improving these projects.

jira-database-etl's People

Contributors

toddbirchard avatar renovate-bot avatar imgbotapp avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.