Giter VIP home page Giter VIP logo

oss-dashboard's Introduction

Amazon Open Source Program GitHub Dashboard

There is a distinct lack of tooling for GitHub admins and Open Source program managers who oversee many GitHub organizations. The oss-dashboard provides a view over many GitHub projects, allowing for custom reporting on the synchronized data.

Screenshot


High level description of the architecture

There are three parts to the oss-dashboard.

Part 1 - Syncing

  1. Ruby is used to connect to GitHub, pull down the latest data, and update a PostgreSQL Database.
  2. Git is used to check out the latest version of the source from each GitHub repository.

Phase 2 - Reviewing

  1. Ruby is used to run reports on the database.
  2. Ruby is also used to run reports on the checked out source code.

Phase 3 - Generate the dashboard

An HTML dashboard is generated from the data and reviews.

Further Instructions

The following documents provided further information

Project Direction

The high level plan is to work on the following items:

  • Custom dashboards. This is mostly implemented and needs the configuration approach implemented.
  • Move away from XSLT to ERB (or another Ruby based HTML generator). Reality is that XSLT seems to scare people.
  • Move away from a single HTML page to many HTML pages. It was useful to wrap up a report into a single HTML and mail around, but now the lack of deep linking is a pain.
  • Add a Repository dashboard. Currently repositories are only viewed in the multiple (ie: each dashboard displays multiple repositories at once). Having a dashboard for each repository will allow for more data.
  • Merge the reporting. Currently there are Database and Source code report structures. The plan is to merge them together.
  • Run a live demo, possibly using Apache MXNet and its dependencies as the data.
  • Consider using GHTorrent as the source of data; though the SQL database is still very attractive and simplifies the reporting step.
  • Add more reports!
  • Pull the source apart. Currently it's all driven from one high level ruby file. Splitting into independent pieces (syncing, reports, dashboard generation) will allow each section to have more focus.
  • Custom charting + custom visualization in general so that a custom-viz plugin can pair with a custom-report plugin, without the engine in the middle having to understand their format.

Contributing

Contributions are much appreciated, be they bug reports, ideas or patches. See the Contributing file for more information.

Other Projects of Interest

Or rather, some useful lists of Other Projects of Interest:

oss-dashboard's People

Contributors

adamdmharvey avatar ahetawal-p avatar ahetawal-salesforce avatar chorankates-sfdc avatar dependabot[bot] avatar gesellix avatar hyandell avatar jamesiri avatar jpeddicord avatar kaakaa avatar luxifer avatar mainec avatar miguelgf avatar ysknmt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

oss-dashboard's Issues

Query for specific org

Add, or enhance existing script, such that there is a query of database rows for a specific org.

Alternatively, show for each org.

Show badges for repos

Need to parse the content out of the README files. There appear to be two ways to indicate the badges:

Examples:

[![Code Climate](https://codeclimate.com/github/aws/aws-codedeploy-agent.png)](https://codeclimate.com/github/aws/aws-codedeploy-agent)

.. image:: https://travis-ci.org/aws/aws-cli.png?branch=develop
   :target: https://travis-ci.org/aws/aws-cli
   :alt: Build Status

Members not showing in public mode

Even when an organization has public members, running in public mode isn't picking them up. Need to investigate what the API is sending back.

No report names

The report names don't show if the issues aren't downloaded yet.

Get Started mode

Add a get started script, or a flag to refresh_dashboard that takes an org and a directory rather than a config file.
It begins by creating that directory, if needed, and generating a standard configuration file for it. A github file is also made and the user instructed to:

a) Updated the github config file with their personal access token.
b) Then run in the normal way.

Alternatively it could wait on user input like a wizard, though I would always leave the personal access token adding to the file to the user.

Allow for multiple phases at once

Currently the code lets you do everything, run in light mode or run a particular phase. Change it to allow multiple phases to be listed (at which point 'light' is just an alias for an internal list, as is the default being everything).

Stop checking out the code?

Currently the source reports are run directly on the checked out code.

Consider moving this to the GitHub search API:

https://developer.github.com/v3/search/

While there might be a reason for the source checkout (for example running existing tooling over the source), the current source reports all look like they can be implemented as search requests.

Downside - yet more running out of rate limited requests.

Use the API to pull the source for private repositories

Currently it uses git on the command line to pull the source. Need to move to GitHub API, or at least do so for private source if that proves a performance issue. Alternatively, allow a flag that says "I'm logged in, checkout private code".

PostgreSQL

Either move to, or also support, PostgreSQL.

Private access breaks

I'm getting a dashboard with Team (-1) and broken navigation. I think it's when you start with an org in public mode, then add that you have private access later.

Fonts, CSS etc

Looks ugly currently; needs a look and feel. Essentials are:

  • Font, text size
  • Table spacing
  • Page margins

Stop filtering out Owners

Now that the Owners team is frowned upon, there's no reason to keep it filtered out of the dashboard. Remove that logic.

Optimize the issue syncing

Currently the issues sync by looping over every repository and pulling any changed issues for that repository.

This could be optimized by pulling only the changed issues for an organization.

Pluggable charts

Currently charts are hard coded. The data should be moved out to JSON files rather than inline, and there should be a way to add your own charts.

Document database changes

I've been adding columns and tables. Need to be more helpful and show what tweaks to do to update a db.

Support Outside Collaborators

Need to show these in Members, with something to indicate they're an outside collaborator.

Also show them in Teams.

Document/design a custom data section

One may have internal data; for example an internal url for a private wiki page, or the name of the manager owning the software, or an address in the ticketing system.

With the current setup, these could easily go into the db in their own table (as designed by the user), and the db-reporting could pull them out. They wouldn't show on other tabs, so you'd have db reports of Repository Managers, and maybe 'Lacks Repository Manager'. Including these in other tabs would be preferable but would involve having to integrate custom data to those tabs.

Command line args

Move from positional args to proper -- args.

Keep the config as required.

DB Reports: Metadata API

Unlike the source reports (whose context is github files), the db reports aren't typed.

Have an API that gets recorded in the metadata header. It should cover:

  • Pretty Name of DB Reporter (Source Reporter need this too)
  • Title for each column
  • Type of each column so some special logic can be offered. For example, type of 'repository' would get a link to the repo. Same for members etc.

Open question is whether to encourage additional fields. Like a 'send email' link to tell no 2fa folk to turn on 2fa.

Adjust theme to match the Org avatar

Assuming only one org being shown, use the colours in the avatar to define the theme of the page and the colours in the charting.

This could include: "Make the colours in the charting depend on the bootstrap theme".

AllOrgs Members aren't distinct

The members tab shows a member multiple times if they are a member of multiple orgs. Same for the member related reports.

Support repository filtering

Some mechanism to define a list of repositories to be looked at.

Rather than filtering, having a repositories : [ ] entry in the config may be the simplest. Would be in org/repo format.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.