racker / gutsy Goto Github PK

internal dashboard bootstrap

License: Apache License 2.0

JavaScript 97.67% Shell 2.33%

gutsy's Introduction

Gutsy DevOps Dashboard

Gutsy is an app for an out-of-the-box DevOps Dashboard built on top of DevOps JSON data.

The goal is to reflect not only the most recent raw devops.json data, but also to realize related APIs such as version control, issue trackers, build systems and on-call rotations.

Communication

irc.freenode.org#gutsy

Installation

Node Setup:

Make sure you have node installed. We require at least node 0.8.11. We recommend 'n' the node version manager (similar to ruby's rvm). Forever helps keep the app running if anything goes boink.

npm install [email protected] -g
npm install n forever -g
sudo n 0.8.11

Installing Gutsy

git clone https://github.com/racker/gutsy.git
git submodule update --init
npm install (you will require gcc for the sqlite3 db build)
cp ./lib/settings.js.example ./lib/settings.js
cp -r keys_example keys

Website

Setup your ports:

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 443 -j REDIRECT --to-port 8443

If you want different ports, change them in lib/web/app.js

Run the web app:

forever start ./bin/gutsy

Add a project using the Add Project button. You're set!

Pages are currently cached for 5 minutes in the local node process to avoid lengthy API calls.

gutsy's People

Stargazers

Watchers

Forkers

din0bot kami mkimhj kans sqwerl peteper2001 shaunmclernon sundarshankar

gutsy's Issues

Add ability to introspect high scores

When a user clicks on their name on the high scores page, it should show the break-down of why they have the score they have.

Respond Markdown formatting in pull request descriptions on release notes

Add point rules to highscores

The rules for where points come from---let's state those on this page somewhere.

gutsy and highscores common code

This includes most of utils and perhaps a few others. We should so something sane.

Tests

Create some fake devops fixtures, and render them through index.jade to make sure the index can at least compile when different optional fields are present or absent.
Crawler tests? Seems like a basic download that we don't have to touch again. If we continue to add different auth methods, we may want tests. Need to make sure it fails gracefully when endpoint is down (doesn't overwrite with empty file)
Each new middleware should probably have unit tests sine munging data is easy to test. Could use the same fake fixtures, too.
Potentially an integration test on the index router/controller.

Create Simple Cache

Certain apis take a long time and/or are rate limited and don't have a fine resolution in time. Use a simple internal cache with timestamps to sorta go around this.

deployment script

The goal of gutsy is to make having a basic internal dashboard one-click (once a devops.json endpoint is created).

The deployment script should:

ask for devops.json URL
ask for project github URL
extract project name from github URL
create DNS, eg .rax.io
cloud server:
- by default create public cloud server?
- ask for host, port of existing cloud server? (for rax would be internal-cloud info)
- ask for creds to provider? (for rax would be internal-cloud info)
- maybe we have a private repo with rax credentials to be used with gutsy; or just fork gutsy simply to add rax credentials
deploy code to server 1-off
create post commit hook with github to deploy code?

As a proof of concept, let's clone reach and use this script.

Make dashboard more dashboardy--- glance useful, colors, compact

Front dashboard: dependency chart

use logmagic

See usage in dreadnot: https://github.com/racker/dreadnot/blob/master/lib/web/app.js#L23

http://journal.paul.querna.org/articles/2011/12/26/log-for-machines-in-json/

Show latest metric values in service health page

This looks close, but the metrics aren't so clear when there are a lot of them.

I wish --coverage worked with the whiskey tests

--coverage is commented out from bin/tester because it doesn't work----shows NaN after the tests run. Would be fun if this worked. jscoverage port via whiskey test runner.

add pager duty middleware

middleware takes devops.json object and retrieves pager duty info (if any) and overwrites or adds to? ops contacts with on call rotation. Or create new field?

pager duty devops.json spec:

url (include the appropriate schedule)
http://<your_subdomain>.pagerduty.com/schedule/rotations/<schedule_id>
api key

Recreate reports.slicehost.com for Reach

Recreate some of the reports that reports.slicehost.com provides for Reach. Screenshots attached.

[Redacted screenshots of proprietary info.]

use nock for tests

get rid of mock request_maker and use nock https://github.com/flatiron/nock

Sometimes it would be great to have a full stack trace and the like for errors. Many errors are currently dumped at as a simple string to the user. Instead, we could dump out all sorts of things. Maybe node already has a good way of doing this?

On defects page: avg time a bug is open

Something like "avg open time for bugs" or "weekly rate of defects opened and closed"

Use setters & getters for body payload

This should improve error-handling quite a bit.

create json validator

Must use https://github.com/racker/devopsjson to validate.

Defect list showing closed defects

Example: Reach's first defect on the list. It is closed. We should hide all closed defects.

More people in leaderboard

Something funny is going on so that only a single person shows up in leaderboard.

Front dashboard has list of recent happenings (better than github's)

river of recent happenings - project based happenings in git(pull
requests) and v1 and dreadnot and...

Api errors are dumped out to the user with no context

The errors list should probably include the API that generated the error so its easy to tell what setting is wrong, etc.

Continue to show range form on release notes page after GO!ing? or a back button?

Edit me on Github! button on each dashboard links back to devops.json

This only works for devops.json data that is hosted by github, rather than being a well known URI like originally envisioned.

Still it would be a nice feature when possible.

Info about projects on metadashboard

display programming languages from devops.json tags, github API?
summarize README.md?

Update URL with start/end dates for release notes

Would be cool to send a URL to someone to share a list of release notes.

Make dashboards more usable and dashboardy

Involve designers and otherwise rethink dashboard layout and UX

default the release notes time period to last week, not last month

Client-side bootstrap sort of service health tables using bootstrap

APIs with no related_api are just ignored.

As it turns out, this is bad if you misspell a credential, as in version_one instead of versionone. Nothing happens and no feedback is given. Perhaps we should emit a warning somewhere if some related_api is specified but not used. This is obvious after the fact but surprisingly difficult to determine at the time.

Fix linting errors.

I think most of these are just missing declarations and semicolons.

Better project search on front dashboard listing of projects

show tags and description on projects listing

readable timestamps in devops.json

maybe this issue should go into devops.json not gutsy, but since gutsy is the only known user of devops.json....

but yeah, it would be nice if devops.json events were human readable.

Document how to enable gutsy features via devops.json (and gutsy settings if necessary as temporary soln)

add last deploy date meta data to dashboard

fix failing / timing-out test

Using nock to mock API interaction might fix this, but it still seems wrong that this failing test would error out.

lucy@beethoven ~/raxbox/gutsy
(master) $ npm test

> [email protected] test /Users/lucy/raxbox/gutsy
> ./bin/test

/Users/lucy/raxbox/gutsy/tests/view_tests/test_index.js
  test_example_minimum_none                                                  [OK]
  test_example_simple_none                                                   [OK]
  test_example_full_none                                                     [OK]
/Users/lucy/raxbox/gutsy/tests/view_tests/test_defects.js
  test_example_minimum_only_versionone                                       [OK]
  test_example_simple_only_versionone                                        [OK]
  test_example_full_only_versionone                                          [OK]
/Users/lucy/raxbox/gutsy/tests/view_tests/test_devhealth.js
  test_example_minimum_only_github                                           [OK]
  test_example_simple_only_github                                            [OK]
  test_example_full_only_github                                              [OK]
/Users/lucy/raxbox/gutsy/tests/middleware_tests/test_load_devops.js
  test_example_minimum                                                       [OK]
  test_example_simple                                                        [OK]
  test_example_full                                                          [OK]
/Users/lucy/raxbox/gutsy/tests/middleware_tests/test_devops_directory_setter.js
  test_example_minimum                                                       [OK]
  test_example_simple                                                        [OK]
  test_example_full                                                          [OK]
/Users/lucy/raxbox/gutsy/tests/middleware_tests/test_navbar.js
  test_example_minimum                                                       [OK]
  test_example_simple                                                        [OK]
  test_example_full                                                          [OK]
/Users/lucy/raxbox/gutsy/tests/middleware_tests/test_github.js
  test_example_minimum                                                       [OK]
  test_example_simple                                                        [OK]
  test_example_full                                                          [OK]
/Users/lucy/raxbox/gutsy/tests/middleware_tests/test_versionone.js
  test_example_minimum                                                       [OK]
  test_example_simple                                                        [OK]
  test_example_full                                                          [OK]
/Users/lucy/raxbox/gutsy/tests/middleware_tests/test_newrelic.js
  test_example_minimum                                                       [OK]
  test_example_simple                                                        [OK]
  test_example_full                                                          [OK]
/Users/lucy/raxbox/gutsy/tests/middleware_tests/test_pagerduty.js
  test_example_minimum                                                       [OK]
  test_example_simple                                                        [OK]
  test_example_full__expected_timeout                                      [FAIL]

Exception
AssertionError: null === "{\"message\":\"Account Not Found\",\"code\":2007}"
    at Object.strictEqual (/Users/lucy/raxbox/gutsy/node_modules/whiskey/lib/assert.js:333:29)
    at Object.isNull (/Users/lucy/raxbox/gutsy/node_modules/whiskey/lib/assert.js:74:14)
    at /Users/lucy/raxbox/gutsy/tests/middleware_tests/base.js:40:16
    at /Users/lucy/raxbox/gutsy/node_modules/async/lib/async.js:190:13
    at /Users/lucy/raxbox/gutsy/node_modules/async/lib/async.js:116:25
    at /Users/lucy/raxbox/gutsy/node_modules/async/lib/async.js:187:17
    at /Users/lucy/raxbox/gutsy/node_modules/async/lib/async.js:491:34
    at /Users/lucy/raxbox/gutsy/lib-cov/web/middleware/pagerduty.js:105:3
    at IncomingMessage.<anonymous> (/Users/lucy/raxbox/gutsy/lib-cov/utils.js:73:3)
    at IncomingMessage.emit (events.js:88:20)

  timeout                                                               [TIMEOUT]
---------------------------------------------------------------------------------
Ran 30 tests in 22.085s

Successes: 29
Failures: 1
Timeouts: 1
Skipped: 0

FAILED

Test Coverage
   +------------------------------------------+----------+------+------+--------+
   | filename                                 | coverage | LOC  | SLOC | missed |
   +------------------------------------------+----------+------+------+--------+
   | web/middleware/index.js                  |   100.00 |   28 |    1 |      0 |
   | web/middleware/logger.js                 |     8.70 |   59 |   23 |     21 |
   | web/middleware/devops_directory_setter.js |   100.00 |   10 |    4 |      0 |
   | web/middleware/load_devops.js            |    65.00 |   58 |   20 |      7 |
   | utils.js                                 |    96.67 |   65 |   30 |      1 |
   | web/urls.js                              |   100.00 |    5 |    5 |      0 |
   | web/middleware/navbar.js                 |   100.00 |   19 |    6 |      0 |
   | web/middleware/pagerduty.js              |    68.75 |   78 |   32 |     10 |
   | web/middleware/versionone.js             |    39.02 |   98 |   41 |     25 |
   | web/middleware/github.js                 |    56.03 |  288 |  141 |     62 |
   | web/middleware/newrelic.js               |    80.00 |   32 |   15 |      3 |
   +------------------------------------------+----------+------+------+--------+
                                              |    59.43 |  740 |  318 |    129 |
                                              +----------+------+------+--------+

npm ERR! [email protected] test: `./bin/test`
npm ERR! `sh "-c" "./bin/test"` failed with 2
npm ERR! 
npm ERR! Failed at the [email protected] test script.
npm ERR! This is most likely a problem with the gutsy package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     ./bin/test
npm ERR! You can get their info via:
npm ERR!     npm owner ls gutsy
npm ERR! There is likely additional logging output above.
npm ERR! 
npm ERR! System Darwin 10.8.0
npm ERR! command "node" "/usr/bin/npm" "test"
npm ERR! cwd /Users/lucy/raxbox/gutsy
npm ERR! node -v v0.6.11
npm ERR! npm -v 1.1.2
npm ERR! code ELIFECYCLE
npm ERR! message [email protected] test: `./bin/test`
npm ERR! message `sh "-c" "./bin/test"` failed with 2
npm ERR! errno {}
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /Users/lucy/raxbox/gutsy/npm-debug.log
npm not ok

Make event timeline clearer

Would be cool if events were a vertical timeline with the timestamps spaced out appropriately, and where now showed up, with a little countdown or color changing as now gets close to events.

Test settings/json

Add tests for settings against the schema from devops.json.

Alter load_devops to test against schema too maybe.

Add consistency checks for settings file.

use last 100 closed pull requests in github middleware in addition to open pull requests

Add unit tests for injector middlewear.

add version one defect middleware

Merge idjson, devops, and polling_data

Kinda silly to have all three since they share a lot of data.

crawler hangs after crawling everything

Add dreadnot info to release notes

Would be nice to see dreadnot status on the release notes page to determine which merged branches are deployed.

try/catch in JS is nuts-> did we do this correctly everywhere?

Javascript handles try/catch very strangely. See this article for the full story. This basically effects all of our async calls which may or may not fail silently now. A full review would probably not take long and be a good idea.

join project button that automatically signs user up to various systems

This is an idea that I originally saw as a "new member init" script in a project. I would interactively take a person through entering passwords and usernames, sign them up for various systems (or at least email people who could), and then output a summary at the end.

Checking out code and running scripts works great for developers, but sometimes others folks want to be involved in various ways, eg leadership or customer support.

The idea here is that each dashboard would have a [join] button that did various things depending on creds and contacts in devops.json---or in some spec anyway:

Sign user up for email lists
Creates accounts or htpasswd creds for ticketing, logging, deployment, staging, demo systems.
Join github teams and orgs
Share google doc collections
Join G+ circles

There would also be a [leave] button.

For what can't be automated, either opens tabs at the correct URLs for the user, and/or emails appropriate people with a todo list and helpful urls + information.

Thoughts?

fix tests

Fix url capitalization problem

Urls are case sensitive. This probably is not desired. For example this does not work:

http://dashboard.rax.io/p/reach

but this does:

http://dashboard.rax.io/p/Reach

Invalid v1 credentials for reach.

401 - Unauthorized: Access is denied due to invalid credentials.

\r\n

You do not have permission to view this directory or page using the credentials that you supplied.

Figure out where this happened and revert it.