Giter VIP home page Giter VIP logo

spotlight's Introduction

Build status

Dependency status

Spotlight

Hybrid rendering app for the GOV.UK Performance Platform using Backbone and D3. JavaScript is shared between the client and server, and the app makes use of progressive enhancement to provide a great experience in every browser.

Building and running the app

Development

Just Spotlight: The simplest way to get started is to run just this app, against production data.

Firstly, it is recommended that you set up Node Version Manager on your host. See the (nvm) README for installation instructions.

Next checkout the Spotlight repo and create an .nvmrc file in its root directory containing the version of node specified in the 'engines' entry in package.json e.g. 6.11.2.

Now install the specified version of node using nvm:

nvm install 6.11.2

To check you have the correct version of node installed:

nvm which

Found '/Users/<username>/<path to>/spotlight/.nvmrc' with version <6.11.2>
/Users/<username>/.nvm/versions/node/v6.11.2/bin/node

Now tell nvm to use the version of node specified in the .nvmrc file:

nvm use

You can then run the app as follows:

npm install
npm start

Now you should be able to connect to the app at http://localhost:3057.

The app uses node-supervisor and grunt-contrib-watch to monitor changes, automatically restart the server and recompile Sass.

By default, this will look at production data, but perhaps you want to connect to a different data source. You can do that by creating your own config file in /config/config.development_personal.json that mimics /config/config.development.json with a different backdropUrl property. It'll be ignored by Git.

Full stack: if you're using our development environment then you can run all our apps in one go and use a real database for development. As a bonus, this will let you test the image fallbacks using the screenshot-as-a-service app.

First, you need to set up the Performance Platform development environment.

Once you have a machine with the required system-level dependencies, you can run the application with:

cd /var/apps/pp-puppet/development
bowl performance

Running tests

Command line

The Jasmine tests are divided into ones that work on the client (test/spec/client), and ones that work on the server (test/spec/server and test/spec/shared). The client tests are run using Jasmine v2.x, while the server tests are using Jasmine v1.x. It used to be that both were written for Jasmine v1, but after upgrading node versions our client tests needed to be upgraded.

npm test runs both client and server tests, as well as linting the codebase:

  • npm run jasmine_node executes server Jasmine tests in Node.js
  • npm run jasmine executes client Jasmine tests in PhantomJS
  • npm run shell:cheapseats executes feature tests using cheapseats with a small subset of dashboards, for speed
  • npm run shell:cheapseats_full_run runs cheapseats with all dashboards
  • npm run test:functional executes functional tests using [nightwatch][https://github.com/beatfactor/nightwatch]
Functional tests

As part of the CI (travis) npm run test:functional:ci is run. This spins up an instance of spotlight, nightwatch and phantomjs to run the tests in a headless environment.

To assist with debugging the functional tests can also be run in a selenium webdriver using the following command npm run test:functional:ff

If you want to run against firefox,chrome and phantom you can also do npm run test:functional:all.

All the functional tasks except ci will require a server to be running already.

Debugging locally

Install node-inspector where the app runs with sudo npm install -g [email protected] and run it with node-inspector.

Start the app with node --debug app/server.js and visit http://spotlight.perfplat.dev:8080/debug to view the console.

Production

npm run build:production to create a production release.

NODE_ENV=production node app/server.js to run the app in production mode.

Heroku

If you want to deploy the app to Heroku, follow these instructions.

Create an app on Heroku

Using the web interface, or the CLI:

heroku create <app-name>

Set the app to use the node-grunt buildpack

The app runs on Heroku using a custom buildpack for Grunt.js support.

This means it will run the grunt commands we need to compile the app when deploying code.

 heroku config:set BUILDPACK_URL=https://github.com/mbuchetics/heroku-buildpack-nodejs-grunt.git

Set configuration vars

heroku config:set NODE_ENV=development # makes app run in development mode
heroku config:set npm_config_production=true # does not install dev dependencies

Deploy the code

If the code you're deploying is not in master, then you'll need to make sure you specify your local branch to push to master. Otherwise it will just deploy your local master (and probably not work as expected).

git push heroku <your-branch-name>:master
heroku open # opens the freshly deployed app in a browser

Or just...

Deploy

If you want the Heroku app to be password-protected, set config variables as follows, before pushing the code.

heroku config:set BASIC_AUTH_USER=xxxx
heroku config:set BASIC_AUTH_PASS=xxxx
heroku config

Logging

You might also want to enable some logging in your Heroku app to assist with debugging. You can use logentries to do that:

heroku addons:add logentries

You can then access the logs from your app's dashboard on Heroku (under the "Add-ons" section).

Contributing

For Javascript, follow the styleguide (apart from the sections on GOV.UK modules as we don't use these)

Functionality should work without Javascript where possible.

All content should work well with screenreaders (at least Voiceover and JAWS). 'Work well' means

  • a screenreader user can orientate themselves effectively and use the page.
  • async updates are reported to the user (an 'accessibility' module exists for this).

Notes for Developers

Tables

Tables are used in the following places:

  1. To display a list of dashboards on the services, web-traffic and other dashboards pages
  2. To display data on a dashboard, e.g. a web-traffic dashboard
  3. To be displayed instead of a graph if javascript is disabled.

Configuration

As there are more tables underpinning graphs than any other tables in spotlight, table columns are configured as though they are the axes on a graph.

Columns defined in the x-axis will appear before those in the y-axis.

For example, axes defined as:

axes: {
    x: {
      key: 'key_X',
      label: 'Label X'
    },
    y: [
      {
        key: 'key_Y_one',
        label: 'Label Y1',
        format: 'integer'
      },
      {
        key: 'key_Y_two',
        label: 'Label Y2',
        format: 'integer'
      }
    ]
}

Will be displayed as:

Label X Label Y1 Label Y2
Value X Value Y One Value Y Two

The y-axis only accepts a list of column configuration.

The x-axis will accept a list or a single value. If a list is provided, the columns will appear in reverse order in the table. For example, x-axis columns defined as:

x: [
  {
    key: 'key_X_one',
    label: 'Label X1'
  },
  {
    key: 'key_X_two',
    label: 'Label X2'
  }
]

Will be displayed as:

Label X2 Label X1
Value X Two Value X One

Axes configuration is most commonly found in the module visualisation settings, and can be edited via the admin app.

Configuration for tables displaying lists of dashboards can be found in their respective controllers.

Route service

A route service is deployed in front of spotlight to limit access to the spotlight origin application to only the concourse and GDS office IPs.

To alter the list of approved IPs you can do the following in the production space:

cf set-env performance-platform-spotlight-rtsvc ALLOWED_IPS '1.2.3.4/32; 5.6.7.8/32'

spotlight's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spotlight's Issues

Error: Cannot find module 'graceful-fs'

Trying to run master now, but it won't start. I've run npm cache clear under spotlight to no avail.

09:38:17 spotlight.1               | npm WARN unmet dependency /var/apps/spotlight/node_modules/grunt/node_modules/findup-sync requires glob@'~3.2.9' but will load
09:38:17 spotlight.1               | npm WARN unmet dependency /var/apps/spotlight/node_modules/grunt/node_modules/glob,
09:38:17 spotlight.1               | npm WARN unmet dependency which is version 3.1.21
09:38:17 spotlight.1               | npm WARN unmet dependency /var/apps/spotlight/node_modules/grunt/node_modules/findup-sync requires lodash@'~2.4.1' but will load
09:38:17 spotlight.1               | npm WARN unmet dependency /var/apps/spotlight/node_modules/grunt/node_modules/lodash,
09:38:17 spotlight.1               | npm WARN unmet dependency which is version 0.9.2
09:38:17 spotlight.1               | npm WARN unmet dependency /var/apps/spotlight/node_modules/grunt-contrib-watch/node_modules/gaze/node_modules/globule requires lodash@'~1.0.1' but will load
09:38:17 spotlight.1               | npm WARN unmet dependency /var/apps/spotlight/node_modules/lodash,
09:38:17 spotlight.1               | npm WARN unmet dependency which is version 2.4.1
09:38:17 spotlight.1               | npm WARN unmet dependency /var/apps/spotlight/node_modules/node-sass/node_modules/mocha/node_modules/jade requires mkdirp@'0.3.0' but will load
09:38:17 spotlight.1               | npm WARN unmet dependency /var/apps/spotlight/node_modules/node-sass/node_modules/mkdirp,
09:38:17 spotlight.1               | npm WARN unmet dependency which is version 0.3.5
09:38:18 spotlight.1               | 
09:38:18 spotlight.1               | module.js:340
09:38:18 spotlight.1               |     throw err;
09:38:18 spotlight.1               |           ^
09:38:18 spotlight.1               | Error: Cannot find module 'graceful-fs'
09:38:18 spotlight.1               |     at Function.Module._resolveFilename (module.js:338:15)
09:38:18 spotlight.1               |     at Function.Module._load (module.js:280:25)
09:38:18 spotlight.1               |     at Module.require (module.js:364:17)
09:38:18 spotlight.1               |     at require (module.js:380:17)
09:38:18 spotlight.1               |     at Object.<anonymous> (/var/apps/spotlight/node_modules/grunt/node_modules/glob/glob.js:39:10)
09:38:18 spotlight.1               |     at Module._compile (module.js:456:26)
09:38:18 spotlight.1               |     at Object.Module._extensions..js (module.js:474:10)
09:38:18 spotlight.1               | exited with code 8
09:38:18 system                    | sending SIGTERM to all processes

Links in tables

We need to be able to present these as links:

image

In the future we may have an additional requirement to provide a separate value for the content of the <a> tag vs the href=.

It may be possible to linkify the current URLs with a formatter, but the future reqiurement may require something different.

Presenting values as a percentage from 0-100

In backdrop, our values are expressed as a percentage from 0-100, but spotlight treats "format": "percent" as values ranging from 0-1. I've discussed this with @alexmuller and we think maybe the best way to fix this is to introduce another "format" which has the range needed here.

This is what currently happens:

image

Add a filter-by parameter to completion_numbers

It is possible to get by without it by specifying the filter in the numerator/denominator regular expression, but the net result is that rather than just fetching the data from backdrop for one department, it is getting them for all of them, resulting in fetching a factor >20x more data than necessary.

It's also counter-intuitive, I just expected to be able to specify a filter-by.

String substitutions in tables

In this table:

image

There are strings like t.co. Our product manager would like to see these presented as "twitter.com". So we need a method to do string substitutions at presentation time.

Module stubs should support relative links

The following JSON snippet fails validation:

  "relatedPages": {
    "transaction": {
      "title": "Apply for a UK visa",
      "url" : "/apply-uk-visa"
    }
  },

I think I'd be ok with this passing.

raw.scss isn't being used so should be refactored out...

Previously, appending a query parameter of ?raw would apply a class of raw to the body element. This would use the styles in raw.scss.

I can't find when this got dropped, but we should remove the styles if they're no longer being used.

Table contents are double-html-escaped

See below. I can't link to a live backdrop instance because the data hasn't been backfilled yet, but I can assure you that in backdrop the contents aren't escaped.

image

npm install failing

On the advice of @theotherurmy, I've been trying to setup spotlight outside of the VM.

It doesn't work inside the VM, and it doesn't work outside either. The current problem is the log I get below.

If I run it repeatedly, each time I see a different error. It seems to have stabilised at the last of these.

npm ERR! peerinvalid The package grunt does not satisfy its siblings' peerDependencies requirements!
npm ERR! peerinvalid Peer [email protected] wants grunt@~0.4.0
npm ERR! peerinvalid Peer [email protected] wants grunt@~0.4.0
npm ERR! peerinvalid Peer [email protected] wants grunt@~0.4.0
npm ERR! peerinvalid Peer [email protected] wants grunt@~0.4.0
npm ERR! peerinvalid Peer [email protected] wants grunt@~0.4.0
npm ERR! peerinvalid Peer [email protected] wants grunt@~0.4.0
npm ERR! peerinvalid Peer [email protected] wants grunt@~0.4.0
npm ERR! peerinvalid Peer [email protected] wants grunt@~0.4.1
npm ERR! peerinvalid Peer [email protected] wants grunt@^0.4.0
npm ERR! peerinvalid Peer [email protected] wants grunt@~0.4.0
npm ERR! peerinvalid Peer [email protected] wants grunt@~0.4.0

npm ERR! System Linux 3.11.0-20-generic
npm ERR! command "/home/pwaller/.nvm/v0.10.15/bin/node" "/home/pwaller/.nvm/v0.10.15/bin/npm" "install"
npm ERR! cwd /home/pwaller/sw/gds/spotlight
npm ERR! node -v v0.10.15
npm ERR! npm -v 1.3.5
npm ERR! code EPEERINVALID
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/pwaller/sw/gds/spotlight/npm-debug.log
npm ERR! not ok code 0

At this point, I ran npm cache clear; rm -R node_modules and tried from scratch

npm ERR! Error: No compatible version found: each-async@'^0.1.2'
npm ERR! Valid install targets:
npm ERR! ["0.1.0","0.1.1","0.1.2","0.1.3"]
npm ERR!     at installTargetsError (/home/pwaller/.nvm/v0.10.15/lib/node_modules/npm/lib/cache.js:719:10)
npm ERR!     at /home/pwaller/.nvm/v0.10.15/lib/node_modules/npm/lib/cache.js:638:10
npm ERR!     at saved (/home/pwaller/.nvm/v0.10.15/lib/node_modules/npm/node_modules/npm-registry-client/lib/get.js:142:7)
npm ERR!     at /home/pwaller/.nvm/v0.10.15/lib/node_modules/npm/node_modules/graceful-fs/polyfills.js:133:7
npm ERR!     at Object.oncomplete (fs.js:107:15)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <[email protected]>

npm ERR! System Linux 3.11.0-20-generic
npm ERR! command "/home/pwaller/.nvm/v0.10.15/bin/node" "/home/pwaller/.nvm/v0.10.15/bin/npm" "install"
npm ERR! cwd /home/pwaller/sw/gds/spotlight
npm ERR! node -v v0.10.15
npm ERR! npm -v 1.3.5
npm http 304 https://registry.npmjs.org/node-sass
npm http 304 https://registry.npmjs.org/chalk
npm http 304 https://registry.npmjs.org/pad-stdio
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/pwaller/sw/gds/spotlight/npm-debug.log
npm ERR! not ok code 0
npm ERR! Error: No compatible version found: chalk@'^0.4.0'
npm ERR! Valid install targets:
npm ERR! ["0.1.0","0.1.1","0.2.0","0.2.1","0.3.0","0.4.0"]
npm ERR!     at installTargetsError (/home/pwaller/.nvm/v0.10.15/lib/node_modules/npm/lib/cache.js:719:10)
npm ERR!     at /home/pwaller/.nvm/v0.10.15/lib/node_modules/npm/lib/cache.js:638:10
npm ERR!     at saved (/home/pwaller/.nvm/v0.10.15/lib/node_modules/npm/node_modules/npm-registry-client/lib/get.js:142:7)
npm ERR!     at /home/pwaller/.nvm/v0.10.15/lib/node_modules/npm/node_modules/graceful-fs/polyfills.js:133:7
npm ERR!     at Object.oncomplete (fs.js:107:15)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <[email protected]>

npm ERR! System Linux 3.11.0-20-generic
npm ERR! command "/home/pwaller/.nvm/v0.10.15/bin/node" "/home/pwaller/.nvm/v0.10.15/bin/npm" "install"
npm ERR! cwd /home/pwaller/sw/gds/spotlight
npm ERR! node -v v0.10.15
npm ERR! npm -v 1.3.5
npm http 304 https://registry.npmjs.org/pad-stdio
npm http 304 https://registry.npmjs.org/node-sass
npm http 304 https://registry.npmjs.org/async
npm http 304 https://registry.npmjs.org/each-async
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/pwaller/sw/gds/spotlight/npm-debug.log
npm ERR! not ok code 0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.