####Built by Katie Schick
####The Project
CommonClarity transforms standardized test data into a visual, interactive reporting dashboard for teachers, to help inform and target instruction.
Data-driven instruction has grown in popularity with the rise of standardized testing. In many districts around the country, students take standardized tests nearly every other month. These tests generate a huge amount of data, which teachers are expected to utilize to inform instruction; however, they lack the time and the tools to make sense of it.
CommonClarity makes it easy for teachers to upload their data and instantly generate a dashboard of reports, showing their students' strengths and weaknesses according to Common Core State Standards. Data are grouped in useful ways to expose patterns and track progress, so that teachers can better help students improve.
####Table of Contents
- Technologies Used
- How It Works
- Product Details
- Product Screenshots
- [Try It Yourself!] (#try-it-yourself)
####Technologies Used
CommonClarity was written using AngularJS, D3, JavaScript, Python, Flask, HTML5, CSS3/Sass, jQuery, BeautifulSoup, SQLAlchemy, and Postgresql.
Click here for a full dashboard view.
####How It Works
Teachers can quickly set up their classes of students in the system and easily import the CSV files of their test data. Those files are then parsed by matching student names and Common Core State Standards to the database and entering scores for each student according to each standard.
Reports are generated by aggregating the data across all classes, by individual class, and by individual student and generating an interactive dashboard. This dashboard shows teachers the following information:
-
- The top Common Core State Standards students are struggling with, and which students have not met each of those standards
-
- The students who are struggling the most and the percentage of standards each of those students has not met
-
- A high-level overview of student performance on the most recent test
-
- A detailed view of student performance, broken out by all tests and by Common Core State Standard
-
- Student performance on the most recent test, compared to the rest of the school and the district overall
-
- A list of the standards tested on the most recent test
-
- An individual class's performance on the most recent test, broken out by student
-
- Student improvement on the most recent test from the test prior
####Product Details
#####The Database
The Postgres database uses SQLAlchemy as its ORM and contains tables for users, cohorts, student cohorts, tests, Common Core State Standards, scores, and normed scores.
To seed the Common Core State Standards table, I scraped the standards from the CCSS website using Beautiful Soup and wrote the data into a CSV file. The data were cleaned using the script parse.py and seeded into the database table using the script standards_seed.py.
The script scores_seed.py was used to seed the tests and scores tables before the test upload function was built.
#####The RESTful API
The module api.py contains the RESTful API for CommonClarity. Each function is called by a Flask route in app.py and, when called, queries the server, manipulates and aggregates the data, and converts the response into JSON to display on the front-end.
######Optimizing Queries
Upon reviewing my RESTful API, I noticed some inefficiencies in my queries: namely, that I wasn't using the backref relationships in my database as effectively as I could, and I had quite a lot of nested for-loops. After targeting those issues, I was able to significantly improve performance time for the queries that were taking a long time:
Query Pre-optimization Post-optimization Improvement
All cohorts by standard 1912ms 133ms 93% faster
Top "falling behind" standards 1940ms 370ms 81% faster
All single cohort data 3659ms 852ms 77% faster
#####AngularJS and D3
CommonClarity is built as a single-page webapp using AngularJS for the front end. Flask is used to serve the initial template (index.html) and the other pages are in partials. The routes and controllers are configured in app.js. The controllers in controllers.js make calls to RESTful API endpoints and Angular directives are used to display the JSON.
The reports are drawn using D3 and a custom Angular directive (directives.js). Each directive contains a render and a watch function: to watch for changes to the data and re-render the graph when changes are detected. Each graph is scaled according to the amount of data returned and features legends and tooltips.
####Product Screenshots
Click here to see a screenshot of the full dashboard, and scroll down for more product screenshots.
#####Student Set-up
Quickly set up your classes of students, either by uploading a CSV file or by manually entering each name.
#####Easy Data Import
Easily import your standardized test data into the system.
#####Reporting Dashboard
Once your data is imported, the reporting dashboard will automatically update with your new data.
Click here for a full dashboard view.
Use the drill-down menus to view all your classes at once, one class individually, or one particular student.
Target an individual student to see how that student is performing.
See how students are performing, according to specific Common Core State Standards.
Get a visual overview of how all the students in a class are performing.
####Try It Yourself!
#####Environment
- Clone the repository:
$ git clone https://github.com/kkschick/CommonClarity.git
- Create and activate a virtual environment in the same directory:
$ pip install virtualenv
$ virtualenv env
$ . env/bin/activate
- Install the required packages using pip:
(env)$ pip install -r requirements.txt
#####Database
-
To run the Postgres server, download and run postgres.app, and follow the instructions to set up Postgres on your machine—instructions for Mac.
-
Create the database in PostgreSQL:
$ psql
# CREATE DATABASE clarity;
- And then, in your virtual environment, create the database using Python:
(env)$ python -i model.py
(env)$ create_db()
- Still in your virtual environment, create the database tables and seed the standards table:
(env)$ python model.py
(env)$ python standards_seed.py
- Run the app:
(env)$ python app.py
- Point your browser to:
http://localhost:5000/