piotrl / master-thesis Goto Github PK

Concept of master degree includes presenting data.

• Examples of charts with use of RescueTime: http://ilbonte.github.io/rescuetime-again/

We should decide about thesis title by the end of the March 2016.
Also the first sight on the idea of the thesis should be presented on seminary.

For now, aggregation is running only on start of the server. We want it running every hour as RescueTime provides new data every one hour.

Plan is to allow aggregations for different users, not only one.

Action points

• Login / Registration modules - #1
• Aggregation config in database - #3
• Users table
• Configuration page

Resources

Lots of web-config related things might be covered by applying Spring starter archetype:

Activities vs time

Can be used as starting point for looking at corelation between music.

Currently for development purpose, I've hardcoded API keys in config codes.
It should be stored in database and be configurable from UI.

For every track, we need external API calls for detailed information.
A number of requests are huge and slows down aggregation process.

Current situation

• Data from 14 days
• Number of requests: 1500
• Takes around 5 minutes

Most of the requests aren't needed as we save copies in the database.

• Builder for API calls (low level)
• Aggregate from specific period
• Save activities to database with time series (e.g. from 10:01 to 10:17 I was browsing facebook)
• Extract common activity categories (e.g. browsing, shopping, scheduling)

As we base on external APIs, part of users won't be familiar with last.fm or RescueTime.

There's need to write complete instruction how to get API keys from those services - with screenshots.

There's need to create account on which you'll manage yout API keys and settings.

• Registration
• Login
• Settings

Action points

• Works per user
• Status & handling failed aggregation
• Everyday schedule - #2
• Independent from the reports mechanism
• Handling errors - repeating requests / queue

Database schema

• aggregations: per user, contains status when started and finished.
• aggregations_requests: backups API calls - do we need it? Might be useful for error handling. It might be called queue mechanism.

Scrobbles

• Scrobbles per user
• Scrobbles from specific period (e.g. last month till today)

Global data details

• Tracks details globally
• Artists details globally
• Tags

Base for all other front-end features

• layout
• architecture

Previous solution based on node.js and MongoDB was complicated and not predictable.

Now we need Java & PostgreSQL stack.

• Spring boot
• PostgreSQL

Keep in mind thins to do before first public deployment

• Re-enable CRON
• After aggregation finished, refresh DB views
• JavaScript calls shouldn't be hard-coded to one domain
• Loader for AJAX events
• Fix RescueTime API connections
• Settings page