palpitate's People
Forkers
coopie kren1 mm8112 markezorvaldez tlfrd jimmyswimmy consti stormmind jwmneu lijian8 icicle99 robertogemartin kevinwenya nabilbk akanshu22 afcarlpalpitate's Issues
No module named server.heart_rate_imposer
Hi, I'm trying to run Palpitate in Docker on Mac however after uploading the video and waiting for some time I'm getting this traceback
Traceback (most recent call last):
File "heart_rate_server.py", line 9, in
from server.heart_rate_imposer import HeartRateImposer
ImportError: No module named server.heart_rate_imposer
I've tried to use your Bill Clinton example and that also gives the same error.
CNN-RNNs not learning anything
@eadward I've now done the CNN-RNN approach discussed in the meeting.
It is behaving very similarly to the subject independent RNN aproach, meaning it is not learning anything.
For example here is a training trace:
Train on 10721 samples, validate on 10410 samples
Epoch 1/15
10721/10721 [==============================] - 56s - loss: 0.9948 - val_loss: 1.5181
Epoch 2/15
10721/10721 [==============================] - 56s - loss: 0.9000 - val_loss: 1.4525
Epoch 3/15
10721/10721 [==============================] - 55s - loss: 0.8459 - val_loss: 1.4802
Epoch 4/15
10721/10721 [==============================] - 55s - loss: 0.7673 - val_loss: 1.4254
Epoch 5/15
10721/10721 [==============================] - 55s - loss: 0.7109 - val_loss: 1.4720
Epoch 6/15
10721/10721 [==============================] - 55s - loss: 0.7307 - val_loss: 1.4763
Epoch 7/15
10721/10721 [==============================] - 55s - loss: 0.6144 - val_loss: 1.4643
Epoch 8/15
10721/10721 [==============================] - 54s - loss: 0.5660 - val_loss: 1.4857
#Final result on validation set:
Model r: (0.044959106007683075, 4.4554423862127374e-06)
Model rmse: 1.2189017562850681
#Final result on test set:
Model r: (-0.0059427638053850106, 0.48934621677575696)
Model rmse: 1.3999194428042123
The loss (root mean square error in this case) keeps decreasing, but the training has no effect on the validation set performance. This is exactly the same thing as it is happening with the direct RNN approach.
The architecture of the network is (tinkered keras syntax for clarity), shape comments represent the shape of the network at that point:
# shape (n_images, frequencies, time)
# shape (1,200,158)
model.add(Convolution2D(16,in_shape[1],5)) #doing 1d convolutions on the time asix with filter size 5
model.add(MaxPooling2D((1,2)))
# shape (16,1,77)
model.add(Convolution2D(32,1,10))
model.add(MaxPooling2D((1,2)))
# shape (32,1,34)
model.add(Reshape(dims=(...)))
#shape (32,34)
model.add(Permute((2,1)))
#shape (time, features)
#(34,32)
model.add(LSTM(200))
model.add(Dense(1))
model.add(Activation('linear'))
Video Feature Extractor
For the video we will be looking at colour changes in specific areas of the face (do a bit of literature review on that to see what you can learn from previous attempts).
This program must be able to take a video as an input and return relevant information about the facial color change
OpenSMILE research
Look into how OpenSMILE can help us. You don't need to actually put it in our build system. I'll deal with that at some point.
CNN using spectrogram of sound
Statistical Machine Learning Model: WEKA
Using WEKA, we must have a comprehensive machine learning model.
rnnAproach assesment
For each configuration of the network, let the model be trained with 10 different parameters (10 iterations of the loop in rnnAproach) for 20 epochs with early stopping of 2.
Each configuration should be tested on both user dependant and user independent data set.
Network Configurations:
- 2 layers stacked LSTM, 1 MLP at output, with dropout and Gaussian noise
- 1 layers LSTM, 1 MLP at output, with dropout and Gaussian noise
- 2 layers stacked LSTM, 1 MLP at output, with dropout and Gaussian noise, with MLP also before LSTMs
- 2 layers stacked LSTM, 1 MLP at output, with dropout and Gaussian noise with MLP between LSTMs
- 2 layers stacked LSTM, 1 MLP at output, with dropout and Gaussian noise with MLP between LSTMs and before LSTMs
- all of the above with l2 regularization
- all of the above with GRUs instead of LSTMs (if this one performs really badly, you don't have to do all of them)
So create a google docs spreadsheet. For each configuration you will have two tables to fill in, one for dependant and one for independent user data. Tables should have the following columns:
(param1, param2, param3, ... , r, rmse)
You should also copy the get_RNN_model function for each configuration, so there is no ambiguity as to what was used and can be easily reproduced. Also record the best parameters and the r and rmse on the test set.
For example the first configuration should look something like:
2 layers stacked LSTM, 1 MLP at output, with dropout and Gaussian noise - [link to get_RNN_model function]
results for dependant data:
(lstm_hidden, mlp_hidden, dropout1, dropout2, r, rmse)
(256,100,0.5,0.5, 0.63, 0.75)
.
.
.
(450,150,0.4,0.7, 0.59, 0.77)
best parameters: (256,100,0.5,0.5)
performance on test set r=0.65, rmse=0.75
results for independant data:
(lstm_hidden, mlp_hidden, dropout1, dropout2, r, rmse)
(256,100,0.5,0.5, 0.03, 1.2)
.
.
.
(450,150,0.4,0.7, 0.03, 1.2)
best parameters: (256,100,0.5,0.5)
performance on test set r=0.10, rmse=1.2
The way you change between dependant data and independent data is to change the NormalizedSubjectSplitSpectrograms class. The current versions gets the independent one. There is some commented code that can get the dependant one. The main idea is to merge train and validation set shuffle them and then split them again to get the dependant data. Same can be done with the test set but there might be no need.
Analysis of the excel Spreadsheets
There needs to be a script which returns a dicitionary of <subjectID>_<state> -> [[description, time, bpm]]
, from the data in the excel spreadsheets in the HeartAV data
Database Documentations and Sampling
Fully hosted website
Here are some suggestions, feel free to add/remove/change them, as we did not discuss these at the meeting:
- Website can upload video to be processed by the server
- Video is shown on the page.
Software
We must decide on an angle we are pursuing for software. An android app, or a website?
By the end of the sprint, we will need to have a prototype/mock of the decided upon software
Database Manipulation
We need a program to preprocess all of the data we have.
this will require an agreed upon input/output for both the audio and video extraction.
This program must be able to traverse the database(es) we have and return a directory/other database full of preprocessed data from the audio and video extractor.
Machine Learning Library Reasearch
Incorporate a machine learning library in our build system. Make a test case that uses it to learn something trivial to illustrate how to work with it. It would also be nice to have examples for different ML tecniques.
Travis CI integration with Docker
The tests need to be run in a docker context to ensure that the dependencies are all correct.
Youtube compatibility for video
This is also assigned to @kren1 , but github only allows one person assigned to each thing
openSMILE features database inserter
We need a script that given a list of (subjectID; audio_filepath) paris. Inserts a new batch into the database describing it. Then extract audio features from the audio_file (and adds those features to the database if appropriate). Then for each feature, for all timeslices insert the tuple
(subjectID, featuerID, batchID, featureValue, timeslice) into the database.
Demo video
First Frontend Prototype
My current idea is to make something like this:
http://deepdreamgenerator.com/
Where the first tile is a black background, with the video centred in the middle, then scroll down to post a url for us to analyse
HeartRate inserter
This is script should read the files (either xls or the tab seperated ones) and extract the bpm for subjects in the appropriate slice for given batch.
inputs: batchID, files(either an xls file or a the root of dataset), timeslice.
outputs: SQL statements (or direct insert into the database) that inserts the tuples
(subjectID; bpmFeatureID, batchID, bpm in that timeslice, timeslice)
Video Streaming for Backend Service
Audio Feature Extractor
To check previous approaches using audio check this (and related) papers:
You will note that the voice recording can be sustained vowels, reading sentences and other types of activities. Some work better than others (see papers that cite the MBC corpus), and it is your goal also to determine which ones work better.
This program should be able to take audio as input and return the features extracted from it using a config file (I assume that OpenSMILE already has some form of configuration)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.