Comments (3)
Hi @MoustafaAlissa, although I am not the author. I'll try to answer if that's okay with you. From the plot above and the information that you have given about using 30 sec. videos. I assume that your video is recorded at about 30 FPS provided that a 30-sec video has almost 900 values (from your plot). The MTTS-CAN model gives a prediction for each frame; hence, you get about 900 values in your plot.
For post-processing, first, follow the steps given in predict_vitals.py.
Additional note about the cutoff freq. of the Butterworth bandpass filter used by the authors:-
- For heart rate- fl = 0.75 Hz and fh = 2Hz (So the authors assume that the average HR will lie in between [0.75 x60, 2x60 ] BPM or [45,120] BPM
- For Resp. rate- fl = 0.08 Hz and fh = 0.5Hz (normal resp. rate [ 4.8, 30 ] Breaths per min.
After filtering the signals as given in predict_vitals.py you can follow these steps to get values per min:-
- Calculate the power spectral density (can use this) of both the signals
- Filter out all PSD for all freq outside your desired range [0.75,2] Hz for HR and so on
- Calculate at which freq. does the maximum PSD lie
- Multiply the freq obtained by 60 (to convert from Hz to BPM)
Finally, I'm sorry if you have already tried this and asked the question from a different context. I tried explaining to the best of my ability. Others are welcome to correct me. Have fun learning.
from mtts-can.
Hello, again @MoustafaAlissa. I would strongly advise you and your team to focus on the basics of signal processing. Finding Remote PPG is essentially a signal processing problem so to use amazing solutions like this repo efficiently you should be familiar with some of the concepts from this field. Now coming to the problem to calculate the average HR of the video.
The wrong way:-
After detrending and filtering the signal, the mean will lie around zero. Think like you are calculating the mean of a sine wave here that is oscillating around 0. This is why you are getting your weird values ( like -0.15).
The right way:-
After filtering the HR signal, the signal you have will be like we added many sine ways of different frequencies together. Now your task is to find the frequency which has maximum power and you will choose this frequency as the HR (in Hz). The steps are the same as I mentioned in my previous comment. I will also add code this time from a very popular toolbox for such a task- rPPG-Toolbox. Have a look at the _calculate_fft_hr function here and you will get your answer.
Have fun learning...
from mtts-can.
Well, this is our code and the problem is, that we are still getting weird values. We have tried to print out the pulse value, but we are not getting understandable numbers like (-0.15).... pls help
import tensorflow as tf
import numpy as np
import scipy.io
import os
import sys
import argparse
sys.path.append('../')
from model import Attention_mask, MTTS_CAN
import matplotlib.pyplot as plt
from scipy.signal import butter
from inference_preprocess import preprocess_raw_video, detrend
def predict_vitals(args):
img_rows = 36
img_cols = 36
frame_depth = 10
model_checkpoint = 'C:/Users/aliss/OneDrive/Desktop/MTTS-CAN-main/mtts_can.hdf5'
batch_size = args.batch_size
fs = args.sampling_rate
sample_data_path = args.video_path
dXsub = preprocess_raw_video(sample_data_path, dim=36)
print('dXsub shape', dXsub.shape)
dXsub_len = (dXsub.shape[0] // frame_depth) * frame_depth
dXsub = dXsub[:dXsub_len, :, :, :]
model = MTTS_CAN(frame_depth, 32, 64, (img_rows, img_cols, 3))
model.load_weights(model_checkpoint)
yptest = model.predict((dXsub[:, :, :, :3], dXsub[:, :, :, -3:]), batch_size=batch_size, verbose=2)
pulse_pred = yptest[0]
pulse_pred = detrend(np.cumsum(pulse_pred), 100)
[b_pulse, a_pulse] = butter(1, [0.75 / fs * 2, 2.5 / fs * 2], btype='bandpass')
pulse_pred = scipy.signal.filtfilt(b_pulse, a_pulse, np.double(pulse_pred))
avg_pulse_per_seq = np.mean(pulse_pred)
fps = args.sampling_rate / frame_depth
seq_per_min = 60 / fps
avg_pulse_per_min = avg_pulse_per_seq * seq_per_min
print("Durchschnittlicher Puls pro Minute: {:.2f}".format(avg_pulse_per_min))
########## Plot ##################
plt.subplot(211)
plt.plot(pulse_pred)
plt.title('Pulse Prediction')
plt.show()
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('--video_path', type=str, help='processed video path')
parser.add_argument('--sampling_rate', type=int, default=30, help='sampling rate of your video')
parser.add_argument('--batch_size', type=int, default=100, help='batch size (multiplier of 10)')
args = parser.parse_args()
predict_vitals(args)
from mtts-can.
Related Issues (20)
- dataset HOT 3
- Enquiry about training on UBFC-RPPG and PURE dataset HOT 27
- Enquiry about training time and project code HOT 3
- Real-Time Query HOT 1
- Request for infos regarding environment setup (strange results with tensorflow 2.8.0) HOT 3
- Test respiration rate HOT 1
- Data input HOT 1
- TS-CAN and Hybrid-Can dataloaders HOT 1
- dataset problem HOT 1
- Could you provide more details how you pre-process the dataset? HOT 1
- Dimension Error in predict Vitals HOT 2
- Data preprocessing and interface HOT 1
- Using time series data HOT 1
- Pure dataset HOT 1
- Batch size for model predict. HOT 1
- About the order of the Butterworth filter
- labeling the training data
- can i get the result as numbers of BPM? HOT 1
- how to rewrite the train.py for other DataSet
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mtts-can.