Strange map glitches on edited fit file about fitplotter HOT 16 CLOSED

Well, I know where is the problem (did not notice *.fit files in downloads.zip)

from fitplotter.

The original *.fit file is ok, hence, the effect is coming from the artificial editing.

One does not need to edit spikes. There is droplist "zoom" at the bottom right corne (check whether you have latest version - if not, download latest index.html and srs/main.js ). Select "y axes" zoom, and then impose the desirable y-axis range with the spikes exculded. In the future version, I shall probably put an automatical spikes removal, it's not hard, but you can work already by using current functionality.

Also please check new fitplotter version online
https://karaul.github.io/fitplotter/index.html
it's slower that a local version, but it has all the newest features.

from fitplotter.

(did not notice *.fit files in downloads.zip)

Odd, I just downloaded it and saw the two files?

Tried the new online version with my edited file same issue.

Oh I see how the zoom works - cool.

from fitplotter.

Should I concentrate on the spike filtering or "y"-zoom selection is enough already?

It is hard to imagine an effect of afterward editing, the simplet is just to ignore the edited fields, if they corrupt the structure, but then this editing is meaningless

I worked a lot with the spikes in our physical experiments with real-time dynamics, and I know when the proper zoom eleminates the problem, no forced data correction is needed. It might be needed only for the final quaity-ready publication, which is always static (one could not change zoom), but rarely for analysis. In the course of the analysis one has to work with the true raw data rather than the edited data. One needs to have a very sharp understanding how the data were corrected to avoid self-cheating.

from fitplotter.

from fitplotter.

what is HRE and how is it calculated

HRE = heart_rate * pace = number of heart beats per kilometer, it is explained here with example of the analysis
https://drive.google.com/file/d/17wK0y5p7rYlMRBogpZ9yicdnV191E1qx/view?usp=sharing

It is important for marathoners. Below image shows my HRE progress in 2020

from fitplotter.

from fitplotter.

Also Karoo, despite my efforts and their protestations

I have read their answer. I intended to reply too and offer my service, but in order to reply I needed to register, and I do not want. Anyway, if they are offerring any sort of collaboration, I will be glad. This project has been started several years ago in matlab, then with python, and finally with javascript. In terms of thinking it is the matured project, and JS is just syntaxis, very convenient and much faster than in python in terms GUI. No run-time libraries is needed, everything is inside the browser.

There are few more ideas I want to realize, so I continue to work on it

from fitplotter.

Filter to remove spikes is implemented. Tested with "Karoo-14th_ride-Feb-21-2021-123031.fit"

Below are raw data with spikes. The controls to remove spikes are highlighted:

(1) Select in the droplist "Click in the legend" option "Apply filter ..."
(2) Behind the droplist, set size of the filter more than 1 (at 0 no effect)
(3) Click in the legend on the curve to be filtered, here, "speed 21.02"
When everything is as should be, there will appear a filtered curve in addition to the raw one. Other options will be restored, because the filtered operation is dangerous and quickly can make the plot dirty. So filter size will be set 0, and default "Hide/show .." option will be restored

Below is screenshot with the raw and filtered curves

Check that in the droplist "Click in the legend" is option "Hide/show..." and click the original raw curve with spike to hide it, the result is shown below

The filtered curve does not replace the raw one in fit-file dataset, i.e. if you click "Clean plot", the filtered curve disappear, the original raw remains in the y-axis drop list

It is unclear at the moment whether the filtered curve shoul be autmatically added to the the y-axis drop list. The fitplotter has no option to edit and save fit file.

It is an one-dimensional median filter implemented, see https://www.mathworks.com/help/signal/ref/medfilt1.html. This filter is to remove spikes only, it does not smooth noisy signal (smooth insignificantly). In order to smooth, one has to use moving average filter, which is not implemented. I don't think that it is needed because of smoothing filter is implemented already in the hardware. The first Garmin data 10 years ago were much more noisy than nowadays.

from fitplotter.

I haven't had time yet to play with the filter, though it looks amazing. However, I can confirm that the glitches are to do with the edited file as on my own (very simple) plotter I get the same glitches with the simplest of maps and polylines. I think you can close this as it's not your software. I'll have a play with the filter later on. :)

from fitplotter.

I've just had a quick play and looked it up, the wikipedia entry is helpful: https://en.wikipedia.org/wiki/Median_filter

Very clever. Am I right in saying that it does change the data slightly?

from fitplotter.

Yes, it changes just slightly rather than drastically as with Moving_average, For completeness it is necessarry to have both filters. To implement the moving average filter is simpler as the median one .

PS: I did not know about the Wikiedia paper about the median :)

from fitplotter.

Another approach I've just tried with my very basic app, is if I find any value greater than 150 ignore it when I scan through the fit object to create my data set for use with D3. (I'm assuming that I'll never go faster than 150km/h even if I fell off a mountain!!! :) The value it discovered was 237 at 1.39351km Defo a spike! I might have fallen into the canal I was riding alongside! ;)

from fitplotter.

any value greater than 150 ignore

it's a naive approach to set an allowed range :), it depends always on the parameter under the consideration and leads to nowhere, because there might be always candidates running away.

from fitplotter.

Fair point. Do you want me to close this, I'm not sure of the protocol for that?

from fitplotter.

I do not either, I close :) Probably I should open "Discussion" for general themes - I have seen on option in the settings, but before I need to read manual to understand what is it.

Thank you for the testing.

from fitplotter.