I am using Miniconda for a fresh installation with JPype installed manually. bottleneck is missing. Think requirement.txt is still needed.

The default command line tool is using zsh instead of bash from macOS 10.15 Catalina. May need modify the installation scripts for Mac users.

Should gracefully handle empty files. First error occurs: line 220, in _read_device info['StartTime'] = t.iloc[0].strftime(strftime)

Numpy np.dtype does not support timestamps with tz:

  File ".../actipy/src/actipy/processing.py", line 561, in mmap_like
    dtype = np.dtype([
TypeError: Cannot interpret 'datetime64[ns, UTC]' as a data type

Test file:

time,x,y,z
2024-02-29T22:00:05.672174Z,-0.758056640625,0.364013671875,0.598876953125
2024-02-29T22:00:05.682174Z,-0.7578125,0.36327630889886,0.597167145233143
2024-02-29T22:00:05.692174Z,-0.757810847165282,0.361814753415282,0.596943662141614
2024-02-29T22:00:05.702174Z,-0.75755845381054,0.36155245449608,0.599841133669425
2024-02-29T22:00:05.712174Z,-0.756591796875,0.359632363302743,0.598622895491693
2024-02-29T22:00:05.722174Z,-0.756595926041592,0.360599832291592,0.597912778124776
2024-02-29T22:00:05.732174Z,-0.756885465322301,0.36083984375,0.59859814302439
2024-02-29T22:00:05.742174Z,-0.759259997746071,0.360920791768336,0.596900700119761
2024-02-29T22:00:05.752174Z,-0.758531710562023,0.364244601187023,0.596072773098281

So I am in the process of deploying the sleepNet on UKBB, one step of which is to share our model with collaborators with external validity. Having the Python only workflow will be something much easier especially on a Windows machine which unfortunately many of our collaborators still use. This could be a good opportunity to test out the pywear package.

Questions:

1. Is the Pywear package ready for cwa parsing?
2. Is the Python version of feature extractor finished, should be similar to what we used for the SSL work. But this could also be included as part of the SSL release having one manual features and one machine-learnt features. Any advice? @aidendoherty @chanshing
3. Lastly, I will presume that the Python derived features going to differ from the Java derived features. @R-Walmsley probably would be the best person to answer this but is it possible for us to have a capture-24 model trained using these set of features?

After installing java, (java -version returns openjdk version "19.0.1" 2022-10-18) and python (python --version returns Python 3.10.6), trying to run actipy returns:
Traceback (most recent call last):
File "/Users/aschonfeldt/Desktop/brams data/match_annotations.py", line 4, in
data, info = actipy.read_device("raw_data/CWA-DATA.CWA",
File "/Users/aschonfeldt/Desktop/brams data/.venv/lib/python3.10/site-packages/actipy/reader.py", line 49, in read_device
data, info_read = _read_device(input_file, verbose)
File "/Users/aschonfeldt/Desktop/brams data/.venv/lib/python3.10/site-packages/actipy/reader.py", line 167, in _read_device
info_java = java_read_device(input_file, tmpout, verbose)
File "/Users/aschonfeldt/Desktop/brams data/.venv/lib/python3.10/site-packages/actipy/reader.py", line 203, in java_read_device
setupJVM()
File "/Users/aschonfeldt/Desktop/brams data/.venv/lib/python3.10/site-packages/actipy/reader.py", line 230, in setupJVM
jpype.startJVM(convertStrings=False)
File "/Users/aschonfeldt/Desktop/brams data/.venv/lib/python3.10/site-packages/jpype/_core.py", line 184, in startJVM
jvmpath = getDefaultJVMPath()
File "/Users/aschonfeldt/Desktop/brams data/.venv/lib/python3.10/site-packages/jpype/_jvmfinder.py", line 74, in getDefaultJVMPath
return finder.get_jvm_path()
File "/Users/aschonfeldt/Desktop/brams data/.venv/lib/python3.10/site-packages/jpype/_jvmfinder.py", line 212, in get_jvm_path
raise JVMNotFoundException("No JVM shared library file ({0}) "
jpype._jvmfinder.JVMNotFoundException: No JVM shared library file (libjli.dylib) found. Try setting up the JAVA_HOME environment variable properly.

It'd make more sense that CalibOK=0 when iteration is maxed out.

https://github.com/activityMonitoring/pywear/blob/4395b79aee0eeb26e319c538b5c32d95af2a4cce/pywear/processing.py#L171

https://github.com/activityMonitoring/pywear/blob/4395b79aee0eeb26e319c538b5c32d95af2a4cce/pywear/processing.py#L219-L220

The docs for detect_nonwear is out of date
https://github.com/OxWearables/actipy/blob/b0405411f213adbe4d0b040bc9a058b0c1972e72/src/actipy/processing.py#L177C12-L177C32

Current output doesn't include TZ. But we should include this somewhere.

What does column T mean?

(pywear) hangy@NDPH8334 pywear % cat data.csv 
time,x,y,z,T
2014-05-07 13:29:51.000,0.35189518,-0.44426882,0.92247915,20.0
2014-05-07 13:29:51.020,0.28776413,-0.40227732,0.8134708,20.0
2014-05-07 13:29:51.040,0.33098006,-0.3132904,0.8620407,20.0
2014-05-07 13:29:51.060,0.4280428,-0.32581726,0.86628675,20.0
2014-05-07 13:29:51.080,0.37453285,-0.37421665,0.8595783,20.0

Same issue as OxWearables/stepcount#49, which is causing stepcount to fail.

Issue

It seems as though the detect non-wear has issues when timedelta has too few

Data to reproduce

P30_wrist100.csv.gz

This is from https://ora.ox.ac.uk/objects/uuid:19d3cb34-e2b3-4177-91b6-1bad0e0163e7.

Code to reproduce

import actipy
import pandas as pd

filepath = "P30_wrist100.csv.gz"
data = pd.read_csv(
  filepath,
  usecols=['time', 'x', 'y', 'z'],
  parse_dates=['time'],
  index_col='time',
  dtype={'x': 'f4', 'y': 'f4', 'z': 'f4'},
)

data, info = actipy.process(
  data, 
  sample_rate=15,
  lowpass_hz=None,
  calibrate_gravity=True,
  detect_nonwear=True,
  resample_hz="uniform",
  verbose=True
)

With output and error

>>> import actipy
>>> import pandas as pd
>>> 
>>> filepath = "P30_wrist100.csv.gz"
>>> data = pd.read_csv(
...   filepath,
...   usecols=["time", "x", "y", "z"],
...   parse_dates=["time"],
...   index_col="time",
...   dtype={"x": "f4", "y": "f4", "z": "f4"},
... )
>>> 
>>> data, info = actipy.process(
...   data, 
...   sample_rate=15,
...   lowpass_hz=None,
...   calibrate_gravity=True,
...   detect_nonwear=True,
...   resample_hz="uniform",
...   verbose=True
... )
/Users/johnmuschelli/miniconda3/envs/stepcount/lib/python3.9/site-packages/actipy/processing.py:294: UserWarning: Skipping calibration: Insufficient stationary samples: 1 < 50
  warnings.warn(f"Skipping calibration: Insufficient stationary samples: {len(xyz)} < {calib_min_samples}")
Gravity calibration... Done! (0.04s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/johnmuschelli/miniconda3/envs/stepcount/lib/python3.9/site-packages/actipy/reader.py", line 144, in process
    data, info_nonwear = P.detect_nonwear(data)
  File "/Users/johnmuschelli/miniconda3/envs/stepcount/lib/python3.9/site-packages/actipy/processing.py", line 201, in detect_nonwear
    stationary_segment_ids
AttributeError: 'Timedelta' object has no attribute 'astype'

Without detecting non-wear:

>>> data, info = actipy.process(
...   data, 
...   sample_rate=15,
...   lowpass_hz=None,
...   calibrate_gravity=True,
...   detect_nonwear=False,
...   resample_hz="uniform",
...   verbose=True
... )
actipy/processing.py:294: UserWarning: Skipping calibration: Insufficient stationary samples: 1 < 50
  warnings.warn(f"Skipping calibration: Insufficient stationary samples: {len(xyz)} < {calib_min_samples}")
Gravity calibration... Done! (0.04s)
Resampling... Done! (0.04s)

Hi all,

I am currently processing AX6 data (acceleration and gyro). I was wondering if the read_device() function could be extended without too much effort to read the acceleration data from AX6 .cwa files as well. Right now, the function does work on these .cwa files but the resulting acceleration values in the pandas dataframe are incorrect.

Hi!

After checking the Python and Java versions (3.11.2 and 1.8.0_311) I installed actipy.

When I try to process the data as presented in the tutorial:

data, info = actipy.read_device("data/76985_01.cwa.gz",
                                 lowpass_hz=20,
                                 calibrate_gravity=True,
                                 detect_nonwear=True,
                                 resample_hz=50)

I have an error in line 49 of reader.py:

data, info_read = _read_device(input_file, verbose)
The error writes:

JVMNotFoundException: No JVM shared library file (libjli.dylib) found. Try setting up the JAVA_HOME environment variable properly.

I am running the code in a Macbook Pro M1 (2020) with 16 Gb of ram and MacOS Big Sur (11.4)

I tried to fix the error by changing the JAVA_HOME environmental variable bu none of the solutions I found worked out.

Is it fixable?