[REVIEW]: PsychoPhysioCollector about joss-reviews HOT 23 CLOSED

Sounds doable. Go ahead and add me to the team :)

from joss-reviews.

While the above seems like a lot, I'd call this "minor revisions" modulo some clarification on the things that break the "analysis software" model - most critically that I can't actually test this on the biosensor hardware.

from joss-reviews.

/ cc @openjournals/joss-reviewers - would anyone be willing to review this submission?

If you would like to review this submission then please comment on this thread so that others know you're doing a review (so as not to duplicate effort). Something as simple as :hand: I am reviewing this will suffice.

Reviewer instructions

• Please work through the checklist at the start of this issue.
• If you need any further guidance/clarification take a look at the reviewer guidelines here http://joss.theoj.org/about#reviewer_guidelines
• Please make a publication recommendation at the end of your review

Any questions, please ask for help by commenting on this issue! 🚀

Please note #41 looks to be a related submission. It may make sense to have both of these reviewed by the same individual.

from joss-reviews.

@openjournals/joss-editors - any suggestions for potential reviewers here?

from joss-reviews.

I pinged a friend that I think has the right expertise ...
https://twitter.com/lorenaabarba/status/758974889680261121

from joss-reviews.

So you just need someone to answer the reviewer questions on this page for the PDF? Here on this issue?

from joss-reviews.

@davclark - basically yes. There's an associated GitHub repository (https://github.com/sbogutzky/PsychoPhysioCollector) which is the main focus of this review together with the short PDF. See our reviewer guidelines here http://joss.theoj.org/about#reviewer_guidelines

If you think you might be able to help out, I'll add you to the @openjournals/joss-reviewers team so that you have permissions to update the checkboxes at the top of this issue.

from joss-reviews.

@davclark 🎉 . You should have an invite to join the '@joss-reviewers' team in your inbox. This link should also work.

Please shout if anything isn't clear about the review process here.

from joss-reviews.

@arfon do we verify that the license is enforceable? I.e., that the copyright holder indeed has copyright?

@whedon I see you've copyrighted the work to yourself. If produced at an american university, this is often not legal - we need to assign copyright to the institution if we worked on it as a student. For free software, this is usually not a big deal - especially for a very liberal license.

from joss-reviews.

Cool stuff. This is important for social science research - and warrants a change in the language above from the JOSS review checklist:

(ideally to solve real-world analysis problems)

Data collection is also part of science - much harder than analysis in many experiments. To this end, other guidelines are strained. What is the significance of "API documentation?"

A few more notes:

• The tag 2.0.4 does not include the paper.
• I cannot verify all functionality of this software as it requires specialized hardware. I do have an android phone.
• I don't think I have time to install android development tooling. Is there a ready-made APK that I can directly access via the web for installation on my phone? I know I can use android file transfer, but this is a barrier for some users (and would make my life faster too).
• I would clarify that anyone can install the existing APK easily, but that an android development environment is needed to create new questionnaires. In the paper proper, it would also be good to clarify that a few self-report surveys are provided, and that more can be created with proper android development tools.
• Link to android developer tools / docs?
• Could a brief developer doc be provided that points to top-level classes / files for major functionality? This would greatly help with contributions.
• There is no advise on getting help or contributing, except that you are welcome.
• Please spell out acronyms. These include ESM, IMU and ECG.
• Your audience is almost certainly larger than people running "ESM protocols." Please provide a general level of technical aptitude required to use the app and get the data off an analyze it. Give a general sense of quality of biosignals.
• What are existing solutions like? Why wouldn't I just use them?
• Could you say a little more about how "its visualization component allows checking the experimental setup in real time." This would address the question of manual steps to verify operation.
• Is there more written about Flow-Machines?
• The paper appears to be missing the required link to the repo.

from joss-reviews.

@arfon do we verify that the license is enforceable? I.e., that the copyright holder indeed has copyright?

@davclark - If you know of a reason why the author may have go this wrong then I think it's worth raising the question but we don't want to give legal advice to those submitting.

@whedon I see you've copyrighted the work to yourself. If produced at an american university, this is often not legal - we need to assign copyright to the institution if we worked on it as a student. For free software, this is usually not a big deal - especially for a very liberal license.

@davclark - @whedon is actually the bot that made the submission issue. @sbogutzky - this question is for you 😉

from joss-reviews.

(ideally to solve real-world analysis problems)

Data collection is also part of science - much harder than analysis in many experiments. To this end, other guidelines are strained. What is the significance of "API documentation?"

@davclark agreed and thanks for raising this point. Would you be willing to open an issue on https://github.com/openjournals/joss outlining your concerns with the language used in the checklist?

from joss-reviews.

The tag 2.0.4 does not include the paper.

Yes, this is right. tag 2.0.4 is the latest version of the PPC without the paper.

I cannot verify all functionality of this software as it requires specialized hardware. I do have an android phone.

But you can test the basic functionality, because data will always collect from the internal sensors of the smartphone. In this way, you can for example track the arm motion of a participant with a sport armband while running.

I don't think I have time to install android development tooling. Is there a ready-made APK that I can directly access via the web for installation on my phone? I know I can use android file transfer, but this is a barrier for some users (and would make my life faster too).

I would clarify that anyone can install the existing APK easily, but that an android development environment is needed to create new questionnaires.

So, you find the APK. See Installation Instructions of the readme.md

In the paper proper, it would also be good to clarify that a few self-report surveys are provided, and that more can be created with proper android development tools.

Ok.

Link to android developer tools / docs?

See Used Libraries in the readme.md

Could a brief developer doc be provided that points to top-level classes / files for major functionality? This would greatly help with contributions.
There is no advise on getting help or contributing, except that you are welcome.

Please spell out acronyms. These include ESM, IMU and ECG.

Ok.

Your audience is almost certainly larger than people running "ESM protocols." Please provide a general level of technical aptitude required to use the app and get the data off an analyze it. Give a general sense of quality of biosignals.

Ok, but this depends on the sensors and the configuration, which you use.

What are existing solutions like? Why wouldn't I just use them?

Maybe there are other solutions, but I these solutions do not support the hardware of our equipment.

Could you say a little more about how "its visualization component allows checking the experimental setup in real time." This would address the question of manual steps to verify operation.

E. g. if you place electrodes on the human body, you can check the leakage of the ECG based on the realtime visualization of the heart activity and the pattern of the signal.

Is there more written about Flow-Machines?

Yes, of course, unfortunately the most is in german. There one paper of the beginning: "Gait Biometrics in Interaction Design: A Work in Progress Report" and presentation: "Analysis of ‚flow while walking and running’: psychophysiological correlates and activity-based results".
In this year we will also published some scientific papers about our research in the project.

The paper appears to be missing the required link to the repo.

Is that needed?

from joss-reviews.

I think I may have inter-mixed some different kinds of issues above, so I'll respond to a few things again here. Let me also add that assuming it works (as I do), this is a very useful contribution. My comments are simply directed at making the publication / documentation better, and opening this up to more potential users and perhaps developers.

Things that are up to the editors

The things in this section will have no bearing on my recommendation.

The tag 2.0.4 does not include the paper.

Yes, this is right. tag 2.0.4 is the latest version of the PPC without the paper.

I'm not sure if there are rules here, and I'll leave this to the editors. Seems better to have the paper in the tagged release, though.

The paper appears to be missing the required link to the repo.

Is that needed?

This is in the guidelines for reviewers (emphasis mine):

JOSS papers should contain the following:

...

• A list of key references including a link to the software archive

As a reviewer, I feel my job is done by pointing out that this is not done.

Things that are "mine"

I cannot verify all functionality of this software as it requires specialized hardware. I do have an android phone.

But you can test the basic functionality, because data will always collect from the internal sensors of the smartphone. In this way, you can for example track the arm motion of a participant with a sport armband while running.

I've opened openjournals/joss#156 on this point. Given the current discussion, I think we're gonna let the things I can't test slide.

I don't think I have time to install android development tooling. Is there a ready-made APK that I can directly access via the web for installation on my phone? I know I can use android file transfer, but this is a barrier for some users (and would make my life faster too).

I would clarify that anyone can install the existing APK easily, but that an android development environment is needed to create new questionnaires.

So, you find the APK. See Installation Instructions of the readme.md

It seems like there is either a disagreement or lack of understanding here. Let me give a more concrete example:

I am at work with my fancy/annoying new macbook retina. To connect my phone, I need to unplug the hub from the single USB-C port, which means I need to rearrange my desk and move my ergo keyboard out of the way so I can plug into the USB-C port on my phone. I then grab the cable from the power adapter to connect my phone to my computer. The future is going to look more like this for more people.

What I'm suggesting is easy enough that I just installed the APK by creating a
fork and a new release here. Just like what you have, except including a direct download of the APK.

Link to android developer tools / docs?

See Used Libraries in the readme.md

Those don't include https://developer.android.com/studio/index.html

Compiling this would not be hard for most people with rudimentary development skills. You are not currently giving the basic pieces that would facilitate this.

What are existing solutions like? Why wouldn't I just use them?

Maybe there are other solutions, but I these solutions do not support the hardware of our equipment.

Here is another solution. Why wouldn't I just use that? I can at least partially answer this question, but I'm asking you to at a minimum cite primary existing solutions in your paper per JOSS guidelines.

Could you say a little more about how "its visualization component allows checking the experimental setup in real time." This would address the question of manual steps to verify operation.

E. g. if you place electrodes on the human body, you can check the leakage of the ECG based on the realtime visualization of the heart activity and the pattern of the signal.

It seems there is no similar readout for phone sensors? Or am I missing something?

I am able to verify the contents of the CSV files manually. Note that there is a slight type-o in your documentation. You mention psychophysiocollector/PARTICIPANT_NAME/ACTIVITY_NAME, but you actually get them in psychophysiocollector/ACTIVITY_NAME/PARTICIPANT_NAME/DATETIME. Maybe just say PsychoPhysioCollector and leave the rest to the user (it's obvious from there).

Is there more written about Flow-Machines?

Yes, of course, unfortunately the most is in german. There one paper of the beginning: "Gait Biometrics in Interaction Design: A Work in Progress Report" and presentation: "Analysis of ‚flow while walking and running’: psychophysiological correlates and activity-based results".
In this year we will also published some scientific papers about our research in the project.

You should cite these - It doesn't hurt anything, and it facilitates discovery of your work! But I won't demand it.

from joss-reviews.

@davclark - First of all, that's for your feedback.

@arfon - Now some questions to the editors.

The tag 2.0.4 does not include the paper.

Yes, this is right. tag 2.0.4 is the latest version of the PPC without the paper.

I'm not sure if there are rules here, and I'll leave this to the editors. Seems better to have the paper in the tagged release, though.

Is there a rule?

The paper appears to be missing the required link to the repo.

Is that needed?

This is in the guidelines for reviewers (emphasis mine):

JOSS papers should contain the following:

...

A list of key references including a link to the software archive
As a reviewer, I feel my job is done by pointing out that this is not done.

I do not get it, sorry? Should I at the repo as webpage to the bib file and reference the entry in the paper? I do not see this in other paper of the journal.

There, I see Paper DOI; Software repository and Software Archive. But these things will be added by the editors, right? After accepting?

from joss-reviews.

@davclark -

It seems like there is either a disagreement or lack of understanding here. Let me give a more concrete example:

I am at work with my fancy/annoying new macbook retina. To connect my phone, I need to unplug the hub from the single USB-C port, which means I need to rearrange my desk and move my ergo keyboard out of the way so I can plug into the USB-C port on my phone. I then grab the cable from the power adapter to connect my phone to my computer. The future is going to look more like this for more people.

What I'm suggesting is easy enough that I just installed the APK by creating a
fork and a new release here. Just like what you have, except including a direct download of the APK.

Sorry, there is a lack of understanding. What is the difference of my APK and your fork? In my case, I downloaded the APK from the repo and put it in my Google Drive or Dropbox folder and installed it on the smartphone.

from joss-reviews.

@davclark -

What are existing solutions like? Why wouldn't I just use them?

Maybe there are other solutions, but I these solutions do not support the hardware of our equipment.
Here is another solution. Why wouldn't I just use that? I can at least partially answer this question, but I'm asking you to at a minimum cite primary existing solutions in your paper per JOSS guidelines.

Ok. Right. This is a quite new solution of Shimmer for a newer type of IMUs. We began our studies in 2013 and build the existing solution for an older type of Shimmer IMUs (R2). These solution seems not to meet our requirements of mobility and automatic time synchronization of more data streams. A solution, which can you use, if you find it, is that: "Psychophysiological Correlates of Flow During Daily Activities" by Gaggioli et al. (2013)

Could you say a little more about how "its visualization component allows checking the experimental setup in real time." This would address the question of manual steps to verify operation.
E. g. if you place electrodes on the human body, you can check the leakage of the ECG based on the realtime visualization of the heart activity and the pattern of the signal.
It seems there is no similar readout for phone sensors? Or am I missing something?

No, because we worked in our studies primary with the Shimmer R2 IMUs.

I am able to verify the contents of the CSV files manually. Note that there is a slight type-o in your documentation. You mention psychophysiocollector/PARTICIPANT_NAME/ACTIVITY_NAME, but you actually get them in psychophysiocollector/ACTIVITY_NAME/PARTICIPANT_NAME/DATETIME. Maybe just say PsychoPhysioCollector and leave the rest to the user (it's obvious from there).

That is right, I fixed it!

from joss-reviews.

@sbogutzky I am pretty sure at this point that I've communicated my concerns / recommendations - so please make any remaining revisions and I'll wait for you to tell me you're done. I'll also wait for @arfon or another editor to chime in on the questions for them.

Regarding the google drive solution, that makes good sense. You've illustrated that there is a pretty easy way for individuals to install the APK without a wire. My solution was the same APK as yours, just providing a direct link from the github release page. I still think that'd be a nice-to-have, but I don't think this is required.

from joss-reviews.

@davclark - Thank for your recommendations. I'm ready with my revision.

from joss-reviews.

Looks good to me, and I hope the changes are helpful to others and (perhaps more importantly) for you substantiating this work as a genuine academic contribution.

So as reviewer, I recommend acceptance. It's up to the editors about the verification issue described above (that I can't verify all functionality without the sensors), but I recommend the "low bar" of accepting based on verification "as possible" and trusting the authors for the rest (including evidence from the provided references in which this software was used with the sensors).

Editors, please also note that the tag is updated to 2.0.5.

from joss-reviews.

Editors, please also note that change the title to "PsychoPhysioCollector: A Smartphone-Based Data Collection App for Psychophysiological Research"

It is possible to change that on the Paper page?

from joss-reviews.

Thanks for the review @davclark and for being responsive @sbogutzky. Some responses to questions that have come up in this thread:

Is there a rule? (about the tagged releases)

Ideally the release should include the paper but I don't think this is a hard rule right now. More important to me is that the release matches the version that has been archived (e.g. with Zenodo).

There, I see Paper DOI; Software repository and Software Archive. But these things will be added by the editors, right? After accepting?

That's correct. The final version of the PDF will include a direct link to the software repository and the archive of the code.

Editors, please also note that change the title to "PsychoPhysioCollector: A Smartphone-Based Data Collection App for Psychophysiological Research"

This is now changed.

So as reviewer, I recommend acceptance. It's up to the editors about the verification issue described above (that I can't verify all functionality without the sensors), but I recommend the "low bar" of accepting based on verification "as possible" and trusting the authors for the rest (including evidence from the provided references in which this software was used with the sensors).

👍

from joss-reviews.

@sbogutzky you paper is now accepted into JOSS and your DOI is http://dx.doi.org/10.21105/joss.00040 🎉 🚀 💥

Thanks for all of your help @davclark!

from joss-reviews.