The BIDS validator was giving us the error "You have to define 'TaskName' for this file" for all 6 of our functional scans (3 tasks, each acquired in both AP and PA directions). We tried to address this by adding "SidecarChanges": {"TaskName" : ""} to each functional scan in our conversion_config.json

We've tried multiple iterations of our task names (see conversion_config_AP.json attached
conversion_config_AP.pdf
) but the validator is still showing an error (see BIDS validator output attached
Output-BIDSValidator-jobid-65139645.pdf
). I checked the tasks' sidecar .json files and found that a TaskName had not been added to that file after running convert2bids, which explains the validator errors (see the attached sidecar .json for our REW task
sub-1068_task-REW_acq-AP1_bold.pdf
).

Do you know why the command "SidecarChanges" isn't actually working to edit our sidecar .json files? Is it a formatting issue in our conversion_config file, or maybe something else?

I am rerunning analyses with a newer version of fmriprep (previously 'fmriprep_1.5.0.sif', now using 'fmriprep-version-20.0.7.simg' per a superuser recommendation). When I try to run "fmriprep_process -config_file /proj/cohenlab/projects/MAP/clpipe_config_map_012021.json -submit 0011", the job quits bc of a bids validator error that I have had with the previous version of fmriprep:

I remember this having to be fixed to allow the fieldmap IntendedFor line be more flexible to file name types. Is this the same issue again with this newer version of the pipeline?

Thanks,
Mac

Hey Teague,
I would like to request that documentation be added to the readthedocs regarding the fmapcleanup field. It looks like the default is 3 but in my clpipe_config.json you had set the field to -1. I think the fmapcleanup field specifies how many volumes to use in the distortion correction, but I am not sure that is correct. My guess is that -1 means this feature is turned off because we are not collecting fieldmaps. However, I would like to request that you add some explanation about possible values to be in this field and what they mean.
Thanks! Mackenzie

Hi Teague!

I reinstalled clpipe on Friday after Mackenzie let everyone know of the updates to the clpipe launch scripts with respect to UNC's Longleaf partitions. After logging out and then back in, I'm still running into the error below when I try to submit a subject for preprocessing:

It looks like "qos specification" points to a partition issue remaining in the clpipe scripts. Let me know if you want me to try anything else!

Could you add a setting in the config file so that custom fmriprep flags can be added?

In the DICOM to BIDS Conversion documentation showing the example conversion_config.json, can that include example sections for designating fmap images to assign to specific functional scans? Including how to write BIDS-acceptable "customLabels" for them

And in that same section, can there be an example/description of adding "SidecarChanges: {TaskName: "[task name from custom label]"}" to functional scan sections on the conversion_config.json? Not designating "Task Names" was an error that popped up in our BIDS Validator output for all of our functional scans when we didn't have this SidecarChanges line.

Thank you!

Hi! I've tried running one participant through glm_setp for one task ~5pm yesterday (after the clpipe reinstall), and the log output file has only shown "INFO:root:Running Subject 1044” the entire time, while our directory for the resulting image files and confounds.tsv is still empty. The slurm info on this batch job says it completed in 10 seconds. I tried running the command again with the option "-debug" and the log shows this

So it looks like it may be a Nipype problem?

If there is a blank EV (not in the last position), glm_l1_preparefsf seems to pull the last EV .txt file of the run prior.

For example, if I have only specified 4 .txt files when there are 5 EV spaces in the Feat template (EV2 is blank), EV1 is good, EV2 is still blank, EV3 has the EV4.txt file, EV4 has the EV5.txt file, and EV5 contains an incorrect EV5.txt (from the prior subject).

Hi Teague, I wanted to ask if you were planning to update the default version of fMRIPrep that is prepopulated in the clpipe_config.json files to be the stable version of fMRIPrep. This came up in trainings and I wanted to ask! I think this would be a useful edit if you dont have other concerns with updating it!
Mackenzie

Can you add a flag to only create fmriprep QC reports for specific subjects?

I'm not sure how to check our clpipe version, but I think it's from before the GLM component was incorporated. I realized this when I tried running glm_setup on a subject and got a weird traceback message (pictured below). How can I install the latest version of clpipe (if it's any different from initial installation), and will this process affect any of the directories and files that were already created using the first version? Can this process also be described in the documentation?

Along the same lines, does the newest version of clpipe include an fMRIPrep update? The one our clpipe version currently uses is v1.5.0, and our fMRIPrep logs all display a concerning message that this version was flagged for significant issues. One of those issues was discussed here: nipreps/fmriprep#1819 and it seems pretty similar to the errors shown for some of our subjects who failed fmriprep. It looks like a new major release 20.0.0+ came out right before the pandemic in Feb. But I'm not sure if that is in the works to be implemented in clpipe or not

In the config file documentation, add documentation saying that in longleaf, the working directory is /pine/scr/[first letter of onyen]/[second letter of onyen]/onyen

Can you clarify where the slurm files end up?

Specify which of the options need to be added by the user versus which will be filled in automatically.

Hi Teague,

In setting up post processing options in clpipe_config.json, I am a little confused about the GlobalSignalRegression field. This field does not automatically appear in the "PostProcessingOptions" when a config file is generated. However, I can see that you have implemented it in "ProcessingStreams".

I can see that "global_signal" is default included in the "Confounds", "ConfoundsQuad", "ConfoundsDerive", and "ConfoundsQuadDerive" fields. I am assuming this is the equivalent of "GlobalSignalRegression: True/False. Include global signal into nuisance regression. Defaults to True." as noted in the documentation, but I find it confusing that this field doesnt appear as noted in the documentation. I am hoping you could clarify that in the documentation/config file.

Thanks! Mackenzie

Hi Teague! We are currently preparing for Level 2 analysis, where we'll be combining L1 analyses from the AP sequence of our EmoReg task with the L1 of the PA sequence of the task. But we're running into issues using the glm_l2_preparefsf command. We're finding that if one user in our group ran a subject's Level 1 EmoReg_AP feat job, and a different user ran the Level 1 EmoReg_PA feat job, then the glm_l2_preparefsf command won't permit either user to create the Level 2 .fsf file for that subject. This error persists even after we've made sure that file permissions throughout both the L1 AP and L1 PA .feat directories are recursively set to chmod 775.

For example, my team member Lucy ran subject 1002's AP data through Level 1. I ran the subject's PA data through Level 1. Both Level 1 .feat directories have identical permissions, with full group permissions, but when I try to run the glm_l2_preparefsf command (just for this subject), I get the following error that I don't have permission to access one of the registration files in the AP data:

Even though the permissions on that file are full for the group:

Lucy gets a reciprocal error, saying that she doesn't have permissions to the PA data (that I own, but she should have access to as a group member). When I run the clpipe command on another subject whose AP and PA data were both run through Level 1 analysis by me, it ends up working.

We're not sure if this is a clpipe issue, or a deeper problem with file permissions either via FSL's FEAT programming or with Longleaf in general, but we figured we'd reach out to you first!

Right now it sets those TPs to 0.

Discussed in 2/6 meeting

Hey Teague, The documentation currently specifies grab_config_file. This is not currently working on the most recent version of the pipeline. I recommended that Andrea use get_config_file as that is what I have used on previous versions of the pipeline. Is get_config_file going to be updated to grab_config_file or is this a typo in the readthedocs? Thanks! Mackenzie

Users are given an uncaught TypeError stack trace when giving an invalid path.

We should probably use a try/catch to provide the user with a more helpful message.

Hi Teague! We tried to run fMRIPrep version 20.2.1 on a subject via clpipe, and the job failed due to a permissions error. The weird thing is that in our fMRIPrepOptions in our config file, our chosen atlas is the MNIPediatricAsym, cohort 2. Our config file is below, and everything is the same as when we submitted it except the FMRIPrepPath pointed to "fmriprep-20.2.1.simg".

But the "permissions denied" error was for a file in the WHS atlas directory, which we don't specify using anywhere in our config file.

We are now running the same subject with fMRIPrep version 20.0.7 and it's running successfully, so I wonder if the issue is with v20.2.1? Any help would be much appreciated!

--Amanda (CIRCLE Lab with Dr. Sheridan at UNC)

Can there be a way to add custom confound regressors? For example, if we wanted to run compcor or if we had physio regressors that we wanted to add.

The clpipe readthedocs seem to suggest that additional documentation for postprocessing has been created. However the pages listed below are not shared. This documentation is also referenced in the configuration files sections. Please update these sections when you have a chance. [[I noticed this because I was looking for additional information on the clpipe_config.json fields "OversamplingFreq" and "PercentFreqSample".]]

From clpipe readthedocs:
For a description of the various postprocessing steps, along with references, please see the following documentation:

1. Nuisance Regression
2. Frequency Filtering
3. Scrubbing
4. Spectral Interpolation

Hi Teague!

Apologies for another post but I was hoping for some clarification on the documentation in the configuration file (here: https://clpipe.readthedocs.io/en/latest/configfiles.html)

For context, I was looking for more information concerning Spectral Interpolation and PropVoxel. I noticed that some of the field explanations say something like: "For more information on spectral interpolation see Postprocessing/Spectral Interpolation." However, when going to the Postprocessing fMRI I can't find any further information (https://clpipe.readthedocs.io/en/latest/postprocessing.html#postprocessing-fmri). Am I going to the wrong section for more information?

Additionally, some of the field explanations are missing (e.g. PropVoxel) from the configuration file documentation. Is there somewhere else that I can find this information?

Apologies if this will all be updated soon, I know there have been a few updates to the config files!

-Hannah Morgan

If specify the TR in the command line, receive an error (below). If the -TR flag is not used, it pulls it in from the file and runs successfully.

Command line:
fmri_postprocess -config_file /proj/cohenlab/projects/kki_adhd/scripts/GNGsConfig.json -task GNGs -TR 2.5 -log_output_dir /proj/cohenlab/projects/kki_adhd/clpipe_batchOutput_GNGs -submit 0905

What is submitted to SLURM:
sbatch -n 1 --mem=20000 --time=8:0:0 --cpus-per-task=1 --job-name="PostProcessing0905" --output=/proj/cohenlab/projects/kki_adhd/clpipe_batchOutput_GNGs/Output-PostProcessing0905.out --wrap="fmri_postprocess -config_file=/proj/cohenlab/projects/kki_adhd/data_BIDS/derivatives/fmriprep/GNGsConfig.json -target_dir=/proj/cohenlab/projects/kki_adhd/data_BIDS/derivatives/fmriprep/ -target_suffix=MNI152NLin2009cAsym_preproc.nii.gz -output_dir=/proj/cohenlab/projects/kki_adhd/data_BIDS/derivatives/fmriprep -output_suffix=clpipe_noscrub -log_output_dir=/proj/cohenlab/projects/kki_adhd/clpipe_batchOutput_GNGs -task=GNGs -tr=2.5 -single 0905"

Error (in batchOutput log):
Error: no such option: -t

Expanding off Issue #134

Convert2Bids should use a try/catch here to provide a more user-friendly error message:

Hi,

I have a question about the fmri_roi_extraction function. When I try to run it, it can't find the brain mask produced by fmriprep because it is searching for the wrong name.
Is there a possibility to change that name?

Thanks a lot for your help!
Best,
Antonia

UPDATE:
I found a work-around by renaming the brain mask files in the fmriprep-directory (deleting "space-MNI152NLin2009cAsym_").

I believe the log output folder paths that are set in the config file are not being used. I am testing several processing streams and the log directories were all the same even though I changed them in the config files. (As you know, I am using this at Duke so it might be a non-UNC issue.)

Hellow! I was wondering if there is information about the default size of the spheres of the atlases in the documentation? For example, I left the -radius tag out of my command line (see below) but did not specify a radius size in my configuration file (see ROI Extraction Option below). Everything ran fine but I am wondering what radius sphere size was used.

Command Line:
fmri_roi_extraction -config_file clpipe_config.json -task rest -atlas_name power -submit

Config File Section - ROI Extraction
"ROIExtractionOptions": {
"TargetDirectory": ".../data_postproc/postproc_default",
"TargetSuffix": "postproc_noScrubGSR.nii.gz",
"OutputDirectory": "..../postproc_noScrubGSR",
"Atlases": [
"power"
],
"RequireMask": true,
"PropVoxels": 0,
"MemoryUsage": "8000",
"TimeUsage": "2:0:0",
"NThreads": "1",
"LogDirectory": ".../logs/ROI_extraction_logs"
},

Thank you!

-Hannah Morgan

Hello, I am new to neuroimaging (and linux) and my colleague and I are trying to set up directories in the BIDS format through clpipe. When running the convert2bids command, we're encountering the following error: " 'NoneType' object is not subscriptable."

However, we have had success using the helper to identify image types from DICOMS; and we've subsequently been able to update the configuration file.

I'm not sure if it's related, but there also appeared to be a python error during installation of clpipe, though the requisite directories and files appeared as we expected. In using the suggested command from the warning message to upgrade the pip version (see image below), we are met with the exact same warning message about the pip versions. In other words, version 20.0.2 doesn't appear to upgrade to 21.0.1 using this command, and using the command to upgrade just repeats the warning message.

We would appreciate any help you can provide! Thank you very much in advance!

-Sam

Since some studies have extra time at the end of tasks after the task is over but the scanner is still running, it would be useful to be able to specify whether we want to cut off x number of volumes from the end of each scan.

Hey Teague, I haven't checked this myself, but I've heard that the highpass and lowpass filters in clpipe differ from what we have in ours.
Cohen Lab settings: "FilteringHighPass": 0.009, "FilteringLowPass": 0.08,
I was told that the Highpass filter was set at a default of 0.8. They might have meant Lowpass, but I'm not sure. I am reaching out to ask if the default settings need adjusting or what the reasoning is behind ours differing from the defaults.
Thanks! Mackenzie

Hi Teague! submitting this new issue since it's a separate topic

In the glm_setup section of our clpipe_glm_config.json, I wanted to clarify some questions about the Reference Image and the Brain Mask (with MaskFolderRoot and MaskSuffix). Our standard space is the MNIPediatricAsym from cohort 2. After looking through hng's image files in singularity_imgs/template_flow, I've only found one image with the "desc_fMRIPrep_boldref" suffix, and it's for MNI152NLin2009cAsym. Is this still the file we should be using as our "ReferenceImage", and then use each subject's newly-created brain mask file from data_fmriprep as our brain mask? After reading the documentation, it's still a little unclear to me if we can use the MNI152 as the reference image if it's determining our dimensions and resolution, since it's presumably a larger brain than the pediatric one.

(Also please let me know if I should be submitting more "user questions" like this in another place other than Issues!)

Hi there,

Apologies for another question but I am having trouble with the fsl_onset_extraction function in clpipe. I made a BIDS formatted events file (nback_events.tsv below) but I had some questions about the GLM Config File as well as the fsl_onset_extraction function.

1. When I try to run fsl_onset_extraction, I get an error saying "Error: no such option: -s" does this mean that submit should not be included? When I tried to run it without submit, nothing seemed to happen. Here is what I tried to run below:
   fsl_onset_extract -config_file /nas02/depts/hopfinger/hlmorgan/lumosity_output_2020/clpipe_config.json -glm_config_file /nas02/depts/hopfinger/hlmorgan/lumosity_output_2020/GLMConfigFile_1back.json -submit

2. Where should the event files be stored? I point the event file suffix to the correct document but I am confused about if it should be in the data_BIDS folder, fmri_prep folder, main folder etc.? Should it be in the target directory that I specified in the GLMSetupOptions portion of my config file?

3. For the TrialTypetoExtract, is how I did it below correct?

"Level1Onsets": {
"EventFileSuffix": "nback_events.tsv",
"TrialTypeVariable": "trial_type",
"TrialTypeToExtract": [
"on",
"off"
],
"TimeConversionFactor": 1,
"ParametricResponseVariable": "1",
"EVDirectory": "/nas02/depts/hopfinger/hlmorgan/lumosity_output_2020/data_glm_1back/level1_1back/onsets_1back"
},

Thank you in advance for any help!

-Hannah Morgan

If a file is already present, the conversion will fail, so if ses-1 is present, ses-2 will fail on a new conversion. Either add overwrite flag or try-catch statement

Add to documentation the list of required modules i.e. python/3.6.6, fsl, dcm2niix

It would be useful to be able to run the processing checker after every stage of data manipulation (i.e., after BIDS conversion, after fmriprep, after postprocessing, etc.)

project_setup will not run without the source_dir option, which is not listed as required in the documentation:

We should probably just change the click settings and documentation to indicate that this argument is required.

Hi Teague, I am putting in this issue to ask about the status of SUSAN Smoothing in the glm_setup function. Right now I think the script is setup to require one brightness and fwhm for all subjects, is this correct? Are there plans to implement a function more like SUSAN in FSL where it is adjusted per run?
Thanks! Mackenzie