Cascading multi-tissue volume fraction maps issues about dmipy HOT 5 OPEN

Hey @villalonreina ,

Thanks for the feedback!
Can you perhaps give me a minimum working script that illustrates the bug? It's a bit hard to debug just from the description;-)

from dmipy.

Hi @rutgerfick, after further considerations this issue has nothing to do with normalization of the DWIs. Please just scratch it.
To spell out our model for the cortical GM, here is a step-by-step description:
Step 1: 2 compartment SMT model Stick+Zeppelin with tortuosity constraint and a fixed parDiff for stick=zep (Note: ICVF outputs for GM S0 vs non-multi-tissue version are very similar and very smooth)

from dmipy.signal_models import cylinder_models, gaussian_models
from dmipy.core.modeling_framework import MultiCompartmentSphericalMeanModel

stick = cylinder_models.C1Stick()
zeppelin = gaussian_models.G2Zeppelin()

mc_mdi_model = MultiCompartmentSphericalMeanModel(models=[stick, zeppelin],
                                                  S0_tissue_responses=[S0_gm, S0_gm])[
](url)mc_mdi_model.set_tortuous_parameter('G2Zeppelin_1_lambda_perp',
                                    'C1Stick_1_lambda_par',
                                    'partial_volume_0',
                                    'partial_volume_1')
mc_mdi_model.set_equal_parameter('C1Stick_1_lambda_par',
                                 'G2Zeppelin_1_lambda_par')
mc_mdi_model.set_fixed_parameter('C1Stick_1_lambda_par', 1.1e-9)

Step 2: we cascade the ICVF into a 3 compartment model Stick+Zep+Ball without tortuosity and no equality constraint

from dmipy.signal_models import cylinder_models, gaussian_models
from dmipy.core import modeling_framework

stick = cylinder_models.C1Stick()
zeppelin = gaussian_models.G2Zeppelin()
ball = gaussian_models.G1Ball()

mcmdi_mod = modeling_framework.MultiCompartmentSphericalMeanModel(
    models=[stick, zeppelin, ball], S0_tissue_responses=[S0_gm, S0_gm, S0_csf])
mcmdi_mod.set_fixed_parameter('G1Ball_1_lambda_iso', 3e-9)
mcmdi_mod.set_fractional_parameter('G2Zeppelin_1_lambda_perp',
                                   'G2Zeppelin_1_lambda_par')
mcmdi_mod.set_fixed_parameter('C1Stick_1_lambda_par', 1.1e-9)

The non multi-tissue ICVF for step 2 is very smooth and quite similar to step 1 (as expected since it was an initial guess). However, the MT version is very patchy and pixelated. Moreover, when we switched from the older Dmipy version 0.1dev to the newer 1.0.3, there is no difference in the non-MT version but a HUGE difference in the MT outputs (see screenshots below):


Thanks so much for helping us.

from dmipy.

Thanks for the detailed descriptions!

I am surprised by these differences between versions, I am not sure yet how to explain it.
What do you call "huge" differences? As in is the ICVF completely of a different order?
Do the other parameter maps also look different?
I agree it looks more noisy for the 1.0.3 version, even though it seems most of the noise is appears to be on the boundaries between tissue and water.

Can you also show the error maps of the two versions?
from the fittted model you can call fitted_model.mean_squared_error(self, data).
Is the error really high for one version over the other?

Does this look like something you recognize @matteofrigo ?
If it's still unexplained, maybe you can send us only this slice of the data so we can see if the problem is reproducible here...

from dmipy.

@villalonreina please correct me if I'm wrong. You get the ICVF from the single-tissue Stick+Zeppelin (SZ) model of step1, and you fix the computed ICVF right into the multi-tissue Stick+Zeppelin+Ball (SZB) model of step2. At this point you fit the SZB model and you look again at the ICVF. I guess that what you expect is to see the same VF that you computed from the SZ model, both for the single- and the multi-tissue ICVF.

I'm actually ok with having the gray spots in the white matter, since the ICVF is lower where the fibers are expected to cross. For this reason I would blame the fact that the defined SZB model describes a parallel-fibers configuration. Another thing that I would keep in mind is that when the MT formulation is used in a model that includes a CSF compartment, the differences between the ICVF and the CSFVF are brutally amplified with respect to the single-tissue case. That's the reason why I there's a bigger contrast in the MT image than in the single tissue one.

About the pixelated pattern, there are two distinct problems:

1. The difference between the versions.
2. The presence of almost-random values on the exterior layer of the gray matter.

For the version problem, it would help a lot to get the exact commit of the two specific versions you are comparing, or a way to install them on our machines (you can also send us the source directly).

As far as the random-looking values are concerned, let's start by considering that the SZ and the SZB models are both designed to describe the white matter, hence anything that we get outside of it must be taken with extra caution. In particular, the interface between the GM and the CSF is a very difficult terrain for the models. If I understood correctly what you are doing, you are interested at what happens in the GM, and you may want to visualize only the results there (set ICVF=0 inside the WM and the CSF masks). This could sensibly improve the contrast.

Another thing I would check is the ECVF, which could be stealing some space to the CSF (or vice-versa). In regions where the directional component is not predominant, the absence of the tortuosity constraint could make the zeppelin look like a ball, making it impossible to distinguish one from the other and inducing degeneracy in the fitting. The huge difference between the S0 of the two compartments only increases this effect.

I'll try to reproduce the issue on some data that I have on my PC. In the meantime, as @rutgerfick said, I think we should start by looking at the error maps.

from dmipy.

@matteofrigo @rutgerfick @TMNir With regards to our expectations as you said we expected the ICVF from the SZ model to be similar to the ICVF of SZB model, especially since it was cascaded and in both steps the grey matter (GM) S0 was used. In addition we expected in step 1 (SZ model) the single-tissue S0_GM ICVF to look different than the standard ICVF.

Question 2. Random GM values:
To your point about the dark patches in the WM where fibers cross, that makes sense for the Corona Radiata, however we also see very bright patches in the GM (see below)

As you mentioned we are masking the GM for our analyses, but it's still patchy. Also, we are not sure about the Zeppelin looking like a CSF ball since we fixed them with different diffusivity values and used the fractional parameter constraint for the Zeppelin. To us the ECVF and ISOVF are less patchy. We also tried the single-tissue SZB model with S0_GM as a sanity check and found, as you predicted, that without S0_CSF the output is closer to the standard (no crazy white patches). At this point I think we could really use your advice on how to use the multi-tissue approach to fit the three compartments =/

Question 1. Differences between versions:
It's difficult to quantify how different the images are between the versions, but you can see the SZB multi-tissue ICVF histograms below. The other multi-tissue maps are also different, interestingly the standard non multi-tissue outputs are not (not sure if that is a clue):

In terms of the MSE between the two dmipy versions, the newer 1.0.3 has a higher error than the 0.1dev. See below:

This is the detailed information about the versions we have tried. We don't have the exact commit IDs though.
The older version was downloaded from github on December 4th, 2019. And installed with
pip install -e /path/to/package
When retrieving the version in the code it says Version: 0.1.dev0. I have the source folder and I could share it with you.
The newer version was installed on the lab’s system by admin on Oct 19th, 2020 using pip (see below the installation output). Can you know the exact commit of the master branch just from the date?

(base) [root@c2004 ~]# . /usr/local/pyenv-bins/.pyenv-bashrc-usrlocal;
(base) [root@c2004 ~]# pyenv activate miniconda3-latest;
(base) [root@c2004 ~]# conda env list | egrep diffusion_std
diffusion_std            /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std
(base) [root@c2004 ~]# conda activate diffusion_std;
(diffusion_std) [root@c2004 ~]# which python
/usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/bin/python
(diffusion_std) [root@c2004 ~]# which python3
/usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/bin/python3
(diffusion_std) [root@c2004 ~]# python3 -m pip install dmipy
Collecting dmipy
 Downloading dmipy-1.0.3-py3-none-any.whl (28.0 MB)
    |████████████████████████████████| 28.0 MB 10.5 MB/s
Requirement already satisfied: dipy in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from dmipy) (1.2.0)
Requirement already satisfied: scipy in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from dmipy) (1.5.2)
Collecting pathlib
 Downloading pathlib-1.0.1.tar.gz (49 kB)
    |████████████████████████████████| 49 kB 1.5 MB/s
Requirement already satisfied: boto in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from dmipy) (2.49.0)
Requirement already satisfied: numpy>=1.13 in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from dmipy) (1.19.1)
Requirement already satisfied: cvxpy in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from dmipy) (1.1.3)
Requirement already satisfied: nibabel>=3.0.0 in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from dipy->dmipy) (3.1.1)
Requirement already satisfied: h5py>=2.5.0 in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from dipy->dmipy) (2.10.0)
Requirement already satisfied: osqp>=0.4.1 in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from cvxpy->dmipy) (0.6.1)
Requirement already satisfied: ecos>=2 in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from cvxpy->dmipy) (2.0.7.post1)
Requirement already satisfied: scs>=1.1.3 in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from cvxpy->dmipy) (2.1.2)
Requirement already satisfied: packaging>=14.3 in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from nibabel>=3.0.0->dipy->dmipy) (20.4)
Requirement already satisfied: six in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from h5py>=2.5.0->dipy->dmipy) (1.15.0)
Requirement already satisfied: future in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from osqp>=0.4.1->cvxpy->dmipy) (0.18.2)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/pyenvs/versions/miniconda3-latest/envs/diffusion_std/lib/python3.7/site-packages (from packaging>=14.3->nibabel>=3.0.0->dipy->dmipy) (2.4.7)
Building wheels for collected packages: pathlib
 Building wheel for pathlib (setup.py) ... done
 Created wheel for pathlib: filename=pathlib-1.0.1-py3-none-any.whl size=14348 sha256=33707d3fba151bd288e73a3f55c46638d590a3ed2c0421174d19d66a27490e2a
 Stored in directory: /root/.cache/pip/wheels/6e/96/b8/10037fe231e23970bac58361d7c93571ab983a7bbc55e68550
Successfully built pathlib
Installing collected packages: pathlib, dmipy
Successfully installed dmipy-1.0.3 pathlib-1.0.1
(diffusion_std) [root@c2004 ~]#

from dmipy.