dmpelt / msdnet Goto Github PK

Dear author:
Thanks for sharing the code. This work is excellent. I would like to ask why the gradient is not subtracted in the function "updategradients_internal" of the update parameters? And how does the 'lib' function work in operations.py?

Is your feature request related to a problem? Please describe.
It is currently not possible to use a loss function other than MSE for training or validation.

Describe the solution you'd like
I think that two solutions could work
(i) Implementation of more loss functions
(ii) A simple guide on how users could write their own loss functions

There are pros and cons to each. Esentially flexibility vs off-the-shelf usability.

Describe alternatives you've considered
n/a

Additional context

We have been testing a number of networks for noise removal from microscopy images. One of our early findings is that if one uses a self similarity metric (SSIM) for network training then the result is a network that performs better on all metrics. I.e. a network trained with SSIM loss obtains a better MSE than a network trained with MSE loss. Therefore, we find that SSIM is the best loss functions for training our denoising networks.

We found the MSD performs almost as well as U-Net for denoising, despite the fact U-Net was trained with SSIM loss and MSD trained with MSE loss. So it is possible MSD could push even further.

I also note that in cases where we used very few training examples MSD was the best performing network.

Describe the bug
Example 01: Train a network for regression and Example 05: Train a network for regression (tomography) fail with
AttributeError: 'NoneType' object has no attribute 'deriv'

It looks like loss in train.pyis never instantiated and remains None, leading to a null pointer exception-like error (I am coming from a Java background).

L66
def __init__(self, network, loss = None, a = 0.001, b1 = 0.9, b2 = 0.999, e = 10**-8):
L73
self.loss = loss
L85
err = self.loss.deriv(out, tar) <- this line throws the NoneType error

To Reproduce
Steps to reproduce the behavior:

1. Set up msdnet in a new conda environment

conda config --add channels conda-forge
conda create --name tomo msdnet
conda activate tomo

3. Install dependencies
4. Run Supplementary 02: Generate data for tomography examples
5. Run :~$ python3 train_regr_tomography.py
6. See error:

Traceback (most recent call last):
  File "/home/mdoube/train_regr_tomography.py", line 93, in <module>
    msdnet.train.train(n, t, val, bprov, 'tomo_regr_params.h5',loggers=[consolelog,filelog,imagelog], val_every=len(datsv))
  File "/home/mdoube/miniconda3/envs/tomo/lib/python3.10/site-packages/msdnet/train.py", line 187, in train
    trainalg.step(network, batch)
  File "/home/mdoube/miniconda3/envs/tomo/lib/python3.10/site-packages/msdnet/train.py", line 85, in step
    err = self.loss.deriv(out, tar)
AttributeError: 'NoneType' object has no attribute 'deriv'

Expected behavior
Script completes and training continues until it is stopped manually

Desktop (please complete the following information):

• Dell 7920 workstation with NVIDIA RTX A4000 GPU
• OS: Ubuntu 20.04
• CUDA 11.8.0
• NVIDIA driver 525.85.12
• Miniconda 23.1.0
• Python 3.10.8
• ASTRA 2.1.0
• tomopy 1.12.2

Additional context
tomopy and ASTRA are working properly. This demo completed successfully.

Describe the bug
A clear and concise description of what the bug is.
So, I have tried to run the example code for train_regr (Example 2) on a GPU enabled computer and my laptop, but whenever I run it, it produces the error:

To Reproduce
I have not edited any of the code and am running the code on Jupyter Lab. The same error appeared when I tried running the code on Jupyter Notebook as well. I downloaded the MS-D Net line and I believe I installed the correct cuda-toolkit version.

To get this error, I ran generatedata, and then ran train_regr. The error popped up after running train_regr.

Desktop:

• OS: Windows 10

I have tried tracing the error back to the train functions but there does not seem to be anything wrong with them. Again, I have not made any edits to the code yet, so I am not sure why it is not running.

PR #14 removed openmp support for windows because of build problems. Hopefully it is possible to re-enable in the future, but I need to figure out what the underlying cause is first...

Hi,

i have a few questions about the normalization in Example 01: train a network for regression.
a. Is it correct that only the training data is normalized in this example?, i would expect that the validation- & test-data should be normalized as well?
b. how is the normalization performed exactly, do you normalize the noisy & noiseless (training) data separately (calculating and applying mean & std of both sets separately), or do you calculate a single mean & std-value that is applied to both?

many thanks in advance,
best regards,
Rolf

Describe the bug
Not enough arguments for Train_regr

To Reproduce
So, through a previous issue, I was able to run all of the non-tomography examples. However, my python version was too high to install astra-toolkit. Next, I tried many different options for allowing myself to download astra, but it always created incompatibility errors with CudaToolkit. So, I came back to the GitHub page to see if there were similar issues

I lowered my python version to 3.6 and created a new environment using code in a previous argument:
conda create -n msdnettest -c conda-forge python=3.6 msdnet

However, when I activated the new environment in anaconda and attempted to run train_regr in Jupyter Lab, I received an error:
TypeError: not enough arguments for format string

Screenshots

Desktop (please complete the following information):

• OS: Windows 10

Seems related to openmp...

Describe the bug
The version of msdnet on conda-forge still 1.0.0, but without loss.py. So train examples didn't work.

https://stats.stackexchange.com/a/336077/293880

Describe the bug
When trying to install with conda there are python package conflicts (my python is 3.5.4) also when tried to install over clean conda environment (python 3.5.4)
There is solution for that?

Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies. Conda may not use the correct pip to install your packages, and they may end up in the wrong place. Please add an explicit pip dependency. I'm adding one for you, but still nagging you.
Collecting package metadata (repodata.json): done
Solving environment: failed

UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:

• astra-toolbox==1.8 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0']
• joblib==0.11 -> python[version='2.7.,3.5.,3.6.*']
• keras==2.2.0 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0']
• msdnet -> python[version='>=3.6,<3.7.0a0,>=3.7,<3.8.0a0']
• numba==0.37.0 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0']
• numpy==1.13.1 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0']
• pandas==0.23.4 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0']
• pywin32==220 -> python[version='2.7.,3.4.,3.5.,3.6.']
• scikit-image==0.13.0 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0']
• scikit-learn==0.20.0 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0']
• scipy==0.19.1 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0']
• semantic_version==2.6.0 -> python[version='2.7.,3.4.,3.5.,3.6.,>=3.7,<3.8.0a0']
• theano==0.9.0 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0']
• tqdm==4.31.1 -> python[version='>=3.6,<3.7.0a0,>=3.7,<3.8.0a0']

If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Your current python version
is (python=3.5). Note that conda will not change your python version to a different minor version
unless you explicitly specify that.

The following specifications were found to be incompatible with each other:

Package setuptools conflicts for:
pywin32==220 -> python[version='2.7.,3.4.,3.5.,3.6.'] -> pip -> wheel -> setuptools
scikit-learn==0.20.0 -> scipy -> numpy[version='1.11.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip -> wheel -> setuptools
joblib==0.11 -> python[version='2.7.,3.5.,3.6.'] -> pip -> wheel -> setuptools
scikit-image==0.13.0 -> scipy[version='>=0.17'] -> numpy[version='1.11.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel -> setuptools
semantic_version==2.6.0 -> python[version='2.7.,3.4.,3.5.,3.6.,>=3.7,<3.8.0a0'] -> pip -> wheel -> setuptools
tqdm==4.31.1 -> python[version='>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip -> wheel -> setuptools
theano==0.9.0 -> scipy[version='>=0.14,>=0.14.0'] -> numpy[version='1.10.,1.11.,1.9.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel -> setuptools
numba==0.37.0 -> numpy[version='>=1.11.3,<2.0a0,>=1.13,<1.14.0a0,>=1.14,<1.15.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel -> setuptools
python=3.5 -> pip -> wheel -> setuptools
pandas==0.23.4 -> numpy[version='>=1.11.,>=1.11.3,<2.0a0,>=1.12.1,<2.0a0,>=1.14.6,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip -> wheel -> setuptools
keras==2.2.0 -> keras-base=2.2.0 -> keras-preprocessing==1.0.1 -> scipy[version='>=0.14'] -> numpy[version='1.10.,1.11.,1.9.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip -> wheel -> setuptools
numpy==1.13.1 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel -> setuptools
astra-toolbox==1.8 -> scipy -> numpy[version='1.10.,1.11.,1.9.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel -> setuptools
scipy==0.19.1 -> numpy[version='1.11.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel -> setuptools
msdnet -> scikit-image -> scipy[version='>=0.17'] -> numpy[version='>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip -> wheel -> setuptools
plotly==3.2.1 -> requests -> urllib3[version='>=1.21.1,<1.22,>=1.21.1,<1.23,>=1.21.1,<1.24,>=1.21.1,<1.25'] -> pyopenssl[version='>=0.14'] -> cryptography[version='>=1.3,>=1.9,>=2.1.4,>=2.2.1'] -> cffi[version='>=1.4.1,>=1.7'] -> pycparser -> python[version='3.5.,>=3.5,<3.6.0a0'] -> pip -> wheel -> setuptools
pip -> wheel -> setuptools
Package pip conflicts for:
numba==0.37.0 -> numpy[version='>=1.11.3,<2.0a0,>=1.13,<1.14.0a0,>=1.14,<1.15.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip
astra-toolbox==1.8 -> scipy -> numpy[version='1.10.,1.11.,1.9.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip
semantic_version==2.6.0 -> python[version='2.7.,3.4.,3.5.,3.6.,>=3.7,<3.8.0a0'] -> pip
scikit-learn==0.20.0 -> scipy -> numpy[version='1.11.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip
msdnet -> scikit-image -> scipy[version='>=0.17'] -> numpy[version='>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip
joblib==0.11 -> python[version='2.7.,3.5.,3.6.'] -> pip
pywin32==220 -> python[version='2.7.,3.4.,3.5.,3.6.'] -> pip
scipy==0.19.1 -> numpy[version='1.11.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip
numpy==1.13.1 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip
scikit-image==0.13.0 -> scipy[version='>=0.17'] -> numpy[version='1.11.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip
keras==2.2.0 -> keras-base=2.2.0 -> keras-preprocessing==1.0.1 -> scipy[version='>=0.14'] -> numpy[version='1.10.,1.11.,1.9.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip
python=3.5 -> pip
tqdm==4.31.1 -> python[version='>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip
plotly==3.2.1 -> requests -> urllib3[version='>=1.21.1,<1.22,>=1.21.1,<1.23,>=1.21.1,<1.24,>=1.21.1,<1.25'] -> pyopenssl[version='>=0.14'] -> cryptography[version='>=1.3,>=1.9,>=2.1.4,>=2.2.1'] -> cffi[version='>=1.4.1,>=1.7'] -> pycparser -> python[version='3.5.,>=3.5,<3.6.0a0'] -> pip
pandas==0.23.4 -> numpy[version='>=1.11.,>=1.11.3,<2.0a0,>=1.12.1,<2.0a0,>=1.14.6,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip
theano==0.9.0 -> scipy[version='>=0.14,>=0.14.0'] -> numpy[version='1.10.,1.11.,1.9.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip
Package six conflicts for:
msdnet -> scikit-image -> matplotlib[version='>=1.3.1,>=2.0.0'] -> python-dateutil -> six[version='>=1.5']
keras==2.2.0 -> keras-base=2.2.0 -> keras-preprocessing==1.0.1 -> six[version='>=1.9.0']
numba==0.37.0 -> singledispatch -> six
scikit-image==0.13.0 -> matplotlib[version='>=1.3.1,>=2.0.0'] -> python-dateutil -> six[version='>=1.5']
theano==0.9.0 -> pygpu[version='>=0.6.5,>=0.6.5,<0.7'] -> libgpuarray[version='0.6.8,0.6.9,0.7.0,0.7.1,0.7.3'] -> six
pandas==0.23.4 -> python-dateutil[version='>=2.5.'] -> six[version='>=1.5']
astra-toolbox==1.8 -> six
plotly==3.2.1 -> requests -> urllib3[version='>=1.21.1,<1.22,>=1.21.1,<1.23,>=1.21.1,<1.24,>=1.21.1,<1.25'] -> pyopenssl[version='>=0.14'] -> cryptography[version='>=1.3,>=1.9,>=2.1.4,>=2.2.1'] -> six[version='>=1.4.1']
Package llvmlite conflicts for:
msdnet -> numba[version='>=0.41'] -> llvmlite[version='>=0.27.0,<0.28.0a0,>=0.28.0,<0.29.0a0,>=0.29.0,<0.30.0a0']
numba==0.37.0 -> llvmlite=0.22
Package wheel conflicts for:
keras==2.2.0 -> keras-base=2.2.0 -> keras-preprocessing==1.0.1 -> scipy[version='>=0.14'] -> numpy[version='1.10.,1.11.,1.9.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip -> wheel
python=3.5 -> pip -> wheel
numba==0.37.0 -> numpy[version='>=1.11.3,<2.0a0,>=1.13,<1.14.0a0,>=1.14,<1.15.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel
astra-toolbox==1.8 -> scipy -> numpy[version='1.10.,1.11.,1.9.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel
joblib==0.11 -> python[version='2.7.,3.5.,3.6.'] -> pip -> wheel
tqdm==4.31.1 -> python[version='>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip -> wheel
pandas==0.23.4 -> numpy[version='>=1.11.,>=1.11.3,<2.0a0,>=1.12.1,<2.0a0,>=1.14.6,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip -> wheel
pywin32==220 -> python[version='2.7.,3.4.,3.5.,3.6.'] -> pip -> wheel
scikit-image==0.13.0 -> scipy[version='>=0.17'] -> numpy[version='1.11.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel
scikit-learn==0.20.0 -> scipy -> numpy[version='1.11.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip -> wheel
pip -> wheel
semantic_version==2.6.0 -> python[version='2.7.,3.4.,3.5.,3.6.,>=3.7,<3.8.0a0'] -> pip -> wheel
numpy==1.13.1 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel
plotly==3.2.1 -> requests -> urllib3[version='>=1.21.1,<1.22,>=1.21.1,<1.23,>=1.21.1,<1.24,>=1.21.1,<1.25'] -> pyopenssl[version='>=0.14'] -> cryptography[version='>=1.3,>=1.9,>=2.1.4,>=2.2.1'] -> cffi[version='>=1.4.1,>=1.7'] -> pycparser -> python[version='3.5.,>=3.5,<3.6.0a0'] -> pip -> wheel
msdnet -> scikit-image -> scipy[version='>=0.17'] -> numpy[version='>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=3.6,<3.7.0a0,>=3.7,<3.8.0a0'] -> pip -> wheel
theano==0.9.0 -> scipy[version='>=0.14,>=0.14.0'] -> numpy[version='1.10.,1.11.,1.9.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel
scipy==0.19.1 -> numpy[version='1.11.,>=1.11.3,<2.0a0'] -> mkl_random -> numpy-base[version='>=1.0.2,<2.0a0'] -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'] -> pip -> wheel
Package psutil conflicts for:
scikit-image==0.13.0 -> dask[version='>=0.5'] -> distributed[version='>=1.19.0,>=1.20.0,>=1.21.0,>=1.23.2,>=1.28.0'] -> psutil
msdnet -> scikit-image -> dask[version='>=0.5'] -> distributed[version='>=1.19.0,>=1.20.0,>=1.21.0,>=1.23.2,>=1.28.0'] -> psutil
Package numba conflicts for:
msdnet -> numba[version='>=0.41']