xamyzhao / brainstorm Goto Github PK

Hello, I encountered the following problems while running the main.py. Do you know why？
2020-06-26 11:00:27.306985: F .\tensorflow/core/util/mkl_util.h:636] Check failed: dims == sizes.size() (5 vs. 4)

Hi,
Thank you for sharing the code.
I use your data in this github and use your default setting to run

python main.py trans --gpu 0 --data mri-100unlabeled --model flow-fwd
python main.py trans --gpu 0 --data mri-100unlabeled --model flow-bck

get the spatial transform models,and edit main.py then run

python main.py trans --gpu 0 --data mri-100unlabeled --model color-unet

get the appearence model

and then train the segmentation network

python main.py seg --gpu 0 --data mri-100unlabeled --aug_tm

have error messages:

Traceback (most recent call last):
  File "main.py", line 367, in <module>
    test_every_n_epochs=test_every_n_epochs)
  File "/home/aven/Research/brainstorm-master/src/experiment_engine.py", line 133, in run_experiment
    run_metadata=None,
  File "/home/aven/Research/brainstorm-master/src/experiment_engine.py", line 184, in train_batch_by_batch
    joint_loss, joint_loss_names = exp.train_on_batch()
  File "/home/aven/Research/brainstorm-master/src/segmenter_model.py", line 876, in train_on_batch
    self.X_train_batch, self.Y_train_batch, self.ul_train_ids = next(self.aug_train_gen)
  File "/home/aven/Research/brainstorm-master/src/segmenter_model.py", line 599, in _generate_augmented_batch
    colored_vol, color_delta, _ = self.color_aug_model.predict([source_X, X_colortgt_src, source_contours])
  File "/home/aven/anaconda3/envs/oneshot/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict
    check_batch_axis=False)
  File "/home/aven/anaconda3/envs/oneshot/lib/python3.6/site-packages/keras/engine/training.py", line 86, in _standardize_input_data
    str(len(data)) + ' arrays: ' + str(data)[:200] + '...')
ValueError: Error when checking model : the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 4 array(s), but instead got the following list of 3 arrays: [array([[[[[0.],
          [0.],
          [0.],
          ...,
          [0.],
          [0.],
          [0.]],

         [[0.],
          [0.],
          [0.],
          ...,
          [0.],
       ...

i think the error is in src/segmenter_model.py _generate_augmented_batch

colored_vol, color_delta, _ = self.color_aug_model.predict([source_X, X_colortgt_src, source_contours])

this function need 4 arrays but you only give source_X, X_colortgt_src, source_contours 3 arrays.

Is it your code have problem? or something wrong i have?
Hope u can help me to solve this problem.
My enviroment is:

OS: Ubuntu 16.04 x86_64
Python version: Python 3.6.8 :: Anaconda, Inc.
tensorflow-gpu version: 1.9.0
Keras version: 2.1.6

Hi,

Would you like you show the data list you used in your experiment, so that I can reproduce your results. I tried to download the eight datasets your mentioned, but I found they contains too many MRI images. Besides, in your experiments, only 250 MRI images were used, so would you like to list these 250 images names? Thank you very much.

@xamyzhao Hi, thanks for your excellent work. But when I run python main.py trans --gpu 1 --data mri-100unlabeled --model flow-bidir by default, there are following errors.

stdout, stderr:
b''
b'Format: "jpg" not recognized. Use one of: canon cmap cmapx cmapx_np dot dot_json eps fig gv imap imap_np ismap json json0 mp pdf pic plain plain-ext png pov ps ps2 svg svgz tk vml vmlz xdot xdot1.2 xdot1.4 xdot_json\n'

Traceback (most recent call last):
File "main.py", line 337, in
test_every_n_epochs=test_every_n_epochs)
File "/root/userfolder/workspace_python/brainstorm/src/experiment_engine.py", line 109, in run_experiment
exp.create_models()
File "/root/userfolder/workspace_python/brainstorm/src/transform_models.py", line 442, in create_models
super(TransformModelTrainer, self).create_models()
File "/root/userfolder/workspace_python/brainstorm/src/experiment_base.py", line 77, in create_models
self._print_models()
File "/root/userfolder/workspace_python/brainstorm/src/experiment_base.py", line 161, in _print_models
show_shapes=True)
File "/root/userfolder/anaconda3/lib/python3.6/site-packages/keras/utils/vis_utils.py", line 138, in plot_model
dot.write(to_file, format=extension)
File "/root/userfolder/anaconda3/lib/python3.6/site-packages/pydot.py", line 1817, in write
s = self.create(prog, format, encoding=encoding)
File "/root/userfolder/anaconda3/lib/python3.6/site-packages/pydot.py", line 1945, in create
assert process.returncode == 0, process.returncode
AssertionError: 1

Hi,Amy!
Thank you for your excellent work. Now I'm using your code to train a segmentation network. There are two questions I want to ask you. (1) During the segmentation training, the number of steps is set to 10000, and a Tesla V100 PCIe 32GB GPU card is used to execute" python main.py fss --gpu 0
-- data mri-100unlabeled -- aug_tm".This process takes about 57 hours, which is too long.So, what kind of GPU cards do you use for training? How long does 100000 steps take?
(2) whether the verification set provided by the project can evaluate the performance of the segmented network?Training the segmented network, setting 10000 steps, using the verification set test, the segmentation result of "- -aug _ sas" is slightly worse than that of the directly trained the segmentation network. In this case, I don't know whether it's a step number problem or a verification set problem?Can you provide a test set?

I have installed pynd==0.3.6，why report this error？please

I am not using voxelmorph labels for my mri dataset. I have removed the line "#'use_labels': voxelmorph_labels". then there is an error.

i have run this script.
python main.py seg --gpu 0 --data mri-100unlabeled --aug_tm

File "main.py", line 330, in
exp = segmenter_model.SegmenterTrainer(data_params, arch_params, debug=args.debug)
File "C:\Users\Administrator\Downloads\brainstorm-master\brainstorm-master\src\segmenter_model.py", line 144, in init
flow_epoch = int(int(re.search('(?<=_iter)[0-9]*', self.arch_params['tm_flow_model']).group(0)) / 100)
AttributeError: 'NoneType' object has no attribute 'group'

Please tell me, how to resolve this error.

Is there any way to get the label point of an mri image like voxelmorph?

Hi,

I am trying to use your code on a different dataset.
I have several segmented images that I would like to use for training.
I run the following command:
python3 main.py trans --gpu 1 --data mri-supervised --model flow-fwd
I have changed src/mri_loader.py to correctly load my data (I also had to change the vol_size in src/transform_models.py ).
I have slightly changed the 'mri-supervised' configuration:

    'mri-supervised': {  # supervised experiment
        'use_labels': my_labels,
        'use_atlas_as_source': False,
        'use_subjects_as_source': [],
        'do_load_test': False,
        'img_shape': (181, 217, 181, 1),
        'n_shot': 3, 
        'n_unlabeled': 0,
        'n_validation': 2,
        'do_preload_vols': True,
        'aug_in_gen': True,
        'n_tm_aug': None,
        'n_flow_aug': None,
        'warp_labels': True,
    },

Thus in my loaded dataset, I have:

vols_unlabeled_train.shape = (0, 181, 217, 181, 1)
vols_labeled_train.shape = (3, 181, 217, 181, 1)
vols_labeled_valid.shape = (2, 181, 217, 181, 1)

When I run the script, I have the following error:

[2019-05-07 14:38:31] TransformModel_mri_source_3l_0ul_flow_fwd_grad_l2-regfwt1_cc_vm-win9-wt1 training epoch 0/501
Traceback (most recent call last):
  File "main.py", line 358, in <module>
    test_every_n_epochs=test_every_n_epochs)
  File "/home/mansenca/VOLBRAIN/brainstorm_git/src/experiment_engine.py", line 133, in run_experiment
    run_metadata=None,
  File "/home/mansenca/VOLBRAIN/brainstorm_git/src/experiment_engine.py", line 222, in train_batch_by_batch
    if batch_count >= 10:
UnboundLocalError: local variable 'batch_count' referenced before assignment

(If I set batch_count to 0 outside the "for bi" loop, I then get the same kind of error for bi).

I suppose I am not using your code correctly.
Could you tell me where I am wrong ?

Besides could you tell me how long an epoch is supposed to take approximately ?

Thanks,
Boris.

Thank you very much for sharing the code.
I have many questions about evaluate_segmenters.py. Could you tell me more details about how to run the evaluate_segmenters.py? And I can't find the src/losses, so I import the voxelmorph package. However I find I cannot find the binary_dice. So how can I do?
And if I let the ds_key=mri-100unlabeled-test then will "IndexError: list index out of range"
If I let the ds_key=mri-100unlabeled then will "KeyError: 'n_test'"
Thanks for your help

Thank you very much for your excellent work. Now I am running your code. The sample data provided by you are used in the dataset. The trained models are all in the experience / folder. There are the following confusion points. Please help me to answer them: 1. Python main.py fss -- gpu 0 -- data mri-100unlabeled (’ fss’ should be changed to’ seg’). If the execution does not specify flags, but I specify space and appearance models in seg_model_arch_params, does the resulting segmentation network model use the
aug_tm method by default?
2. After completing the training of network segmentation, I tried to evaluate. I readed the code of evaluate_segmenters.py and found that there was no voxelmorph file. The loss.py file only had the project https://github.com/voxelmorph/voxelmorph. How can I solve this problem.

Hi, thanks for your excellent work! Since the repo is released recently, I wonder if it is complete and ready to run the code and train the model till now?

I would like to use your method to perform data augmentation on my dataset but I am running into a lot of errors due to package dependencies e.g pytools-lib is now called pystrum or the new version of neurite does not have the SpatialTransformer layer. What is the best approach to run your code?

I am able to run the train the forward transform model, however, I'm not able to understand the result after 100 epochs of training. It seems to me the transformed image looks the same as the original one, is it because I'm training too less epochs?

Thank you for your excellent work!
Now I want to have a try on my own dataset (2D gray CT image of lung). Can this transform model work on this type data?
If you can give me some advice, I'll be very appreciate:)

Thank you very much for your work! I am working on the generation of 2D heart slices. I plan to use your code "--sas" to generate data, but when visualizing the data, I found that the shape of the data is the shape of the source data. Could you tell me which step is set wrong?

Hi Amy, I think it would be good to clear state the dependencies: Python 3 (3.5+?), a number of python packages, such as $ pip install opencv-python nibabel tqdm pydot, and GraphViz software

can you tell me how to solve this question????
AttributeError: 'ResourceSummaryWriter' object has no attribute 'add_summary'

Just question about the appearance transformer.

As shown in the paper, the intensity variance of brain T1 MR images seems to be caused by no histogram matching. Could traditional pre-processing overcomes the mentioned appearance variance, especially of brain MR image? What kind of specific appearance variance could the appearance transformer solve?

Best,
Dongming

hello xamyzhao,if I just want to get the augmented data sets,what operations should I take?

Can you please share the freesurfer commands to do the Preprocessing told in the paper?

Reference from the paper : "we resample the brains to 256 × 256 × 256 with 1mm isotropic voxels,
and affinely align and crop the images to 160 × 192 × 224. We do not apply any intensity corrections, and we perform
skull-stripping by zeroing out voxels with no anatomical label".

Hi,
Thank you for sharing the code.
I run

python main.py seg --gpu 0 --data mri-100unlabeled --aug_sas

has a error message:

Traceback (most recent call last):
  File "main.py", line 367, in <module>
    test_every_n_epochs=test_every_n_epochs)
  File "/home/aven/Research/brainstorm-master/src/experiment_engine.py", line 121, in run_experiment
    exp.create_generators(batch_size=run_args.batch_size)
  File "/home/aven/Research/brainstorm-master/src/segmenter_model.py", line 252, in create_generators
    self._create_augmented_examples()
  File "/home/aven/Research/brainstorm-master/src/segmenter_model.py", line 496, in _create_augmented_examples
    aug_im = utils.concatenate_with_pad([
AttributeError: module 'src.utils' has no attribute 'concatenate_with_pad'

After reading the code，u don,t have definition concatenate_with_pad in src/utils.py.
Is it your code have problem? or something wrong i have?
I try my best to ask in English.Hope u can know what i mean.

My enviroment is:

OS: Ubuntu 16.04 x86_64
Python version: Python 3.6.8 :: Anaconda, Inc.
tensorflow-gpu version: 1.9.0
Keras version: 2.1.6

In the 35nd line of the main.py, there is a key "use_subjects_as_source" whose content is ['OASIS_OAS1_0327_MR1_mri_talairach_orig'], I have searched all the files and can't find this file. I'm not sure if this is a MRI data or not, and I don't understand what it is used for, please help me, thank you very much !

Thanks for sharing a good job!

I have a question about network architecture. As I understanding, given an input image I, you first feed it to the appearance network to change the intensity. Then feed the result of the appearance network to the registration network to register it with atlas.

Do you try to investigate another direction likes, first you feed the network to registration network, then the moved image feeds to the appearance network? Which one is better?

Thank you very much for your experiments and new ideas. So far, I have successfully run your code and successfully trained to get the output image, but I want to use the newly generated image. How do I save the output image separately? Instead of just saving the long picture in figures

Hi,
Thank you for sharing the code.
I run

python main.py trans --gpu 0 --data mri-100unlabeled --model flow-fwd
python main.py trans --gpu 0 --data mri-100unlabeled --model flow-bck

to get the spatial transform models, edit main.py then run

python main.py trans --gpu 0 --data mri-100unlabeled --model color-unet

get the appearance/color transform model.
I edit the code at line 265:276, and then train the segmentations network,

python main.py fss --gpu 0 --data mri-100unlabeled --aug_tm

but get Graph disconnected error.

After reading the code, I find that may be the problem in segmenter_model.py, line 417

self.color_aug_model = Model(
                        inputs=self.color_aug_model.inputs[:-1],
                        outputs=[
                            self.color_aug_model.outputs[0],
                            self.color_aug_model.get_layer('add_color_delta').output,
                            self.color_aug_model.outputs[2]],
                        name='color_model_wrapper')

here the input flow_srctotgt is dropped, but the outputs[0], which is transformed_out, still connect to flow_srctotgt by transformed_out = SpatialTransformer(indexing='xy')([transformed_out, flow_srctotgt]) in line 292 at file networks.py. So the program crashed.

According to line 576 in file segmenter_model.py,

color_delta, colored_vol, _ = self.color_aug_model.predict([source_X, X_colortgt_src, source_contours])

Line 417 in file segmenter_model.py should be

self.color_aug_model = Model(
                        inputs=self.color_aug_model.inputs[:-1],
                        outputs=[
                            self.color_aug_model.outputs[1], # edit 0 to 1
                            self.color_aug_model.get_layer('add_color_delta').output,
                            self.color_aug_model.outputs[2]],
                        name='color_model_wrapper')

After change code here, I can train the segment network.

So, is this the correct way to fix that problem?
Thanks.:thumbsup:
####
My enviroment is:

OS: Ubuntu 18.10 x86_64
Python version: Python 3.6.7 :: Anaconda, Inc.
tensorflow-gpu version: 1.12.0
Keras version: 2.2.4

Hi, thank you for your excellent work. I have a parameter that is not very clear. Does the parameter

do_preload_vols = True

mean that I will use the source as the target? And if I use different source and target set, should I set it to false?

The GPU is "GeForce GTX 1080 Ti" got the out-of-memory (OOM) error running:

$ python3 main.py trans --gpu 0 --data mri-100unlabeled --model flow-fwd

Error:

OOM when allocating tensor with shape[1,16,160,192,224] and type float on
/job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc

When I was trying to reproduce the project, I have come across the bug 'AttributeError: 'InputLayer' object has no attribute 'outbound_nodes' . I use the tensorflow-gpu 1.13.1 and keras with version 2.2.4.

This bug is possible to be triggered by the version of keras, could the authors publish the environment that you used in a requirement.txt? 👍

Hi,

Thank you for the great work. I have a question about whether this data augmentation method could be applied to natural medical images such as images that are taken by a endoscope? Thank you!

First thanks for your contribution. In your introduction, you said you use a GPU with 12 GB, but I run the program in a GPU with 24 GB and use nearly 18 GB. I dont know why, can you help me?
And the GPU Fan mabye is broken, but it is not the reason.

Hello! Thank you very much for sharing the code!
I have a question for you, in networks.py ,''from ext.neuron.neuron.layers import SpatialTransformer'', I didn't find the 'SpatialTransformer' in the layers.py.
Is there a problem with the file I downloaded? Can you share me the correct layers.py file? Thank you very much!

[2023-04-14 22:46:13] TransformModel_mri_source-atlas_0l_100ul_flow_fwd_grad_l2-regfwt1_cc_vm-win9-wt1 training epoch 0/501
Traceback (most recent call last):
File "main.py", line 343, in
test_every_n_epochs=test_every_n_epochs)
File "C:\Users\Administrator\Downloads\brainstorm-master\brainstorm-master\src\experiment_engine.py", line 131, in run_experiment
run_args=run_args,
File "C:\Users\Administrator\Downloads\brainstorm-master\brainstorm-master\src\experiment_engine.py", line 213, in train_batch_by_batch
if batch_count >= 10:
UnboundLocalError: local variable 'batch_count' referenced before assignment

Code is working properly for size 256,256,46. When i changed the files with size 256,256,19 then the above error is coming.
any idea?

Hi, thanks for your excellent work. But when I run python main.py trans --gpu 0 --data mri-100unlabeled --model flow-fwd by default, there are following errors.
ValueError: Error when checking target: expected flow to have shape (None, 160, 192, 224, 3) but got array with shape (1, 160, 192, 224, 1)

In the first example:

python main.py trans --gpu 0 --data mri-100-csts2 --model flow-bds

where is the model flow-bds? If there is no, where can I find a substitute?
Thanks

I compressed my nii files into an npz file and placed them in the train folder.
I didn't even touch the validation folder and the following error occurs.

When performing a warp operation on a label, does the label require a one-hot form? If not, will you get any other values other than ground truth?

Hello! I have tried a long time but I still can't figure out this problem. It seems like I have some problems with my dataset. I use standard OASIS dataset, and I convert the 'nii.gz' files to '.npz' files myself. And I also modify the key value of the npz files. Could you give me any help?

'flow-fwd' can learn spatial transform,and 'color-unet' learn appearence transform, what about 'flow-bck' , 'flow-bidir '? And what's the different between these models?