Giter VIP home page Giter VIP logo

ista-net's Introduction

ISTA-Net

Implementing ISTA-Net, an interpretable optimization-inspired deep network for image compressive sensing as proposed by Zhang and Ghanem in this paper.

@inproceedings{zhang2018ista,
  title={ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing},
  author={Zhang, Jian and Ghanem, Bernard},
  booktitle={CVPR},
  pages={1828--1837},
  year={2018}
}

How to run

  • Download the dataset Training_Data_Img91.mat from this link, place it in the current directory and then run ista_net.py. This file contains detailed comments.
  • Interactive training and experimentation was done in Google Colab, and the Jupyter notebook has been uploaded in the current directory as ista_net_jupyter.ipynb. This file does not contain any comments, and was used by me for iterating over diffferent models quickly. If Github doesn't preview this file for you, please view it here.
  • Both these files contain the same code. I recommend reading both files, one for a detailed thought process, and the other to see running outputs line by line.

Observations and assumptions

  • The number of parameters mentioned in the paper is = 336,978. That is for Nb = 9. For Nb = 5 (as implemented in their Github code), number of parameters = (336,978x5)/9 = 187,210, consistent with my Keras implementation.
  • I didn't have the processing power to train the model for 300 epochs, so I ran the training for 25 epochs and compared the results against the first 25 epochs of the code provided by the authors. Their implementation seemed to converge faster, but suffered from irregular gradients. My model converged slower, but had a gentle cost gradient. These epochs are detailed in the Jupyter notebook, and the authors' observations are well-described in their paper.

Model visualisation

Author's visualisation:

Author's visualisation

Visualisation of my implementation:

ISTA-Net model

ista-net's People

Contributors

hansinahuja avatar

Stargazers

avatar avatar avatar avatar

Watchers

avatar avatar

ista-net's Issues

Expected float32 passed to parameter 'y' of op 'Equal', got 'accuracy' of type 'str' instead.

Thanks for your code! When I trained ista_net.py, I met an error:

batch_size=64)
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\keras\engine\training.py", line 66, in _method_wrapper
return method(self, *args, **kwargs)
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\keras\engine\training.py", line 848, in fit
tmp_logs = train_function(iterator)
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\eager\def_function.py", line 580, in call
result = self._call(*args, **kwds)
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\eager\def_function.py", line 627, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\eager\def_function.py", line 506, in _initialize
*args, **kwds))
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\eager\function.py", line 2446, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\eager\function.py", line 2777, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\eager\function.py", line 2667, in _create_graph_function
capture_by_value=self._capture_by_value),
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\framework\func_graph.py", line 981, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\eager\def_function.py", line 441, in wrapped_fn
return weak_wrapped_fn().wrapped(*args, **kwds)
File "D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\framework\func_graph.py", line 968, in wrapper
raise e.ag_error_metadata.to_exception(e)
TypeError: in user code:

D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\keras\engine\training.py:571 train_function  *
    outputs = self.distribute_strategy.run(
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:951 run  **
    return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2290 call_for_each_replica
    return self._call_for_each_replica(fn, args, kwargs)
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2649 _call_for_each_replica
    return fn(*args, **kwargs)
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\keras\engine\training.py:543 train_step  **
    self.compiled_metrics.update_state(y, y_pred, sample_weight)
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\keras\engine\compile_utils.py:391 update_state
    self._build(y_pred, y_true)
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\keras\engine\compile_utils.py:322 _build
    self._metrics, y_true, y_pred)
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\util\nest.py:1118 map_structure_up_to
    **kwargs)
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\util\nest.py:1214 map_structure_with_tuple_paths_up_to
    *flat_value_lists)]
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\util\nest.py:1213 <listcomp>
    results = [func(*args, **kwargs) for args in zip(flat_path_list,
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\util\nest.py:1116 <lambda>
    lambda _, *values: func(*values),  # Discards the path arg.
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\keras\engine\compile_utils.py:421 _get_metric_objects
    return [self._get_metric_object(m, y_t, y_p) for m in metrics]
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\keras\engine\compile_utils.py:421 <listcomp>
    return [self._get_metric_object(m, y_t, y_p) for m in metrics]
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\keras\engine\compile_utils.py:439 _get_metric_object
    if metric not in ['accuracy', 'acc', 'crossentropy', 'ce']:
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\ops\math_ops.py:1491 tensor_equals
    return gen_math_ops.equal(self, other, incompatible_shape_error=False)
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\ops\gen_math_ops.py:3224 equal
    name=name)
D:\anaconda\install\envs\tf2.2gpunew\lib\site-packages\tensorflow\python\framework\op_def_library.py:479 _apply_op_helper
    repr(values), type(values).__name__, err))

TypeError: Expected float32 passed to parameter 'y' of op 'Equal', got 'accuracy' of type 'str' instead. Error: Expected float32, got 'accuracy' of type 'str' instead.

I think the reason that the error happened is our tensorflows do not match. I will be grateful if you tell me your tensorflow version. Or there exsits other problems I have not realized, Can you give me some advices?

Thank you very much!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.