yaroslavvb / chain_constant_memory Goto Github PK

View Code? Open in Web Editor NEW

42.0 42.0 13.0 28 KB

Example of backprop which uses constant memory

Python 100.00%

chain_constant_memory's People

Contributors

Stargazers

Watchers

Forkers

fatmas1982 leotam qingsong99 hunslater-deeplearning kvechera stjordanis mrojasabregu shawwn fagan2888 psobot rvanw fantasyrain shadowkun

chain_constant_memory's Issues

Error running mem_util_test.py

Hi,

Windows 7
Python 3.5

Python crushes when runs mem_util_test.py

First the script writes:

2018-02-19 18:31:16.886501: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to u
se: AVX
2018-02-19 18:31:17.002507: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1105] Found device 0 with properties:
name: GeForce GTX 660 major: 3 minor: 0 memoryClockRate(GHz): 1.0975
pciBusID: 0000:01:00.0
totalMemory: 2.00GiB freeMemory: 1.75GiB
2018-02-19 18:31:17.003507: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX
660, pci bus id: 0000:01:00.0, compute capability: 3.0)
2018-02-19 18:31:17.393530: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\common_runtime\gpu\gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX
660, pci bus id: 0000:01:00.0, compute capability: 3.0)
2018-02-19 18:31:17.401530: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\stream_executor\dso_loader.cc:129] Couldn't open CUDA library cupti64_90.dll
2018-02-19 18:31:17.402530: F C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\35\tensorflow/stream_executor/lib/statusor.h:212] Non-OK-status: status_ status: Failed precondition: could not dlopen DSO: cupti64_90
.dll; dlerror: cupti64_90.dll not found

Then Python crushes with this error message:

Problem signature:
  Problem Event Name:	APPCRASH
  Application Name:	python.exe
  Application Version:	3.5.4150.1013
  Application Timestamp:	59891fd7
  Fault Module Name:	ucrtbase.DLL
  Fault Module Version:	10.0.10586.1171
  Fault Module Timestamp:	59ae5010
  Exception Code:	40000015
  Exception Offset:	00000000000699df
  OS Version:	6.1.7601.2.1.0.768.3
  Locale ID:	2057
  Additional Information 1:	fd1b
  Additional Information 2:	fd1b385075e60b88ead45d673b86dde4
  Additional Information 3:	970e
  Additional Information 4:	970e86229f1ab91001ad8d3665413eaf

Read our privacy statement online:
  http://go.microsoft.com/fwlink/?linkid=104288&clcid=0x0409

If the online privacy statement is not available, please read our privacy statement offline:
  C:\Windows\system32\en-US\erofflps.txt

Linearize creating cyclic graph.

I tried to use the lineare.linearize() function, but it created cyclic graph.
I tried that to optimize CycleGAN, which has lots of gradients.
Without the usage of lineare.linearize() everything works, but after adding lineare.linearize() after the graph building, the sess.run(tf.global_variables_initializer()) fails with tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph is invalid, contains a cycle with 107 nodes, including: Adam/real-anime_gen/Select/e, Adam/real-anime_gen/Select, Adam/real-anime_gen/learning_rate.
But the linearize() function itself worked and did not detect any cycles.
models.zip
I am adding original graph and linearized version as .pb files in zip (in this example generators and discriminators are only dummy one-layer networks to simplify the problem) and also html visualization of linearized graph (based on your visualizer from https://github.com/yaroslavvb/stuff/blob/master/simple_rewiring.ipynb which is really nice btw, thanks for that).

It creates cycles between individual generators, apparently it does not handle well the cycle loss.

I also modified the end of the linearize function and added

toposort(copy_graph(get_graph(tf.get_default_graph())))
toposort(copy_graph(reversed_graph(get_graph(tf.get_default_graph()))))

before the return statement. In case of cyclic graph, at least one of them should throw the CircularDependencyError, right?
But no error was thrown there and still the cycle remains and tensorflow complains and throws exception.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.