Error: Run on GPU about nomeroff-net HOT 7 CLOSED

Hi,
Pay attention to

Registered devices: [CPU, XLA_CPU]

you have to use tensorflow-gpu to work on your GPU device.

Please run this code to make sure
#29 (comment)

from nomeroff-net.

I am get:

/home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/ubuntu/.local/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING: Logging before flag parsing goes to stderr.
W0729 12:36:46.215900 140500138620736 deprecation_wrapper.py:119] From 1.py:2: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

W0729 12:36:46.216178 140500138620736 deprecation_wrapper.py:119] From 1.py:2: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

2019-07-29 12:36:46.232007: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-07-29 12:36:46.237208: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2095145000 Hz
2019-07-29 12:36:46.238391: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x17fb770 executing computations on platform Host. Devices:
2019-07-29 12:36:46.238431: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): ,
Device mapping:
/job:localhost/replica:0/task:0/device:XLA_CPU:0 -> device: XLA_CPU device
2019-07-29 12:36:46.239695: I tensorflow/core/common_runtime/direct_session.cc:296] Device mapping:
/job:localhost/replica:0/task:0/device:XLA_CPU:0 -> device: XLA_CPU device

MatMul: (MatMul): /job:localhost/replica:0/task:0/device:CPU:0
2019-07-29 12:36:46.243862: I tensorflow/core/common_runtime/placer.cc:54] MatMul: (MatMul)/job:localhost/replica:0/task:0/device:CPU:0
a: (Const): /job:localhost/replica:0/task:0/device:CPU:0
2019-07-29 12:36:46.243895: I tensorflow/core/common_runtime/placer.cc:54] a: (Const)/job:localhost/replica:0/task:0/device:CPU:0
b: (Const): /job:localhost/replica:0/task:0/device:CPU:0
2019-07-29 12:36:46.243914: I tensorflow/core/common_runtime/placer.cc:54] b: (Const)/job:localhost/replica:0/task:0/device:CPU:0
2019-07-29 12:36:46.244432: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
[[22. 28.]
[49. 64.]]

from nomeroff-net.

Yes, you use CPU version tensorflow.

If your device support GPU follow this guide
https://www.tensorflow.org/install/gpu

from nomeroff-net.

Thank's. I was able to run on GPU.
I have 16 VRAM. I want to run in three threads. But when I run one thread, it uses all the memory.
I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 15190 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:06.0, compute capability: 6.0)
How can I limit the amount of VRAM per thread?

from nomeroff-net.

I get a out of memory when I run the second thread:

2019-07-29 15:14:46.026772: I tensorflow/core/common_runtime/bfc_allocator.cc:800] InUse at 0x7f5a8a927d00 next 439 of size 1024
2019-07-29 15:14:46.026781: I tensorflow/core/common_runtime/bfc_allocator.cc:800] InUse at 0x7f5a8a928100 next 440 of size 1024
2019-07-29 15:14:46.026790: I tensorflow/core/common_runtime/bfc_allocator.cc:800] InUse at 0x7f5a8a928500 next 441 of size 1024
2019-07-29 15:14:46.026800: I tensorflow/core/common_runtime/bfc_allocator.cc:800] InUse at 0x7f5a8a928900 next 442 of size 4096
2019-07-29 15:14:46.026809: I tensorflow/core/common_runtime/bfc_allocator.cc:800] InUse at 0x7f5a8a929900 next 443 of size 4096
2019-07-29 15:14:46.026819: I tensorflow/core/common_runtime/bfc_allocator.cc:800] InUse at 0x7f5a8a92a900 next 444 of size 589824
2019-07-29 15:14:46.026829: I tensorflow/core/common_runtime/bfc_allocator.cc:800] InUse at 0x7f5a8a9ba900 next 445 of size 589824
2019-07-29 15:14:46.026838: I tensorflow/core/common_runtime/bfc_allocator.cc:800] InUse at 0x7f5a8aa4a900 next 446 of size 589824
2019-07-29 15:14:46.026849: I tensorflow/core/common_runtime/bfc_allocator.cc:800] InUse at 0x7f5a8aada900 next 18446744073709551615 of size 1070848
2019-07-29 15:14:46.026860: I tensorflow/core/common_runtime/bfc_allocator.cc:809] Summary of in-use Chunks by size:
2019-07-29 15:14:46.026873: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 83 Chunks of size 256 totalling 20.8KiB
2019-07-29 15:14:46.026886: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 39 Chunks of size 512 totalling 19.5KiB
2019-07-29 15:14:46.026897: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 136 Chunks of size 1024 totalling 136.0KiB
2019-07-29 15:14:46.026907: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 1 Chunks of size 1280 totalling 1.2KiB
2019-07-29 15:14:46.026918: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 52 Chunks of size 2048 totalling 104.0KiB
2019-07-29 15:14:46.026929: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 68 Chunks of size 4096 totalling 272.0KiB
2019-07-29 15:14:46.026940: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 21 Chunks of size 8192 totalling 168.0KiB
2019-07-29 15:14:46.026951: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 1 Chunks of size 12288 totalling 12.0KiB
2019-07-29 15:14:46.026962: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 1 Chunks of size 16384 totalling 16.0KiB
2019-07-29 15:14:46.026973: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 1 Chunks of size 24576 totalling 24.0KiB
2019-07-29 15:14:46.026983: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 1 Chunks of size 37632 totalling 36.8KiB
2019-07-29 15:14:46.026995: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 2 Chunks of size 65536 totalling 128.0KiB
2019-07-29 15:14:46.027005: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 7 Chunks of size 262144 totalling 1.75MiB
2019-07-29 15:14:46.027016: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 3 Chunks of size 524288 totalling 1.50MiB
2019-07-29 15:14:46.027029: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 3 Chunks of size 589824 totalling 1.69MiB
2019-07-29 15:14:46.027041: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 23 Chunks of size 1048576 totalling 23.00MiB
2019-07-29 15:14:46.027055: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 1 Chunks of size 1070848 totalling 1.02MiB
2019-07-29 15:14:46.027066: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 1 Chunks of size 2097152 totalling 2.00MiB
2019-07-29 15:14:46.027077: I tensorflow/core/common_runtime/bfc_allocator.cc:812] 3 Chunks of size 4194304 totalling 12.00MiB
2019-07-29 15:14:46.027087: I tensorflow/core/common_runtime/bfc_allocator.cc:816] Sum Total of in-use chunks: 43.88MiB
2019-07-29 15:14:46.027102: I tensorflow/core/common_runtime/bfc_allocator.cc:818] total_region_allocated_bytes_: 46006272 memory_limit_: 46006272 available bytes: 0 curr_region_allocation_bytes_: 92012544
2019-07-29 15:14:46.027116: I tensorflow/core/common_runtime/bfc_allocator.cc:824] Stats:
Limit: 46006272
InUse: 46006272
MaxInUse: 46006272
NumAllocs: 447
MaxAllocSize: 4194304

2019-07-29 15:14:46.027167: W tensorflow/core/common_runtime/bfc_allocator.cc:319] ***************************************************************************************************x
2019-07-29 15:14:46.027209: W tensorflow/core/framework/op_kernel.cc:1502] OP_REQUIRES failed at assign_op.h:117 : Resource exhausted: OOM when allocating tensor with shape[1024] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc

from nomeroff-net.

use this code:

import keras
import tensorflow as tf


config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction = (1 / 3)
config.gpu_options.allow_growth = True
keras.backend.set_session(tf.Session(config=config))

where 3 it ts your count of threads

from nomeroff-net.

Thank's it is working!

from nomeroff-net.