pattio / deepswarm Goto Github PK
View Code? Open in Web Editor NEWNeural Architecture Search Powered by Swarm Intelligence ๐
License: MIT License
Neural Architecture Search Powered by Swarm Intelligence ๐
License: MIT License
Is there a place to set the learning rate?
Since it is not referenced, so I have this question in the paper.
If so that would be very amazing, the comparison between PSO and Bayesian Optimization has been validated in SigOpt's paper where two of them were very close instead of such a big gap in your paper. However, they didn't optimize topology but only the number of hidden units. I think maybe this is the main reason why NAS requires a different techniques than traditional ML where topology is hard-wired in the model itself.
def save_object(self, data, name):
"""Saves given object to the object backup directory.
Args:
data: object that needs to be saved.
name: string value representing the name of the object.
"""
with open(str(self.current_path / Storage.DIR["OBJECT"] / name), 'wb') as f:
pickle.dump(data, f, pickle.HIGHEST_PROTOCOL)
this yeilds a MemoryError when trying to do the pickle. It takes up at least 20GB of RAM. My model is fairly small. See log excerpt following:
-------------------------------NEW BEST ANT FOUND-------------------------------
---------------------------BEST ANT DURING ITERATION----------------------------
=======
Ant: 0x5eaa4358
Loss: 0.246397
Accuracy: 0.908695
Path: InputNode(shape:(256, 256, 1)) -> Conv2DNode(kernel_size:3, filter_count:64, activation:ReLU) -> FlattenNode() -> OutputNode(output_size:1, activation:Sigmoid)
Hash: 0f8bcb2b4fbd88fb3c5ffca7be08add054c8d35df1f122f4db0f043a91f7c901
=======
When I try to run the program I get following error:
ValueError Traceback (most recent call last)
in
2 import tensorflow as tf
3
----> 4 from deepswarm.backends import Dataset, TFKerasBackend
5 from deepswarm.deepswarm import DeepSwarm
6
~\AppData\Local\Programs\Python\Python38\Scripts\DeepSwarm-master\deepswarm\backends.py in
7
8 from abc import ABC, abstractmethod
----> 9 from sklearn.model_selection import train_test_split
10 from tensorflow.keras import backend as K
11
c:\users\user\appdata\local\programs\python\python38\lib\site-packages\sklearn_init_.py in
62 else:
63 from . import __check_build
---> 64 from .base import clone
65 from .utils._show_versions import show_versions
66
c:\users\user\appdata\local\programs\python\python38\lib\site-packages\sklearn\base.py in
12 from scipy import sparse
13 from .externals import six
---> 14 from .utils.fixes import signature
15 from .utils import _IS_32BIT
16 from . import version
c:\users\user\appdata\local\programs\python\python38\lib\site-packages\sklearn\utils_init_.py in
10 from scipy.sparse import issparse
11
---> 12 from .murmurhash import murmurhash3_32
13 from .class_weight import compute_class_weight, compute_sample_weight
14 from . import _joblib
init.pxd in init sklearn.utils.murmurhash()
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject
I use:
python 3.8
numpy 1.20.1
I would appreciate your help.
WindowsPath doesnt integrate nicely into open until python 3.6.
EDIT 1: you could also wrap the open(windowsPath) methods with open(str(windowsPath))
In evaluate_model, the code below can be used to return metrics which can only be computed on all of the data as opposed to averaged by batches as currently done. For simplicity, you can set numThreads and qSize to 1.
def evaluate(model, generator, steps, numThreads=2, qSize=5):
numItemsPushed_predict = 0
dataQueue = queue.Queue(maxsize=qSize)
mutex = threading.Lock()
def producer(steps):
nonlocal numItemsPushed_predict
killMe = False
while True:
mutex.acquire()
if numItemsPushed_predict < steps:
numItemsPushed_predict += 1
else:
killMe = True
myUid = numItemsPushed_predict
mutex.release()
if killMe:
return
#
x, y = generator.next(myUid-1)
dataQueue.put((x,y,myUid-1))
#
#
#
tVec = []
for k in range(numThreads):
t = threading.Thread(target=producer, args=(steps,))
t.daemon = True
t.start()
tVec.append(t)
resultVec = []
batchSize = None
pBar = tqdm.tqdm(range(steps), desc='EVALUATE')
for k in pBar:
currentQSize = dataQueue.qsize()
item = dataQueue.get()
x = item[0]
y = item[1]
uid = item[2] # For debug
if batchSize is None:
if type(x) is list:
batchSize = x[0].shape[0]
else:
batchSize = x.shape[0]
#
resultVec = np.zeros(steps)
r = model.evaluate(x, y, batch_size = batchSize, verbose=0)
resultVec[k] = r
#if type(y_pred) is list:
# predVec[k*batchSize : (k+1)*batchSize] = y_pred[0].flatten()
#else:
# predVec[k*batchSize : (k+1)*batchSize] = y_pred.flatten()
pBar.set_description('EVALUATE | QSize: {0}/{1}'.format(currentQSize, qSize))
#
return resultVec
evaluate_model(self, model) becomes:
y_true, y_pred = evaluate(model, data -- will have to convert from generator (easy), 1, 1)
loss = lossFunction(y_true, y_pred)
accuracy can be computed from sklearn.accuracy_score
You could also support losses like AUC now.
First up, thanks for this wonderfully structured code. Works perfectly with Tensorflow 1.13.1
I am currently working on adapting this for time series data for my use-case. In the backends.py there is a provision to extend this to other backends than TF. Is there a PyTorch implementation available for the same?
Hi, thanks for your awesome tool! It would be great if there were an option to train models for sequences or 1d inputs - will there be a future version with 1D CNN layer options? Thanks!
Got this error after running a bit more.
Stack:
File "D:\ASASINATR\projects\muscle_deep_swarm\trainer.py", line 332, in
main()
File "D:\ASASINATR\projects\muscle_deep_swarm\trainer.py", line 325, in main
topology = deepswarm.find_topology()
File "c:\python35\Lib\site-packages\deepswarm\deepswarm.py", line 43, in find_topology
best_ant = self.aco.search()
File "c:\python35\Lib\site-packages\deepswarm\aco.py", line 61, in search
self.storage.perform_backup()
File "c:\python35\Lib\site-packages\deepswarm\storage.py", line 67, in perform_backup
self.save_object(self.deepswarm, Storage.ITEM["BACKUP"])
File "c:\python35\Lib\site-packages\deepswarm\storage.py", line 213, in save_object
pickle.dump(data, f, pickle.HIGHEST_PROTOCOL)
File "c:\python35\Lib\site-packages\tensorflow\python\ops\resource_variable_ops.py", line 923, in reduce
return (ResourceVariable, (self.numpy(),))
File "c:\python35\Lib\site-packages\tensorflow\python\ops\resource_variable_ops.py", line 687, in numpy
"numpy() is only available when eager execution is enabled.")
builtins.NotImplementedError: numpy() is only available when eager execution is enabled.
I get the following error everytime after generating ant 1
builtins.ValueError: Tensor("Adam/iterations:0", shape=(), dtype=resource) must be from the same graph as Tensor("training/Adam/Const:0", shape=(), dtype=int64).
Prior output is:
---------------------------Current search depth is 1----------------------------
--------------------------------GENERATING ANT 1--------------------------------
Train on 5270 samples, validate on 586 samples
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.