gmontamat / gentun Goto Github PK

Hyperparameter tuning for machine learning models using a distributed genetic algorithm

License: Apache License 2.0

Python 100.00%

convolutional-neural-networks cross-validation distributed-algorithm distributed-genetic-algorithm gene-encoding genetic-algorithm genetic-algorithms grid-search hyperparameter-optimization hyperparameter-tuning keras machine-learning master-worker rabbitmq xgboost

gentun's People

Contributors

Stargazers

Watchers

Forkers

colinsongf msshazlynaszemi sushantjha8 mepland hashemadl lokiluciferase ahmadmobeen charantezy rashmiar laokpa amimul uelideschwert mountains-high mahi818 jennifer19 jedrzejpolaczek easy-forks sigireddybalasai

gentun's Issues

Add github actions job to publish to DockerHub

Trouble while installing package

I use GoogleColab
First i installed using this command :

%%bash
git clone https://github.com/gmontamat/gentun
cd gentun
python setup.py install

then i imported package using this command

from gentun import GeneticAlgorithm, Population, XgboostIndividual

i face this error

ImportError                               Traceback (most recent call last)
<ipython-input-2-a67f03c5af3f> in <module>()
      1 from sklearn.datasets import fetch_california_housing
----> 2 from gentun import GeneticAlgorithm, Population, XgboostIndividual

ImportError: cannot import name 'GeneticAlgorithm' from 'gentun' (unknown location)

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

after excuting this command output is

import gentun
dir(gentun)

['__doc__',
 '__file__',
 '__loader__',
 '__name__',
 '__package__',
 '__path__',
 '__spec__']

Add a method to validate result produced by workers

The distributed version of the algorithm receives results from the workers without validating their work. This is safe when one has control of all the instances running the worker nodes but it is dangerous when one shares the queue of jobs on the internet.
When opening the genetic algorithm to the world, a malicious user may return a fake metric claiming to have trained the model using k-fold cross validation, corrupting the overall result of the parameter tuning.
Some options to prevent this from happening are to:

Send the workers a random dev set after they finish evaluating an individual and ask them to make predictions using the model trained
Use different workers at random to evaluate the same individual and cross-check their results
Find a proof of work like the time it used to run, maybe ask for partial results
Blockchain? (j/k, no way!)

lack of mnist.py

Hi, in this test file,

import mnist

but no module named mnist.

I guess probably that some directory or files named as mnist.py is missing?

Which version of RabbitMQ is compatible with this code?

Hi, thanks for the code.
I am trying to run the distributed training however, I have got below error:

Using TensorFlow backend.
Initializing a random population. Size: 20
Starting genetic algorithm...

Evaluating generation #1...
Traceback (most recent call last):
  File "/media/vip/Program/mobeen/gentun/tests/mnist_server.py", line 23, in <module>
    ga.run(50)
  File "/media/vip/Program/mobeen/gentun/gentun/algorithms.py", line 29, in run
    self.evolve_population()
  File "/media/vip/Program/mobeen/gentun/gentun/algorithms.py", line 72, in evolve_population
    fittest = self.population.get_fittest()
  File "/media/vip/Program/mobeen/gentun/gentun/server.py", line 105, in get_fittest
    self.evaluate_in_parallel()
  File "/media/vip/Program/mobeen/gentun/gentun/server.py", line 113, in evaluate_in_parallel
    RpcClient(None, None, **self.credentials).purge()
  File "/media/vip/Program/mobeen/gentun/gentun/server.py", line 31, in __init__
    result = self.channel.queue_declare(exclusive=True)
TypeError: queue_declare() missing 1 required positional argument: 'queue'

Process finished with exit code 1

I solved this error as suggested here giving an empty queue.
After this I got another error as below:

Using TensorFlow backend.
Initializing a random population. Size: 20
Starting genetic algorithm...

Evaluating generation #1...
Traceback (most recent call last):
  File "/media/vip/Program/mobeen/gentun/tests/mnist_server.py", line 23, in <module>
    ga.run(50)
  File "/media/vip/Program/mobeen/gentun/gentun/algorithms.py", line 29, in run
    self.evolve_population()
  File "/media/vip/Program/mobeen/gentun/gentun/algorithms.py", line 72, in evolve_population
    fittest = self.population.get_fittest()
  File "/media/vip/Program/mobeen/gentun/gentun/server.py", line 105, in get_fittest
    self.evaluate_in_parallel()
  File "/media/vip/Program/mobeen/gentun/gentun/server.py", line 113, in evaluate_in_parallel
    RpcClient(None, None, **self.credentials).purge()
  File "/media/vip/Program/mobeen/gentun/gentun/server.py", line 33, in __init__
    self.channel.basic_consume(self.on_response, no_ack=True, queue=self.callback_queue)
TypeError: basic_consume() got an unexpected keyword argument 'no_ack'

Process finished with exit code 1

By the feel of these errors I think I am using some other version of RabbitMQ.
I installed RabbitMQ using:
sudo apt-get install rabbitmq-server

Is it a version mismatch or am I missing something?

Publish as pip package

Publish this library in PyPI.

Connection failure on second generation

@ahmadmobeen I was able to replicate your problem, thanks for reporting it. It's due to pika updating its API. Please use pika==1.1.0 (which should be compatible with your rabbitmq version) and the dev branch code of client.py and server.py. I will merge with master sometime later.

edit: I'm also working on the dockerfile for the gentun server to simplify running the distributed version of the genetic algorithm.

I tried this method and it worked for generation # 1.

@gmontamat Thank you for your reply. I was able to run the code by doing some small changes in server.py and client.py as shown here
I tried this and it worked for generation # 1.

got the following error in both cases:


Using TensorFlow backend.
Initializing a random population. Size: 20
Starting genetic algorithm...

Evaluating generation #1...
 [*] Got fitness for individual 0
 [*] Got fitness for individual 2
 [*] Got fitness for individual 1
 [*] Got fitness for individual 4
 [*] Got fitness for individual 3
 [*] Got fitness for individual 6
 [*] Got fitness for individual 8
 [*] Got fitness for individual 10
 [*] Got fitness for individual 5
 [*] Got fitness for individual 9
 [*] Got fitness for individual 12
 [*] Got fitness for individual 11
 [*] Got fitness for individual 7
 [*] Got fitness for individual 13
 [*] Got fitness for individual 16
 [*] Got fitness for individual 15
 [*] Got fitness for individual 17
 [*] Got fitness for individual 14
 [*] Got fitness for individual 19
 [*] Got fitness for individual 18
Fittest individual is:
{'S_1': '011', 'S_2': '1000010100'}
Fitness value is: 0.9978

Evaluating generation #2...
Traceback (most recent call last):
  File "/media/vip/Program/mobeen/gentun/tests/mnist_server.py", line 23, in <module>
    ga.run(50)
  File "/media/vip/Program/mobeen/gentun/gentun/algorithms.py", line 29, in run
    self.evolve_population()
  File "/media/vip/Program/mobeen/gentun/gentun/algorithms.py", line 72, in evolve_population
    fittest = self.population.get_fittest()
  File "/media/vip/Program/mobeen/gentun/gentun/server.py", line 105, in get_fittest
    self.evaluate_in_parallel()
  File "/media/vip/Program/mobeen/gentun/gentun/server.py", line 113, in evaluate_in_parallel
    RpcClient(None, None, **self.credentials).purge()
  File "/media/vip/Program/mobeen/gentun/gentun/server.py", line 28, in __init__
    self.connection = pika.BlockingConnection(self.parameters)
  File "/home/vip/anaconda3/envs/keras/lib/python3.7/site-packages/pika-1.1.0-py3.7.egg/pika/adapters/blocking_connection.py", line 359, in __init__
    self._impl = self._create_connection(parameters, _impl_class)
  File "/home/vip/anaconda3/envs/keras/lib/python3.7/site-packages/pika-1.1.0-py3.7.egg/pika/adapters/blocking_connection.py", line 450, in _create_connection
    raise self._reap_last_connection_workflow_error(error)
pika.exceptions.ProbableAuthenticationError: ConnectionClosedByBroker: (403) 'ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.'

Process finished with exit code 1

Originally posted by @ahmadmobeen in #17 (comment)