znclub-pa-ml-ai / octopy-predictor Goto Github PK

Generic platform for Prediction using basic machine learning models

Python 13.80% Jupyter Notebook 86.20%

machine-learning machine-learning-as-a-service

octopy-predictor's Introduction

                    ___                                      
                   (   )                                     
  .--.     .--.     | |_       .--.        .-..    ___  ___  
 /    \   /    \   (   __)    /    \      /    \  (   )(   ) 
|  .-. ; |  .-. ;   | |      |  .-. ;    ' .-,  ;  | |  | |  
| |  | | |  |(___)  | | ___  | |  | |    | |  . |  | |  | |  
| |  | | |  |       | |(   ) | |  | |    | |  | |  | '  | |  
| |  | | |  | ___   | | | |  | |  | |    | |  | |  '  `-' |  
| '  | | |  '(   )  | ' | |  | '  | |    | |  ' |   `.__. |  
'  `-' / '  `-' |   ' `-' ;  '  `-' /    | `-'  '   ___ | |  
 `.__.'   `.__,'     `.__.    `.__.'     | \__.'   (   )' |  
                                         | |        ; `-' '  
                                        (___)        .__.'

Generic platform for Prediction using basic machine learning models

Status

branch	status
master
datagatherer
Architecture/Core-Design-loosely-coupled-classes
develop

Index

Structure
Components
Architecture
Technology Stack
Setup
Input File Constraints
Algorithms supported

Structure

data: All data to be stored/ loaded
demo
- bokeh_demo: Visualizer Component Demo - to be moved out
- flask_demo: WebInterface Demo - to be moved out
src: All working component files
tests

Components

Architecture

Setup

setup conda env

conda env list # show current environments
conda create --name OctoPy # if OctoPy is NOT listed
conda list # show all libraries
conda install pip # if pip is NOT listed
conda list > versions.txt # store all versions post install
pip install -r requirements.txt # install from requirements
conda list > versions.txt # store all versions post install

Input File Constraints

Algorithms supported

Support Vector Machine

TODO

NameError: name 'fetch_all_features' is not defined :: currently converted to self.method_name(). Need to learn about static methods in Python Class
Give options as REGRESSION / CLASSIFICATION. Else regression data fails for classification model

SETUP/ CONFIGURATIONS

GENERAL

Setup path variables

Windows : double check to make sure you see python in your PATH. You can find your path by opening your control panel -> System and Security -> System -> Advanced System Settings -> Environment Variables -> Selecting Path -> Edit ->

Find programs running on port number

netstat -a -o -n | findstr port_number

Kill task using PID

taskkill /F /PID 6136

FLASK

python app.py

octopy-predictor's People

Contributors

Stargazers

Watchers

octopy-predictor's Issues

Python circular dependency

Different modules are:

context : application level objects
util : application utility methods
config : configuration file dictionary, resides in context

I require config values inside util. Hence I have used

in context.py
context.VARIABLE = config.VARIABLE
in util.py

import context
def util():
   use context.VARIABLE

Circular dependency is created while loading any app_module:

web_runner depends on context
context depends on WebInterface
WebInterface depends on util
util depends on context
breaks

File not accepted while using Restlet Client

File upload using Restlet Client is not accepted by server, however it's accepted when using a web browserhttp://localhost:port_no/data-file

Flask: logging pattern, Query parameters, Environment variables

Enhancements

Logger class to enable patterns and customizations
Query parameters - test with different patterns
Environment variables - read from environment to make it generic

File upload tests via HTML/ REST Client

File upload successful via HTML ui
File upload throws error via REST client

Cannot read from file without saving on disk

#read from upload File viz FileStorage object

def _read_from_file(file):
    _file_content = None
    try:
        _file_content = file.read()
        util.debug_store['file_content at datagatherer'] = _file_content
    except IOError as io_error:
        util.debug_store['io_error at datagatherer'] = io_error.__traceback__
        raise io_error
    else:
        return _file_content

Core: Design loosely coupled classes

Revisit current coupling of classes
List down dependency graph (find automated solution)
Finalize I/O for each classs
Remove redundant

Gatherer: CSV File to dataframe

read file from temporary path
read file from bytes

controller set repeating

controller set is printed for debugging proper init of context methods.
currently, the debug statement is printed twice.

Create Test Suite with multiple Test classes

TypeError: TestCases and TestSuites must be instantiated before passing them to addTest()

Pytest local run failure

On Travis:
`
�[0K$ pytest
�[1m============================= test session starts ==============================�[0m
platform linux -- Python 3.8.0, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /home/travis/build/ZNClub-PA-ML-AI/OctoPy-Predictor, inifile: pytest.ini, testpaths: octopy_predictor/tests
�[1mcollecting ... �[0m�[1m
collecting 3 items �[0m�[1m
collected 4 items �[0m

octopy_predictor/tests/test_analyser.py �[32m.�[0m�[32m.�[0m�[36m [ 50%]�[0m
octopy_predictor/tests/test_datagatherer.py �[32m.�[0m�[32m.�[0m�[36m [100%]�[0m

�[32m�[1m============================== 4 passed in 0.53s ===============================�[0m
travis_time:end:05c48c34:start=1580630442976354918,finish=1580630443794582636,duration=818227718,event=script
�[0K�[32;1mThe command "pytest" exited with 0.�[0m
`

On local:
`
$ pytest
======================================================= test session starts ========================================================
platform linux -- Python 3.7.3, pytest-4.3.1, py-1.8.0, pluggy-0.9.0
rootdir: /home/znevzz/Desktop/Projects/ZNevzz/OctoPy-Predictor, inifile: pytest.ini
plugins: remotedata-0.3.1, openfiles-0.3.2, doctestplus-0.3.0, arraydiff-0.3
collected 0 items / 2 errors

============================================================== ERRORS ==============================================================
_____________________________________ ERROR collecting octopy_predictor/tests/test_analyser.py _____________________________________
ImportError while importing test module '/home/znevzz/Desktop/Projects/ZNevzz/OctoPy-Predictor/octopy_predictor/tests/test_analyser.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
octopy_predictor/tests/test_analyser.py:3: in
from octopy_predictor.src.analyser import Analyser, AnalyserMetricsRegistry, REGRESSION_MODEL
E ModuleNotFoundError: No module named 'octopy_predictor'
___________________________________ ERROR collecting octopy_predictor/tests/test_datagatherer.py ___________________________________
ImportError while importing test module '/home/znevzz/Desktop/Projects/ZNevzz/OctoPy-Predictor/octopy_predictor/tests/test_datagatherer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
octopy_predictor/tests/test_datagatherer.py:6: in
from octopy_predictor.src.datagatherer import *
E ModuleNotFoundError: No module named 'octopy_predictor'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
===================================================== 2 error in 0.41 seconds ======================================================

Setter not working

@values.getter
not working in DataGathererInput

Gatherer: SQL to dataframe

create connection with database
execute query and return a dataframe
support relational database: sqllite3