Giter VIP home page Giter VIP logo

pymilo's Introduction



Codecov PyPI version built with Python3 Discord Channel

Overview

PyMilo is an open source Python package that provides a simple, efficient, and safe way for users to export pre-trained machine learning models in a transparent way. By this, the exported model can be used in other environments, transferred across different platforms, and shared with others. PyMilo allows the users to export the models that are trained using popular Python libraries like scikit-learn, and then use them in deployment environments, or share them without exposing the underlying code or dependencies. The transparency of the exported models ensures reliability and safety for the end users, as it eliminates the risks of binary or pickle formats.

PyPI Counter
Github Stars
Branch main dev
CI
Code Quality CodeFactor codebeat badge

Installation

PyPI

Source code

Conda

Usage

Imagine you want to train a LinearRegression model representing this equation: $y = x_0 + 2x_1 + 3$. You will create data points (X, y) and train your model as follows.

>>> import numpy as np
>>> from sklearn.linear_model import LinearRegression
>>> X = np.array([[1, 1], [1, 2], [2, 2], [2, 3]])
>>> y = np.dot(X, np.array([1, 2])) + 3
 # y = 1 * x_0 + 2 * x_1 + 3
>>> model = LinearRegression().fit(X, y)
>>> pred = model.predict(np.array([[3, 5]]))
# pred = [16.] (=1 * 3 + 2 * 5 + 3)

Using PyMilo Export class you can easily serialize and export your trained model into a JSON file.

>>> from pymilo import Export
>>> Export(model).save("model.json")

You can check out your model as a JSON file now.

{
    "data": {
        "fit_intercept": true,
        "copy_X": true,
        "n_jobs": null,
        "positive": false,
        "n_features_in_": 2,
        "coef_": {
            "pymiloed-ndarray-list": [
                1.0000000000000002,
                1.9999999999999991
            ],
            "pymiloed-ndarray-dtype": "float64",
            "pymiloed-ndarray-shape": [
                2
            ],
            "pymiloed-data-structure": "numpy.ndarray"
        },
        "rank_": 2,
        "singular_": {
            "pymiloed-ndarray-list": [
                1.618033988749895,
                0.6180339887498948
            ],
            "pymiloed-ndarray-dtype": "float64",
            "pymiloed-ndarray-shape": [
                2
            ],
            "pymiloed-data-structure": "numpy.ndarray"
        },
        "intercept_": {
            "value": 3.0000000000000018,
            "np-type": "numpy.float64"
        }
    },
    "sklearn_version": "1.4.2",
    "pymilo_version": "0.8",
    "model_type": "LinearRegression"
}

You can see all the learned parameters of the model in this file and change them if you want. This JSON representation is a transparent version of your model.

Now let's load it back. You can do it easily by using PyMilo Import class.

>>> from pymilo import Import
>>> model = Import("model.json").to_model()
>>> pred = model.predict(np.array([[3, 5]]))
# pred = [16.] (=1 * 3 + 2 * 5 + 3)

This loaded model is exactly the same as the original trained model.

Supported ML models

scikit-learn PyTorch
Linear Models ✅ -
Neural networks ✅ -
Trees ✅ -
Clustering ✅ -
Naïve Bayes ✅ -
Support vector machines (SVMs) ✅ -
Nearest Neighbors ✅ -
Ensemble Models ✅ -
Pipeline Model ✅ -
Preprocessing Models ✅ -

Details are available in Supported Models.

Issues & bug reports

Just fill an issue and describe it. We'll check it ASAP! or send an email to [email protected].

  • Please complete the issue template

You can also join our discord server

Discord Channel

Show your support

Star this repo

Give a ⭐️ if this project helped you!

Donate to our project

If you do like our project and we hope that you do, can you please support us? Our project is not and is never going to be working for profit. We need the money just so we can continue doing what we do ;-) .

PyMilo Donation

pymilo's People

Contributors

ahreccese avatar dependabot[bot] avatar sadrasabouri avatar sepandhaghighi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

pymilo's Issues

[Feature]: scikit-learn preprocessing module support

Describe the feature you want to add

Support serialization and deserialization of all Scikit-learn preprocessing transformers

  • Binarizer
  • FunctionTransformer
  • KBinsDiscretizer
  • KernelCenterer
  • LabelBinarizer
  • LabelEncoder
  • MultiLabelBinarizer
  • MaxAbsScaler
  • Normalizer
  • OneHotEncoder
  • OrdinalEncoder
  • PolynomialFeatures
  • PowerTransformer
  • QuantileTransformer
  • RobustScaler
  • SplineTransformer
  • StandardScaler
  • TargetEncoder

Describe your proposed solution

No response

Describe alternatives you've considered, if relevant

No response

Additional context

No response

PyTorch Support

Description

Support serialization and deserialization of PyTorch models

scikit-learn Neighbors support

Description

add Neighbors support

  • KNeighborsClassifier
  • KNeighborsRegressor
  • NearestNeighbors
  • NearestCentroid
  • RadiusNeighborsClassifier
  • RadiusNeighborsRegressor
  • LocalOutlierFactor

Upload to IPFS

Description

Ability to upload the exported model to IPFS and opensea.

Exported File Compression

Description

  • Ability to compress the exported file
  • Set the compression method in the Export-Import function constructor parameters

Exported File Splitting

Description

Support splitting (and merging) the exported (and imported) model files to enhance memory usage efficiency.

Birch:: root_ before dummy_leaf_

Description

There should be constraints that force serialization of the "root_" before the "dummy_leaf_" field in Birch model.

[Feature]: Add post PyMilo-Import function execution

Describe the feature you want to add

Add ability to call specific functions after PyMilo Import

Describe your proposed solution

No response

Describe alternatives you've considered, if relevant

No response

Additional context

No response

Server/Client (RESTful API)

Description

  • Support the Client-Server model to provide the possibility of ML model abstraction and remote access to the models
  • PyMilo ML model wrapper to wrap around ML models to abstract model function calls.
  • Add fallback function to PyMilo ML model wrapper to delegate function calls to the remote server to get executed in the main model

Document modification

Description

Add more detailed descriptions to the current document and provide a few detailed tutorial pages.

[Feature]: PyMilo documentation website

Describe the feature you want to add

PyMilo documentation website, using Sphinx using a theme provided by Read the Docs.

something like pydocstyle documentation website

Describe your proposed solution

No response

Describe alternatives you've considered, if relevant

No response

Additional context

No response

JS/TS SDK

Description

Provide a JavaScript/TypeScript SDK to use ML models through PyMilo in a web browser

[Feature]: Ability to sign and encrypt the exported file

Describe the feature you want to add

Add symmetric & Asymmetric encryption for the exported file

Describe your proposed solution

No response

Describe alternatives you've considered, if relevant

No response

Additional context

No response

[Feature]: numpy.intc serialization support

Describe the feature you want to add

Serialization for numpy.intc data structure should be added(it is used in scikit svm models)

Describe your proposed solution

No response

Describe alternatives you've considered, if relevant

No response

Additional context

No response

Model Versioning

Description

  • Support model versioning to enable PyMilo to support different model versions

[Feature]: Publish to Conda

Describe the feature you want to add

add associated GitHub action to publish to conda(like PyPI auto-upload)

Describe your proposed solution

No response

Describe alternatives you've considered, if relevant

No response

Additional context

No response

[Feature]: Ability to download model from remote url

Describe the feature you want to add

Ability to give specific url to PyMilo Import instance and it will then download the given model during the instantiation part.

Describe your proposed solution

No response

Describe alternatives you've considered, if relevant

No response

Additional context

No response

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.