This repository shows an implementation of a reinforcement learning algorithm in the ERP Software ERPNext.
Setup process:
- Install ERPNext + Dependencies
- Configure ERPNext and Import Data
- Setup a Virtual Environment and the OpenAI Gym Environment
- Edit Token
- Start Training an Agent
- Using the Trained Model
Tested on a virtual machine with the following specifications:
- Ubuntu 20.04.2.0 LTS (Server Version)
- Python 3.7
- 4 vCPUs
- 8GB RAM
- 40GB storage volume
apt update
apt -y upgrade
apt install software-properties-common
add-apt-repository ppa:deadsnakes/ppa
apt install python3.7 python3.7-dev python3.7-doc python3-pip
pip3 install ansible
apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8
add-apt-repository 'deb [arch=amd64,i386,ppc64el] http://mirror.nodesdirect.com/mariadb/repo/10.3/ubuntu bionic main'
If an error about the architecture appears:
nano /etc/apt/sources.list
change
deb [arch=ppc64el,amd64,i386] http://mirror.nodesdirect.com/mariadb/repo/10.3/ubuntu bionic main
to
deb [arch=ppc64el,amd64,arm64] http://mirror.nodesdirect.com/mariadb/repo/10.3/ubuntu bionic main
sudo apt update
sudo apt -y install mariadb-server libmysqlclient-dev
sudo apt install curl nano git pv
sudo nano /etc/mysql/my.cnf
Add the following snippet below [mysqld]
innodb-file-format=barracuda
innodb-file-per-table=1
innodb-large-prefix=1
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
Add the following snippet below [mysql]
default-character-set = utf8mb4
Change innodb_buffer_pool_size to 2GB
innodb_buffer_pool_size = 2000M
Finnish MariaDB Installation
sudo systemctl restart mariadb
sudo systemctl enable mariadb
sudo service mysql restart
sudo mysql_secure_installation
sudo curl --silent --location https://deb.nodesource.com/setup_12.x | sudo bash -
curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor | sudo tee /usr/share/keyrings/yarnkey.gpg >/dev/null
echo "deb [signed-by=/usr/share/keyrings/yarnkey.gpg] https://dl.yarnpkg.com/debian stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update && sudo apt-get install yarn
sudo apt -y install nginx nodejs redis-server supervisor
sudo systemctl start nginx
sudo systemctl enable nginx
sudo systemctl start redis-server
sudo systemctl enable redis-server
sudo apt -y install libxrender1 libxext6 xfonts-75dpi xfonts-base fontconfig
wget https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6-1/wkhtmltox_0.12.6-1.focal_amd64.deb
sudo dpkg -i wkhtmltox_0.12.6-1.focal_amd64.deb
sudo apt -f install
sudo dpkg -i wkhtmltox_0.12.6-1.focal_amd64.deb
sudo adduser frappe
sudo usermod -a -G sudo frappe
sudo su - frappe
sudo pip3 install frappe-bench
bench init --frappe-branch=version-13 frappe-bench --python=python3.7
cd frappe-bench
bench new-site site1.local
bench get-app --branch version-13 erpnext
bench new-site site1.local
bench --site site1.local install-app erpnext
Increase system's file watchers limit
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
Start the bench to access from it from client
bench start
Eneable developer mode
bench set-config developer_mode 1
Eneable custom scripting
bench --site site1.local set-config server_script_enabled true
Not necessary but very useful: https://github.com/proenterprise/bench-stop
Command for stopping the bench
python3 stop.py
Either Setup ERPNext by your own or use the following restore command to restore my version
bench --site site1.local --force restore \
--mariadb-root-username root \
--mariadb-root-password admin \
--with-public-files ./sites/site1.local/private/backups/*files.tar \
--with-private-files ./sites/site1.local/private/backups/*private-files.tar \
./sites/site1.local/private/backups/*database.sql.gz
Also copy the Encryption key to the new site_config.json file
Next add the api file to the ERPNext directory.
Setup a virtual environment following this tutorial: https://janakiev.com/blog/jupyter-virtual-envs/
Clone the Custom-Environment directory from this repository
Enter the virtual envionment
virtualenv <env_name>
source <env_name>/bin/activate
Install dependencies from the requirements text file
pip install -r path/to/requirements.txt
Install the custom environment and its dependencies
pip install -e .
Enter the ERPNext Token and the IP-Adress in the token file
Start training an agent by running the PPO.py file
python PPO.py
- Insert the path to the model in the continual training file
- Continue to train the model by running the training file
- The Environment receives the current state of the ERP system, i.e. the current stock levels taking into account goods receipts and goods issues. For example [93, 120, 17]
- The modell decides which product should be purchased and emits an action. For example, [12, 0, 20]
- As a result of the action, a Material Request is created via the API in the ERP system.
- The user can now view this Material Request and make changes, for example to the order quantities. The user then confirms the Material Request and a Purchase Order is created from the Material Request in the ERP system. 5.
- The Gym Environment recognizes this and calculates the difference between the proposed order quantity and the order quantity executed by the user. The difference results in the feedback to the agent. If there is no difference between the quantity suggested by the agent and the quantity in the Purchase Order, the reward is r=1. If the quantity is different, a negative reward is calculated from the difference. Note that this reward is in the same order of magnitude as the rewards from the environment for training.
SQL-Statements for removing Data during the learning process
SET SQL_SAFE_UPDATES = 0;
DELETE FROM `tabPurchase Order Item`;
DELETE FROM `tabPurchase Order`;
DELETE FROM `tabMaterial Request`;
Backup and Restore: https://wiki.erpnext.org/pages/viewpage.action?pageId=40337533
Guide for installing ERPNext on Ubuntu: https://www.vultr.com/docs/how-to-install-erpnext-open-source-erp-on-ubuntu-17-04-22771
Guide for Installing Specific Python Versions: https://docs.python-guide.org/starting/install3/linux/
Stable Baselines3 Documentation: https://stable-baselines3.readthedocs.io/en/master/
OpenAI Gym: https://gym.openai.com/
Soccer Custom Environment: https://github.com/openai/gym-soccer