Ay20/21 Capstone Project Group B18
- 1. Setting up
- 2. Hardware
- 3. Comms Internal
- 4. Comms External
- 5. Machine Learning Software
- 6. Machine Learning FPGA Hardware
- 7. Dashboard Software
- Appendix A: Product Scope
- Git clone this repository, if you are using a zipped version, unzip it to view the contents
- Ensure you have the relevant packages installed from the different subcomponents stated below
โ Return to Table of Contents
- Open Beetle_with_Position file in Arduino IDE
- Click "Upload"
- See the data by clicking "Serial Monitor" (Magnifying glass at the top right of the IDE)
- Open /comms_internal/combine/combine.ino in Arduino IDE
- Upload Code On to Bluno
- sudo hcitool lescan
- locate device with name "bluno"
- Record the MAC address to be used in dancer_client.py
Ensure pip3 is installed in your system.
To install pip3: apt-get install python3-pip
On system with local database where /comms_external/FINAL/laptop-sql.py is run, ensure all the requirements are installed by running:
pip3 install -r laptop-sql_requirements.txt
On dancers' laptop where /comms_external/FINAL/dancer_client.py is run, ensure all the requirements are installed by running:
pip3 install -r dancer_client_requirements.txt
On Ultra96, git clone this project. Inside the realtime_eval directory where /realtime_eval/ultra96_client.py is run, ensure all the requirements are installed by running:
pip3 install -r ultra96_client_requirments.txt
The requirement files can be found here:
- Inside the realtime_eval directory, /realtime_eval/ultra96_client.py run the following command:
python3 ultra96_client.py [IP address of evaluation server] [Port number to connect with for evaluation server]
- Enter the port number for SQL and Ultra96 socket connection. Any port that's free can be used. The same port number has to be used for laptop-sql.py afterwards
- Enter the port number of laptop and Ultra96 socket connection. Any port that's free can be used. The same port has to be used for dancer_client.py afterwards.
- Enter 3 for number of dancers under evaluation mode.
- Start the evaluation server after step 4. The secret key for evaluation server is "abcdefghijklmnop" without the quotation marks.
- Press enter when prompted on ultra96_client.py after the evaluation server has started.
- Inside the directory with laptop-sql.py on your local machine where the database is hosted, run the following command:
python3 laptop-sql.py
- When prompted for the port number for connection to Ultra96, enter the same port number that was given in step 2 under Running Ultra96 Client section.
- Inside the directory with dancer_client.py, run the following command:
python3 dancer_client.py
- It will prompt for the dancer number. After discussing with the 3 dancers, each dancer should take on an unique number from 1 to 3. Enter this unique dancer number
- There are 6 set of gloves for dancing labelled a to f. Enter the glove letter being used by the dancer in lower case and press enter.
- Finally, the dancer will have to enter the port number for the laptop to Ultra96 client connection. This port number must be the same as the one provided in step 3 under Running Ultra96 Client section.
All connections will be up and the system will be ready.
Pytorch + Ignite for MLP
Scikit-learn for Extra Trees Classifier
- The Machine Learning code is available in the sw1_ml folder with the following directory structure-
-
The data sub-directory stores processed data used for the individual subcomponent test.
-
The notebooks sub-directory stores the implementation of the various models
-
The src folder stores the various utility files used for processing data, feature extraction as well as the different Machine Learning models.
Run the notebook file step by step to process the relevant dataset, train the model and create a confusion matrix
Vivado + Vivado HLS 2019.2
Pynq OS on Ultra96 board
-
HLS projects are stored in the path /hardware_FPGA/Ultra96/Vivado/vivadoHLS. They contain zip files that can be imported into HLS and be exported to a IP that can be imported in Vivado. The algorithm for the models are stored in mlpHLS.cpp in the zip file.
-
The Vivado project is stored in /hardware_FPGA/Ultra96/Vivado/vivadoHLS/CG4002_B18\hardware_FPGA\ultra96\Vivado\MLPvivado.xpr.zip. This project archive can be imported from Vivado. Click open block diagram and import the mlpHLS IP file generated bu Vivado HLS,and generate a bitstream
-
Scripts to export and study the algorithms are in/hardware_FPGA/Ultra96/Vivado/vivadoHLS/CG4002_B18\hardware_FPGA\Notebooks
Helper functions are in \realtime_eval\MLP_predict_fpga.py E.g to program the FPGA with the bitstream ETC_Start() can be called, and then ETC_Predict(inputs) can be called to get prediction.
The architecture of the dashboard and an in depth explanation of each feature component can be viewed in the Final Report Document under Section 5.2.1 Dasboard - Software Engineering Aspect.
- Ensure that the latest node package manager is installed. If you are using ubuntu, you can use the command
sudo apt install nodejs
andsudo apt install npm
. Check the version usingnodejs -v
. - Change directory to
/dashboard/dancedance/
and runnpm install
to install the node module dependencies required for the front-end dashboard. - Change directory to the node server
./api/
and runnpm install
to install the node module dependencies required for the back-end server. - Once again, change directory back to the front-end via
./dancedance/
and runnpm run dev
. This command will run both React as well as the node server at the same time using npm-watch library. - In order for the back end database to work, install MySQL using the command
sudo apt install mysql-server
andsudo mysql_secure_installation
to easily set up a new MySQL server if you haven't done so. - Import the database
toAws.sql
found in./dashboard
- You should be able to register your very own account and log into the dasboard!
- To simulate sensor data being displayed onto the evaluation mode tab, change directory to
./dashboard/api/
and runpython sendingDataSimulator.py
. This will populate your Predictions and RawSensorData table in the database, and you should be able to view the evaluation feature with data pouring in. - Alternatively, visit this website which this dashboard is currently being hosted!