• Clouds
  • Table of Contents
  • Overview
  • Features
  • Requirements
  • Installation
    • 1. Clone the repository
    • 2. Change directory into repository folder
    • 3. Setup AWS credentials for artifact download from S3
    • 4. Install required packages (required for local implementation)
  • Usage
    • 1. Local
      • Pipeline only
      • Unit Test
    • 2. Docker
      • Pipeline only
        • Build the Docker image
        • Run the application
      • Unit Test
        • Build the Docker image for unit test
        • Run the tests
  • Customization
    • AWS

This repository contains the Clouds project, an interactive web application built with Streamlit, which allows users to select and run different versions of trained machine learning models. This document includes a guide for setting up, installing, retrieving models, setting up the Streamlit application, usage, customization, and troubleshooting.

The main feature of this project is the Streamlit-based web application, which allows users to load, select and run different versions of trained machine learning models. The application also includes a comprehensive testing framework, logging and error handling mechanisms to ensure a smooth and robust user experience. Each module can be easily customized to accommodate specific requirements or preferences using config.yaml. The entire application and its unit tests can be run inside a docker container. The application has also been deployed on AWS ECS, but has been stopped mainly for cost reasons.

• Python 3.7 or higher
• Streamlit
• Docker
• AWS credentials
• See requirements.txt.
• The application assumes that an S3 bucket with trained model objects and training data has already been setup. It will not set one up for you.

git clone https://github.com/MSIA/423-2023-hw3-hwl6390.git

cd 423-2023-hw3-hwl6390

This guide assumes you have installed the AWS CLI. If you have not configured an AWS profile, run the following.

aws configure sso --profile my-sso

For the purposes of this guide, the name of the AWS profile will be my-sso. The user can name it however they like.

After configuring the sso, run the following to login.

aws sso login --profile my-sso

After logging in, export the profile as an environment variable.

export AWS_PROFILE=my-sso

If you run aws configure list and are able to see my-sso in the list of profiles, the environment variable has been set correctly.

pip install -r requirements.txt

Verify you are in the same directory as main.py. Then, run

streamlit run main.py

in the terminal.

Run

pytest

in the terminal.

docker build -t clouds-app -f dockerfiles/docker-main .

docker run -p 80:80 -v ~/.aws:/root/.aws -e AWS_PROFILE=my-sso clouds-app

docker build -t unittest-cloud -f dockerfiles/docker-test .

docker run unittest-cloud

To customize settings within the pipeline, edit config.yaml.

Modify aws section of config.yaml to achieve desired bucket name and prefixes.