This is a container repository that makes developing against the QMK web services (Configurator, API, and the Compiler) easier.

We use Docker Compose to start and run the services required for QMK Configurator. Setting up a dev environment is a pretty simple process.

1. Install and run Docker.
2. Clone this repository:

   git clone --recurse-submodules https://github.com/qmk/qmk_web_stack
   cd qmk_web_stack

3. Fix the submodules (checks out each one to master or main):

   ./fix-submodules.sh

4. Build and start the Docker containers:

   docker-compose build
   docker-compose up

5. Populate the database:

   ./populate_api.sh

6. Visit your local configurator: http://localhost:5000/

You are now ready to develop on your local environment. There are some things you can do outside the normal Docker Compose workflow that are helpful when developing features.

There are some functional tests for the qmk_compiler code. You can run them with the provided script:

./run_tests.sh

If you want to run a subset of tests you can pass the fully resolved name to the test. Usually this is in the format <test_module>.<test_function_name>. For example you could pass in test_qmk_compiler.test_0010_keyboard_list to run only that test.

You should be aware that test_qmk_compiler.test_0000_checkout_qmk_skip_cache and test_qmk_compiler.test_9999_teardown are always run because some of the tests depend on them.

Example:

./run_tests.sh test_qmk_compiler.test_0010_keyboard_list

The QMK API will clone and update qmk_firmware as needed. By default this means the master branch of https://github.com/qmk/qmk_firmware. You can set environment variables for qmk_compiler and qmk_api inside docker-compose.yml to change this default. If you want to change the git branch for all repositories at once (qmk_firmware, chibios, chibios-contrib, lufa, vusb) you can set GIT_BRANCH. Otherwise you can change the branch on a repo-by-repo basis. You must set the git URL for all repos individually.

The default environment variables are:

One of the key features of the API is the parsing of qmk_firmware to build a database about QMK keyboards and keymaps. There are two ways to start this process:

1. populate_api.sh: This script will run the update process directly, bypassing the normal infrastructure.
2. trigger_update.sh: This script triggers an update using the same pathway as qmk_bot- it sets a flag in redis which is acted upon by qmk_api_tasks.

Normally you will want to use method #1. If you need to test method #2 you will also need to start the qmk_api_tasks service, which is not started by default.

QMK API uses Discord's webhook API to send messages to #configurator_log in our server. If you'd like to use this with your own server you will need to configure a webhook and set that URL here. By default this feature is disabled.

The qmk_api_tasks service serves two roles: it continually tests keyboards to ensure they're compatible with Configurator, and it handles routine maintenance such as qmk_firmware updates and S3 cleanup. You typically will not need to run this service except for specific purposes, such as working on qmk_api_tasks itself or exercising the backend infrastructure.

To start qmk_api_tasks, use the run_qmk_api_tasks.sh script:

./run_qmk_api_tasks.sh

When running a local copy of Configurator, you'll likely be unable to download firmware or source due to internal Docker hostnames.

The MinIO web interface is made available at http://localhost:9001/, and for now can be used to manually access the resulting files. You'll need to cross-reference the Configurator's job ID in the build log with the equivalent prefix in the qmk-api bucket.

You can log into MinIO with the minio container's root user and password as specified in the docker-compose.yml file.

If you need to manually clean your S3 storage you can use the cleanup_storage.sh script:

./cleanup_storage.sh