Some classes and tools for Over-The-Air (OTA) firmware updates on ESP32. I wanted a simple and flexible interface for managing and running OTA updates for ESP32* devices. These tools are for managing OTA updates of the micropython firmware installed in the device flash storage (not the python files in the mounted filesystem).
- Contents: Usage | Installation | How it Works:
- API docs: ota.update | ota.rollback | ota.status
Write a new micropython image from a web server to the next OTA partition on the flash storage:
>>> import ota.update
>>> ota.update.from_firmware_file("http://nas.local/micropython.bin", reboot=True)
Writing new micropython image to OTA partition 'ota_0'...
Device capacity: 384 x 4096 byte blocks.
Opening firmware file http://nas.local/micropython.bin...
Writing 380 blocks + 2032 bytes.
BLOCK 380 + 2032 bytes
Verifying SHA of the written data...Passed.
SHA256=7920d527d578e90ce074b23f9050ffe4ebbd8809b79da0b81493f6ba721d110e
OTA Partition 'ota_0' updated successfully.
Micropython will boot from 'ota_0' partition on next boot.
Remember to call ota.rollback.cancel() after successful reboot.
Rebooting in 10 seconds (ctrl-C to cancel)
Print the current status of the OTA partitions on the device:
>>> import ota.status
>>> ota.status.status()
Micropython firmware v1.20.0 has booted from partition 'ota_0'.
The next OTA partition is 'ota_1'.
The / filesystem is mounted from partition 'vfs'.
Partition table:
# Name Type SubType Offset Size (bytes)
nvs data nvs 0x9000 0x4000 16,384
otadata data ota 0xd000 0x2000 8,192
phy_init data phy 0xf000 0x1000 4,096
ota_0 app ota_0 0x10000 0x180000 1,572,864
ota_1 app ota_1 0x190000 0x180000 1,572,864
vfs data fat 0x310000 0xf0000 983,040
>>>
NOTE: After performing an OTA update, the device must be hard_reset()
or
power cycled before it will boot into the new firmware. A soft_reset()
(including pressing ctrl-D
at the repl prompt) will not load the new
firmware.
After booting up successfully, stop the esp32 from rolling back to the previous firmware on next boot (should do this on every successful boot and app startup):
import ota.rollback
ota.rollback.cancel()
Install ota
package with mpremote
into /lib/ota/
on the device (as .py modules):
mpremote mip install github:glenn20/micropython-esp32-ota/mip/ota
or, install module as byte-compiled .mpy files
mpremote mip install github:glenn20/micropython-esp32-ota/mip/ota/mpy
Remember to ensure /lib
is in your sys.path
.
To support Over-The-Air updates, a micropython image requires a special partition table, such as:
Partition table:
# Name Type SubType Offset Size (bytes)
nvs data nvs 0x9000 0x4000 16,384
otadata data ota 0xd000 0x2000 8,192
phy_init data phy 0xf000 0x1000 4,096
ota_0 app ota_0 0x10000 0x180000 1,572,864
ota_1 app ota_1 0x190000 0x180000 1,572,864
vfs data fat 0x310000 0xf0000 983,040
- Use
ota.status.status()
to print the full partition table of your device.
For micropython, an OTA-enabled partition table usually includes:
- one partition with a subtype of ota (usually named otadata)
- where the bootloader saves metadata about the state of the ota partitions
- two app partitions with subtypes of ota_0 and ota_1.
- The OTA updater writes new micropython firmware images into these partitions
- and usually one data partition named vfs or fat.
Any micropython image built with BOARD_VARIANT=OTA
will have a partition table
like this (including the official OTA images at
https://micropython.org/download/ESP32_GENERIC).
You can also add an OTA-enabled partition table to a non-ota micropython
firmware file with mp-image-tool-esp32 --ota
.
The partition table has to make room for two app partitions on the device (instead of the normal one). This means space is tight on a 4MB flash device. The OTA partition usually has less room for each micropython firmware image (1.5MB instead of 2MB) and much less room for the vfs filesystem partition (<1MB instead of 2MB). Devices with more than 4MB of flash can use larger app and vfs partitions.
Micropython will boot from one of the ota_X app partitions and write new firmware to the other one. After writing new firmware to the other partition, it will be set as the boot partition for the next reboot. The old firmware image is still available in case it is necessary to rollback to the previous firmware.
After booting from either ota partition, the micropython firmware will
automatically mount the /
filesystem from the vfs partition.
An OTA partition should be updated with a "micropython app image". The
micropython firmware (.bin
files) downloaded from the MicroPython downloads
page combine the bootloader, partition table
and the micropython app image, so can not be used for micropython OTA
updates.
There are three ways to obtain a micropython.bin
you can use for OTA updates:
- Download a
.app-bin
file from the MicroPython downloads page, - Use the
micropython.bin
file in theports/esp32/build_XXX
folder- if you build your own micropython firmware, or
- Extract the
.app-bin
firmware from a combined firmware file with:mp-image-tool-esp32 --extract-app ESP32_GENERIC-20231005-v1.21.0.bin
The ota.update
module provides the OTA
class which can be used to write new
micropython firmware to the next ota partition on the device and two
convenience functions which use OTA
to perform simple OTA firmware updates:
from_file()
and from_json()
.
-
function
ota.update.from_file(url: str, sha="", length=0, verify=True, verbose=True, reboot=True)
Read a micropython firmware from url and write it to the next ota partition. sha and length are used to validate the data written to the partition. Returns the number of bytes written to the partition.
url
is a http[s] url or a filename on the devicesha
(optional) is the expected sha256sum of the firmware filelength
(optional) is the expected length of the firmware file (in bytes)verify=True
(optional) Read back the data written to the flash storage and veryify the sha256sum checksum.verbose=True
(optional) prints out verbose information of what it is doingreboot=True
(optional) Performs amachine.hard_reset()
10 seconds after a successful OTA update.
-
function
ota.update.from_json(url: str, sha="", length=0, verify=True, verbose=True, reboot=True)
Read a JSON file from url (must end in ".json") containing the url, sha and length of the firmware file. Then, read the firmware file and write it to the next ota partition. Returns the number of bytes written to the partition.
-
The JSON file should specify an object including the firmware, sha, and length keys, eg:
{ "firmware": "micropython.bin", "sha": "7920d527d578e90ce074b23f9050ffe4ebbd8809b79da0b81493f6ba721d110e", "length": 1558512 }
The firmware key provides a url (or filename) for the firmware image. This may be specified relative to the basename of the url for the json file.
-
The functions above use the methods in the OTA
class to perform the OTA
updates.
-
class
ota.update.OTA(verify=True, verbose=True, reboot=False, sha="", length=0)
- Create an OTA class instance which can be used to write new micropython firmware on the device. May be used as a context manager in a with statement.
- Checks that:
- The bootloader is OTA-enabled (CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE=y)
- There are OTA partitions available to write the new firmware
(
esp32.Partition.get_next_update()
)
- Arguments:
verify
: if true, read back the data from the partition on close() to verify the sha256sum matches the written dataverbose
: if true, print out useful progress and diagnostic informationreboot
: if true, reboot the device on close() - if all checks passsha
: optionally provide the expected sha256sum of the firmwarelength
: optionally specify the length of the firmware file and check it will fit on device- sha and length may instead be provided as arguments to some methods below.
-
Method:
OTA.from_firmware_file(url: str, sha="", length=0) -> int
-
Read a micropython firmware from url and write it to the next ota partition. sha and length are used to validate the data written to the partition. Returns the number of bytes written to the partition.
url
is a http[s] url or a filename on the devicesha
(optional) is the expected sha256sum of the firmware filelength
(optional) is the expected length of the firmware file (in bytes)
-
-
Method:
OTA.from_json(url: str) -> int
-
Read a JSON file from url (must end in ".json") containing the url, sha and length of the firmware file. Then, read the firmware file and write it to the next ota partition. Returns the number of bytes written to the partition.
The JSON file should specify an object including the firmware, sha, and length keys, eg:
{ "firmware": "micropython.bin", "sha": "7920d527d578e90ce074b23f9050ffe4ebbd8809b79da0b81493f6ba721d110e", "length": 1558512 }
The firmware key provides a url (or filename) for the firmware image. This may be specified relative to the basename of the url for the json file.
-
-
Method:
OTA.from_stream(f, sha="", length=0) -> int
- Read a micropython firmware from an open file/stream,
f
, and write it to the next ota partition. Returns the number of bytes written to the partition. sha and length are used to validate the data written to the partition. f
is an io stream (file-like object) which supports the readinto() methodsha
(optional) is the expected sha256sum of the firmware filelength
(optional) is the expected length of the firmware file (in bytes)
- Read a micropython firmware from an open file/stream,
-
Method:
OTA.write(data: bytes | bytearray) -> int
- Copy data to the end of the firmware file being written to the ota partition.
-
Method:
OTA.close()
- Flush buffered data to the ota partition
- Compute the sha256sum of data written
- Check length of firmware matches expected length (if provided)
- Check sha256sum of firmware matches expected hash (if provided)
- If
verify
is true:- read back firmware from partition and check sha256sum matches
- Validate the new firmware image and set the new OTA partition as boot partition (esp32.Partition.set_boot())
- If
reboot
is true, perform a hard reset of the device after a delay of 10 seconds.
If all checks pass, the new firmware will be loaded after the next reboot. If any checks fail, a
ValueError
exception will be raised.OTA.close()
will be called automatically ifOTA
is used in a with statement (as a context manager).
- Flush buffered data to the ota partition
from ota.update import OTA
# Write firmware from a url provided in a JSON file
with OTA() as ota:
ota.from_json("http://nas.local/micropython/micropython.json")
# Write firmware from a url or filename and reboot if successful and verified
with OTA(reboot=True) as ota:
ota.from_firmware_file(
"http://nas.local/micropython/micropython.bin",
sha="7920d527d578e90ce074b23f9050ffe4ebbd8809b79da0b81493f6ba721d110e",
length=1558512)
# Write firmware from an open stream:
with OTA() as ota:
with open("/sdcard/micropython.bin", "rb") as f:
ota.from_stream(f)
# Read a firmware file from a serial uart
remaining = 1558512
sha = "7920d527d578e90ce074b23f9050ffe4ebbd8809b79da0b81493f6ba721d110e"
with OTA(length=remaining, sha=sha) as ota:
data = memoryview(bytearray(1024))
gc.collect()
while remaining > 0:
n = uart.readinto(data[:min(remaining, len(data))]):
ota.write(data[:n])
remaining -= n
# Used without the "with" statement - must call close() explicitly
ota = OTA()
ota.from_json("http://nas.local/micropython/micropython.json")
ota.close()
When booting a new OTA firmware for the first time, you need to tell the
bootloader if it is OK to continue using the new firmware. Otherwise, the
bootloader will assume something went wrong and automatically rollback
to the previous firmware on the next reboot. You can use ota.rollback.cancel()
to tell the bootloader not to rollback to the previous firmware.
If the new firmware fails to startup or your app does not operate correctly with
the new firmware, reboot the device without cancelling the rollback and the
old firmware will be restored. You may also wish to use a watchdog timer
(WDT) during
your app startup sequence to force a reboot if the startup hangs or fails before
you call ota.rollback.cancel()
.
Note: the rollback mechanism is only available if the bootloader was
compiled with CONFIG_BOOTLOADER_ROLLBACK_ENABLE=y
.
-
function
ota.rollback.cancel()
- Tell the bootloader to continue booting this ota firmware partition by invoking esp_ota_mark_app_valid_cancel_rollback().
- A reasonable approach is to call
ota.rollback.cancel()
on every successful boot (eg. inmain.py
or after your app has successfully started up). Theota.rollback
module is designed to be lightweight so you can call it every time your device boots up.
-
function
ota.rollback.force()
- Manually set the next reboot to boot from the other *ota partition. This bypasses the bootloader's OTA rollback provisions, but lets you switch between the two firmwares on the device as you need.
-
function
ota.rollback.cancel_force()
- Cancel any previous call to
ota.rollback.force()
. This function will manually set the boot partition for future boots to the currently booted partition.
- Cancel any previous call to
-
function
ota.status.status()
- Check the current firmware is OTA-capable
- Print the current partition table
- Show which is the currently booted partition
- Show the partition which will be used on next reboot
-
function
ota.status.ready() -> bool
- Return
True
if the current device supports OTA firmware updates:- the bootloader was compiled with CONFIG_BOOTLOADER_ROLLBACK_ENABLE=y, and
- the partition table supports OTA updates.
- Return