Developing for Reliability in CircuitPython

reliability.md

# SPDX-FileCopyrightText: 2023 anecdata
#
# SPDX-License-Identifier: MIT

try / except all the things (even the main loop)

with all the things (where available and appropriate)

manage volatile memory:

• heap: gc.mem_free() & gc.collect()
  • settings.toml: CIRCUITPY_HEAP_START_SIZE=(must be a multiple of 4; defaults to 8192) (added in CP 9)
• C stack: supervisor.runtime.next_stack_limit (removed in CP 9)
• pystack:
  • settings.toml: CIRCUITPY_PYSTACK_SIZE= (defaults to 1536 bytes for most boards)
• espidf memory (espressif port):
  • espidf.heap_caps_get_largest_free_block()
  • espidf.heap_caps_get_free_size()
  • espidf.heap_caps_get_total_size()
  • settings.toml: CIRCUITPY_RESERVED_PSRAM= (espressif port with external PSRAM) (removed in CP 9)

safemode.py

• example: https://gist.github.com/anecdata/fe35dc6a94069fc920edf61a64750b53
• check supervisor.runtime.safe_mode_reason
• store any useful info for boot.py (or code.py) in microcontroller.nvm (where available), or write to CIRCUITPY
• microcontroller.reset, or alarm.exit_and_deep_sleep_until_alarms() (where available)

boot.py

• check microcontroller.cpu.reset_reason
• store any useful info for code.py in alarm.sleep_memory (where available), microcontroller.nvm (where available), or write to CIRCUITPY
• print() from boot.py is appended to boot_out.txt (512 bytes total)

code.py

• check supervisor.runtime.run_reason
• check boot_out.txt for boot.py exceptions
• check any useful info stored from safemode.py and boot.py
• supervisor.get_previous_traceback(): could be from boot.py (reset) or code.py (reload)
• microcontroller.watchdog (where available)
• supervisor.set_next_code_file(reload_on_*)
• if networked, try various escalations to restore function:
  • espressif and raspberrypi ports native wifi (where available):
    • wifi.radio.enabled = False / wifi.radio.enabled = True
    • Station: check connection to AP: wifi.rado.connected
    • Station: wifi.rado.stop_station() / wifi.rado.start_station()
    • AP: check that the AP is active: wifi.radio.ap_active
    • AP: wifi.rado.stop_AP() / wifi.rado.start_AP()
  • esp32spi:
    • esp._debug = val # 0,1,2,3
    • esp.set_esp_debug(val) # True,False
    • check for connection: esp.is_connected or esp.status
    • esp.disconnect() / esp.connect()
    • esp.reset()
  • wiznet:
    • eth.link_status
    • eth.sw_reset()
    • hardware reset pin (where available)
• when everything above fails: supervisor.reload()
• keep a count of consecutive reloads in alarm.sleep_memory (where available)
• when a failure continues after a reload, or when the consecutive reload count gets too high: microcontroller.reset()