Implement reliable bootloader version detection using rollback capability for OTA updates

docs/bootloader-compatibility.md

@@ -0,0 +1,82 @@
+# Bootloader Compatibility Checking
+
+As of WLED 0.16, the firmware includes bootloader version checking to prevent incompatible OTA updates that could cause boot loops.
+
+## Background
+
+ESP32 devices use different bootloader versions:
+- **V2 Bootloaders**: Legacy bootloaders (ESP-IDF < 4.4)
+- **V3 Bootloaders**: Intermediate bootloaders (ESP-IDF 4.4+)
+- **V4 Bootloaders**: Modern bootloaders (ESP-IDF 5.0+) with rollback support
+
+WLED 0.16+ requires V4 bootloaders for full compatibility and safety features.
+
+## Checking Your Bootloader Version
+
+### Method 1: Web Interface
+Visit your WLED device at: `http://your-device-ip/json/bootloader`
+
+This will return JSON like:
+```json
+{
+  "version": 4,
+  "rollback_capable": true,
+  "esp_idf_version": 50002
+}
+```
+
+### Method 2: Serial Console
+Enable debug output and look for bootloader version messages during startup.
+
+## OTA Update Behavior
+
+When uploading firmware via OTA:
+
+1. **Compatible Bootloader**: Update proceeds normally
+2. **Incompatible Bootloader**: Update is blocked with error message:
+   > "Bootloader incompatible! Please update to a newer bootloader first."
+3. **No Metadata**: Update proceeds (for backward compatibility with older firmware)
+
+## Upgrading Your Bootloader
+
+If you have an incompatible bootloader, you have several options:
+
+### Option 1: Serial Flash (Recommended)
+Use the [WLED web installer](https://install.wled.me) to flash via USB cable. This will install the latest bootloader and firmware.
+
+### Option 2: Staged Update
+1. First update to WLED 0.15.x (which supports your current bootloader)
+2. Then update to WLED 0.16+ (0.15.x may include bootloader update)
+
+### Option 3: ESP Tool
+Use esptool.py to manually flash a new bootloader (advanced users only).
+
+## For Firmware Builders
+
+When building custom firmware that requires V4 bootloader:
+
+```bash
+# Add bootloader requirement to your binary
+python3 tools/add_bootloader_metadata.py firmware.bin 4
+```
+
+## Technical Details
+
+- Metadata format: ASCII string `WLED_BOOTLOADER:X` where X is required version (1-9)
+- Checked in first 512 bytes of uploaded firmware
+- Uses ESP-IDF version and rollback capability to detect current bootloader
+- Backward compatible with firmware without metadata
+
+## Troubleshooting
+
+**Error: "Bootloader incompatible!"**
+- Use web installer to update via USB
+- Or use staged update through 0.15.x
+
+**How to check if I need an update?**
+- Visit `/json/bootloader` endpoint
+- If version < 4, you may need to update for future firmware
+
+**Can I force an update?**
+- Not recommended - could brick your device
+- Use proper upgrade path instead

tools/add_bootloader_metadata.py

@@ -0,0 +1,73 @@
+#!/usr/bin/env python3
+"""
+Simple script to add bootloader requirement metadata to WLED binary files.
+This adds a metadata tag that the OTA handler can detect.
+
+Usage: python add_bootloader_metadata.py <binary_file> <required_version>
+Example: python add_bootloader_metadata.py firmware.bin 4
+"""
+
+import sys
+import os
+
+def add_bootloader_metadata(binary_file, required_version):
+    """Add bootloader metadata to a binary file"""
+    if not os.path.exists(binary_file):
+        print(f"Error: File {binary_file} does not exist")
+        return False
+
+    # Validate version
+    try:
+        version = int(required_version)
+        if version < 1 or version > 9:
+            print("Error: Bootloader version must be between 1 and 9")
+            return False
+    except ValueError:
+        print("Error: Bootloader version must be a number")
+        return False
+
+    # Create metadata string
+    metadata = f"WLED_BOOTLOADER:{version}"
+
+    # Check if metadata already exists
+    try:
+        with open(binary_file, 'rb') as f:
+            content = f.read()
+
+        if metadata.encode('ascii') in content:
+            print(f"File already contains bootloader v{version} requirement")
+            return True
+
+        # Check for any bootloader metadata
+        if b"WLED_BOOTLOADER:" in content:
+            print("Warning: File already contains bootloader metadata. Adding new requirement.")
+    except Exception as e:
+        print(f"Error reading file: {e}")
+        return False
+
+    # Append metadata to file
+    try:
+        with open(binary_file, 'ab') as f:
+            f.write(metadata.encode('ascii'))
+        print(f"Successfully added bootloader v{version} requirement to {binary_file}")
+        return True
+    except Exception as e:
+        print(f"Error writing to file: {e}")
+        return False
+
+def main():
+    if len(sys.argv) != 3:
+        print("Usage: python add_bootloader_metadata.py <binary_file> <required_version>")
+        print("Example: python add_bootloader_metadata.py firmware.bin 4")
+        sys.exit(1)
+
+    binary_file = sys.argv[1]
+    required_version = sys.argv[2]
+
+    if add_bootloader_metadata(binary_file, required_version):
+        sys.exit(0)
+    else:
+        sys.exit(1)
+
+if __name__ == "__main__":
+    main()

tools/bootloader_metadata_README.md

@@ -0,0 +1,54 @@
+# Bootloader Metadata Tool
+
+This tool adds bootloader version requirements to WLED firmware binaries to prevent incompatible OTA updates.
+
+## Usage
+
+```bash
+python3 tools/add_bootloader_metadata.py <binary_file> <required_version>
+```
+
+Example:
+```bash
+python3 tools/add_bootloader_metadata.py firmware.bin 4
+```
+
+## Bootloader Versions
+
+- **Version 2**: Legacy bootloader (ESP-IDF < 4.4)
+- **Version 3**: Intermediate bootloader (ESP-IDF 4.4+)  
+- **Version 4**: Modern bootloader (ESP-IDF 5.0+) with rollback support
+
+## How It Works
+
+1. The script appends a metadata tag `WLED_BOOTLOADER:X` to the binary file
+2. During OTA upload, WLED checks the first 512 bytes for this metadata
+3. If found, WLED compares the required version with the current bootloader
+4. The update is blocked if the current bootloader is incompatible
+
+## Metadata Format
+
+The metadata is a simple ASCII string: `WLED_BOOTLOADER:X` where X is the required bootloader version (1-9).
+
+This approach was chosen over filename-based detection because users often rename firmware files.
+
+## Integration with Build Process
+
+To automatically add metadata during builds, add this to your platformio.ini:
+
+```ini
+[env:your_env]
+extra_scripts = post:add_metadata.py
+```
+
+Create `add_metadata.py`:
+```python
+Import("env")
+import subprocess
+
+def add_metadata(source, target, env):
+    firmware_path = str(target[0])
+    subprocess.run(["python3", "tools/add_bootloader_metadata.py", firmware_path, "4"])
+
+env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", add_metadata)
+```

wled00/fcn_declare.h

@@ -466,6 +466,10 @@ void handleBootLoop();   // detect and handle bootloops
 #ifndef ESP8266
 void bootloopCheckOTA(); // swap boot image if bootloop is detected instead of restoring config
 #endif
+#ifndef WLED_DISABLE_OTA
+uint32_t getBootloaderVersion(); // get current bootloader version
+bool isBootloaderCompatible(uint32_t required_version); // check bootloader compatibility
+#endif
 // RAII guard class for the JSON Buffer lock
 // Modeled after std::lock_guard
 class JSONBufferGuard {

wled00/util.cpp

@@ -10,6 +10,11 @@
 #elif ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(3, 3, 0)
   #include "soc/rtc.h"
 #endif
+#ifndef WLED_DISABLE_OTA
+
+  #include "esp_flash.h"     // for direct flash access
+  #include "esp_log.h"       // for error handling
+#endif
 #endif
 
 
@@ -857,6 +862,65 @@ void handleBootLoop() {
   ESP.restart(); // restart cleanly and don't wait for another crash
 }
 
+#ifndef WLED_DISABLE_OTA
+#ifdef ESP32
+
+// Get bootloader version for OTA compatibility checking
+// Uses rollback capability as primary indicator since bootloader description 
+// structure is only available in ESP-IDF v5+ bootloaders
+uint32_t getBootloaderVersion() {
+  static uint32_t cached_version = 0;
+  if (cached_version != 0) return cached_version;
+
+  DEBUG_PRINTF_P(PSTR("Determining bootloader version...\n"));
+
+  #ifndef WLED_DISABLE_OTA
+  bool can_rollback = Update.canRollBack();
+  #else
+  bool can_rollback = false;
+  #endif
+
+  DEBUG_PRINTF_P(PSTR("Rollback capability: %s\n"), can_rollback ? "YES" : "NO");
+
+  if (can_rollback) {
+    // Rollback capability indicates v4+ bootloader
+    cached_version = 4;
+    DEBUG_PRINTF_P(PSTR("Bootloader v4+ detected (rollback capable)\n"));
+  } else {
+    // No rollback capability - check ESP-IDF version for best guess
+    #if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(4, 4, 0)
+      cached_version = 3;
+      DEBUG_PRINTF_P(PSTR("Bootloader v3 detected (ESP-IDF 4.4+)\n"));
+    #elif ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(4, 0, 0)
+      cached_version = 2;
+      DEBUG_PRINTF_P(PSTR("Bootloader v2 detected (ESP-IDF 4.x)\n"));
+    #else
+      cached_version = 1;
+      DEBUG_PRINTF_P(PSTR("Bootloader v1/legacy detected (ESP-IDF 3.x)\n"));
+    #endif
+  }
+
+  DEBUG_PRINTF_P(PSTR("getBootloaderVersion() returning: %d\n"), cached_version);
+  return cached_version;
+}
+
+// Check if current bootloader is compatible with given required version
+bool isBootloaderCompatible(uint32_t required_version) {
+  uint32_t current_version = getBootloaderVersion();
+  bool compatible = current_version >= required_version;
+
+  DEBUG_PRINTF_P(PSTR("Bootloader compatibility check: current=%d, required=%d, compatible=%s\n"), 
+                 current_version, required_version, compatible ? "YES" : "NO");
+
+  return compatible;
+}
+#else
+// ESP8266 compatibility functions - always assume compatible for now
+uint32_t getBootloaderVersion() { return 1; }
+bool isBootloaderCompatible(uint32_t required_version) { return true; }
+#endif
+#endif
+
 /*
  * Fixed point integer based Perlin noise functions by @dedehai
  * Note: optimized for speed and to mimic fastled inoise functions, not for accuracy or best randomness

wled00/wled.h

@@ -106,6 +106,10 @@
     #include <LittleFS.h>
   #endif
   #include "esp_task_wdt.h"
+
+  #ifndef WLED_DISABLE_OTA
+    #include "esp_ota_ops.h"
+  #endif
 
   #ifndef WLED_DISABLE_ESPNOW
     #include <esp_now.h>

wled00/wled_server.cpp

@@ -388,6 +388,28 @@ void initServer()
 
   createEditHandler(correctPIN);
 
+  // Bootloader info endpoint for troubleshooting
+  server.on("/bootloader", HTTP_GET, [](AsyncWebServerRequest *request){
+    AsyncJsonResponse *response = new AsyncJsonResponse(128);
+    JsonObject root = response->getRoot();
+
+    #ifdef ESP32
+    root[F("version")] = getBootloaderVersion();
+    #ifndef WLED_DISABLE_OTA
+    root[F("rollback_capable")] = Update.canRollBack();
+    #else
+    root[F("rollback_capable")] = false;
+    #endif
+    root[F("esp_idf_version")] = ESP_IDF_VERSION;
+    #else
+    root[F("rollback_capable")] = false;
+    root[F("platform")] = F("ESP8266");
+    #endif
+
+    response->setLength();
+    request->send(response);
+  });
+
   static const char _update[] PROGMEM = "/update";
 #ifndef WLED_DISABLE_OTA
   //init ota page
@@ -426,6 +448,41 @@ void initServer()
     if (!correctPIN || otaLock) return;
     if(!index){
       DEBUG_PRINTLN(F("OTA Update Start"));
+
+      #ifndef WLED_DISABLE_OTA
+      // Check for bootloader compatibility metadata in first chunk
+      if (len >= 32) {
+        // Look for metadata header: "WLED_BOOTLOADER:X" where X is required version
+        const char* metadata_prefix = "WLED_BOOTLOADER:";
+        size_t prefix_len = strlen(metadata_prefix);
+
+        // Search for metadata in first 512 bytes or available data, whichever is smaller
+        size_t search_len = (len > 512) ? 512 : len;
+        for (size_t i = 0; i <= search_len - prefix_len - 1; i++) {
+          if (memcmp(data + i, metadata_prefix, prefix_len) == 0) {
+            // Found metadata header, extract required version
+            char version_char = data[i + prefix_len];
+            if (version_char >= '1' && version_char <= '9') {
+              uint32_t required_version = version_char - '0';
+
+              DEBUG_PRINTF_P(PSTR("OTA file requires bootloader v%d\n"), required_version);
+
+              if (!isBootloaderCompatible(required_version)) {
+                DEBUG_PRINTF_P(PSTR("Bootloader incompatible! Current: v%d, Required: v%d\n"), 
+                              getBootloaderVersion(), required_version);
+                request->send(400, FPSTR(CONTENT_TYPE_PLAIN), 
+                             F("Bootloader incompatible! This firmware requires bootloader v4+. "
+                               "Please update via USB using install.wled.me first, or use WLED 0.15.x."));
+                return;
+              }
+              DEBUG_PRINTLN(F("Bootloader compatibility check passed"));
+              break;
+            }
+          }
+        }
+      }
+      #endif
+
       #if WLED_WATCHDOG_TIMEOUT > 0
       WLED::instance().disableWatchdog();
       #endif