sliding_window_moving_average causes boot looping.

[tomlut]

Operating environment/Installation (Hass.io/Docker/pip/etc.):

Hassio 0.85.1, ESPhome 1.10.0
ESP (ESP32/ESP8266/Board/Sonoff):

ESP8266 (Wemos D1 mini)

Affected component:

https://esphomelib.com/esphomeyaml/components/sensor/index.html#sensor-filters

Description of problem:
Including the recommended configuration to bring back a windowed moving average causes validation errors:

  - platform: adc
    pin: A0
    name: "Master Bed Sensor"
    update_interval: 15s
    filters:
      - multiply: 3.25
      - sliding_window_moving_average:

sensor.adc: [source /config/esphomeyaml/master_bed.yaml:37]
  platform: adc
  pin: A0
  name: Master Bed Sensor
  update_interval: 15s
  filters:  [source /config/esphomeyaml/master_bed.yaml:42]
    - multiply: 3.25
    - [source /config/esphomeyaml/master_bed.yaml:43]
      
      expected a dictionary.
      sliding_window_moving_average: 

Including the full sliding window filter configuration causes boot looping:

  - platform: adc
    pin: A0
    name: "Master Bed Sensor"
    update_interval: 15s
    filters:
      - multiply: 3.25
      - sliding_window_moving_average:
          window_size: 15
          send_every: 15

[18:51:56] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[18:51:56]
[18:51:56]wdt reset
[18:51:56]load 0x4010f000, len 1384, room 16 
[18:51:56]tail 8
[18:51:56]chksum 0x2d
[18:51:56]csum 0x2d
[18:51:56]vbb28d4a3
[18:51:56]~ld
[18:51:56][I][logger:071]: Log initialized
[18:51:56][C][ota:461]: There have been 3 suspected unsuccessful boot attempts.
[18:51:56][I][application:053]: Running through setup()...
[18:51:56][D][binary_sensor:027]: 'Master Bed Sensor Status': Sending state OFF
[18:51:56][C][sensor.adc:028]: Setting up ADC 'Master Bed Sensor'...
[18:51:56][C][wifi:029]: Setting up WiFi...
[18:51:56][D][wifi:260]: Starting scan...

Removing the sliding window average configuration compiles and runs as expected. Though the sensor is not averaged (as is wanted):

  - platform: adc
    pin: A0
    name: "Master Bed Sensor"
    update_interval: 15s
    filters:
      - multiply: 3.25

Problem-relevant YAML-configuration entries:

PASTE YAML FILE HERE

As per descrition above.
Traceback (if applicable):

Additional information:

[OttoWinter]

You're right about the first part, indeed only specifying sliding_window_moving_average: is invalid.

But with the configuration you gave here, I don't see any boot looping. The log you provided only contains the last part of the log, if it were to crash there would be a long stacktrace printed before that. I would need that stacktrace to debug this issue.

[tomlut]

I'll give it another go tomorrow and post the full log. Getting late here now.

[tomlut]

I tried updating to the following (clean build) OTA:

esphomeyaml:
  name: master_bed
  platform: ESP8266
  board: d1_mini

wifi:
  ssid: 'WAPLO'
  password: !secret wifi_pwd
  manual_ip:
    static_ip: 10.1.1.71
    gateway: 10.1.1.1
    subnet: 255.255.255.0

mqtt:
  discovery: true
  discovery_prefix: homeassistant
  broker: '10.1.1.100'
  username: !secret mqtt_usr
  password: !secret mqtt_pwd

logger:
  # level: WARN

ota:
  password: !secret esp_pwd

binary_sensor:
  - platform: status
    name: "Master Bed Sensor Status"

sensor:
  - platform: wifi_signal
    name: "Master Bed Sensor WiFi Signal"
    update_interval: 15s
    filters:
      - sliding_window_moving_average: {}

  - platform: adc
    pin: A0
    name: "Master Bed Sensor"
    update_interval: 4s
    filters:
      - multiply: 3.25
      - sliding_window_moving_average: {}

I can ping the device but I receive no sensor updates from it via the log or in HA. The device is shown as offline in ESPhome and the status is 'unavailable' in HA. I waited until the log terminated and a further 5 minutes for HA to show any sign of this sensor.

I removed the sliding window filters and tried re-uploading a clean build OTA. The ESP then correctly started logging to ESPhome and sending sensor readings again.

There really does seem to be something amiss with the sliding window average filter.

[tomlut]

Update, to simplify things I tried removing the multiplication filter and the sliding average from the wifi sensor leaving just this sensor:

  - platform: adc
    pin: A0
    name: "Master Bed Sensor"
    update_interval: 4s
    filters:
      - sliding_window_moving_average: {}

Same problem. The ESP never publishes any mqtt messages. It can be seen as online by ESPhome and can be pinged and updated OTA. Removing the sliding window filter and re-flashing resumes normal operation.

[tomlut]

I tried this using the exponential average instead.

  - platform: adc
    pin: A0
    name: "Master Bed Sensor"
    update_interval: 3s
    filters:
      - multiply: 3.25
      # - sliding_window_moving_average: {}
      - exponential_moving_average:
          alpha: 0.1
          send_every: 15

With this result:

[12:52:16]
[12:52:16] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[12:52:16]
[12:52:16]wdt reset
[12:52:16]load 0x4010f000, len 1384, room 16 
[12:52:16]tail 8
[12:52:16]chksum 0x2d
[12:52:16]csum 0x2d
[12:52:16]vbb28d4a3
[12:52:16]~ld
[12:52:16][E][ota:466]: Boot loop detected. Proceeding to safe mode.
[12:52:23][W][ota:136]: Last Boot was an unhandled reset, will proceed to safe mode in 0 restarts

Show Logs:

INFO Reading configuration...
INFO Starting log output from /dev/ttyUSB0 with baud rate 115200
[12:54:23][E][ota:485]: No OTA attempt made, restarting.
[12:54:23][W][wifi:727]: Event: Disconnected ssid='WAPLO' bssid=FC:EC:DA:38:94:62 reason='Association Leave'
[12:54:23]
[12:54:23] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[12:54:23]
[12:54:23]wdt reset
[12:54:23]load 0x4010f000, len 1384, room 16 
[12:54:23]tail 8
[12:54:23]chksum 0x2d
[12:54:23]csum 0x2d
[12:54:23]vbb28d4a3
[12:54:23]~ld
[12:54:32]
[12:54:32] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[12:54:32]
[12:54:32]wdt reset
[12:54:32]load 0x4010f000, len 1384, room 16 
[12:54:32]tail 8
[12:54:32]chksum 0x2d
[12:54:32]csum 0x2d
[12:54:32]vbb28d4a3
[12:54:32]~ld
[12:54:40]
[12:54:40] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[12:54:40]
[12:54:40]wdt reset
[12:54:40]load 0x4010f000, len 1384, room 16 
[12:54:40]tail 8
[12:54:40]chksum 0x2d
[12:54:40]csum 0x2d
[12:54:40]vbb28d4a3
[12:54:40]~ld
[12:54:49]
[12:54:49] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[12:54:49]
[12:54:49]wdt reset
[12:54:49]load 0x4010f000, len 1384, room 16 
[12:54:49]tail 8
[12:54:49]chksum 0x2d
[12:54:49]csum 0x2d
[12:54:49]vbb28d4a3
[12:54:49]~ld

[tomlut]

I just tried a new Wemos D1 Mini board. Same result. Boot loops with sliding_window_moving_average: {}, for the A0 sensor, works fine without the filter.

[tomlut]

PROGRESS!

I removed the wifi and status sensors. I could then use the filtered A0 sensor.

Adding either the wifi sensor or status sensor back into the configuration results in failure to connect to wifi and a watchdog time out / restart.

Removing the filter from the A0 sensor allows both the status and wifi sensor to be included.

Data and memory use when compiling does not seem to be the problem (about 50% and 30%).

I really like to be able to monitor the state of my sensors easily and this was working in the previous version (with built-in default filtering of sensors). Any chance you could investigate this Otto?

SUMMARY:
Working configs:

esphomeyaml:
  name: master_bed
  platform: ESP8266
  board: d1_mini

wifi:
  ssid: 'WAPLO'
  password: !secret wifi_pwd
  manual_ip:
    static_ip: 10.1.1.71
    gateway: 10.1.1.1
    subnet: 255.255.255.0

mqtt:
  discovery: true
  discovery_prefix: homeassistant
  broker: '10.1.1.100'
  username: !secret mqtt_usr
  password: !secret mqtt_pwd

logger:
  # level: WARN

ota:
  password: !secret esp_pwd

sensor:
  - platform: adc
    pin: A0
    name: "Master Bed Sensor"
    update_interval: 4s
    filters:
      - multiply: 3.25
      - sliding_window_moving_average:
          window_size: 15
          send_every: 15

OR

esphomeyaml:
  name: master_bed
  platform: ESP8266
  board: d1_mini

wifi:
  ssid: 'WAPLO'
  password: !secret wifi_pwd
  manual_ip:
    static_ip: 10.1.1.71
    gateway: 10.1.1.1
    subnet: 255.255.255.0

mqtt:
  discovery: true
  discovery_prefix: homeassistant
  broker: '10.1.1.100'
  username: !secret mqtt_usr
  password: !secret mqtt_pwd

logger:
  # level: WARN

ota:
  password: !secret esp_pwd

binary_sensor:
  - platform: status
    name: "Master Bed Sensor Status"

sensor:
  - platform: wifi_signal
    name: "Master Bed Sensor WiFi Signal"
    update_interval: 3min
    filters:
      - sliding_window_moving_average:
          window_size: 15
          send_every: 15

  - platform: adc
    pin: A0
    name: "Master Bed Sensor"
    update_interval: 60s

Configs that upload but fail to run:

esphomeyaml:
  name: master_bed
  platform: ESP8266
  board: d1_mini

wifi:
  ssid: 'WAPLO'
  password: !secret wifi_pwd
  manual_ip:
    static_ip: 10.1.1.71
    gateway: 10.1.1.1
    subnet: 255.255.255.0

mqtt:
  discovery: true
  discovery_prefix: homeassistant
  broker: '10.1.1.100'
  username: !secret mqtt_usr
  password: !secret mqtt_pwd

logger:
  # level: WARN

ota:
  password: !secret esp_pwd

sensor:
  - platform: wifi_signal
    name: "Master Bed Sensor WiFi Signal"
    update_interval: 3min
    filters:
      - sliding_window_moving_average:
          window_size: 15
          send_every: 15

  - platform: adc
    pin: A0
    name: "Master Bed Sensor"
    update_interval: 60s
    filters:
      - multiply: 3.25
      - sliding_window_moving_average:
          window_size: 15
          send_every: 15

OR

esphomeyaml:
  name: master_bed
  platform: ESP8266
  board: d1_mini

wifi:
  ssid: 'WAPLO'
  password: !secret wifi_pwd
  manual_ip:
    static_ip: 10.1.1.71
    gateway: 10.1.1.1
    subnet: 255.255.255.0

mqtt:
  discovery: true
  discovery_prefix: homeassistant
  broker: '10.1.1.100'
  username: !secret mqtt_usr
  password: !secret mqtt_pwd

logger:
  # level: WARN

ota:
  password: !secret esp_pwd

binary_sensor:
  - platform: status
    name: "Master Bed Sensor Status"

sensor:

  - platform: adc
    pin: A0
    name: "Master Bed Sensor"
    update_interval: 60s
    filters:
      - multiply: 3.25
      - sliding_window_moving_average:
          window_size: 15
          send_every: 15

[OttoWinter]

@tomlut Ok so your logs provide some good info. Specifically, rst cause:4 means that there was a hardware WDT reset.

However, what is weird that nothing is displayed on boot. If the code would even reach the point where the moving average or anything else for that matter is initialized, the logs are set up.
And so I would except to see a line like Logs initialized! on each reboot. But that message is not there, so that means there's something fundamentally wrong with the ESP.
Do you have another ESP to check if it's something to do with the ESP?

I also tried your configuration files and was not able to replicate the issue.

[tomlut]

Hi Otto, I have 4 new Wemos D1 minis on order. Should be a week or two away.

I did try it with another D1 mini and got similar results, though thinking about it now, failure to connect to wifi might have been a router / assigned static IP issue as I was using this D1 for another sensor previously (so MAC address mismatch in the router). Doing this test also absolutely trashed my HA mqtt integration (duplicate ids / no unique ids from device). Even stopping HA and removing the entries from the .storage files did not help. I ended up having to do a snapshot restore.

Next time I test this I will disable discovery.

Just FYI, I also tried from the command line on Windows with similar results

I’m reasonably sure it is not a faulty device as this was working with the previous version of ESPhome. Though your inability to reproduce the issue is worrying. What hardware device are you using to replicate this?

[OttoWinter]

It was a wemos d1 mini as well. What's the most weird to me is the missing Logs initialized message, I cannot think of a reason why that would not show up.

[tomlut]

Ok if you can use A0 with the filter and other sensors specified in my config on the same hardware then it has to be a problem with my D1 board. I’ve tried reinstalling the addon and using the alternate command line approach on a whole different platform (Windows) so that rules out a problem with my ESPhome installation.

I’ll dig out the other D1 and try again tomorrow - with discovery disabled and a correct static IP assignment.

Thanks for your time.

[tomlut]

I deleted all reference to my other (spare-ish) D1 mini from my router and uploaded this config:

esphomeyaml:
  name: test_d1_mini
  platform: ESP8266
  board: d1_mini

wifi:
  ssid: 'WAPLO'
  password: !secret wifi_pwd
  # manual_ip:
    # static_ip: 10.1.1.71
    # gateway: 10.1.1.1
    # subnet: 255.255.255.0

mqtt:
  # discovery: true
  # discovery_prefix: homeassistant
  broker: '10.1.1.100'
  username: !secret mqtt_usr
  password: !secret mqtt_pwd

logger:
  # level: WARN

ota:
  password: !secret esp_pwd

binary_sensor:
  - platform: status
    name: "D1 Test Sensor Status"

sensor:
  - platform: wifi_signal
    name: "D1 Test WiFi Signal"
    update_interval: 15s
    filters:
      - sliding_window_moving_average:
          window_size: 15
          send_every: 15

  - platform: adc
    pin: A0
    name: "D1 A0 Test Sensor"
    update_interval: 4s
    filters:
      - multiply: 3.25
      - sliding_window_moving_average:
          window_size: 15
          send_every: 15

And it works. No issue at all. Looks like my initial D1 mini must have an issue with flash memory at one of the higher addresses - assuming that putting back discovery or a static IP address is not going to reintroduce the fault. But I cant test that until I can remove the existing device integration from HA. EDIT: see next post

I will close this issue now but would appreciate guidance on how best to remove the master_bed sensor from my HA integration. I'll ask over on the HA community forums though, to prevent cluttering up your issues feed.

Link: https://community.home-assistant.io/t/how-to-remove-an-integrated-esp-device/92764

Thanks for your help Otto.

[tomlut]

Oh damn. Buoyed by the above success I decided to update anotehr ESP D1 mini. And immediately lost contact with it. The config:

esphomeyaml:
  name: lounge_dht
  platform: ESP8266
  board: d1_mini

wifi:
  ssid: 'WAPLO'
  password: !secret wifi_pwd
  manual_ip:
    static_ip: 10.1.1.75
    gateway: 10.1.1.1
    subnet: 255.255.255.0

mqtt:
  discovery: true
  discovery_prefix: homeassistant
  broker: '10.1.1.100'
  username: !secret mqtt_usr
  password: !secret mqtt_pwd

logger:
  # level: WARN

ota:
  password: !secret esp_pwd

binary_sensor:
  - platform: status
    name: "Lounge Room DHT Status"

sensor:
  - platform: wifi_signal
    name: "Lounge Room DHT WiFi Signal"
    update_interval: 15s
    filters:
      - sliding_window_moving_average:
          window_size: 15
          send_every: 15

  - platform: dht
    pin: D5
    model: DHT22
    update_interval: 15s
    temperature:
      name: "Lounge Room Temperature"
      filters:
        - sliding_window_moving_average:
            window_size: 15
            send_every: 15
    humidity:
      name: "Lounge Room Humidity"
      filters:
        - sliding_window_moving_average:
            window_size: 15
            send_every: 15

The log:

NFO Reading configuration...
INFO Starting log output from /dev/ttyUSB0 with baud rate 115200
[14:49:16][I][ota:154]: Boot seems successful, resetting boot loop counter.
[14:49:24][E][ota:485]: No OTA attempt made, restarting.
[14:49:24][I][helpers:277]: Forcing a reboot... Reason: 'ota-safe-mode'
[14:49:24][W][wifi:727]: Event: Disconnected ssid='WAPLO' bssid=FC:EC:DA:38:94:62 reason='Association Leave'
[14:49:24]
[14:49:24] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[14:49:24]
[14:49:24]wdt reset
[14:49:24]load 0x4010f000, len 1384, room 16 
[14:49:24]tail 8
[14:49:24]chksum 0x2d
[14:49:24]csum 0x2d
[14:49:24]vbb28d4a3
[14:49:24]~ld
[14:49:24][I][logger:071]: Log initialized
[14:49:24][C][ota:461]: There have been 0 suspected unsuccessful boot attempts.
[14:49:24][I][application:053]: Running through setup()...
[14:49:24][D][binary_sensor:027]: 'Lounge Room DHT Status': Sending state OFF
[14:49:24][C][sensor.dht:030]: Setting up DHT...
[14:49:24][C][wifi:029]: Setting up WiFi...
[14:49:24][D][wifi:260]: Starting scan...
[14:49:32]
[14:49:32] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[14:49:32]
[14:49:32]wdt reset
[14:49:32]load 0x4010f000, len 1384, room 16 
[14:49:32]tail 8
[14:49:32]chksum 0x2d
[14:49:32]csum 0x2d
[14:49:32]vbb28d4a3
[14:49:32]~ld
[14:49:32][I][logger:071]: Log initialized
[14:49:32][C][ota:461]: There have been 1 suspected unsuccessful boot attempts.
[14:49:32][I][application:053]: Running through setup()...
[14:49:32][D][binary_sensor:027]: 'Lounge Room DHT Status': Sending state OFF
[14:49:32][C][sensor.dht:030]: Setting up DHT...
[14:49:32][C][wifi:029]: Setting up WiFi...
[14:49:32][D][wifi:260]: Starting scan...
[14:49:41]
[14:49:41] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[14:49:41]
[14:49:41]wdt reset
[14:49:41]load 0x4010f000, len 1384, room 16 
[14:49:41]tail 8
[14:49:41]chksum 0x2d
[14:49:41]csum 0x2d
[14:49:41]vbb28d4a3
[14:49:41]~ld
[14:49:41][I][logger:071]: Log initialized
[14:49:41][C][ota:461]: There have been 2 suspected unsuccessful boot attempts.
[14:49:41][I][application:053]: Running through setup()...
[14:49:41][D][binary_sensor:027]: 'Lounge Room DHT Status': Sending state OFF
[14:49:41][C][sensor.dht:030]: Setting up DHT...
[14:49:41][C][wifi:029]: Setting up WiFi...
[14:49:41][D][wifi:260]: Starting scan...
[14:49:49]
[14:49:49] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[14:49:49]
[14:49:49]wdt reset
[14:49:49]load 0x4010f000, len 1384, room 16 
[14:49:49]tail 8
[14:49:49]chksum 0x2d
[14:49:49]csum 0x2d
[14:49:49]vbb28d4a3
[14:49:49]~ld
[14:49:49][I][logger:071]: Log initialized
[14:49:49][C][ota:461]: There have been 3 suspected unsuccessful boot attempts.
[14:49:49][I][application:053]: Running through setup()...
[14:49:49][D][binary_sensor:027]: 'Lounge Room DHT Status': Sending state OFF
[14:49:49][C][sensor.dht:030]: Setting up DHT...
[14:49:49][C][wifi:029]: Setting up WiFi...
[14:49:49][D][wifi:260]: Starting scan...
[14:49:58]
[14:49:58] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[14:49:58]
[14:49:58]wdt reset
[14:49:58]load 0x4010f000, len 1384, room 16 
[14:49:58]tail 8
[14:49:58]chksum 0x2d
[14:49:58]csum 0x2d
[14:49:58]vbb28d4a3
[14:49:58]~ld
[14:49:58][I][logger:071]: Log initialized
[14:49:58][C][ota:461]: There have been 4 suspected unsuccessful boot attempts.
[14:49:58][I][application:053]: Running through setup()...
[14:49:58][D][binary_sensor:027]: 'Lounge Room DHT Status': Sending state OFF
[14:49:58][C][sensor.dht:030]: Setting up DHT...
[14:49:58][C][wifi:029]: Setting up WiFi...
[14:49:58][D][wifi:260]: Starting scan...
[14:50:06]
[14:50:06] ets Jan  8 2013,rst cause:4, boot mode:(3,6)
[14:50:06]