Modbus RTU and ESPHome

[GeorgeIoak]

I've got an installation that doesn't afford connecting the inverter to ethernet easily. As a workaround I flashed a module with ESPHome and loaded Modbus and added sensors like this:

  - platform: modbus_controller
    modbus_controller_id: solaredge
    name: "L1 Voltage"
    id: l1_voltage
    register_type: holding
    address: 40076
    register_count: 1
    unit_of_measurement: "V"
    value_type: U_WORD
    accuracy_decimals: 1
    filters:
      - multiply: 0.1
      - filter_out: 6553.5
      - filter_out: 1413.1

That works for now but I was wondering if anyone had any ideas on easily converting my setup in order to use this HA integration. The only idea I thought of was to drop ESPHome and write a program that uses RTU to read the inverter registers and then host a Modbus TCP to mimic an inverter. In theory that would work but that seems like a lot of work.

I know there's WiFi to Ethernet adapters that people have used but I already had this hardware so I thought I'd try it before throwing money at the "problem".

The other idea is that perhaps if I just create sensors with the same names as this integration then everything else would just work but there seems to be a bunch of sensors created and I'm not sure if all of them is needed.

[mpredfearn]

What you really want to do is turn the ESP into a modbus proxy, so that the HA integration talks to the ESP over WiFi and then the ESP proxies the modbus-TCP into modbus over RS485. Surprisingly few hits on Github to do that, you'd think it would be quite a common thing to want to do?
To be honest, the amount of effort that might be involved in inventing and maintaining a system like that, you're probably better off with either getting a hold of the Solaredge WiFi module, or using a cheap TP-Link router as a WiFi to Ethernet bridge. It'll be much less effort.

[GeorgeIoak]

I read that perhaps a recent SolarEdge firmware update enabled having MODBUS over WiFi. I haven't read more about this but you mentioned a SE WiFi module so I'm guessing you need that the WiFi they already have for the AP is different and can't be used for the MODBUS interface?

I am a bit surprised that the solution I came up with doesn't appear to be used by anyone. It's far from ideal but it seems to be working and it a quick and somewhat easy solution for those not able to get an Ethernet cable to the inverter. That's still my ultimate goal but I was hoping for an interum solution for now.

[mpredfearn]

Maybe I got lucky, but my inverter firmware never disabled modbus over WiFi. My inverter has the SE WiFi module in it and that enables the connection to solaredge cloud for the app and also allows modbus over TCP for this integration. I just had to enable modbus TCP in the inverter settings and then it just worked.

Since it was installed I've pulled Ethernet round the house and now have a switch quite nearby the inverter, so at some point I should get around to cabling it in....

[GeorgeIoak]

I'm curious how you were able to get the inverter onto your network. I don't recall seeing anything like that when you put it into AP mode and go into the settings.

[WillCodeForCats]

My inverters were installed with hardwired Ethernet from day 1 so I've never had to look at any other options, but if I had to use something wireless I would probably try like a TP Link TL-WR802N since they're $23 on Amazon in the US. For me at that price it's not worth trying to do any custom workarounds.

I could look into adding modbus/rtu serial support to the integration in addition to modbus/tcp if that would help. Maybe there's something that transparently bridges serial between two esp units instead of trying to convert it to tcp.

[GeorgeIoak]

Your lucky! I thought I had a good solution but it is creating some work. I'm a bit stubborn so I might keep plugging away at this for a bit more just in the name of science!

I do have a question though. I've worked out the signed and unisigned registers and I'm getting what looks to be sane values but I'm not seeing the data track what comes up with my Tesla app. I don't have a meter and only single phase. Do you happen to know what should track with the app (which happens to match the standard dial meter on the side of the house). I know there will be a lag but the values don't seem close.

[WillCodeForCats]

I've never used or seen the Tesla app so can't really say where it gets its data from.

[GeorgeIoak]

Looks like it tracks AC Power (Register 40083) but it's more delayed than I thought it would be which through me off initially.

[GeorgeIoak]

In case anyone comes across this thread I'll post what appears to be working for me:

  - platform: modbus_controller
    modbus_controller_id: solaredge
    name: "AC Voltage Scale"
    id: solaredge_ac_voltage_scale
    register_type: holding
    address: 40082
    register_count: 1
    unit_of_measurement: ""
    value_type: S_WORD
    filters:
      - filter_out: 6553.5
      - filter_out: 1413.1

  - platform: modbus_controller
    modbus_controller_id: solaredge
    name: "AC Voltage Phase AB Voltage"
    id: solaredge_ab_voltage
    register_type: holding
    address: 40076
    register_count: 1
    unit_of_measurement: "V"
    value_type: U_WORD
    accuracy_decimals: 1
    filters:
#      - multiply: 0.1
      - filter_out: 6553.5
      - filter_out: 1413.1
      - lambda: return x * powf(10.0, id(solaredge_ac_voltage_scale).state);

I need to still clean up some sensors that were created during my testing that I've renamed and I think because I had this integration installed my sensor names are not coming through as solaredge_xxxx

Note that in the SE spec if the register is int16 then the ESPHome type is S_WORD and uint16 is U_WORD. Total AC Lifetime Energy Production (Register 40093) is acc32 and for that you use U_DWORD

[WillCodeForCats]

Maybe this would work:

https://github.com/emelianov/modbus-esp8266/tree/master/examples/Bridge

ModbusRTU to ModbusTCP bridge

[GeorgeIoak]

Those do look interesting, I'll have to check them out later today when I have more time.

I did work more on this over the weekend and finally got the bugs worked out and so far today it looks like it's handling the change in scale factors correctly. Even though ESPHome was reading multiple registers in one read when I used the lambda function on a sensor it wasn't using the current read value but the last read value. Since the scale factor registers come after the value registers that makes sense and my code above work have worked if SE had reversed the order of the values and scales.
So I defined internal register reads and then at the end created a template sensor to perform the above calculation. I originally was trying to simplify things but I couldn't find a way around this if you're using ESPHome.

  - platform: modbus_controller
    modbus_controller_id: solaredge
    name: "Total Real Power Raw"
    id: solaredge_ac_power_raw
    internal: true
    register_type: holding
    address: 40083
    register_count: 1
    device_class: "power"
    state_class: "measurement"
    unit_of_measurement: "W"
    value_type: S_WORD
    accuracy_decimals: 1
    filters:
      - filter_out: nan
      - filter_out: -32768

...
  
  - platform: template
    name: "Total Power"
    id: solaredge_ac_power
    device_class: "power"
    state_class: "measurement"
    unit_of_measurement: "W"
    lambda: |-
      return id(solaredge_ac_power_raw).state * powf(10.0, id(solaredge_ac_power_scale).state);

I did the same technique for status so in HA it displays the text version:

  - name: "Se Inverter Status"
    platform: template
    id: solaredge_text_status
    lambda: |-
      switch (int(id(solaredge_status).state)) {
        case 1: return std::string("Off");
        case 2: return std::string("Sleeping");
        case 3: return std::string("Grid Monitoring");
        case 4: return std::string("ON, Producing Power");
        case 5: return std::string("Production");
        case 6: return std::string("Shutting Down");
        case 7: return std::string("FAULT");
        case 8: return std::string("Maintenance");
        default: return std::string("Unknown");
        }

Probably would have been much easier just writing C code and not relying on ESPHome but I wasn't sure how to easily integrate with HA unless I piped the MODBUS to MQTT.

Now I bet after I look at the 2 bridges you found I'll probably have wished I did more research before continuing down the ESPHome path!

[GeorgeIoak]

Man, I spoke too soon! I made it through almost a complete day without any scale factor errors but I just got one:

I'm wondering with your implementation do you ever see this happen? Right now I'm reading every 10 seconds but it should be readings all the registers at the same time so there shouldn't be a mis-match. It initially goes from a scale of -1 to -2 at 3:59:34 and then for 1 reading at 4:00:14 it changes back to -1 and now it's at -2 again

[GeorgeIoak]

Supposedly that's what ESPHome does when I defined ALL of the sensors sequentially. Initially I was doing a read of the scale first and then the values but I changed and that seemed to work except for this one time today.

[WillCodeForCats]

FYI the HA core modbus integration cannot do block reads, so it's not possible to read the sensors and scale factor in a single read.

[WillCodeForCats]

For inverter current you need to read 40071 through 40075 in a single register read transaction to get the scale factor right. Anytime you see the scale factor is wrong it's a clear sign that whatever is doing the reading is not actually reading the registers in a single modbus transaction. When read in a single transaction all the values in it are fixed at that point in time.

[WillCodeForCats]

Actually yours was watts so that's 40083 and 40084, but the two have to be in a single transaction: never separately.

[GeorgeIoak]

I wish I could connect a Logic Analyzer or a scope (easily that is) so I can see exactly what ESPHome is doing. According to the documentation, it should be "converting" a series of sequentially defined sensors into 1 MODBUS read. I have 40071-40093 and then 40095-40101 but 40093 is the 32-bit Lifetime Energy so it really should be reading 40071 to 40101 all in 1 read.

[GeorgeIoak]

So It's not a scope but I was able to enable VERY_VERBOSE logging and in the console it does appear that it is sending a long read starting from 40071 (0x(C87) for 31 registers:

[18:50:28][V][modbus_controller:036]: Sending next modbus command to device 1 register 0x9C87 count 31
[18:50:28][VV][uart.arduino_esp32:151]:     Flushing...
[18:50:28][V][modbus:194]: Modbus write: 01.03.9C.87.00.1F.9A.7B (8)
[18:50:28][V][modbus_controller:486]: Command sent 3 0x9C87 31
[18:50:28][V][modbus_controller:036]: Sending next modbus command to device 1 register 0x9C87 count 31
[18:50:28][VV][uart.arduino_esp32:151]:     Flushing...
[18:50:28][V][modbus:194]: Modbus write: 01.03.9C.87.00.1F.9A.7B (8)
[18:50:28][V][modbus_controller:486]: Command sent 3 0x9C87 31
[18:50:28][V][modbus:042]: Modbus received Byte  1 (0X1)
[18:50:28][V][modbus:042]: Modbus received Byte  3 (0X3)
[18:50:28][V][modbus:042]: Modbus received Byte  62 (0X3e)
[18:50:28][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:28][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  254 (0Xfe)
[18:50:29][V][modbus:042]: Modbus received Byte  9 (0X9)
[18:50:29][V][modbus:042]: Modbus received Byte  105 (0X69)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  4 (0X4)
[18:50:29][V][modbus:042]: Modbus received Byte  182 (0Xb6)
[18:50:29][V][modbus:042]: Modbus received Byte  4 (0X4)
[18:50:29][V][modbus:042]: Modbus received Byte  179 (0Xb3)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  234 (0Xea)
[18:50:29][V][modbus:042]: Modbus received Byte  64 (0X40)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  253 (0Xfd)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  10 (0Xa)
[18:50:29][V][modbus:042]: Modbus received Byte  24 (0X18)
[18:50:29][V][modbus:042]: Modbus received Byte  130 (0X82)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  128 (0X80)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  255 (0Xff)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  0 (0X0)
[18:50:29][V][modbus:042]: Modbus received Byte  249 (0Xf9)
[18:50:29][V][modbus:042]: Modbus received Byte  214 (0Xd6)
[18:50:29][V][modbus_controller:055]: Modbus response queued
[18:50:29][V][component:200]: Component modbus took a long time for an operation (0.35 s).
[18:50:29][V][component:201]: Components should block for at most 20-30ms.
[18:50:29][V][modbus_controller:063]: Process modbus response for address 0x9C87 size: 62
[18:50:29][V][modbus_controller:098]: data for register address : 0x9C87 : 
[18:50:29][D][modbus_controller.sensor:025]: Sensor new state: 0.00

It does take 350ms for that to happen and it has a warning that it should only block for 20-30ms so I'm not sure what the best approach here is. I have the baud rate at 115,200 which I think is the highest SolarEdge supports. I could break up current, voltage, and power but then they would be out of sync when/if you tried to look at them together.

I might have to pop on the ESPHome Developer Discord tomorrow and see if I can get anyone to take a quick look and offer any advice. There were maybe 20 scale changes today for power and the code worked for all of them except that one so at least that's a big improvement from the other day.