Enqueue messages

* queue outgoing messages
* add optional debug logging
* add simple "mutex" for ESP8266
* add connection states
* add CI (Github Actions)
* update deps
* pass disconnectreason to user
* update docs (#252)
* various bugfixes

Author: bertmelis

.github/workflows/build_examples_pio.yml

@@ -0,0 +1,27 @@
+name: Build with Platformio
+
+on: [push, pull_request]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v1
+    - name: Set up Python
+      uses: actions/setup-python@v1
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install platformio
+    - name: Add libraries
+      run: |
+        platformio lib -g install AsyncTCP
+        platformio lib -g install ESPAsyncTCP
+    - name: Getting ready
+      run: |
+        chmod +x ./scripts/CI/build_examples_pio.sh
+    - name: Build examples
+      run: |
+        ./scripts/CI/build_examples_pio.sh

.github/workflows/cpplint.yml

@@ -0,0 +1,20 @@
+name: cpplint
+
+on: [push, pull_request]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v1
+    - name: Set up Python
+      uses: actions/setup-python@v1
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install cpplint
+    - name: Linting
+      run: |
+        cpplint --repository=. --recursive --filter=-whitespace/line_length,-legal/copyright,-runtime/printf,-build/include,-build/namespace ./src

.gitignore

@@ -1 +1,2 @@
 /config.json
+.vscode/

.travis.yml

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2015 Marvin Roger
+Copyright (c) 2015-2021 Marvin Roger
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

LICENSE

@@ -1,9 +1,10 @@
-Async MQTT client for ESP8266 and ESP32
-=============================
+# Async MQTT client for ESP8266 and ESP32
 
-[![Build Status](https://img.shields.io/travis/marvinroger/async-mqtt-client/master.svg?style=flat-square)](https://travis-ci.org/marvinroger/async-mqtt-client)
+![Build with PlatformIO](https://github.com/marvinroger/async-mqtt-client/workflows/Build%20with%20Platformio/badge.svg)
+![cpplint](https://github.com/marvinroger/async-mqtt-client/workflows/cpplint/badge.svg)
 
 An Arduino for ESP8266 and ESP32 asynchronous [MQTT](http://mqtt.org/) client implementation, built on [me-no-dev/ESPAsyncTCP (ESP8266)](https://github.com/me-no-dev/ESPAsyncTCP) | [me-no-dev/AsyncTCP (ESP32)](https://github.com/me-no-dev/AsyncTCP) .
+
 ## Features
 
 * Compliant with the 3.1.1 version of the protocol

README.md

@@ -2,9 +2,9 @@
 
 To use AsyncMqttClient, you need:
 
-* An ESP8266
-* The Arduino IDE for ESP8266 (version 2.2.0 minimum)
-* Basic knowledge of the Arduino environment (upload a sketch, import libraries, ...)
+* An ESP8266 or ESP32
+* The Arduino IDE or equivalent IDE for ESP8266/32
+* Basic knowledge of the Arduino environment (use the IDE, upload a sketch, import libraries, ...)
 
 ## Installing AsyncMqttClient
 
@@ -15,7 +15,9 @@ There are two ways to install AsyncMqttClient.
 1. Download the [corresponding release](https://github.com/marvinroger/async-mqtt-client/releases/latest)
 2. Load the `.zip` with **Sketch → Include Library → Add .ZIP Library**
 
-AsyncMqttClient has 1 dependency: [ESPAsyncTCP](https://github.com/me-no-dev/ESPAsyncTCP). Download the [.zip](https://github.com/me-no-dev/ESPAsyncTCP/archive/master.zip) and install it with the same method as above.
+AsyncMqttClient has 1 dependency:
+* For ESP8266: [ESPAsyncTCP](https://github.com/me-no-dev/ESPAsyncTCP). Download the [.zip](https://github.com/me-no-dev/ESPAsyncTCP/archive/master.zip) and install it with the same method as above.
+* Fors ESP32: [AsyncTCP](https://github.com/me-no-dev/AsyncTCP). Download the [.zip](https://github.com/me-no-dev/AsyncTCP/archive/master.zip) and install it with the same method as above.
 
 ## Fully-featured sketch
 

docs/1.-Getting-started.md

@@ -156,5 +156,11 @@ Return the packet ID (or 1 if QoS 0) or 0 if failed.
 * **`retain`**: Retain flag
 * **`payload`**: Payload. If unset, the payload will be empty
 * **`length`**: Payload length. If unset or set to 0, the payload will be considered as a string and its size will be calculated using `strlen(payload)`
-* **`dup`**: Duplicate flag. If set or set to 1, the payload will be flagged as a duplicate
-* **`message_id`**: The message ID. If unset or set to 0, the message ID will be automtaically assigned. Use this with the DUP flag to identify which message is being duplicated
+* **`dup`**: ~~Duplicate flag. If set or set to 1, the payload will be flagged as a duplicate~~ Setting is not used anymore
+* **`message_id`**: ~~The message ID. If unset or set to 0, the message ID will be automtaically assigned. Use this with the DUP flag to identify which message is being duplicated~~ Setting is not used anymore
+
+#### bool clearQueue()
+
+When disconnected, clears all queued messages
+
+Returns true on succes, false on failure (client is no disconnected)

docs/2.-API-reference.md

@@ -1,7 +1,12 @@
 # Memory management
 
-AsyncMqttClient does not use an internal buffer, it uses the raw TCP buffer.
+AsyncMqttClient buffers outgoing messages in a queue. On sending data is copied to a raw TCP buffer. Received data is passed directly to the API.
 
-The max receive size is about 1460 bytes per call to your onMessage callback. But the amount of data you can receive is unlimited, as if you receive, say, a 300kB payload (such as an OTA payload), then your `onMessage` callback will be called about 200 times, with the according len, index and total parameters. Keep in mind the library will call your `onMessage` callbacks with the same topic buffer, so if you change the buffer on one call, the buffer will remain changed on subsequent calls.
+## Outgoing messages
 
-You can send data as long as you stay below the available TCP window (which is about 3-4kB on the ESP8266). The data is indeed held in memory by the async TCP code until ACK is received. If the TCP window was sufficient to send your packet, the `publish` method will return a packet ID indicating the packet was sent. Otherwise, a `0` will be returned, and it's your responsability to resend the packet with `publish`.
+You can send data as long as memory permits. A minimum amount of free memory is set at 4096 bytes. You can lower (or raise) this value by setting `MQTT_MIN_FREE_MEMORY` to your desired value.
+If the free memory was sufficient to send your packet, the `publish` method will return a packet ID indicating the packet was queued. Otherwise, a `0` will be returned, and it's your responsability to resend the packet with `publish`.
+
+## Incoming messages
+
+No incoming data is buffered by this library. Messages received by the TCP library is passed directly to the API. The max receive size is about 1460 bytes per call to your onMessage callback but the amount of data you can receive is unlimited. If you receive, say, a 300kB payload (such as an OTA payload), then your `onMessage` callback will be called about 200 times, with the according len, index and total parameters. Keep in mind the library will call your `onMessage` callbacks with the same topic buffer, so if you change the buffer on one call, the buffer will remain changed on subsequent calls.

docs/3.-Memory-management.md

@@ -1,19 +1,19 @@
 # Limitations and known issues
 
-* When the CleanSession is set to `false`, the implementation is not spec compliant. The following is not honored:
+* The library is spec compliant with one limitation. In case of power loss the following is not honored:
 
 > Must be kept in memory:
 * All messages in a QoS 1 or 2 flow, which are not confirmed by the broker
 * All received QoS 2 messages, which are not yet confirmed to the broker
 
-This means retransmission is not honored in case of a failure.
+This means retransmission is not honored in case of a power failure. This behaviour is like explained in point 4.1.1 of the MQTT specification v3.1.1
 
 * You cannot send payload larger that what can fit on RAM.
 
 ## SSL limitations
 
-* SSL requires use of esp8266/Arduino 2.4.0, which is not yet released (platform = espressif8266_stage in PlatformIO).
 * SSL requires the build flag -DASYNC_TCP_SSL_ENABLED=1
 * SSL only supports fingerprints for server validation.
 * If you do not specify one or more acceptable server fingerprints, the SSL connection will be vulnerable to man-in-the-middle attacks.
-* Some server certificate signature algorithms do not work. SHA1, SHA224, SHA256, and MD5 are working. SHA384, and SHA512 will cause a crash. 
+* Some server certificate signature algorithms do not work. SHA1, SHA224, SHA256, and MD5 are working. SHA384, and SHA512 will cause a crash.
+* TLS1.2 is not supported.

docs/4.-Limitations-and-known-issues.md

@@ -1,3 +1,11 @@
 # Troubleshooting
 
-To be completed when issues arise.
+* The payload of incoming messages contains **raw data**. You cannot just print out the data without formatting. This is because Arduino's `print` functions expect a C-string as input and a MQTT payload is not. A simple solution is to print each character of the payload:
+
+```cpp
+for (size_t i = 0; i < len; ++i) {
+    Serial.print(payload[i]);
+}
+```
+
+Further reading: https://en.wikipedia.org/wiki/C_string_handling

docs/5.-Troubleshooting.md

@@ -20,6 +20,11 @@ void connectToWifi() {
   WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
 }
 
+void connectToMqtt() {
+  Serial.println("Connecting to MQTT...");
+  mqttClient.connect();
+}
+
 void onWifiConnect(const WiFiEventStationModeGotIP& event) {
   Serial.println("Connected to Wi-Fi.");
   connectToMqtt();
@@ -31,11 +36,6 @@ void onWifiDisconnect(const WiFiEventStationModeDisconnected& event) {
   wifiReconnectTimer.once(2, connectToWifi);
 }
 
-void connectToMqtt() {
-  Serial.println("Connecting to MQTT...");
-  mqttClient.connect();
-}
-
 void onMqttConnect(bool sessionPresent) {
   Serial.println("Connected to MQTT.");
   Serial.print("Session present: ");

examples/FullyFeatured-ESP8266/FullyFeatured-ESP8266.ino

@@ -33,6 +33,7 @@ disconnect	KEYWORD2
 subscribe	KEYWORD2
 unsubscribe	KEYWORD2
 publish	KEYWORD2
+clearQueue	KEYWORD2
 
 #######################################
 # Constants (LITERAL1)

keywords.txt

@@ -12,18 +12,18 @@
     "type": "git",
     "url": "https://github.com/marvinroger/async-mqtt-client.git"
   },
-  "version": "0.8.2",
+  "version": "0.9.0",
   "frameworks": "arduino",
   "platforms": ["espressif8266", "espressif32"],
   "dependencies": [
     {
       "name": "ESPAsyncTCP",
-      "version": "1.2.0",
+      "version": ">=1.2.2",
       "platforms": "espressif8266"
     },
     {
       "name": "AsyncTCP",
-      "version": "^1.0.0",
+      "version": ">=1.1.1",
       "platforms": "espressif32"
     }
   ]

library.json

@@ -1,5 +1,5 @@
 name=AsyncMqttClient
-version=0.8.2
+version=0.9.0
 author=Marvin ROGER
 maintainer=Marvin ROGER
 sentence=An Arduino for ESP8266 and ESP32 asynchronous MQTT client implementation

library.properties

@@ -0,0 +1,49 @@
+#!/bin/bash
+
+#pip install -U platformio
+#platformio update
+platformio lib -g install AsyncTCP
+platformio lib -g install ESPAsyncTCP
+
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[0;33m'
+NC='\033[0m'
+
+lines=$(find ./examples/ -maxdepth 1 -mindepth 1 -type d)
+retval=0
+while read line; do
+  if [[ "$line" != *ESP8266 && "$line" != *ESP32 ]]
+  then
+    echo -e "========================== BUILDING $line =========================="
+    echo -e "${YELLOW}SKIPPING${NC}"
+    continue
+  fi
+  echo -e "========================== BUILDING $line =========================="
+  if [[ -e "$line/platformio.ini" ]]
+  then
+    # skipping
+    #output=$(platformio ci --lib="." --project-conf="$line/platformio.ini" $line 2>&1)
+    :
+  else
+    if [[ "$line" == *ESP8266 ]]
+    then
+      output=$(platformio ci --lib="." --project-conf="scripts/CI/platformio_esp8266.ini" $line 2>&1)
+    else
+      output=$(platformio ci --lib="." --project-conf="scripts/CI/platformio_esp32.ini" $line 2>&1)
+    fi
+  fi
+  if [ $? -ne 0 ]; then
+    echo "$output"
+    echo -e "Building $line ${RED}FAILED${NC}"
+    retval=1
+  else
+    echo -e "${GREEN}SUCCESS${NC}"
+  fi
+done <<< "$lines"
+
+# cleanup
+platformio lib -g uninstall AsyncTCP
+platformio lib -g uninstall ESPAsyncTCP
+
+exit "$retval"

scripts/CI/build_examples_pio.sh

@@ -0,0 +1,16 @@
+; PlatformIO Project Configuration File
+;
+;   Build options: build flags, source filter
+;   Upload options: custom upload port, speed and extra flags
+;   Library options: dependencies, extra library storages
+;   Advanced options: extra scripting
+;
+; Please visit documentation for the other options and examples
+; https://docs.platformio.org/page/projectconf.html
+
+[env:esp32]
+platform = espressif32
+board = esp32dev
+framework = arduino
+build_flags =
+  -Wall

scripts/CI/platformio_esp32.ini

@@ -0,0 +1,16 @@
+; PlatformIO Project Configuration File
+;
+;   Build options: build flags, source filter
+;   Upload options: custom upload port, speed and extra flags
+;   Library options: dependencies, extra library storages
+;   Advanced options: extra scripting
+;
+; Please visit documentation for the other options and examples
+; https://docs.platformio.org/page/projectconf.html
+
+[env:esp8266]
+platform = espressif8266
+board = esp01_1m
+framework = arduino
+build_flags =
+  -Wall