BLE Basic Infrastructure (#788)

Adds manufacturer-independent infrastructure for a BLE-based connection protocol, and OpenLCB's implemnentation classes and constants.

- Adds abstract classes for defining BLE Advertisements and Services.
- Adds concrete implementation of these with encoding the specific advertisements for GATT-based central and peripheral for OpenLCB, including the necessary UUID constants.
- Adds a Port implementation for DirectHub for sending text data (e.g. gridconnect frames) over BLE.

Since this is all manufacturer-independent, there are no external dependencies involved, but it also means that in itself, this basic infrastructure does not work. A significant amount of additional code is needed to tie this together with a specific manufacturer's BLE stack.

Misc changes:
- Makes SimpleStack be able to expose the PIP bytes via a direct API
- Fixes a compiler warning on LinkedDataBuffer

===

* Initial commit of BLE support.

* Add some BLE tests.

* Move from std::string to std::vector container.

* Add scan data tests.

* Add more tests.

* Fill in update() API logic.

* Move advertisements to use std::basic_string.

* Test advertisement updates.

* Add helpers for grabbing out specific advertisement payload fields.

* Fix include path.

* Add start of OpenLCB BLE advertisement definition.

* Fix trivial comment.

* Fix math bug.

* Add OpenLCB BLE advertisement tests.

* OpenLCB specific BLE definitions.

* Remove redundant parentheses scope.

* Add PIP API to SimpleStack.

* Small updates to openlcb::BLEService.

* Update BLE definitions.

* Add include guards.

* Update the openlcb::BLEService attributes table.

* Add comment.

* Only append name if non-null and length > 0.

* Add simple BLE connections management structures.

* Add OpenLCB BLE Gatt Client manager.

* Adds more convenient read API to databuffer.

* Fix uninitialized variable warning.

* Adds skeleton of BLE hub port base class.

* Adds hub port to the client object.
Changes the client objects to be constant in memory address instead of being moved around.

* Adds an abstract class for BLE protocol engines. This is needed for proper
ownership of objects when BLE client / connections get removed.

* Updates the HUbPOrt to derive from the proper abstract base class
for ownership by the client object.
Refactors the send function to be an std::function instead of a subclass.

* Removes misguided attempt to implement the sending inside this class.
Adds accessor for an abstract protocol engine instead.

* Adds fuzz test to data buffer.
Rewrites conditions in move read pointer forward, because it was incorrect.

* Adds accessor to lookup client by connection and output handle.

* Ensures that try_append_to will always complete for an empty data buffer.

* Adds implementation for input data for BLEHubPort.

* Adds more fuzz testing capability to the data buffer test.

* Fix some crashing cornercases.

* Adds utility function to find clients by the inbound handle.

* Adds more fuzz test debugging.

* Expands the cases where try_append_from succeeds.
- Handles the case when the src or dst buffer is empty.
- Detects when the append happens exactly at a buffer boundary and the
  buffers are already linked buffer.
- Adds a parameter that makes this work even when the link is not there yet

* Adds a debugging facility that verifies that data buffer unrefs are correct.

* Finishes fuzz tests.

* Rewrite data read advance to be much simpler.
This fixes a buffer refcount bug when a buffer ended up with a zero length
and too few refs.

* Fixes crashes in the duffer append code.

* Add API for getting the current active connection count.

* Bump task stack for TWAI watchdog task.

* Ensure we are using the ESP32's logging facility for ESP_PLATFORM.

* Fix bugs that caused data loss over BLE:
- MTU < 500.
- do_write needs to be repeated, so we call it via the executor.

* Fix bug when we take bytes from an appendable data buffer.
We took over o.free_ but this makes the current buffer appendable, which is wrong.

* Fixes bug in tracking pending bytes in the queue.

* Releases the notifiable of the first buffer faster.

* Fixes bug in counting pending bytes.

* Fix bug that might be the cause of the disconnect crash.
Keeping iterators in a vector is not allowed when erase is used.

* Fix comment.

---------

Co-authored-by: Balazs Racz <[email protected]>

etc/config.mk

@@ -2,9 +2,9 @@ ifndef MAKE_INC_CONFIG_MK
 MAKE_INC_CONFIG_MK := 1
 
 ifneq ($(TARGET),bare.pruv3)
-CORELIBS := console utils executor os dcc openlcb withrottle
+CORELIBS := console utils executor os dcc openlcb withrottle ble
 
-LINKCORELIBS = -lconsole -lopenlcb -lwithrottle -ldcc -lexecutor -lutils -lexecutor -los
+LINKCORELIBS = -lconsole -lopenlcb -lwithrottle -ldcc -lexecutor -lutils -lexecutor -los -lble
 endif
 
 endif # MAKE_INC_CONFIG_MK

src/ble/Advertisement.cxx

@@ -0,0 +1,168 @@
+/** @copyright
+ * Copyright (c) 2024, Stuart Baker
+ * All rights reserved
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are strictly prohibited without written consent.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ *
+ * @file Advertisement.cxx
+ *
+ * Advertisement definition.
+ *
+ * @author Stuart Baker
+ * @date 2 March 2024
+ */
+
+#include "ble/Advertisement.hxx"
+
+namespace ble
+{
+
+//
+// Advertisement::concat_service_data_128()
+//
+std::basic_string<uint8_t> Advertisement::concat_service_data_128(
+    const uint8_t uuid[16], const void *buf, size_t size)
+{
+    const uint8_t *data = static_cast<const uint8_t*>(buf);
+    std::basic_string<uint8_t> result(uuid, uuid + 16);
+    result.append(data, size);
+    return result;
+}
+
+//
+// Advertisement::prepend()
+//
+int Advertisement::prepend(
+    Field field, Defs::AdvType type, const void *buf, size_t size, bool clip)
+{
+    const uint8_t *data = static_cast<const uint8_t*>(buf);
+    std::basic_string<uint8_t> *d;
+    size_t max;
+
+    switch (field)
+    {
+        default:
+        case Field::DATA:
+            d = &data_;
+            max = extended_ ?
+                MAX_EXT_DATA_PAYLOAD_SIZE : MAX_SCAN_DATA_PAYLOAD_SIZE;
+            break;
+        case Field::SCAN_DATA:
+            HASSERT(!extended_);
+            d = &scanData_;
+            max = MAX_DATA_PAYLOAD_SIZE;
+            break;
+    }
+
+    size_t space = std::min((size + 2), (max - d->size()));
+    if (space < (size + 2) && clip == false)
+    {
+        // Data doesn't fit and clipping is not allowed.
+        return -1;
+    }
+    d->insert(0, 1, space - 1);
+    d->insert(1, 1, static_cast<uint8_t>(type));
+    d->insert(2, data, space - 2);
+    return space;
+}
+
+//
+// Advertisement::append()
+//
+int Advertisement::append(
+    Field field, Defs::AdvType type, const void *buf, size_t size, bool clip)
+{
+    const uint8_t *data = static_cast<const uint8_t*>(buf);
+    std::basic_string<uint8_t> *d;
+    size_t max;
+
+    switch (field)
+    {
+        default:
+        case Field::DATA:
+            d = &data_;
+            max = extended_ ?
+                MAX_EXT_DATA_PAYLOAD_SIZE : MAX_SCAN_DATA_PAYLOAD_SIZE;
+            break;
+        case Field::SCAN_DATA:
+            HASSERT(!extended_);
+            d = &scanData_;
+            max = MAX_DATA_PAYLOAD_SIZE;
+            break;
+    }
+
+    size_t space = std::min((size + 2), (max - d->size()));
+    if (space < (size + 2) && clip == false)
+    {
+        // Data doesn't fit and clipping is not allowed.
+        return -1;
+    }
+    d->push_back(space - 1);
+    d->push_back(static_cast<uint8_t>(type));
+    d->append(data, space - 2);
+    return space;
+}
+
+//
+// Advertisement::update()
+//
+int Advertisement::update(Field field, Defs::AdvType type, const void *buf,
+                          size_t size, unsigned instance, bool exact_size,
+                          bool clip)
+{
+    const uint8_t *data = static_cast<const uint8_t*>(buf);
+    std::basic_string<uint8_t> *d;
+    size_t max;
+
+    switch (field)
+    {
+        default:
+        case Field::DATA:
+            d = &data_;
+            max = extended_ ?
+                MAX_EXT_DATA_PAYLOAD_SIZE : MAX_SCAN_DATA_PAYLOAD_SIZE;
+            break;
+        case Field::SCAN_DATA:
+            HASSERT(!extended_);
+            d = &scanData_;
+            max = MAX_DATA_PAYLOAD_SIZE;
+            break;
+    }
+
+    uint8_t len;
+    ssize_t pos = Defs::adv_find_data(*d, type, &len, instance);
+    if (pos < 0)
+    {
+        // No matching advertising data found.
+        return -1;
+    }
+    if (exact_size && len != size)
+    {
+        // Found the data, but it is the wrong size.
+        return -1;
+    }
+    size_t space = std::min((size + 2), (max - d->size()) + (len + 2));
+    if (space < size && clip == false)
+    {
+        // Data doesn't fit and clipping is not allowed.
+        return -1;
+    }
+    d->at(pos) = space - 1;
+    d->replace(pos + 2, len, data, space - 2);
+    return space;
+}
+
+} // namespace ble