Documentation: The making of SSH Stamp a (md)book

docs/.gitignore

@@ -0,0 +1 @@
+book/book

docs/book/.gitignore

@@ -0,0 +1 @@
+book

docs/book/README.md

@@ -0,0 +1,11 @@
+# The Making of SSH Stamp book
+
+This is a journey through the development of SSH Stamp, a tool designed to enhance SSH-to-UART bridging security and usability. 
+
+The book covers the design decisions, technical challenges, and solutions implemented throughout the project.
+
+## Quickstart
+
+```
+mdbook serve --open
+```

docs/book/book.toml

@@ -0,0 +1,5 @@
+[book]
+authors = ["Roman Valls Guimera"]
+language = "en"
+src = "src"
+title = "The making of SSH Stamp"

docs/book/src/README.md

@@ -0,0 +1,7 @@
+# Introduction
+
+> TL;DR: we need "Rust-on-embedded" sort of book, that's what I'm saying. Easier said than done, I know but was wondering if you have something like that cooking.
+
+> Definitely don't have anything like that cooking ATM.
+
+> "Embassynomicon"?

docs/book/src/SUMMARY.md

@@ -0,0 +1,13 @@
+# Summary
+[Introduction](README.md)
+
+- [Rules of engagement](./chapter_1.md)
+  - [Knowledge prerequisites](./chapter_1_1.md)
+- [Async firmware structure](./chapter_2.md)
+  - [Config via host environment variables](./chapter_2_1.md)
+  - [Non Volatile Storage (NVS)](./chapter_2_2.md)
+- [Factory defaults?](./chapter_3.md)
+- [Testing](./chapter_4.md)
+- [OTA](./chapter_5.md)
+- [Profiling]()
+- [HAL portability](./chapter_6.md)

docs/book/src/chapter_1.md

@@ -0,0 +1,24 @@
+# Yet another Rust embeeded book?
+
+This is a book sparking from a discussion on the matrix esp-rs channel, where the author was asking for a structured project pattern for async Rust on embedded devices, specifically using the embassy ecosystem AND Espressif's esp-hal as the Hardware Abstraction Layer.
+
+I think that the words from `@ivmarkov` set the stage for this book quite well for what I was looking for:
+
+> After all, folks are likely re-inventing the same thing in their firmware all over again - a bunch of reactive tasks modeled with async tasks scheduled in one or two embassy executors. Same as what you would do with non-async Rust on top of an RTOS, except with async-rust your tasks are async tasks.
+>
+> Some of these tasks have a higher prio / lower latency constraints (dealing with tricky hw), and some "can wait" and "tolerate higher latency" (perhaps like smoltcp networking code).
+>
+> And then also some peripheral-but-important topics come, like OTA updates, NVS storage and so on.
+>
+>Picking one concrete example might help. Back in the days in my initial struggles, I had this "water-meter-with-an-emergency-valve-and-an-lcd-screen" project (now hopelesssly outdated). Perhaps picking one such example for the book and going thru it would be good enough to cover most of the topics.
+
+# Rules of engagement
+
+Somebody said that engineers thrive under constraints? Here are (non negotiable?) "principles" for my project:
+
+* no_std
+* no alloc
+* no unsafe
+* use embassy ecosystem
+
+Hopefully those same self-imposed rules help you in creating lean, performant and safe async Rust firmware for embedded projects.

docs/book/src/chapter_1_1.md

@@ -0,0 +1,33 @@
+# Requirements
+
+**Draft chapter, here are some raw discussion notes (from several third party authors I'll quote if they want to be quoted!) for now as we decide what's "best" (simpler?) for each usecase?**
+
+**This chapter should go through how to safely share data between async tasks without compromising latency, performance and "ergonomics" while implementing whichever approach**
+
+> Are you looking specifically for an "async cancellation" code or are you just trying to figure out how to share stuff amongst tasks "without mutexes"? Where "without mutexes" (that is, critical-section-based mutexes) would only be possible if all your tasks that need access to the data are confined to a single thread-mode embassy executor?
+
+To begin with, I'd be happy with config sharing between async tasks, no cancellation needed nor any other task manipulation logic needed other than reload `uart_task` config(s) when needed (see [chapter 2.1](chapter_2_1.md)).
+
+> - You DO still need a criticalsectionmutex (or similar) when dealing with the `InterruptExecutor` UART task, as that task can preempt the normal embassy tasks.
+> - Unless the data you're sharing access to is large (and therefore you don't want to copy it), its often simpler to have [message passing channels][embassy_channels] that pass updates between tasks rather than any kind of shared state.
+
+<!--
+Stuff like interior mutability Rc vs Arc what 'static really means
+what a blocking_mutex::Mutex<NoopRawMutex, ...> has on top of and comparing to a regular RefCell (hint: nothing; it is just a safer pattern to express the same except that you can't get it wrong - as with RefCell - and erroneously keep a RefCell guard (RefMut) across await points)
+-->
+
+> When you want to share mutable stuff across multiple tasks in the same, "thread-mode" embassy-executor (let's put interrupt executors aside).
+
+Let's not put them aside since I'm using them in SSH Stamp, see points made above.
+
+> 1. For blocking stuff: `embassy_sync::blocking_mutex::Mutex<NoopRawMutex, RefCell<T>>` gives you interior mutability, but since you use NoopRawMutex, it is not a "real" mutex so to say, and you don't pay the "mutex" overhead. You share this mutex either with make_static or with Rc (the latter I have to check, but relatively sure it should work; why not?)
+>
+> 2. For async stuff: `embassy_sync::Mutex<NoopRawMutex, T>` - also gives you interior mutability without a real mutex (so no overhead). Again, either make_static, or Rc should work prior to locking the async "mutex".
+> 
+> By "real" mutex above I mean like a critical section raw mutex (`CriticalSectionRawMutex`, as opposed to `NoopRawMutex`), which you should not need, as long as all this stuff is shared only within a thread-mode executor, created from within the main async task and spawned using the spawner you have as an arg in that main task.
+
+> Also an alternative to embassy_sync::blocking_mutex::Mutex<NoopRawMutex, T> is just a RefCell.
+
+> There are other small complications, like blocking_mutex::Mutex does not have a RefCell inside by defalt (as the async variant has) so you have to put it yourself (or use Cell if you want to store inside small stuff which is Copy), but this is a detail.
+
+[embassy_channels]: https://docs.embassy.dev/embassy-sync/git/default/channel/struct.Channel.html

docs/book/src/chapter_2.md

@@ -0,0 +1,53 @@
+# Async firmware structure
+
+Peripherals should be initialized in the main function [and shared between tasks][embassy_shared_peripherals_pattern], all embassy tasks should be defined in separate files with their corresponding structs and impls.
+
+```rust
+% tree
+.
+├── lib.rs
+├── main.rs
+└── tasks
+    ├── mod.rs
+    ├── net.rs
+    ├── uart.rs
+    └── wifi.rs
+(...)
+```
+
+Each task is defined in its own file, allowing for modularity and separation of concerns. Ideally, initialization of a peripheral and its long-running form should be separate, i.e:
+
+```rust
+(...)
+    spawner.spawn(tasks::uart::init(peripherals)).unwrap();
+    spawner.spawn(tasks::uart::run()).unwrap();
+
+    spawner.spawn(tasks::wifi::init(peripherals)).unwrap();
+    spawner.spawn(tasks::wifi::run()).unwrap();
+(...)
+```
+
+The main function initializes the necessary peripherals and spawns the tasks using the Embassy executor. But in the snippet above some problems start to arise: 
+
+1. Should we pass around `esp_hal::peripherals::Peripherals`?
+1. Or perhaps just initialise the concrete peripherals we need such as `peripheral.UART1` being then passed to the UART task only?
+1. How do we "pack" groups of required peripherals and pass them to the task? Espressif wireless typically requires `peripherals.{WIFI && TIMG0 && RNG && SYSTIMER}`
+1. How do we "unpack" (split AND share) `peripheral.RNG` if we later want to have a task that deals with cryptography (which will require random number generation via the RNG peripheral)?
+1. Can we safely free up unused heap/stack space from, `tasks::wifi::init()` after it has done its job? As opposed to `tasks::wifi::run()`, it's a one-off task... perhaps it should be a free-standing function instead?
+
+Most HALs provide isolated peripheral-oriented examples but no overarching (embassy-based) **project structure that discourages all-in-main.rs anti-pattern**.
+
+Following this structure might seem trivial, but here is where most of the architecture and design decisions start to come into play, such as:
+
+- How to handle shared state between tasks such as critical boot and runtime configuration?
+- Are tasks supposed to be cancelable? Restartable? Resumable?
+- How to handle errors and recover from failures? Which crate provides the smallest RAM/Flash footprint for error handling?
+
+Furthermore, if we are to [decouple IO from compute][sans_io_premise] for easy testing [and also timing][abstracting_time_sansio], which are the [main tenets of a SansIO approach][sans_io_ssh_stamp], we need to consider how [tasks and their state interact with a FSM][fsm_std_tests].
+
+
+[sans_io_ssh_stamp]: https://github.com/brainstorm/ssh-stamp/issues/25
+[abstracting_time_sansio]: https://www.firezone.dev/blog/sans-io#abstracting-time
+[sans_io_premise]: https://www.firezone.dev/blog/sans-io#the-premise-of-sans-io
+[fsm_std_tests]: https://github.com/brainstorm/ssh_fsm_tests
+[embassy_shared_peripherals_pattern]: https://embassy.dev/book/#_sharing_peripherals_between_tasks

docs/book/src/chapter_2_1.md

@@ -0,0 +1,25 @@
+# Config via host environment variables
+
+Before a SSH connection is established, the hosts's environment variables are read to configure the firmware, borrowing a convenient feature from the (Open)SSH client binary and protocol (courtesy of man (1) ssh):
+
+```
+    (...) ssh reads ~/.ssh/environment, and adds lines of the format “VARNAME=value” to the environment if the file exists and users are allowed to change their environment.
+```
+
+This allows for dynamic configuration of the firmware based on the host environment, enabling flexibility and adaptability in different deployment scenarios. For instance, the pin configuration for the UART TX pin can be set via an environment variable, allowing the firmware to adapt to different hardware setups without requiring editing files, performing resets nor recompiling the firmware:
+
+```
+export PIN_UART_TX=2
+
+ssh -o "SendEnv PIN_UART_TX" user@host
+```
+
+Other environment variables can be used to configure the firmware's behavior, such as:
+
+1. Setting the Wi-Fi SSID, password and encryption standard (Open, WPA, WPA2, etc...).
+2. Setting password-based authentication for the SSH server.
+3. Setting public key authentication for the SSH server (and/or generating an new server secret, adding a new (third party?) public key, etc...).
+4. Enabling debugging features such as logging levels or enabling/disabling specific tasks.
+5. Wiping the firmware's configuration, returning to factory defaults, [see chapter 3](chapter_3.md).
+
+Those are a few examples of the host environment variables as firmware configuration mechanism, more can follow as SSH Stamp grows and can also be adapted for custom use cases.

docs/book/src/chapter_2_2.md

@@ -0,0 +1,11 @@
+# Non Volatile Storage (NVS)
+
+1. How should storage be structured for lightweight configuration purposes?
+1. Is [cfg_noodle][cfg_noodle] a good fit for this? Or should we use [sequential_storage][sequential_storage] instead?
+1. Is it worth using a custom NVS implementation for this firmware? Or should we use the existing esp-hal NVS implementation and build on top of it?
+1. Is the serdes approach implemented by [sunset's][sunset] (and SSH Stamp's) SSHConfig a good fit for this? Or are there better solutions?
+1. How should config be accessed/updated/deleted across tasks with the least amount of boilerplate (needing the least amount of .lock(), mutex guards, etc..)?
+
+[cfg_noodle]: https://github.com/tweedegolf/cfg-noodle
+[sequential_storage]: https://github.com/tweedegolf/sequential-storage
+[sunset]: https://github.com/mkj/sunset

docs/book/src/chapter_3.md

@@ -0,0 +1,11 @@
+# Secure factory defaults?
+
+What's a sensible factory default configuration for SSH Stamp in 2025?
+
+The firmware should be able to run out of the box, so it should have a sensible default configuration that allows it to operate without requiring any additional setup... but it also needs to comply with increasingly strict IoT cybersecurity standards... [rightfully criticised!](https://www.youtube.com/@mattbrwn).
+
+**How should provisioning be done for this firmware?**
+
+TBD (D for Discussed and/or Done).
+
+[matt_brown]: https://www.youtube.com/@mattbrwn

docs/book/src/chapter_4.md

@@ -0,0 +1,7 @@
+# Testing
+
+Testing on the host, on the target, both emulated (on CI) and via HIL:
+
+1. How are pins managed across dev boards and ICs?
+1. How is the actual HIL testing jig built? A few pin headers that accept all dev boards and ICs? Custom PCBs?
+1. TBD: Show how CI/CD works for HIL testing. Perhaps custom (GitHub?) runners?

docs/book/src/chapter_5.md

@@ -0,0 +1,13 @@
+# Normal OTA
+
+TBD: Enumerate existing OTA mechanisms (compatible with esp-hal/no_std) and ideally see how big their RAM/Flash footprint is, explain pros and cons, etc.
+
+# Custom OTA: SCP
+
+SSH Stamp could leverage pre-existing OTA mechanisms that involve bootloaders, custom cloud services that distribute new builds to fleets and so on.
+
+But we are writing a SSH server... wouldn't it make sense to just use the SCP protocol to transfer new builds to the device? [That's what Andrew Zonenberg has done on his StaticNet project][ssh_stamp_scp_ota].
+
+[ssh_stamp_scp_ota]: https://github.com/brainstorm/ssh-stamp/issues/24
+
+TBD: Describe SCP protocol, how to implement it, how to use it, how to test it, etc.

docs/book/src/chapter_6.md

@@ -0,0 +1,5 @@
+# HAL portability
+
+When a target is supported in full, can I easily switch between different HALs?
+
+Would `cargo`'s "feature-gating" be enough to support different HALs?