|
| 1 | +# Acquisition and Control |
| 2 | + |
| 3 | +The exercises below will help you become familiar with acquiring and recording data from the [Harp Hobgoblin](https://github.com/harp-tech/device.hobgoblin), as well as issuing commands to connected peripherals using Bonsai. In addition, you will learn how to visualize and manipulate the recorded data in Python. |
| 4 | + |
| 5 | +> [!WARNING] |
| 6 | +> When adding the operators in these tutorials, make sure to use the device-specific operators, e.g. `Device (Harp.Hobgoblin)`. |
| 7 | +
|
| 8 | +## Prerequisites |
| 9 | + |
| 10 | +- Install the [`Bonsai.Gui`](https://bonsai-rx.org/gui/) package from the [Bonsai package manager](https://bonsai-rx.org/docs/articles/packages.html). |
| 11 | + |
| 12 | +## Acquisition |
| 13 | + |
| 14 | +### Exercise 1: Acquiring analog input |
| 15 | + |
| 16 | +In the acquisition section of this tutorial we will record data from a photodiode sensor. The photodiode sensor should be connected to analog input channel `0` (`GP26`) on the `Hobgoblin` as shown below. |
| 17 | + |
| 18 | +> [!TIP] |
| 19 | +> You can use another sensor (such as a potentiometer, push button, etc) and any of the other analog input channels (`GP27` or `GP28`). You may have to adjust properties accordingly. |
| 20 | +
|
| 21 | +{width=400px} |
| 22 | + |
| 23 | +In Bonsai: |
| 24 | + |
| 25 | +:::workflow |
| 26 | + |
| 27 | +::: |
| 28 | + |
| 29 | +- Insert a [`Device`] operator. This operator is used to receive data from Harp devices and send commands to it. |
| 30 | +- Set the `PortName` property of the [`Device`] operator to the serial port `Hobgoblin` is connected to (e.g. `COM7`). |
| 31 | +- Start the workflow. If you visualize the output of the [`Device`] operator you should observe a continuous stream of messages. |
| 32 | + |
| 33 | +> [!NOTE] |
| 34 | +> The [`Device`] operator automatically changes its name to `Hobgoblin` when added to the workflow. In this tutorial, we will be referring to the original name of the operator in the Bonsai toolbox, which will be different from how it appears in your workflow or in the workflow images shown. |
| 35 | +
|
| 36 | +:::workflow |
| 37 | + |
| 38 | +::: |
| 39 | + |
| 40 | +- Insert a [`Parse`] operator. |
| 41 | +- Select the [`Parse`] operator, and pick [`AnalogData`] from the `Register` property dropdown menu. |
| 42 | +- Right-click on the [`Parse`] operator, select `Harp.Hobgoblin.AnalogDataPayload` > `AnalogInput0` from the context menu. This will select data from the first analog input channel. |
| 43 | + |
| 44 | +> [!NOTE] |
| 45 | +> Harp devices send data, and receive commands, encoded as [`HarpMessage`] objects following the [Harp binary protocol](../protocol/BinaryProtocol-8bit.md). The [`Parse`] operator is able to recognize and decode messages arriving from a specific `Register` (e.g. [`AnalogData`]) and recover the data encoded in the message payload (e.g. [`AnalogDataPayload`]). |
| 46 | +> |
| 47 | +> All messages arriving from the device contain a hardware timestamp indicating when data was acquired, which can be recovered by selecting the `Timestamped` version of the register in the [`Parse`] operator, e.g. [`TimestampedAnalogData`]. |
| 48 | +
|
| 49 | +- Run the workflow, open the visualizer for `AnalogInput0`, and shine the flashlight from your phone on the photodiode. |
| 50 | +**What do you see?** |
| 51 | + |
| 52 | +### Exercise 2: Acquiring timestamped data |
| 53 | + |
| 54 | +One of the main advantages of devices in the Harp ecosystem is that all [`HarpMessage`] data are hardware-timestamped, rather than relying on software timestamping by the operating system, which is susceptible to jitter. To access hardware timestamped data, make the follow modications to the previous workflow. |
| 55 | + |
| 56 | +:::workflow |
| 57 | + |
| 58 | +::: |
| 59 | + |
| 60 | +- Change the `Register` property in the [`Parse`] operator from [`AnalogData`] to [`TimestampedAnalogData`]. |
| 61 | +- Replace the member selector at the end of the workflow with a [`RollingGraph`] operator. |
| 62 | +- Select the [`RollingGraph`] operator, and set the `Index` property to use the `Seconds` field of the timestamped data. |
| 63 | +- Set the `Value` property to the `Value` field of the timestamped data. |
| 64 | +- Run the workflow and open the [`RollingGraph`] visualizer. |
| 65 | +**What is the visualizer representing?** |
| 66 | + |
| 67 | +### Exercise 3: Recording timestamped data |
| 68 | + |
| 69 | +For simple use cases, data can be saved to a text file using [`CsvWriter`]. In a later exercise, we will go through why this approach does not scale well for more complicated recordings. |
| 70 | + |
| 71 | +:::workflow |
| 72 | + |
| 73 | +::: |
| 74 | + |
| 75 | +- Add a [`CsvWriter`] operator in between [`Parse`] and [`RollingGraph`]. |
| 76 | +- Configure the `FileName` property of the [`CsvWriter`] with a file name ending in `.csv`, for instance `AnalogData.csv`. |
| 77 | +- Set the `IncludeHeader` property of the [`CsvWriter`] to `True`. This automatically creates column headings for the text file. |
| 78 | +- Run the workflow, shine the light on the photodiode, and then open the resulting text file. |
| 79 | +**How is the data organized?** |
| 80 | + |
| 81 | +> [!TIP] |
| 82 | +> You can set the `Overwrite` property of the `CsvWriter` to `True` to avoid having to delete old files before each run. Be careful to disable this during actual experiment recordings! |
| 83 | +
|
| 84 | +### Exercise 4: Visualizing recorded data |
| 85 | + |
| 86 | +We will take a brief detour from Bonsai to look at how to visualize the data we have recorded. This section assumes you already have a Python environment with [`pandas`](https://pandas.pydata.org/), [`matplotlib`](https://matplotlib.org/) and [`harp-python`](https://github.com/harp-tech/harp-python) installed. You can install these quickly with [`uv`](https://docs.astral.sh/uv/): |
| 87 | + |
| 88 | +```cmd |
| 89 | +uv pip install pandas matplotlib harp-python |
| 90 | +``` |
| 91 | + |
| 92 | +The below script loads the CSV file and inspects the values in analog input 0. |
| 93 | + |
| 94 | +```python |
| 95 | +import pandas as pd |
| 96 | + |
| 97 | +# Load recorded data from the CSV file |
| 98 | +analog_data = pd.read_csv("./AnalogData.csv", index_col = 0) |
| 99 | + |
| 100 | +# Display the first few rows of the DataFrame |
| 101 | +print(analog_data.head()) |
| 102 | + |
| 103 | +# Plot analog input channel 0 |
| 104 | +analog_data["Value.AnalogInput0"].plot() |
| 105 | +``` |
| 106 | + |
| 107 | +**How is the Harp timestamp getting into the X-axis?** |
| 108 | + |
| 109 | +## Control |
| 110 | + |
| 111 | +### Exercise 5: Controlling digital output |
| 112 | + |
| 113 | +In the control section of this tutorial, we will send commands to turn on and off a LED. Connect one of the LED modules to digital output channel `GP15` on the `Hobgoblin`. |
| 114 | + |
| 115 | +> [!TIP] |
| 116 | +> You can use another actuator (such as an active buzzer) and any of the other digital output channels (`GP15` through `GP22`) by changing the appropriate properties. |
| 117 | +
|
| 118 | +{width=400px} |
| 119 | + |
| 120 | +Previously we have been acquiring data from the `Hobgoblin` by placing operators after the [`Device`] operator. In order to send commands to the device, we need to place operators that lead into the [`Device`] operator. |
| 121 | + |
| 122 | +:::workflow |
| 123 | + |
| 124 | +::: |
| 125 | + |
| 126 | +- Insert a [`KeyDown`] operator and set its `Key` property to `A`. We will use this key to turn ON the LED. |
| 127 | +- Insert a [`CreateMessage`] operator, which will construct a [`HarpMessage`] command to send to the device. |
| 128 | +- Configure the `Payload` property to [`DigitalOutputSetPayload`] which will set the digital output to `High`. |
| 129 | +- Configure the [`DigitalOutputSet`] property to select the digital output pin (`GP15`) to send the command to. |
| 130 | + |
| 131 | +Now that we have constructed a [`HarpMessage`] to turn on the digital output, we will construct a similar [`HarpMessage`] to turn it off. |
| 132 | + |
| 133 | +- Insert a [`KeyDown`] operator and set its `Key` property to `S`. We will use this key to turn OFF the LED. |
| 134 | +- Insert a [`CreateMessage`] operator. |
| 135 | +- Configure the `Payload` property to [`DigitalOutputClearPayload`] which will clear the digital output and set it to `LOW`. |
| 136 | +- Configure the [`DigitalOutputClear`] property to the same digital output pin (`GP15`). |
| 137 | + |
| 138 | +> [!NOTE] |
| 139 | +> At this point we are ready to send these [`HarpMessage`] commands into the `Hobgoblin`. However, the [`Device`] operator only accepts a single input sequence transmitting all the [`HarpMessage`] commands. |
| 140 | +
|
| 141 | +- Insert a [`Merge`] operator to combine these two commands into one [`HarpMessage`] sequence. |
| 142 | +- Insert a [`Device`] operator to send the [`HarpMessage`] sequence into the `Hobgoblin`. |
| 143 | +- Run the workflow and press either the `A` or `S` key. |
| 144 | +**What do you observe?** |
| 145 | + |
| 146 | +### Exercise 6: Recording timestamped commands |
| 147 | + |
| 148 | +To know when the digital output of the `Hobgoblin` was turned ON or OFF, we can take advantage of the fact that every Harp device will always echo back any command sent to it with the hardware timestamp of when it was executed in the device. This means we can actually use the exact same format we learned in the acquisition section to receive the [`HarpMessage`] objects which are transmitted by the device when the command was executed. |
| 149 | + |
| 150 | +:::workflow |
| 151 | + |
| 152 | +::: |
| 153 | + |
| 154 | +- Insert a [`Parse`] operator and select [`TimestampedDigitalOutputSet`] from the `Register` property dropdown menu. |
| 155 | +- Insert another [`Parse`] operator as a branch, and select [`TimestampedDigitalOutputClear`] from the `Register` property dropdown menu. |
| 156 | +- Run the workflow, open the visualizers for both of these nodes, and toggle the LED on and off. |
| 157 | +**What do you notice?** |
| 158 | + |
| 159 | +> [!NOTE] |
| 160 | +> For both operators, the [`HarpMessage`] contains the pin number for the digital output that was either turned ON or OFF, as well as the timestamps for those commands. They can be used to report the digital output commands for all pins available on the `Hobgoblin`. |
| 161 | +
|
| 162 | +:::workflow |
| 163 | + |
| 164 | +::: |
| 165 | + |
| 166 | +- Log data from each register with a [`CsvWriter`] operator. |
| 167 | +- Configure the `FileName` property of the [`CsvWriter`] with a file name ending in `.csv`, e.g. `DigitalOutputSet.csv`. |
| 168 | +- Set the `IncludeHeader` property of the [`CsvWriter`] to `True`. |
| 169 | +- Run the workflow, toggle the LED on and off, and then open the resulting text files. |
| 170 | + |
| 171 | +## Integration |
| 172 | + |
| 173 | +### Exercise 7: Combining acquisition and control |
| 174 | + |
| 175 | +You now have all the pieces to create a full workflow that has both acquisition of data and control of peripheral devices. Combine the two workflows together and it should look something like this: |
| 176 | + |
| 177 | +:::workflow |
| 178 | + |
| 179 | +::: |
| 180 | + |
| 181 | +### Exercise 8: Visualizing synchronized recordings |
| 182 | + |
| 183 | +Another main advantage of devices in the Harp ecosystem is that all recorded information streams are timestamped to the same hardware clock, even if they are not sampled with the same period. As such, there is no need for post-hoc alignment during visualization and analysis. We will now take a look at our recorded text files and look at how to visualize them together using Python. |
| 184 | + |
| 185 | +```python |
| 186 | +import pandas as pd |
| 187 | + |
| 188 | +# Load the data |
| 189 | +analog_data = pd.read_csv("./AnalogData.csv", index_col = 0) |
| 190 | +digital_output_set = pd.read_csv("./DigitalOutputSet.csv", index_col = 0) |
| 191 | +digital_output_clear = pd.read_csv("./DigitalOutputClear.csv", index_col = 0) |
| 192 | + |
| 193 | +# Inspect the raw data |
| 194 | +print(analog_data.head()) |
| 195 | +print(digital_output_set.head()) |
| 196 | +print(digital_output_clear.head()) |
| 197 | + |
| 198 | +# Create a plot with all analog channels and vertical lines at digital events |
| 199 | +ax = analog_data.plot() |
| 200 | +adc_min, adc_max = (analog_data.min(axis=None), analog_data.max(axis=None)) |
| 201 | +ax.vlines(digital_output_set["Value"].index, adc_min, adc_max, colors='red', linestyles='dashed') |
| 202 | +ax.vlines(digital_output_clear["Value"].index, adc_min, adc_max, colors='black', linestyles='dashed') |
| 203 | +``` |
| 204 | + |
| 205 | +## Data Interface |
| 206 | + |
| 207 | +### Exercise 9: Streamlining recording |
| 208 | + |
| 209 | +You might have noticed that the approach to recording data in [Exercise 7](#exercise-7-combining-acquisition-and-control) does not scale well, particularly when adding more `Registers` or additional devices. The `Harp.Hobgoblin` package provides a [`DeviceDataWriter`] operator that can be used to record all the data and commands received by the device in a single standardized binary format. |
| 210 | + |
| 211 | +:::workflow |
| 212 | + |
| 213 | +::: |
| 214 | + |
| 215 | +- Copy the final workflow from [Exercise 7](#exercise-7-combining-acquisition-and-control). |
| 216 | +- Delete all the existing [`CsvWriter`] branches. |
| 217 | +- Add a [`DeviceDataWriter`] operator after the [`Device`] operator. |
| 218 | +- Type a name in the `Path` property of [`DeviceDataWriter`]. This name will be used to save all the data coming from the device into a folder with the same name. |
| 219 | +- Run the workflow, then open the folder you specified in the previous step. |
| 220 | +**What do you observe?** |
| 221 | + |
| 222 | +> [!NOTE] |
| 223 | +> The [`DeviceDataWriter`] generates a `device.yml` file that contains device metadata that will be used later for loading data with `harp-python`. In addition, all the data from each `Register` is saved as a separate raw binary file. This includes not just data registers, but other common registers used for device configuration or identification. |
| 224 | +
|
| 225 | +### Exercise 10: Streamlining data analysis |
| 226 | + |
| 227 | +You might also have noticed that the approach to loading data in [Exercise 8](#exercise-8-visualizing-synchronized-recordings) does not scale well as you have to juggle parsing and handling of all data files. The `harp-python` package also simplifies data visualization and analysis by providing a convenient interface to load and read the raw binary files that [`DeviceDataWriter`] records directly into a data frame. |
| 228 | + |
| 229 | +This exercise assumes that you have setup the dependencies from previous exercises, as well as `harp-python`. |
| 230 | + |
| 231 | +```python |
| 232 | +import harp |
| 233 | + |
| 234 | +# Create a device reader object to load Hobgoblin data |
| 235 | +device = harp.create_reader("./Hobgoblin.harp") |
| 236 | + |
| 237 | +# Read data from a register by doing device.<register_name>.read() |
| 238 | +analog_data = device.AnalogData.read() |
| 239 | + |
| 240 | +# The returned data is a pandas.DataFrame that can be easily inspected ... |
| 241 | +print(analog_data.head()) |
| 242 | + |
| 243 | +# ...and visualized |
| 244 | +analog_data.plot() |
| 245 | +``` |
| 246 | + |
| 247 | +> [!NOTE] |
| 248 | +> **Optional** Now that you understand the data loaded by `harp-python`, can you reproduce [Exercise 8](#exercise-8-visualizing-synchronized-recordings)? |
| 249 | +
|
| 250 | +<!--Reference Style Links --> |
| 251 | +[`AnalogData`]: xref:Harp.Hobgoblin.AnalogData |
| 252 | +[`AnalogDataPayload`]: xref:Harp.Hobgoblin.AnalogDataPayload |
| 253 | +[`CreateMessage`]: xref:Harp.Hobgoblin.CreateMessage |
| 254 | +[`CsvWriter`]: xref:Bonsai.IO.CsvWriter |
| 255 | +[`Device`]: xref:Harp.Hobgoblin.Device |
| 256 | +[`DeviceDataWriter`]: xref:Harp.Hobgoblin.DeviceDataWriter |
| 257 | +[`DigitalOutputSet`]: xref:Harp.Hobgoblin.DigitalOutputSet |
| 258 | +[`DigitalOutputClear`]: xref:Harp.Hobgoblin.DigitalOutputClear |
| 259 | +[`DigitalOutputClearPayload`]: xref:Harp.Hobgoblin.CreateDigitalOutputSetPayload |
| 260 | +[`DigitalOutputSetPayload`]: xref:Harp.Hobgoblin.CreateDigitalOutputClearPayload |
| 261 | +[`HarpMessage`]: xref:Bonsai.Harp.HarpMessage |
| 262 | +[`KeyDown`]: xref:Bonsai.Windows.Input.KeyDown |
| 263 | +[`Merge`]: xref:Bonsai.Reactive.Merge |
| 264 | +[`Parse`]: xref:Harp.Hobgoblin.Parse |
| 265 | +[`Parse (Harp.Hobgoblin)`]: xref:Harp.Hobgoblin.Parse |
| 266 | +[`PublishSubject`]: xref:Bonsai.Reactive.PublishSubject |
| 267 | +[`RollingGraph`]: xref:Bonsai.Gui.ZedGraph.RollingGraphBuilder |
| 268 | +[`SubscribeSubject`]: xref:Bonsai.Expressions.SubscribeSubject |
| 269 | +[`TimestampedAnalogData`]: xref:Harp.Hobgoblin.TimestampedAnalogData |
| 270 | +[`TimestampedDigitalOutputSet`]: xref:Harp.Hobgoblin.TimestampedDigitalOutputSet |
| 271 | +[`TimestampedDigitalOutputClear`]: xref:Harp.Hobgoblin.TimestampedDigitalOutputClear |
| 272 | +[`Zip`]: xref:Bonsai.Reactive.Zip |
0 commit comments