micro:bit v2 micropython: method to instantiate/reuse persistent 64-byte buffer

[burksfamly]

Environment: micro:bit v2 micropython (current December 2023 version)
Application: multi-sensor mesh network with repeater (in-use & working)
Note: I must use the existing environment, another language or
in-line assembly is not an option.

Is there a method in this environment to declare and obtain a pointer to
a persistent memory location for a 64-byte buffer that micropython will not
move when the buffer is used, referenced and/or manipulated? In other words,
I want to request a pointer to a 64-byte buffer and manage that memory
myself without the buffer being moved by micropython.

For example:

_global_ba = bytearray(64)
_global_mv_ba = memoryview(_global_ba)

... _global_ba[:] = data ... no go, as item assignment is not supported
... _global_ba.extend() ... will void _global_mv_ba as micropython moves the buffer

def msgDst(): # return a token from the message based on the message protocol
global _global_mv_ba
return(bytes(_global_mv_ba[19:19+4]))

while True:
if msgPending():
_global_ba = getRadioMsgBytes() # this immediately moves the buffer
if msgDst() == myID: # won't work because _global_mv_ba is no longer valid
doSomething():

Based on my research, I am not entirely clear why the bytearray class
has a method to declare the size of the array (e.g., bytearray(64)). When
writing more than 64 bytes to the array, micropython just resizes the array.
In other words, I received no error. Also, I am not entirely clear of this
mutable bytearray() class' purpose, because item assignment is not supported
(e.g., _global_ba[:] = data ... to copy actual data to the buffer). Getting a
memoryview() reference to read from the buffer is useful to reduce memory
copies from slicing, but that pointer must be updated because micropython
keeps moving the buffer.

Obviouly, (micro)python is not my forte, but I am trying to establish control
over a persistent memory location for a comm buffer that I have control over.

If this is not doable, is there another method to achieve the objective that
will minimize memory fragmentation and overhead that micropython has with
other data types (just learned this about strings, lists, etc). By the way,
encoding is entirely ASCII or less.

Thank you!

[mendenm]

@burksfamly you need to see if your getRadioMsgBytes() function has an alternative, which reads into a pre-allocated buffer. SPI, for example, has a read_into() method which uses a pre-allocated buffer. micropython will never really extend a bytearray; instead, it has to create a new one and copy. getRadioMsgBytes is just returning an entirely new buffer and micropython then deallocates the old one since you trampled its name.

note that your line _global_ba[:] = data #... no go, as item assignment is not supported would work fine if you assigned to the memoryview instead, and as long as the RHS was an array, i.e. _global_mv_ba[:] = data.

[burksfamly]

@mendenm Thank You.

I tried your recommendation for the micro:bit at https://python.microbit.org/v/3

a = bytearray(b'0123456789')
m = memoryview(a)
z = m[0:] # z is not a copy
z[0:3] = b'jkl' # ERROR: memoryview object doesn't support item assignment
print(a) # this line never executes due to traceback on error

The above does work at https://www.online-python.com/ , but that is not micropython port for the micro:bit

I am not entirely sure what I am seeking is possible with the referenced environment.

I checked your other suggestion and the micro:bit v2 micropython appears to support read into as follows:

import radio
radio.receive_bytes_info(buffer) ... <= from API

I will explore...thank you...I wonder if others will have any suggestions, as this is trivial in other environments, but seem challenging with the micropython port to micro:bit v2

[mendenm]

@burksfamly OK, you will have to wait for a micro:bit expert to comment on the platform specifics. I thought memoryviews always supported item assignment of that type, unless it is a problem with making a memoryview out of another memoryview on the micro:bit. If you try

a = bytearray(b'0123456789')
m = memoryview(a)
m[0:3] = b'jkl' # ERROR: memoryview object doesn't support item assignment
print(a) # this line never executes due to traceback on error

does that work?

[burksfamly]

@mendenm Thank you...I tried it and the same resulting error...

I tried your recommendation for the micro:bit at https://python.microbit.org/v/3

a = bytearray(b'0123456789')
m = memoryview(a)
m[0:3] = b'jkl' # resulting ERROR: memoryview object doesn't support item assignment
print(a) # this line never executes due to traceback on error

I am hoping a micro:bit vs micropython port expert sees this discussion...

[burksfamly]

@mendenm ... I tested the following psuedo-code on two micro:bits.

This method of using radio.receive_bytes_into(buffer) does in fact allow a global bytearray(64)
and a global pointer to the bytearray(64) to be instantiated and used as a static 64-byte buffer
that can be overwritten and referenced via the memoryview() pointer throughout the application.

Thank you for the tip.

somewhat psuedo-code for the micropython port for micro:bit v2

import radio

_global_ba = bytearray(64)
_global_mv_ba = memoryview(_global_ba)

_myID = b'4097'

def msgDst():
...return a token from the message based on the message protocol
global _global_mv_ba ...really not needed if sketch space is an issue
return(bytes(_global_mv_ba[19:19+4])) ...ref a 4-byte token at a known location in the msg buffer

while True:
numMsgBytes = radio.receive_bytes_into(_global_ba) ...will dump the received bytes directly into _global_ba (overwriting)
if numMsgBytes != None: ...there is a pending msg to process
...numMsgBytes is useful in parsing the last token in the message protocol...that is getting the tail of the last token
if msgDst() == _myID: ...if msg is to me
doSomething()