Viper issues : how to replace ptr8: UNIX port RPI 4

[picpic020960]

Bonjour,
I test micropython app on ESP32 TTGO watch : OK
Then the same app on unix port.
But on unix port no viper decorator .
So comment decorator : ok for basic function but on function with ptr8 I am stuck.
Exists some workarround or trick ?
Thank's for help and link
Regards

@micropython.viper
def init_plte(plte, palette):
    po = ptr8(plte)
    pal = ptr8(palette)
    for i in range(256):
        col = int(rgb(pal[i * 3], pal[i * 3 + 1], pal[i * 3 + 2]))
        po[i * 2 + 1] = col >> 8
        plte[i * 2] = col

[GitHubsSilverBullet]

@micropython.viper
def init_plte(plte, palette):
    po = ptr8(plte)
    pal = ptr8(palette)
    for i in range(256):
        col = int(rgb(pal[i * 3], pal[i * 3 + 1], pal[i * 3 + 2]))
        po[i * 2 + 1] = col >> 8
        plte[i * 2] = col

If plte and palette are declared as bytearrays, then they should be usable almost without a change.

> def init_plte(po, pal): # CHANGE
>     # po = ptr8(plte)
>     # pal = ptr8(palette)
>     for i in range(256):

[picpic020960]

I am confused with "self." or not.

# adapted from https://pyokagan.name/blog/2019-10-14-png/

from tempos import g
from graphics import rgb, BLACK
import deflate
import struct
import micropython
from time import ticks_ms, ticks_diff
import io


def decompress(data):
    f = io.BytesIO(data)
    with deflate.DeflateIO(f, deflate.ZLIB,15) as g:
        return g.read()

@micropython.viper
def init_plte(plte, palette):
    po = ptr8(plte)
    pal = ptr8(palette)
    for i in range(256):
        col = int(rgb(pal[i * 3], pal[i * 3 + 1], pal[i * 3 + 2]))
        po[i * 2 + 1] = col >> 8
        plte[i * 2] = col


class PNG_Tile:
    def __init__(self, tilex, tiley, zoom, stx, sty):
        self._stx = stx
        self._sty = sty
        self._tile = (tilex, tiley, zoom)
        self._data = None
        self._plte = bytearray(512)
        self._decode()
        self._row = 0
        self._col = 0
        self._x = 0
        self._y = 0

    @micropython.viper
    def _init_plte(self, palette):
        po = ptr8(self._plte)
        pal = ptr8(palette)
        for i in range(256):
            col = int(rgb(pal[i * 3], pal[i * 3 + 1], pal[i * 3 + 2]))
            po[i * 2 + 1] = col >> 8
            po[i * 2] = col

    def _decode(self):
        def read_chunk(f):
            # Returns (chunk_type, chunk_data)
            chunk_length, chunk_type = struct.unpack(">I4s", f.read(8))
            chunk_data = f.read(chunk_length)
            (chunk_expected_crc,) = struct.unpack(">I", f.read(4))
            return chunk_type, chunk_data

        # now = ticks_ms()
        # open file and check signature
        try:
            f = open(
                "/sd/tiles/{}/{}/{}.png".format(
                    self._tile[2], self._tile[0], self._tile[1]
                ),
                "rb",
            )
        except:
            return
        PngSignature = b"\x89PNG\r\n\x1a\n"
        if f.read(len(PngSignature)) != PngSignature:
            raise Exception(
                "Invalid PNG Signature {}/{}/{}.png".format(
                    self._tile[2], self._tile[0], self._tile[1]
                )
            )
        chunks = []
        while True:
            chunk_type, chunk_data = read_chunk(f)
            chunks.append((chunk_type, chunk_data))
            if chunk_type == b"IEND":
                break
        _, IHDR_data = chunks[0]  # IHDR is always first chunk
        width, height, bitd, colort, compm, filterm, interlacem = struct.unpack(
            ">IIBBBBB", IHDR_data
        )
        if not (width == 256 and height == 256):
            raise Exception("only 256x256 images")
        if compm != 0:
            raise Exception("invalid compression method")
        if filterm != 0:
            raise Exception("invalid filter method")
        if colort != 3:
            raise Exception("only paletted color for OSM map tiles")
        if bitd != 1 and bitd != 8:
            raise Exception("only bit depth of 1 or 8")
        if interlacem != 0:
            raise Exception("no interlacing support")
        # get palette
        chunk_type, palette = chunks[1]
        if not chunk_type == b"PLTE":
            raise Exception("Palette chunk expected")
        if bitd == 8:
            self._plte = bytearray(512)
            self._init_plte(palette)
            # get image data
            data = b"".join(
                chunk_data for chunk_type, chunk_data in chunks if chunk_type == b"IDAT"
            )
            self._data = decompress(data)
        else:
            self._plte = rgb(palette[0], palette[1], palette[2])
            self._data = b"\x00"
        # print("Decode Time(ms): ",ticks_diff(ticks_ms(),now))

    @micropython.viper
    def _render(self, w: int, h: int):
        baddr = ptr8(g._buf)
        dt = ptr8(self._data)
        pal = ptr8(self._plte)
        rs = int(self._row)
        cs = int(self._col)
        xs = int(self._x)
        ys = int(self._y)
        for j in range(0, h):
            for i in range(0, w):
                col = dt[(rs + j) * 257 + i + cs + 1]
                ind: int = (j + ys) * 480 + (xs + i) * 2
                baddr[ind] = pal[col * 2]
                baddr[ind + 1] = pal[col * 2 + 1]

    def _clip(self, w, h):
        if self._col + w > 256:
            w = 256 - self._col
        if self._x + w > 240:
            w = 240 - self._x
        if self._row + h > 256:
            h = 256 - self._row
        if self._y + h > 240:
            h = 240 - self._y
        return w, h

    def draw(self, c, r, x, y, w, h):
        self._row = r
        self._col = c
        self._x = x
        self._y = y
        w, h = self._clip(w, h)
        # now = ticks_ms()
        if self._data is not None:
            if len(self._data) > 1:
                self._render(w, h)
                g.updateMod(x, y, x + w - 1, y + h - 1)
            else:
                g.fill_rect(x, y, w, h, self._plte)
        else:
            g.fill_rect(x, y, w, h, BLACK)
        # print("Draw Time(ms): ",ticks_diff(ticks_ms(),now))

    def draw_chunk(self, x1, y1, x2, y2):  # coords in 0..511,0..511 space
        def in_tile(x, y):
            return (
                x >= self._stx
                and x < self._stx + 256
                and y >= self._sty
                and y < self._sty + 256
            )

        if in_tile(x1, y1):
            self.draw(
                x1 - self._stx, y1 - self._sty, 0, 0, 240, 240
            )  # rely on clipping
        elif in_tile(x2, y1):
            self.draw(0, y1 - self._sty, 256 - x1, 0, 240, 240)
        elif in_tile(x1, y2):
            self.draw(x1 - self._stx, 0, 0, 256 - y1, 240, 240)
        elif in_tile(x2, y2):
            self.draw(0, 0, 256 - x1, 256 - y1, 240, 240)

[GitHubsSilverBullet]

I am confused with "self." or not.

Don't be.

That's very basic Python knowledge (classes) and can be read in great detail in the official documentation.

[picpic020960]

Not trivial !
After modif:

MicroPython v1.23.0-preview.379.gcfd5a8ea3.dirty on 2024-05-21; linux [GCC 4.2.1] version
Use Ctrl-D to exit, Ctrl-E for paste mode
>>> from apps import maps
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "apps/maps.py", line 115, in <module>
  File "apps/maps.py", line 110, in get_tiles
  File "apps/maps.py", line 87, in create_tile
  File "pngtile.py", line 36, in __init__
  File "pngtile.py", line 105, in _decode
  File "pngtile.py", line 47, in _init_plte
IndexError: mismatch index struct: of buffer

Palette not byte array ...

chunk_type, palette = chunks[1]

The ptr8 viper function has no equivalence in micropython ?

[Josverl]

Then the same app on unix port.
But on unix port no viper decorator .

Which version af micropython are you running , and using what commandline ?
because @micropython.viperjust works on my not too old build : MicroPython v1.28.0-preview.250.g68c2d4e457.dirty on 2026-03-19

Note: the outcomes to NOT match the comments 😕

Sample from : viper @ WiKi

# File: vp8.py 
import array 

@micropython.viper
def myfunction( )->int:
   int32_array = array.array("l", (1,2,3,4))
   x = ptr32( int32_array )
   print(x[0], x[1], x[2] ) # this will print 1, 2, 3
   ba = bytearray(10)
   y = ptr8(ba)
   y[0] = 1 # This will change ba[0]
   return x[1]
   
print("viper result:", myfunction())

ran using:

jos@SB26:~$ micropython  ./vp8.py
1 0 2
viper result : 0

[picpic020960]

MicroPython v1.23.0-preview.379.gcfd5a8ea3.dirty on 2024-05-21; linux [GCC 4.2.1] version
Use Ctrl-D to exit, Ctrl-E for paste mode
>>> from apps import maps

old version with sdl2 function added from russhughes
updates are always not easy.
Sure that is the solution ? not ptr8 equivalence micropython function ?

edit : in this moment

MicroPython v1.29.0-preview.393.gd901e98349 on 2026-06-15; linux [GCC 12.2.0] version
Type "help()" for more information.
>>> 
paste mode; Ctrl-C to cancel, Ctrl-D to finish
=== # File: vp8.py 
=== import array 
=== 
=== @micropython.viper
=== def myfunction( )->int:
===    int32_array = array.array("l", (1,2,3,4))
===    x = ptr32( int32_array )
===    print(x[0], x[1], x[2] ) # this will print 1, 2, 3
===    ba = bytearray(10)
===    y = ptr8(ba)
===    y[0] = 1 # This will change ba[0]
===    return x[1]
===    
=== print("viper result:", myfunction())
=== 
Traceback (most recent call last):
  File "<stdin>", line 4, in <module>
SyntaxError: invalid micropython decorator

Raspberry PI 4 : reason of the issue ?

[peterhinch]

This runs here under the Unix build. Behaviour on RP4 should be identical. Note that declaring the array as "l" produces an incorrect result - the listing below fixes this by declaring 32-bit signed integers:

MicroPython v1.29.0-preview.228.g7d77d3e0dd.dirty on 2026-05-13; linux [GCC 11.4.0] version
Type "help()" for more information.
>>> 
paste mode; Ctrl-C to cancel, Ctrl-D to finish
=== import array 
=== 
=== @micropython.viper
=== def myfunction( )->int:
===    int32_array = array.array("i", (1,2,3,4))
===    x = ptr32( int32_array )
===    print(x[0], x[1], x[2] ) # this will print 1, 2, 3
===    ba = bytearray(10)
===    y = ptr8(ba)
===    y[0] = 1 # This will change ba[0]
===    return x[1]
===    
=== print("viper result:", myfunction())
=== 
1 2 3
viper result: 2
>>> 

[Josverl]

Note that declaring the array as "l" produces an incorrect result

Thanks for figuring that out.
Wiki needs an update then

[picpic020960]

not clear.
micropython unix OK on RPI 4 yes or no ?
@peterhinch "Behaviour on RP4 should be identical" ; are you sure ?
some build config file update ?
as show up micropython v1.29 not accepts viper decorator on my RPI 4. it's nor "l" or "i" issue.

[Josverl]

OK, the plot unravels....
We know:

• Unix port,
• both v1.24, and v1.29.0-preview
• architecture: RPI 4

But we are still missing a few clues to help solve the mystery:

• as that is not a pre-built that can be downloaded from MicroPython.org , you must have built it yourself, or downloaded it from somewhere else.
• if you built it: can you describe in some detail on which OS, and using which toolchain you built it ?
• what OS/build are you running on your RPI-4?

[peterhinch]

"Behaviour on RP4 should be identical" ; are you sure ?

I was, until @Josverl pointed out that there is no official binary for RPI4...

The RPI4 uses ARM Cortex-A72 cores which, though 64-bit, can run Arm Thumb instructions. MicroPython can emit native Arm Thumb. Viper issues native code. I'm not expert on the build system but there is presumably a switch somewhere to tell the resultant firmware what native emitter to use. It would seem that @picpic020960's build can run bytecode but cannot emit native code...

[picpic020960]

First test on android tablette with termux.
Than test on RPI 4 :

uname -a
Linux raspberrypi 6.6.51+rpt-rpi-v6 #1 Raspbian 1:6.6.51-1+rpt3 (2024-10-08) armv6l GNU/Linux

Classic build as readme.md from micropython github.
[https://github.com/micropython/micropython/tree/master/ports/unix]

[picpic020960]

with chatgpt help :

Title

Viper return value broken on unix ARM32 (Thumb) port

Environment

• Raspberry Pi 4
• Raspberry Pi OS 64-bit
• MicroPython unix port built as ARMhf 32-bit
• GCC 12.2.0
  reproduced with v1.28.0 and v1.29.0-preview

Minimal reproducer

import micropython

@micropython.viper
def g() -> int:
    return 42

x = g()
print(x)

Expected:

42

Actual:

Illegal instruction

@micropython.native works correctly.

GDB shows:

Program received signal SIGILL, Illegal instruction.
0xf7fc6744 in ?? ()

=> 0xf7fc6744: udf #241

[Josverl]

Raspberry Pi OS (32-bit)- cannot reproduce

I overlooked that you were using RPIOS64 and focussed on the MicroPython unix port built as ARMhf 32-bit

Based on that I can at least conclude that it works on a 32 bit OS using the below build

• RPI4 hardware
• Clean installed Raspberry Pi OS (32-bit)

sudo apt update
sudo apt install libffi-dev pkg-config
make -C mpy-cross
make -C ports/unix submodules
make -C ports/unix  
ports/unix/build-standard/micropython ./vp42.py 

jos@emst32:~/micropython $ ports/unix/build-standard/micropython ./vp42.py 
- Defining @viper function
- Calling @viper function
 - Result :  42

I only have a single SD card available currently - so I'll wait A BIT before overwriting it with RPIOS64 and retrying.
( Sorry, patience is not my strong point)

Details

jos@emst32:~/micropython $ file ports/unix/build-standard/micropython
readelf -h ports/unix/build-standard/micropython | egrep 'Class|Machine'
ports/unix/build-standard/micropython: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=ab41894398a477666378184a5ee6b9a425832222, for GNU/Linux 3.2.0, stripped
  Class:                             ELF32
  Machine:                           ARM


$ uname -a
Linux emst32 6.18.34+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.18.34-1+rpt1 (2026-06-09) aarch64 GNU/Linux

# file: vp42.py

import micropython

print("- Defining @viper function")
@micropython.viper
def g() -> int:
    return 42

print("- Calling @viper function")
x = g()
print(" - Result : ",x)

[agatti]

Ok, assuming there's no publicly available RPi4 to SSH into, you can try this:

$ gdb micropython
(gdb) set logging enabled
(gdb) set disassemble-next-line on
(gdb) break fun_viper_call
(gdb) start
(gdb) c
# Enter the crashing code to the REPL here, and run it

(gdb) n
(gdb) n
(gdb) n

# It should look something like this at this point:

# 493     in ../../py/objfun.c  
# => 0x00424ab0 <fun_viper_call+22>:      4643            mov     r3, r8  
#    0x00424ab2 <fun_viper_call+24>:      463a            mov     r2, r7  
#    0x00424ab4 <fun_viper_call+26>:      4631            mov     r1, r6  
#    0x00424ab6 <fun_viper_call+28>:      4620            mov     r0, r4  
#    0x00424ab8 <fun_viper_call+30>:      47a8            blx     r5

# Execute `si` commands until the current opcode being executed is `blx r5`

(gdb) si
(gdb) x/30i $pc # <- Increase the number if you don't see `pop {r1, r4, r5, r6, r7, pc}` in the dump.
                # That line marks the end of the generated code

# Execute `si` commands until it crashes or until it reaches `pop {r1, r4, r5, r6, r7, pc}`
# If you end up back into C, run `fin` to exit the running function and get back into assembler.

Do that until you get bored or you get a crash, then send me the log.txt file that gdb should have generated. I've oversimplified the process but that's more or less what I go through each time there's an emitter issue :)

[Josverl]

I could not get a crash single-stepping or with the vp42.py at all

so I created a module to imnport all viper tests (slegdehammer approach)
gdb.txt
allvpt.py

See e-mail for ssh access if you need a subtler approach

[agatti]

Thanks for the help! I've read the log, and yeah, looks like I'll have to dig in myself. Still, appreciate the willingness to go through low-level gdb fun :)

By the way, got the mail, looks like you have to authorise SSH access from within the network - I can reach the host but the port is blocked.

[agatti]

It's more interesting than I thought. Viper for some reason leaves things in such a state on a RPi 4 that it can even crash code that's not really its own (see second execution):

agatti@emst64:~/mpsrc/tests $ gdb ../ports/unix/build-standard/micropython 
GNU gdb (Debian 16.3-1) 16.3
[...]
Reading symbols from ../ports/unix/build-standard/micropython...
(gdb) r micropython/viper_addr.py
Starting program: /home/agatti/mpsrc/ports/unix/build-standard/micropython micropython/viper_addr.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
<class 'bytearray'>
<class 'int'>
bytearray(b'0000')
bytearray(b'1111')
bytearray(b'2222')
bytearray(b'2233')
10
[Inferior 1 (process 338823) exited normally]
(gdb) r micropython/viper_addr.py
Starting program: /home/agatti/mpsrc/ports/unix/build-standard/micropython micropython/viper_addr.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
<class 'bytearray'>
<class 'int'>
bytearray(b'0000')
bytearray(b'1111')
bytearray(b'2222')
bytearray(b'2233')

Program received signal SIGSEGV, Segmentation fault.
0xf7fe3004 in ?? ()
(gdb) bt
#0  0xf7fe3004 in ?? ()
#1  0x004205ba in mp_obj_get_int_truncated (arg=<optimized out>) at ../../py/obj.c:349
#2  0x0042119e in mp_get_buffer (obj=0x4, bufinfo=0xfffef154, flags=4157168032) at ../../py/obj.c:620
#3  0x00000000 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) quit

The most interesting thing here is, however, the arguments to mp_get_buffer in the second execution. The stack gets corrupted but it should be corrupted in exactly the same way, unless there's some 64->32 bits address remapping gone wrong outside MicroPython specific to this CPU? Need to look more into this.

[agatti]

Nope, in this specific configuration cache clearing is not done properly. Manually flush caches in a different way than the ones provided does seem to work.

Almost there :)

[agatti]

This is getting kind of weird though - the OS @picpic020960 reported in https://github.com/orgs/micropython/discussions/19341#discussioncomment-17343092 is 32 bits (armv6l, not aarch64). armv6l uses soft-float, whilst the gnuhf toolchain uses hard-float. That may explain the crash.

@picpic020960 can you please confirm the exact version of the OS you're experiencing the crash on? If uname -m reads armv6l, MicroPython needs different compilation flags (and maybe a different toolchain) - if you can please switch to a proper 64-bits OS and try again.

[picpic020960]

Uname -m ===> AARCH64

[Josverl]

Uname -m ===> AARCH64

I notice that is different from what you reported earlier , are these different devices or a different OS on the same device.

Anyway, on Unix/Aarch64 viper does not exist unfortunately.

An Aarch32 crosscompiled version does have viper, but does not run viper code stably on Aarch64 so far. See earlier comment.

I think there are a few doc and other issues that could be created from this.

[picpic020960]

pi@raspberrypi-4:~ $ uname  -m
aarch64
pi@raspberrypi-4:~ $ uname -a
Linux raspberrypi-4 6.6.62+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.62-1+rpt1 (2024-11-25) aarch64 GNU/Linux

apologies : the above (15 jun) uname is for PI 3 B [+?]
i test the same on android tablette with termux

[agatti]

i test the same on android tablette with termux

I apologise for the question, but what did you do exactly?

As far as I know there's no way to build a 32-bits binary on a 64-bits termux system unless you cross-compiled GCC first. Clang from the termux main repo and GCC from the TUR do not target 32-bits. Copying non-static binaries built elsewhere isn't supposed to work (Glibc vs Bionic libc implementations, at the very least).

If you managed to compile it in 32-bits mode under termux please let me know how, so I can update the Unix port documentation (I sent in the patches to make it work with termux in the first place...).

[picpic020960]

j'ai essayé de produire un MIcropython 32 bit via termux mais j'ai echoué

/data/data/com.termux/files/usr/bin/../../usr/arm-linux-androideabi/lib/crtend_android.o
ld.lld: error: relocation R_ARM_ABS32 cannot be used against symbol 'nlr_push_tail'; recompile with -fPIC

>>> defined in build-standard/py/nlr.o
>>> referenced by nlrthumb.c:41 (../../py/nlrthumb.c:41)
>>>               build-standard/py/nlrthumb.o:(nlr_push)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [../../py/mkrules.mk:233: build-standard/micropython] Error 1

[agatti]

Fair enough, that more or less matches my experience. For a moment I thought you actually built the 32-bits binary under a 64-bits Termux!

[picpic020960]

It's still disappointing, though.

[agatti]

@picpic020960 @Josverl

These changes seem to work for me, can you please give this a try and tell me if it works over there?

diff --git a/py/emitglue.c b/py/emitglue.c
index 9526acac8..491da8768 100644
--- a/py/emitglue.c
+++ b/py/emitglue.c
@@ -108,12 +108,16 @@ void mp_emit_glue_assign_native(mp_raw_code_t *rc, mp_raw_code_kind_t kind, cons
     // so that the generated native code which was created in data RAM will
     // be available for execution from instruction RAM.
     #if defined(__thumb__) || defined(__thumb2__)
+    #if (defined(__linux__) && defined(__GNUC__)) || __ARM_ARCH == 7
+    __builtin___clear_cache((void *)fun_data, (char *)fun_data + fun_len);
+    #else
     #if __ICACHE_PRESENT == 1
     // Flush D-cache, so the code emitted is stored in RAM.
     MP_HAL_CLEAN_DCACHE(fun_data, fun_len);
     // Invalidate I-cache, so the newly-created code is reloaded from RAM.
     SCB_InvalidateICache();
     #endif
+    #endif
     #elif defined(__arm__)
     #if (defined(__linux__) && defined(__GNUC__)) || __ARM_ARCH == 7
     __builtin___clear_cache((void *)fun_data, (char *)fun_data + fun_len);

[Josverl]

@agatti Yes !, that works reliably now

For the Aarch32/ARM binary:

• all tests pass , except extmod/select_poll_fd.py ( pre-existing )
• testing viper tests report 41 tests passed
• the randomness / crashes no longer occur
• 100 repeated tests runs of the viper tests all pass
• the vp42.py sample runs reliably

For the Aarch64 binary:

• all tests pass , except extmod/select_poll_fd.py ( pre-existing )
• testing viper tests report all tests skipped ( Expected - no emitter)

[agatti]

That's great to hear! I'll clean this up, do some more testing on another AArch64 system and then package it up as a PR.

Feel free to un-share the RPi 4 from tailscale - thanks a lot for giving me access to that!