Skip to content

Add additional check for custom array/hash access checking #23399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: blead
Choose a base branch
from

Conversation

khwilliamson
Copy link
Contributor

@khwilliamson khwilliamson commented Jul 3, 2025

PL_check[] can have customized checks for array/hash elements in the
OP_AELEM, OP_EXISTS, and OP_DELETE elements. When these are set, the
multideref optimization is turned off.

It turns out that checking for this always shows that they are not the
standard values on z/OS, because that platform's function pointers may
point to a "function descriptor" and not the actual function, or it may
not. It's not clear to me if this implementation conforms to the C
Standard or not, but it is what it is. As a result our test suite fails
some tests that are expecting non-custom entries in those slots.

This commit builds upon some ideas from Dave Mitchell and Richard Leach
to add 3 entries to the array of function pointers. These entries are
initialized the same way the others are, so that a comparison against
them when no customization has been done should succeed.

The current comparisons are retained, so that it's likely on z/OS one or
the other will succeed when no customization is currently in place.

  • This set of changes does not require a perldelta entry.

PL_check[] can have customized checks for array/hash elements in the
OP_AELEM, OP_EXISTS, and OP_DELETE elements.  When these are set, the
multideref optimization is turned off.

It turns out that checking for this always shows that they are not the
standard values on z/OS, because that platform's function pointers may
point to a "function descriptor" and not the actual function, or it may
not.  It's not clear to me if this implementation conforms to the C
Standard or not, but it is what it is.  As a result our test suite fails
some tests that are expecting non-custom entries in those slots.

This commit builds upon some ideas from Dave Mitchell and Richard Leach
to add 3 entries to the array of function pointers.  These entries are
initialized the same way the others are, so that a comparison against
them when no customization has been done should succeed.

The current comparisons are retained, so that it's likely on z/OS one or
the other will succeed when no customization is currently in place.
I imagine most compilers would know to optimize the reference away when
the first conditional has ruled out all but one possibility, but I think
it is clearer anyway to use that single value.
It's also unlikely that the op_type will be any given value, but I
expect the compiler and libc know that.

This stresses that it is unlikely some module will customize the
handling of these checkers.
@khwilliamson khwilliamson requested a review from iabyn July 3, 2025 21:01
@bulk88
Copy link
Contributor

bulk88 commented Jul 11, 2025

I think its better to figure out which Z/OS C compiler flag/linker flag turns on function pointer equality, and throw that cmd line flag into Configure or hints.sh once and forever. Z/OS linker has 2 different "ld.so" implementations to choose from in its manual. Or Perl on Z/OS simply needs a 1 line macro, to look inside the "function descriptor" C struct, and do 1 or 2 integer math equations to get the PL_ppaddr/PL_check style 64 bit address of the function, and not compare against the 128 bit integer that is represented by typing == Perl_ck_null in source code.

This bug is obvious to understand for any Linux devs. Its basic ELF interposition at work. The integer address you get by typing Perl_ck_null, is libperl.so calling libperl.so without indirection, and without the PLT/GOT, and without the tiny jump shim C function that the PLT/GOT scheme requires, to get a "const literal" function pointer, to a C function whose address WILL NOT BE KNOWN until runtime. AFAIK ELF doesn't allow "re-linking" an SO after it is loaded into address space, so once libperl.so is mapped into address space, the final address of Perl_ck_null inside libperl.so can never change again. AFAIK If a 2nd libfakeperl.so is loaded into the same address space, it will NOT replace the Perl_ck_null from libperl.so with Perl_ck_null from libfakeperl.so. AFAIK LD_PRELOAD/interposition only works until the 1st invocation (LAZY_BIND), or the 1st definition of C symbol Perl_ck_null appears in the process's address space. After that, the ship has sailed for ELF hooking/interposition/PRELOAD feature.

PL_check[PERL_CK_NULL] is the the tiny jump shim C function that the PLT/GOT scheme requires, so that ld.so can interpose/LD_PRELOAD replace Perl_ck_null to point at any machine code address at runtime, yet keep the "address" of the Perl_ck_null function an integer constant that can be duplicated with memcpy() for the purpose of C abstract machine.

https://docs.oracle.com/cd/E19683-01/816-1386/6m7qcobkv/index.html#indexterm-315

A possible fix for Z/OS could also be if ( NUM2PTR(void*,PL_check[o->op_type]) != NUM2PTR(void*,Perl_ck_null)) {. The void * cast will convert Perl_ck_null from a 128 bit function descriptor, or "position independent relative addressing" thing, into a function pointer to a C static jump shim, like on Linux.

1 time hacks like this aren't sustainable, and they won't fix any of the p5p/.git XS mods or any CPAN/PAUSE XS mods either.

For example, this PR didn't catch this line

perl5/pp_hot.c

Line 5121 in 9ef5300

if (PL_op->op_next->op_ppaddr == Perl_pp_and) {

    /* Try to bypass pushing &PL_sv_yes and calling pp_and(); instead
     * jump straight to the AND op's op_other */
    assert(PL_op->op_next->op_type == OP_AND);
    if (PL_op->op_next->op_ppaddr == Perl_pp_and) {
        return cLOGOPx(PL_op->op_next)->op_other;
    }
    else {
        /* An XS module has replaced the op_ppaddr, so fall back to the slow,
         * obvious way. */
        /* pp_enteriter should have pre-extended the stack */

How many more of these lines exist in Perl ecosystem?

Its pretty well documented Z/OS has plenty of different ABIs inside 1 or more address spaces per process. And Z/OS C stack is a linked list if I read this correctly.

https://share.confex.com/share/119/webprogram/Handout/Session11408/Save_area_Conventions.pdf
https://www.ibm.com/docs/en/zos/3.1.0?topic=c-metal-mvs-linkage-conventions#mvslnkcnv

A __far pointer is 128 bits long on Z/OS. Perl DOESN'T SUPPORT 128 bit pointers yet!!! Patches are WELCOME!!!

https://www.ibm.com/docs/en/zos/2.4.0?topic=qualifiers-far-type-qualifier-c-only

The upper half of the pointer contains the access-list-entry token (ALET), which identifies the secondary virtual address space you want to access. The lower half the pointer is the offset within the secondary virtual address space. The size of a __far-qualified pointer is increased to 8 bytes in 31-bit mode and 16 bytes in 64-bit mode. In 31-bit mode, the upper 4 bytes contain the ALET, and the lower 4 bytes is the address within the data space. In 64-bit mode, bytes 0-3 are unused, bytes 4-7 are the ALET, and bytes 8-15 are the address within the data space.

A normal pointer can be converted to a __far pointer explicitly through typecasting or implicitly through assignment. The ALET of the __far pointer is set to zero. A __far pointer can be explicitly converted to a normal pointer through typecasting; the normal pointer keeps the offset of the __far pointer and the ALET is lost. A __far pointer cannot be implicitly converted to a normal pointer.

Pointer arithmetic is supported for __far pointers, with the ALET part being ignored. If the two ALETs are different, the results may have no meaning.

Two __far pointers can be compared for equality and inequality using the == and != operators. The whole pointer is compared. To compare for equality of the offset only, use the built-in function to extract the offset and then compare. To compare for equality of the ALET only, use the built-in function to extract the ALET and then compare. For more information on the set of built-in functions that operate on __far pointers, see z/OS XL C/C++ Programming Guide.

Two __far pointers can be compared using the >, < , >=, and <= relational operators. The ALET parts of the pointers are ignored in this operation. There is no ordering between two __far pointers if their ALETs are different, and between a NULL pointer and any __far pointers. The result is meaningless if they are compared using relational operators.

When a __far pointer and a normal pointer are involved in an operation, the normal pointer is implicitly converted to __far before the operation. There is unspecified behavior if the ALETs are different.
The result of the & (address) operator is a normal pointer, except for the following cases:
If the operand of & is the result of an indirection operator (*), the type of & is the same as the operand of the indirection operator.
If the operand of & is the result of the arrow operator (->, structure member access), the type of & is the same as the left operand of the arrow operator.

maybe this option for ZOS?

https://www.ibm.com/docs/en/zos/2.4.0?topic=qualifiers-fdptr-type-qualifier-c-only

You can declare a function pointer with the __fdptr keyword so that this function pointer can point to a Metal C function descriptor, which is an internal control block that encapsulates all the information that a function call needs to access both the function and the application-specific data.

You use a function pointer that points to a Metal C function descriptor to point to and call functions with their own set of associated data for the particular program or invocation.

So yeah, this is strictly the fault of the Perl 5 interpreter for assuming void * is 8 bytes long. A void * 16 bytes on a Z/OS CPU. The Z/OS C compiler and Z/OS linker, are applying workarounds mechanisms to the Perl 5 interpreter, since Perl 5 is using obsolete/EOL-ed 64 bit 8 byte long void *s, and not can't handle a 128 bit void* pointer. Have a nice day!

Instead of adding extra entries to a P5P created array stored in .rodata or .data, Z/OS Perl needs the right macro or the right cast so the NUM2PTR(void*,Perl_ck_null) inside if ( NUM2PTR(void*,PL_check[o->op_type]) != NUM2PTR(void*,Perl_ck_null)) { is a 64 bit address to a C struct in the __near address space that libperl.so/Glob.so/Utils.so/B.so all share. Not a Z/OS CPU's native 128 bit memory address.

Or properly declare arrays PL_check/PL_ppaddr with a 128 bit pointer type.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants