Skip to content

[Python] Define whether user-overridden to_pylist on Array subclasses is honored for nested children #50604

Description

@viirya

In the review of #50327 it was noted (#50327 (comment)) that the bulk conversion does not consult a user-defined to_pylist override on ExtensionArray subclasses for nested children.

This was already inconsistent before #50327:

  • ListScalar.as_py called child_values.to_pylist(), so an override on a list child's class was honored;
  • StructScalar.as_py converted fields via per-element Scalars, so an override on a struct child's class was ignored.

Top-level overrides dispatch normally through Python method resolution in both the old and the new code, and ExtensionScalar.as_py / __arrow_ext_scalar_class__ — the documented customization point — is honored on all paths.

This issue is to decide the contract:

  1. document that nested to_pylist overrides are not consulted (only ExtensionScalar.as_py is), or
  2. detect custom to_pylist overrides and route nested conversion through them (sketch in the linked discussion).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions