Unsafe overlap in pandas_udf

[zero323]

Unfortunately decorators for Pandas scalar udf overlap unsafely between scalar-to-scalar and scalar-to-struct variants. The culprit is return type:

• scalar-to-scalar uses Union[AtomicDataTypeOrString, ArrayType]
• scalar-to-struct uses Union[StructType, str]

and OrString (in AtomicDataTypeOrString) ans str cause overlap.

One possible solution is to disallow str schema for on of these, preferably scalar-to-scalar:

def pandas_udf(f: Union[AtomicType, ArrayType], returnType: PandasScalarUDFType) -> Callable[[PandasScalarToScalarFunction], UserDefinedFunctionLike]: ...

[zero323]

We can revisit this later but for now, we'll just ignore[misc] overlaps. Quoting JukkaL

Some potentially useful overloaded signatures can't be represented without making them overlapping.