Skip to content

Overload docs #3159

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 12, 2017
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 61 additions & 38 deletions docs/source/function_overloading.rst
Original file line number Diff line number Diff line change
@@ -1,60 +1,83 @@
Function overloading in stubs
=============================
Function Overloading
====================

Sometimes you have a library function that seems to call for two or
more signatures. That's okay -- you can define multiple *overloaded*
instances of a function with the same name but different signatures in
a stub file (this feature is not supported for user code, at least not
yet) using the ``@overload`` decorator. For example, we can define an
``abs`` function that works for both ``int`` and ``float`` arguments:
Sometimes the types in a function depend on each other in ways that
can't be captured with a simple ``Union``. For example, the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/simple//

Copy link
Contributor

@pkch pkch Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe "can't be captured with a Union or generics"?

``__getitem__`` (``[]`` bracket indexing) method can take an integer
and return a single item, or take a ``slice`` and return a
``Sequence`` of items. You might be tempted to annotate it like so:

.. code-block:: python

# This is a stub file!

from typing import overload

@overload
def abs(n: int) -> int: pass

@overload
def abs(n: float) -> float: pass

Note that we can't use ``Union[int, float]`` as the argument type,
since this wouldn't allow us to express that the return
type depends on the argument type.

Now if we import ``abs`` as defined in the above library stub, we can
write code like this, and the types are inferred correctly:
class MyList(Sequence[T]):
def __getitem__(self, index: Union[int, slice]) -> Union[T, Sequence[T]]:
if isinstance(index, int):
... # Return a T here
elif isinstance(index, slice):
... # Return a sequence of Ts here
else:
assert False, "Unsupported argument %r" % (index,)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe just inherit from Generic[T]? Otherwise, this code would fail with Signature of "__getitem__" incompatible with supertype "Sequence".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, that's kind of the point (though I agree it's pretty implicit here).

But this is a little loose, as it implies that when you put in an
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/a little/too/

``int`` you might sometimes get out a single item or sometimes a
sequence. To capture a constraint such as a return type that depends
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like your sentence structure here is a bit convoluted, and it might do to mention a type variable as another option here -- it's preferable when it's possible. Perhaps:

"The return type depends on the parameter type in a way that can't be expressed by a type variable. We can use overloading (link) to give the same function multiple type annotations (signatures) and accurately describe the function's behavior."

Or something -- wordsmithing is hard. My confidence that this version is better than yours is only about 70%.

on a parameter type, we can use `overloading
<https://www.python.org/dev/peps/pep-0484/#function-method-overloading>`_
to give the same function multiple type annotations (signatures).

.. code-block:: python

n = abs(-2) # 2 (int)
f = abs(-1.5) # 1.5 (float)
from typing import Generic, Sequence, overload
T = TypeVar('T')

class MyList(Sequence[T]):

# The @overload definitions are just for the type checker,
# and overwritten by the real implementation below.
@overload
def __getitem__(self, index: int) -> T:
pass # Don't put code here

# All overloads and the implementation must be adjacent
# in the source file, and overload order may matter.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider explaining how overload order matters -- without doing so it feels like a warning against using overloading, because stuff you don't understand might happen.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #1270 (comment) for some background.

@overload
def __getitem__(self, index: slice) -> Sequence[T]:
pass # Don't put code here

# Actual implementation goes last, without @overload.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Actual/The/

# It may or may not have type hints; if it does,
Copy link
Contributor

@pkch pkch Apr 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See #3160 for my concerns with both untyped and typed implementation. Maybe worth warning in the docs that untyped definitions result in the assumption of Any for each of the arguments (at least at the moment) -- that's if the body is even checked at all.

# these are checked against the overload definitions
# as well as against the implementation body.
def __getitem__(self, index):
# This is exactly the same as before.
if isinstance(index, int):
... # Return a T here
elif isinstance(index, slice):
... # Return a sequence of Ts here
else:
assert False, "Unsupported argument %r" % (index,)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raise TypeError("Unsupported index type '%s'" % index.__class__.__name__) or something like that is closer to what Python does in the builtin types.


Overloaded function variants are still ordinary Python functions and
they still define a single runtime object. The following code is
thus valid:

.. code-block:: python

my_abs = abs
my_abs(-2) # 2 (int)
my_abs(-1.5) # 1.5 (float)
they still define a single runtime object. There is no multiple
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/multiple/automatic/

The user does implement dispatch in their implementation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and also: this is single dispatch rather than multiple dispatch, since only one argument determines the implementation. But the sentence should apply to both single and multiple dispatch of course.

dispatch happening, and you must manually handle the different types
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"manually handle the different types in the implementation"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps add that using @functools.singledispatch with @overloaded is not yet supported.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, @overload is not limited to a single argument, and at the type system level you can totally express multi-dispatch with it.

(usually with :func:`isinstance` checks, as shown in the example).

The overload variants must be adjacent in the code. This makes code
clearer, as you don't have to hunt for overload variants across the
file.

Overloads in stub files are exactly the same, except there is no
implementation.

.. note::

As generic type variables are erased at runtime when constructing
instances of generic types, an overloaded function cannot have
variants that only differ in a generic type argument,
e.g. ``List[int]`` versus ``List[str]``.
e.g. ``List[int]`` and ``List[str]``.

.. note::

If you are writing a regular module rather than a stub, you can
often use a type variable with a value restriction to represent
functions as ``abs`` above (see :ref:`type-variable-value-restriction`).
If you just need to constrain a type variable to certain types or
subtypes, you can use a :ref:`value restriction
<type-variable-value-restriction>`.