Generic callable/callback protocols

[chadrik]

Hi,
I'm trying to use TypeVars with callable protocols, as you could with Callable, but I'm not having any luck.

This works using Callable:

import typing

InT = typing.TypeVar('InT')
OutT = typing.TypeVar('OutT')

class CallableHolder(typing.Generic[InT, OutT]):
    def __init__(self, fn: typing.Callable[[InT], OutT]):
        self.fn = fn

    def call(self, arg: InT) -> OutT:
        return self.fn(arg)


def get(f: typing.Callable[[InT], OutT]) -> CallableHolder[InT, OutT]:
    return CallableHolder(f)

def returns_int(x: float) -> int:
    return 1

holder = get(returns_int)
reveal_type(holder)

result = holder.call(1.0)
reveal_type(result)

mypy generates the following output:

test.py:51: error: Revealed type is 'callable_protocol_generic.CallableHolder[builtins.float*, builtins.int*]'
test.py:54: error: Revealed type is 'builtins.int*'

Looks good. However, I want to use callback protocols because of their more expressive handling of star-args, kwargs, etc.

Here's my attempt at recreating the above using callback protocols:

import typing


InT = typing.TypeVar('InT', contravariant=True)  # mypy complained that this needs to be contravariant
OutT = typing.TypeVar('OutT', covariant=True)  # mypy complained that this needs to be covariant

class CallableProtocol(typing.Protocol[InT, OutT]):
    def __call__(self, arg: InT) -> OutT:
        pass


class CallableHolder(typing.Generic[InT, OutT]):
    def __init__(self, fn: CallableProtocol[InT, OutT]):
        self.fn = fn

    def call(self, arg: InT) -> OutT:
        return self.fn(arg)


def get(f: CallableProtocol[InT, OutT]) -> CallableHolder[InT, OutT]:
    return CallableHolder(f)


def returns_int(x: float) -> int:
    return 1

holder = get(returns_int)
reveal_type(holder)

result = holder.call(1.0)
reveal_type(result)

mypy produces the following result:

test.py:50: error: Argument 1 to "get" has incompatible type "Callable[[float], int]"; expected "CallableProtocol[<nothing>, <nothing>]"
test.py:50: error: Need type annotation for 'holder'
test.py:51: error: Revealed type is 'Any'
test.py:51: error: Cannot determine type of 'holder'
test.py:53: error: Cannot determine type of 'holder'
test.py:54: error: Revealed type is 'Any'

I expected to be able to produce the same result as with Callable.

I'm using 0.701.

thanks!

[ilevkivskyi]

Prepare for a surprise: technically mypy is correct that your code is unsafe! The problem is that argument names are part of the protocol, because one can call it as foo(arg=3). To make a variable positional-only (i.e. anonymous) use __arg in the definition of the protocol (or fancy new PEP 570 if you are on Python 3.8+).

The error message is of course horrible. I have seen many people caught by this (and was caught by it myself). Closing as a duplicate of #4530 (already high priority).

[chadrik]

ah, I read about the naming caveat but did not put 2 and 2 together because of the error message. I'll give this a try again tonight. thanks for the quick response.