Skip to content

conformance: change generics_typevartuple_args.py case #2203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JelleZijlstra wants to merge 7 commits into
base: main
Choose a base branch
from
+62 −61
Open

conformance: change generics_typevartuple_args.py case #2203

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1ff577f
conformance: change generics_typevartuple_args.py case
JelleZijlstra Mar 2, 2026
ed33b9c
rerun
JelleZijlstra Mar 2, 2026
dfef23c
Amend spec
JelleZijlstra Mar 5, 2026
1a9dac5
Amend conformance
JelleZijlstra Mar 5, 2026
b08e02f
Merge remote-tracking branch 'upstream/main' into tvt-match
JelleZijlstra Mar 5, 2026
9dcab0e
update results
JelleZijlstra Mar 5, 2026
5841fb4
fix scoring
JelleZijlstra Mar 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions conformance/results/mypy/generics_typevartuple_args.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
conformant = "Partial"
notes = """
Does not enforce that tuples captured by TypeVarTuple are same type.
Does not enforce that tuples captured by TypeVarTuple are of the same length.
"""
output = """
generics_typevartuple_args.py:33: error: Argument 3 to "exec_le" has incompatible type "str"; expected "Env" [arg-type]
Expand All @@ -15,5 +15,4 @@ generics_typevartuple_args.py:67: error: Too few arguments for "func3" [call-ar
conformance_automated = "Fail"
errors_diff = """
Line 75: Expected 1 errors
Line 76: Expected 1 errors
"""
4 changes: 0 additions & 4 deletions conformance/results/mypy/generics_typevartuple_basic.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
conformant = "Partial"
notes = """
Does not enforce that tuples captured by TypeVarTuple are same length.
Does not enforce that tuples captured by TypeVarTuple are same type.
"""
output = """
generics_typevartuple_basic.py:42: error: Argument 1 to "Array" has incompatible type "Height"; expected "tuple[Height, Width]" [arg-type]
Expand All @@ -10,7 +9,6 @@ generics_typevartuple_basic.py:45: error: Argument 1 to "Array" has incompatible
generics_typevartuple_basic.py:52: error: Free type variable expected in Generic[...] [misc]
generics_typevartuple_basic.py:53: error: TypeVarTuple "Shape" is only valid with an unpack [valid-type]
generics_typevartuple_basic.py:56: error: TypeVarTuple "Shape" is only valid with an unpack [valid-type]
generics_typevartuple_basic.py:57: error: Incompatible return value type (got "tuple[*Shape]", expected "tuple[Any]") [return-value]
generics_typevartuple_basic.py:59: error: TypeVarTuple "Shape" is only valid with an unpack [valid-type]
generics_typevartuple_basic.py:65: error: Unexpected keyword argument "covariant" for "TypeVarTuple" [misc]
generics_typevartuple_basic.py:66: error: Too many positional arguments for "TypeVarTuple" [misc]
Expand All @@ -21,7 +19,5 @@ generics_typevartuple_basic.py:106: error: Can only use one type var tuple in a
"""
conformance_automated = "Fail"
errors_diff = """
Line 89: Expected 1 errors
Line 90: Expected 1 errors
Line 57: Unexpected errors ['generics_typevartuple_basic.py:57: error: Incompatible return value type (got "tuple[*Shape]", expected "tuple[Any]") [return-value]']
"""
8 changes: 6 additions & 2 deletions conformance/results/pyrefly/generics_typevartuple_args.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
conformant = "Pass"
conformance_automated = "Pass"
conformant = "Partial"
conformance_automated = "Fail"
notes = """
Does not correctly solve TypeVarTuple with heterogeneous bounds.
"""
errors_diff = """
Line 76: Unexpected errors ["Argument `tuple[Literal['1']]` is not assignable to parameter `*args` with type `tuple[int]` in function `func4` [bad-argument-type]"]
"""
output = """
ERROR generics_typevartuple_args.py:33:12-23: Unpacked argument `tuple[Literal[0], Literal['']]` is not assignable to parameter `*args` with type `tuple[*@_, Env]` in function `exec_le` [bad-argument-type]
Expand Down
3 changes: 2 additions & 1 deletion conformance/results/pyrefly/generics_typevartuple_basic.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
conformant = "Partial"
notes = """
TypeVarTuple is pinned too early when calling generic function
TypeVarTuple is pinned too early when calling generic function.
"""
conformance_automated = "Fail"
errors_diff = """
Line 85: Unexpected errors ['Argument `tuple[float]` is not assignable to parameter `arg2` with type `tuple[int]` in function `func2` [bad-argument-type]']
Line 89: Unexpected errors ["Argument `tuple[Literal['0']]` is not assignable to parameter `arg2` with type `tuple[int]` in function `func2` [bad-argument-type]"]
"""
output = """
ERROR generics_typevartuple_basic.py:42:34-43: Argument `Height` is not assignable to parameter `shape` with type `tuple[Height, Width]` in function `Array.__init__` [bad-argument-type]
Expand Down
12 changes: 8 additions & 4 deletions conformance/results/pyright/generics_typevartuple_args.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,11 @@
conformant = "Pass"
conformant = "Partial"
notes = """
Does not correctly solve TypeVarTuple with heterogeneous bounds.
"""
conformance_automated = "Fail"
errors_diff = """
Line 76: Unexpected errors ['generics_typevartuple_args.py:76:14 - error: Argument of type "tuple[Literal[\\'1\\']]" cannot be assigned to parameter "args" of type "tuple[*Ts@func4]" in function "func4"']
"""
output = """
generics_typevartuple_args.py:33:20 - error: Argument of type "Literal['']" cannot be assigned to parameter of type "Env" in function "exec_le"
  "Literal['']" is not assignable to "Env" (reportArgumentType)
Expand All @@ -17,6 +24,3 @@ generics_typevartuple_args.py:75:13 - error: Argument of type "tuple[Literal[1],
generics_typevartuple_args.py:76:14 - error: Argument of type "tuple[Literal['1']]" cannot be assigned to parameter "args" of type "tuple[*Ts@func4]" in function "func4"
  "Literal['1']" is not assignable to "int" (reportArgumentType)
"""
conformance_automated = "Pass"
errors_diff = """
"""
12 changes: 8 additions & 4 deletions conformance/results/pyright/generics_typevartuple_basic.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,11 @@
conformant = "Pass"
conformant = "Partial"
notes = """
Does not correctly solve TypeVarTuple with heterogeneous bounds.
"""
conformance_automated = "Fail"
errors_diff = """
Line 89: Unexpected errors ['generics_typevartuple_basic.py:89:14 - error: Argument of type "tuple[Literal[\\'0\\']]" cannot be assigned to parameter "arg2" of type "tuple[*Ts@func2]" in function "func2"']
"""
output = """
generics_typevartuple_basic.py:42:34 - error: Argument of type "Height" cannot be assigned to parameter "shape" of type "tuple[*Shape@Array]" in function "__init__"
  "Height" is not assignable to "tuple[*Shape@Array]" (reportArgumentType)
Expand Down Expand Up @@ -30,6 +37,3 @@ generics_typevartuple_basic.py:100:17 - error: Argument of type "Array[Height, W
generics_typevartuple_basic.py:106:14 - error: Generic class can have at most one TypeVarTuple type parameter but received multiple ("Ts1", "Ts2") (reportGeneralTypeIssues)
generics_typevartuple_basic.py:106:29 - error: Type argument list can have at most one unpacked TypeVarTuple or tuple (reportInvalidTypeForm)
"""
conformance_automated = "Pass"
errors_diff = """
"""
16 changes: 8 additions & 8 deletions conformance/results/results.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -371,16 +371,16 @@ <h3>Python Type System Conformance Test Results</h3>
<th class="column col2 conformant">Pass</th>
</tr>
<tr><th class="column col1">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;generics_typevartuple_args</th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not enforce that tuples captured by TypeVarTuple are same type.</p></span></div></th>
<th class="column col2 conformant">Pass</th>
<th class="column col2 conformant">Pass</th>
<th class="column col2 conformant">Pass</th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not enforce that tuples captured by TypeVarTuple are of the same length.</p></span></div></th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not correctly solve TypeVarTuple with heterogeneous bounds.</p></span></div></th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not correctly solve TypeVarTuple with heterogeneous bounds.</p></span></div></th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not correctly solve TypeVarTuple with heterogeneous bounds.</p></span></div></th>
</tr>
<tr><th class="column col1">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;generics_typevartuple_basic</th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not enforce that tuples captured by TypeVarTuple are same length.</p><p>Does not enforce that tuples captured by TypeVarTuple are same type.</p></span></div></th>
<th class="column col2 conformant">Pass</th>
<th class="column col2 conformant">Pass</th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>TypeVarTuple is pinned too early when calling generic function</p></span></div></th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not enforce that tuples captured by TypeVarTuple are same length.</p></span></div></th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not correctly solve TypeVarTuple with heterogeneous bounds.</p></span></div></th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>Does not correctly solve TypeVarTuple with heterogeneous bounds.</p></span></div></th>
<th class="column col2 partially-conformant"><div class="hover-text">Partial<span class="tooltip-text" id="bottom"><p>TypeVarTuple is pinned too early when calling generic function.</p></span></div></th>
</tr>
<tr><th class="column col1">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;generics_typevartuple_callable</th>
<th class="column col2 conformant">Pass</th>
Expand Down
7 changes: 6 additions & 1 deletion conformance/results/zuban/generics_typevartuple_args.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
conformance_automated = "Pass"
conformant = "Partial"
notes = """
Does not correctly solve TypeVarTuple with heterogeneous bounds.
"""
conformance_automated = "Fail"
errors_diff = """
Line 76: Unexpected errors ['generics_typevartuple_args.py:76: error: Argument 2 to "func4" has incompatible type "tuple[Literal[\\'1\\']]"; expected "tuple[int]" [arg-type]']
"""
output = """
generics_typevartuple_args.py:33: error: Argument 3 to "exec_le" has incompatible type "str"; expected "Env" [arg-type]
Expand Down
7 changes: 6 additions & 1 deletion conformance/results/zuban/generics_typevartuple_basic.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
conformance_automated = "Pass"
conformant = "Partial"
notes = """
Does not correctly solve TypeVarTuple with heterogeneous bounds.
"""
conformance_automated = "Fail"
errors_diff = """
Line 89: Unexpected errors ['generics_typevartuple_basic.py:89: error: Argument 2 to "func2" has incompatible type "tuple[Literal[\\'0\\']]"; expected "tuple[int]" [arg-type]']
"""
output = """
generics_typevartuple_basic.py:42: error: Argument 1 to "Array" has incompatible type "Height"; expected "tuple[Height, Width]" [arg-type]
Expand Down
12 changes: 6 additions & 6 deletions conformance/tests/generics_typevartuple_args.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ class Env:
...


def exec_le(path: str, *args: * tuple[*Ts, Env], env: Env | None = None) -> tuple[*Ts]:
def exec_le(path: str, *args: *tuple[*Ts, Env], env: Env | None = None) -> tuple[*Ts]:
raise NotImplementedError


Expand All @@ -39,7 +39,7 @@ def has_int_and_str(x: int, y: str):
# > *args: int, which accepts zero or more values of type int.


def func1(*args: * tuple[int, ...]) -> None:
def func1(*args: *tuple[int, ...]) -> None:
...


Expand All @@ -48,7 +48,7 @@ def func1(*args: * tuple[int, ...]) -> None:
func1(1, "2", 3) # E


def func2(*args: * tuple[int, *tuple[str, ...], str]) -> None:
def func2(*args: *tuple[int, *tuple[str, ...], str]) -> None:
...


Expand All @@ -59,7 +59,7 @@ def func2(*args: * tuple[int, *tuple[str, ...], str]) -> None:
func2("") # E


def func3(*args: * tuple[int, str]) -> None:
def func3(*args: *tuple[int, str]) -> None:
...


Expand All @@ -72,8 +72,8 @@ def func4(*args: tuple[*Ts]):


func4((0,), (1,)) # OK
func4((0,), (1, 2)) # E
func4((0,), ("1",)) # E
Copy link
Member

@carljm carljm Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To me it seems most consistent with how typevar solving works everywhere else in the type system to actually specify that this should not be an error. It looks like that would match the behavior of mypy and zuban, but not of pyrefly or pyright.

If there's opposition to that change, I would still probably favor leaving this in, with E? and a comment -- that seems more useful to future discussion of this case.

(As far as I can see from a quick scan, this behavior was not specified in the PEP.)

Copy link
Collaborator

@erictraut erictraut Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comes from this section of the spec, which was copied from the PEP. This section was added to the PEP late in the process — after I had reviewed the PEP and implemented it in pyright. I was very unhappy when I learned about the addition because, as you point out, it introduces an inconsistency with the way that TypeVar solving works in other cases.

Line 76 of the conformance test is arguably different than the special case spelled out in the spec, which says "If the same TypeVarTuple instance is used in multiple places". Ts appears only once in the signature of func4, but because it is defining the type of a variadic parameter, Ts appears multiple times from the perspective of a constraint solver. Constraints are supplied for each argument that maps to that variadic parameter.

Consider the following example. The spec clearly dictates that the call to func3 is an error. I think the call to func4 should likewise be an error.

def func3[*Ts](a: tuple[*Ts], b: tuple[*Ts], /): ...

func3((1,), ("a",)) # Clearly an error according to the spec

def func4[*Ts](*args: tuple[*Ts]): ...

func4((1,), ("a",)) # For consistency, should be an error too

So, unless we amend the spec, I think the conformance test should remain as is (with the possible addition of a better comment to explain the above).

I'm in favor of modifying the spec and removing this entire section.

Copy link
Member

@carljm carljm Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks @erictraut -- I missed that section when scanning the PEP text. I agree that that section of the PEP/spec text is applicable here, and also that it's unfortunately inconsistent with the usual behavior of Python typevar solving. I'm not sure any of the PEP authors are still active in Python typing, or we could ask what motivated that late addition to the PEP text.

There are use cases for stricter typevar solving that doesn't ever synthesize wider types, but those use cases exist for regular typevars too, not just for variadic ones. So it seems better to go for consistency, and introduce a new orthogonal type system feature for stricter solving. (One possible shape for that feature is a NoInfer qualifier -- though it's not clear how that could be used in the *args case.)

Copy link
Member Author

@JelleZijlstra JelleZijlstra Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the background! I have now amended this PR to propose changing the spec to remove this rule. I will post on Discuss before opening a vote for the Typing Council.

func4((0,), (1, 2)) # E (length mismatch)
func4((0,), ("1",)) # OK


# This is a syntax error, so leave it commented out.
Expand Down
4 changes: 2 additions & 2 deletions conformance/tests/generics_typevartuple_basic.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ def __init__(self, shape: tuple[Shape]): # E: not unpacked
self._shape: tuple[*Shape] = shape

def get_shape(self) -> tuple[Shape]: # E: not unpacked
return self._shape
raise NotImplementedError
Copy link
Member Author

@JelleZijlstra JelleZijlstra Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was causing mypy to emit an error because it resolves the invalid tuple[Shape] to tuple[Any], and then considers self._shape incompatible with a single-element tuple. I don't think the conformance suite should enforce the behavior of type checkers with erroneous annotations.


def method1(*args: Shape) -> None: # E: not unpacked
...
Expand Down Expand Up @@ -86,7 +86,7 @@ def func2(arg1: tuple[*Ts], arg2: tuple[*Ts]) -> tuple[*Ts]:
func2((0.0,), (0,)) # OK
func2((0,), (1,)) # OK

func2((0,), ("0",)) # E
func2((0,), ("0",)) # OK
func2((0, 0), (0,)) # E


Expand Down
35 changes: 9 additions & 26 deletions docs/spec/generics.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1189,51 +1189,34 @@ for two reasons:
* To improve readability: the star also functions as an explicit visual
indicator that the type variable tuple is not a normal type variable.

Variance, Type Constraints and Type Bounds: Not (Yet) Supported
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Variance, Type Constraints and Type Bounds: Not Supported
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""

``TypeVarTuple`` does not yet support specification of:
``TypeVarTuple`` does not currently support specification of:

* Variance (e.g. ``TypeVar('T', covariant=True)``)
* Type constraints (``TypeVar('T', int, float)``)
* Type bounds (``TypeVar('T', bound=ParentClass)``)

We leave the decision of how these arguments should behave to a future PEP, when variadic generics have been tested in the field. As of PEP 646, type variable tuples are
invariant.

Type Variable Tuple Equality
""""""""""""""""""""""""""""

If the same ``TypeVarTuple`` instance is used in multiple places in a signature
or class, a valid type inference might be to bind the ``TypeVarTuple`` to
a ``tuple`` of a union of types:
or class, type checkers may use the same rules for solving the variables as they
would for normal ``TypeVar``\ s. The exact inference behavior is not specified.

::

def foo(arg1: tuple[*Ts], arg2: tuple[*Ts]): ...
def foo(arg1: tuple[*Ts], arg2: tuple[*Ts]) -> tuple[*Ts]: ...

a = (0,)
b = ('0',)
foo(a, b) # Can Ts be bound to tuple[int | str]?

We do *not* allow this; type unions may *not* appear within the ``tuple``.
If a type variable tuple appears in multiple places in a signature,
the types must match exactly (the list of type parameters must be the same
length, and the type parameters themselves must be identical):
reveal_type(foo(a, b)) # May be e.g. tuple[object], tuple[int | str], tuple[Literal[0, "0"]]

::
All usages of the ``TypeVarTuple`` must match in length, however::

def pointwise_multiply(
x: Array[*Shape],
y: Array[*Shape]
) -> Array[*Shape]: ...
foo((1,), (2, 3)) # Error: Expected a tuple of length 1 for arg2, but got a tuple of length 2.

x: Array[Height]
y: Array[Width]
z: Array[Height, Width]
pointwise_multiply(x, x) # Valid
pointwise_multiply(x, y) # Error
pointwise_multiply(x, z) # Error

Multiple Type Variable Tuples: Not Allowed
""""""""""""""""""""""""""""""""""""""""""
Expand Down