Skip to content

(🎁) conformance: check for variance on constrained type variables #2016

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
KotlinIsland wants to merge 2 commits into from
Closed

(🎁) conformance: check for variance on constrained type variables #2016

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
273a90a
conformance: check for variance on constraints
KotlinIsland May 28, 2025
cdb0d02
conformance: update results
KotlinIsland May 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 16 additions & 12 deletions conformance/results/mypy/generics_basic.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,26 @@
conformant = "Pass"
conformant = "Partial"
notes = """
Incorrectly allows variance on contrained type variables.
"""
output = """
generics_basic.py:40: error: Value of type variable "AnyStr" of "concat" cannot be "Sequence[object]" [type-var]
generics_basic.py:41: error: Value of type variable "AnyStr" of "concat" cannot be "Sequence[object]" [type-var]
generics_basic.py:49: error: Type variable must have at least two constrained types [misc]
generics_basic.py:55: error: TypeVar constraint type cannot be parametrized by type variables [misc]
generics_basic.py:69: error: Value of type variable "AnyStr" of "concat" cannot be "Sequence[object]" [type-var]
generics_basic.py:121: error: Duplicate type variables in Generic[...] or Protocol[...] [misc]
generics_basic.py:157: error: Invalid index type "int" for "MyMap1[str, int]"; expected type "str" [index]
generics_basic.py:158: error: Invalid index type "int" for "MyMap2[int, str]"; expected type "str" [index]
generics_basic.py:162: error: Free type variable expected in Generic[...] [misc]
generics_basic.py:163: error: Free type variable expected in Protocol[...] [misc]
generics_basic.py:171: error: If Generic[...] or Protocol[...] is present it should list all type variables [misc]
generics_basic.py:122: error: Duplicate type variables in Generic[...] or Protocol[...] [misc]
generics_basic.py:158: error: Invalid index type "int" for "MyMap1[str, int]"; expected type "str" [index]
generics_basic.py:159: error: Invalid index type "int" for "MyMap2[int, str]"; expected type "str" [index]
generics_basic.py:163: error: Free type variable expected in Generic[...] [misc]
generics_basic.py:164: error: Free type variable expected in Protocol[...] [misc]
generics_basic.py:172: error: If Generic[...] or Protocol[...] is present it should list all type variables [misc]
generics_basic.py:208: error: Dynamic metaclass not supported for "GenericMetaInstance" [misc]
generics_basic.py:208: error: Type variable "generics_basic.T" is unbound [valid-type]
generics_basic.py:208: note: (Hint: Use "Generic[T]" or "Protocol[T]" base class to bind "T" inside a class)
generics_basic.py:208: note: (Hint: Use "T" in function signature to bind "T" inside a function)
generics_basic.py:173: error: If Generic[...] or Protocol[...] is present it should list all type variables [misc]
generics_basic.py:209: error: Dynamic metaclass not supported for "GenericMetaInstance" [misc]
generics_basic.py:209: error: Type variable "generics_basic.T" is unbound [valid-type]
generics_basic.py:209: note: (Hint: Use "Generic[T]" or "Protocol[T]" base class to bind "T" inside a class)
generics_basic.py:209: note: (Hint: Use "T" in function signature to bind "T" inside a function)
"""
conformance_automated = "Pass"
conformance_automated = "Fail"
errors_diff = """
Line 75: Expected 1 errors
"""
2 changes: 1 addition & 1 deletion conformance/results/mypy/version.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
version = "mypy 1.15.0"
test_duration = 1.5
test_duration = 2.1
4 changes: 2 additions & 2 deletions conformance/results/pyright/aliases_typealiastype.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ aliases_typealiastype.py:61:42 - error: Expected class but received "Literal[Tru
aliases_typealiastype.py:62:42 - error: Expected class but received "Literal[1]" (reportGeneralTypeIssues)
aliases_typealiastype.py:63:42 - error: Binary operator not allowed in type expression (reportInvalidTypeForm)
aliases_typealiastype.py:64:42 - error: Type expressions cannot use format string literals (f-strings) (reportGeneralTypeIssues)
aliases_typealiastype.py:66:47 - error: "BadAlias21" is not defined (reportUndefinedVariable)
"""
conformance_automated = "Fail"
conformance_automated = "Pass"
errors_diff = """
Line 66: Expected 1 errors
"""
7 changes: 3 additions & 4 deletions conformance/results/pyright/callables_protocol.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,10 @@ callables_protocol.py:70:7 - error: Type "(**b: str) -> None" is not assignable
  Type "(**b: str) -> None" is not assignable to type "(*vals: bytes, **kwargs: str) -> None"
    Parameter "*vals" has no corresponding parameter (reportAssignmentType)
callables_protocol.py:97:16 - error: Type "(x: int) -> None" is not assignable to declared type "Proto4"
  "function" is incompatible with protocol "Proto4"
    "other_attribute" is not present
    "__call__" is not present (reportAssignmentType)
  "FunctionType" is incompatible with protocol "Proto4"
    "other_attribute" is not present (reportAssignmentType)
callables_protocol.py:121:18 - error: Type "(*vals: bytes, max_len: int | None = None) -> list[bytes]" is not assignable to declared type "NotProto6"
  "function" is not assignable to "NotProto6" (reportAssignmentType)
  "FunctionType" is not assignable to "NotProto6" (reportAssignmentType)
callables_protocol.py:169:7 - error: Type "(x: int) -> Any" is not assignable to declared type "Proto8"
  One or more overloads of "__call__" is not assignable
    Type "(x: int) -> Any" is not assignable to type "(x: str) -> str"
Expand Down
24 changes: 14 additions & 10 deletions conformance/results/pyright/generics_basic.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
conformant = "Pass"
conformant = "Partial"
notes = """
Incorrectly allows variance on contrained type variables.
"""
output = """
generics_basic.py:40:15 - error: Argument of type "bytes" cannot be assigned to parameter "y" of type "AnyStr@concat" in function "concat"
  "bytes" is not assignable to "str" (reportArgumentType)
Expand All @@ -8,19 +11,20 @@ generics_basic.py:49:44 - error: TypeVar must have at least two constrained type
generics_basic.py:55:53 - error: TypeVar constraint type cannot be generic (reportGeneralTypeIssues)
generics_basic.py:69:15 - error: Argument of type "bytes" cannot be assigned to parameter "y" of type "AnyStr@concat" in function "concat"
  "bytes" is not assignable to "str" (reportArgumentType)
generics_basic.py:121:24 - error: Type arguments for "Generic" must be unique (reportInvalidTypeForm)
generics_basic.py:157:5 - error: Argument of type "Literal[0]" cannot be assigned to parameter "key" of type "str" in function "__getitem__"
  "Literal[0]" is not assignable to "str" (reportArgumentType)
generics_basic.py:122:24 - error: Type arguments for "Generic" must be unique (reportInvalidTypeForm)
generics_basic.py:158:5 - error: Argument of type "Literal[0]" cannot be assigned to parameter "key" of type "str" in function "__getitem__"
  "Literal[0]" is not assignable to "str" (reportArgumentType)
generics_basic.py:162:20 - error: Type argument for "Generic" must be a type variable (reportInvalidTypeForm)
generics_basic.py:163:21 - error: Type argument for "Protocol" must be a type parameter (reportInvalidTypeForm)
generics_basic.py:171:7 - error: Generic[] or Protocol[] must include all type variables
  Missing type variables: "T_co" (reportGeneralTypeIssues)
generics_basic.py:159:5 - error: Argument of type "Literal[0]" cannot be assigned to parameter "key" of type "str" in function "__getitem__"
  "Literal[0]" is not assignable to "str" (reportArgumentType)
generics_basic.py:163:20 - error: Type argument for "Generic" must be a type variable (reportInvalidTypeForm)
generics_basic.py:164:21 - error: Type argument for "Protocol" must be a type parameter (reportInvalidTypeForm)
generics_basic.py:172:7 - error: Generic[] or Protocol[] must include all type variables
  Missing type variables: "T_co" (reportGeneralTypeIssues)
generics_basic.py:208:37 - error: Metaclass cannot be generic (reportGeneralTypeIssues)
generics_basic.py:173:7 - error: Generic[] or Protocol[] must include all type variables
  Missing type variables: "T_co" (reportGeneralTypeIssues)
generics_basic.py:209:37 - error: Metaclass cannot be generic (reportGeneralTypeIssues)
"""
conformance_automated = "Pass"
conformance_automated = "Fail"
errors_diff = """
Line 75: Expected 1 errors
"""
9 changes: 8 additions & 1 deletion conformance/results/pyright/generics_syntax_infer_variance.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@ generics_syntax_infer_variance.py:29:35 - error: Type "ShouldBeCovariant1[float]
  "ShouldBeCovariant1[float]" is not assignable to "ShouldBeCovariant1[int]"
    Type parameter "T@ShouldBeCovariant1" is covariant, but "float" is not a subtype of "int"
      "float" is not assignable to "int" (reportAssignmentType)
generics_syntax_infer_variance.py:36:37 - error: Cannot instantiate abstract class "ShouldBeCovariant2"
  "Collection.__len__" is not implemented
  "Sequence.__getitem__" is not implemented (reportAbstractUsage)
generics_syntax_infer_variance.py:37:35 - error: Cannot instantiate abstract class "ShouldBeCovariant2"
  "Collection.__len__" is not implemented
  "Sequence.__getitem__" is not implemented (reportAbstractUsage)
generics_syntax_infer_variance.py:37:35 - error: Type "ShouldBeCovariant2[float]" is not assignable to declared type "ShouldBeCovariant2[int]"
  "ShouldBeCovariant2[float]" is not assignable to "ShouldBeCovariant2[int]"
    Type parameter "T@ShouldBeCovariant2" is covariant, but "float" is not a subtype of "int"
Expand Down Expand Up @@ -57,6 +63,7 @@ generics_syntax_infer_variance.py:155:45 - error: Type "ShouldBeContravariant1[i
    Type parameter "T@ShouldBeContravariant1" is contravariant, but "int" is not a supertype of "float"
      "float" is not assignable to "int" (reportAssignmentType)
"""
conformance_automated = "Pass"
conformance_automated = "Fail"
errors_diff = """
Line 36: Unexpected errors ['generics_syntax_infer_variance.py:36:37 - error: Cannot instantiate abstract class "ShouldBeCovariant2"']
"""
2 changes: 1 addition & 1 deletion conformance/results/pyright/protocols_definition.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ protocols_definition.py:160:22 - error: Type "Concrete3_Bad5" is not assignable
protocols_definition.py:218:22 - error: Type "Concrete4_Bad1" is not assignable to declared type "Template4"
  "Concrete4_Bad1" is incompatible with protocol "Template4"
    "val1" is an incompatible type
      "function" is not assignable to "Sequence[float]" (reportAssignmentType)
      "FunctionType" is not assignable to "Sequence[float]" (reportAssignmentType)
protocols_definition.py:219:22 - error: Type "Concrete4_Bad2" is not assignable to declared type "Template4"
  "Concrete4_Bad2" is incompatible with protocol "Template4"
    "val1" is not present (reportAssignmentType)
Expand Down
4 changes: 2 additions & 2 deletions conformance/results/pyright/version.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
version = "pyright 1.1.400"
test_duration = 1.1
version = "pyright 1.1.401"
test_duration = 1.0
8 changes: 4 additions & 4 deletions conformance/results/results.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,13 +159,13 @@ <h3>Python Type System Conformance Test Results</h3>
<div class="table_container"><table><tbody>
<tr><th class="col1">&nbsp;</th>
<th class='tc-header'><div class='tc-name'>mypy 1.15.0</div>
<div class='tc-time'>1.5sec</div>
<div class='tc-time'>2.1sec</div>
</th>
<th class='tc-header'><div class='tc-name'>pyright 1.1.400</div>
<div class='tc-time'>1.1sec</div>
<th class='tc-header'><div class='tc-name'>pyright 1.1.401</div>
<div class='tc-time'>1.0sec</div>
</th>
<th class='tc-header'><div class='tc-name'>pyre 0.9.23</div>
<div class='tc-time'>10.7sec</div>
<div class='tc-time'>0.6sec</div>
</th>
</tr>
<tr><th class="column" colspan="4">
Expand Down
1 change: 1 addition & 0 deletions conformance/tests/generics_basic.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ def test_concat_subtype(s: str, b: bytes, a: Any, m: MyStr) -> None:
# reveal_type(concat(m, a))
# reveal_type(concat(a, m))

BadConstraint3 = TypeVar("BadConstraint3", str, int, covariant=True) # E: constraints can't have variance
Copy link
Collaborator

@erictraut erictraut May 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find where in the typing spec this is specified. I'm also not sure why it would be the case that value-constrained type variables can't be covariant.

In any case, all conformance tests should derive from the spec itself. You'll see that in most of the tests, the exact text of the spec is quoted to motivate each of the tests.


# Specification: https://typing.readthedocs.io/en/latest/spec/generics.html#user-defined-generic-classes

Expand Down