Skip to content

Add Pyrefly #2080

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
erictraut merged 5 commits into from
Aug 31, 2025
Merged

Add Pyrefly #2080

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d8ad4fd
add pyrefly type checker
yangdanny97 Aug 30, 2025
bd82c91
manually audit pyrefly results
yangdanny97 Aug 30, 2025
19d81f9
update repoort
yangdanny97 Aug 30, 2025
d80e5c1
mark mostly/completely unsupported features as such
yangdanny97 Aug 30, 2025
bbda385
regenerate HTML
yangdanny97 Aug 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions conformance/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ mypy
pip
pyre-check; platform_system != "Windows"
zuban
pyrefly
28 changes: 28 additions & 0 deletions conformance/results/pyrefly/aliases_explicit.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
conformance_automated = "Pass"
errors_diff = """
"""
output = """
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:67:9-28: `TypeAlias[GoodTypeAlias2, type[int | None]]` is not subscriptable [unsupported-operation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:68:9-28: `TypeAlias[GoodTypeAlias3, type[list[int | None]]]` is not subscriptable [unsupported-operation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:69:9-33: Expected 1 type argument for `GoodTypeAlias4`, got 2 [bad-specialization]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:70:9-33: Expected 1 type argument for `GoodTypeAlias8`, got 2 [bad-specialization]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:71:9-33: Expected a valid ParamSpec expression, got `int` [invalid-param-spec]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:79:21-61: function call cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:80:21-31: list literal cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:81:21-34: tuple literal cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:82:21-44: list comprehension cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:83:21-31: dict literal cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:83:23-24: Could not find name `a` [unknown-name]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:83:28-29: Could not find name `b` [unknown-name]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:84:21-36: function call cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:85:21-29: invalid subscript expression cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:86:21-42: if expression cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:87:21-25: Expected `BadTypeAlias9` to be a type alias, got `Literal[3]` [type-alias-error]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:88:22-26: bool literal cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:89:22-23: number literal cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:90:22-33: boolean operation cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:91:22-32: f-string cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:100:5-19: `TypeAlias[ListAlias, type[list[Unknown]]]` is not subscriptable [unsupported-operation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:101:6-20: Expected a callable, got TypeAlias[ListOrSetAlias, type[list[Unknown] | set[Unknown]]] [not-callable]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_explicit.py:102:5-24: `TypeAlias[ListOrSetAlias, type[list[Unknown] | set[Unknown]]]` is not subscriptable [unsupported-operation]
"""
31 changes: 31 additions & 0 deletions conformance/results/pyrefly/aliases_implicit.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
conformant = "Partial"
notes = """
Does not reject invalid syntax in implicit type aliases.
"""
conformance_automated = "Fail"
errors_diff = """
Line 106: Expected 1 errors
Line 111: Expected 1 errors
Line 112: Expected 1 errors
Line 113: Expected 1 errors
Line 117: Expected 1 errors
"""
output = """
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:76:9-28: `type[int | None]` is not subscriptable [unsupported-operation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:77:9-28: `type[list[int | None]]` is not subscriptable [unsupported-operation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:78:9-33: Expected 1 type argument for `GoodTypeAlias4`, got 2 [bad-specialization]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:79:9-33: Expected 1 type argument for `GoodTypeAlias8`, got 2 [bad-specialization]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:80:9-33: Expected a valid ParamSpec expression, got `int` [invalid-param-spec]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:81:9-29: Type `str` is not assignable to upper bound `float` of type variable `TFloat` [bad-specialization]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:107:9-22: Expected a type form, got instance of `list[type[int] | type[str]]` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:108:9-22: Expected a type form, got instance of `tuple[tuple[type[int], type[str]]]` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:109:9-22: Expected a type form, got instance of `list[type[int]]` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:110:9-22: Expected a type form, got instance of `dict[str, str]` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:114:9-22: Expected a type form, got instance of `Literal[3]` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:115:10-24: Expected a type form, got instance of `Literal[True]` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:116:10-24: Expected a type form, got instance of `Literal[1]` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:118:10-24: Expected a type form, got instance of `Literal['int']` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:119:10-24: Expected a type form, got instance of `Literal['int | str']` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:133:6-20: Expected a callable, got type[list[Unknown] | set[Unknown]] [not-callable]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_implicit.py:135:5-24: `type[list[Unknown] | set[Unknown]]` is not subscriptable [unsupported-operation]
"""
19 changes: 19 additions & 0 deletions conformance/results/pyrefly/aliases_newtype.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
conformance_automated = "Pass"
errors_diff = """
"""
output = """
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:11:8-14: Argument `Literal['user']` is not assignable to parameter `_x` with type `int` in function `UserId.__new__` [bad-argument-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:12:14-16: `Literal[42]` is not assignable to `UserId` [bad-assignment]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:18:11-17: `type[UserId]` is not assignable to `type` [bad-assignment]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:23:16-22: NewType `UserId` not allowed in isinstance() [invalid-argument]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:26:21-27: Subclassing a NewType not allowed [invalid-inheritance]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:35:20-29: Expected string literal "GoodName" [invalid-argument]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:41:6-23: Expected 0 type arguments for `GoodNewType1`, got 1 [bad-specialization]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:47:38-47: Second argument to NewType is invalid [invalid-argument]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:50:38-45: Second argument to NewType cannot be an unbound generic [invalid-argument]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:52:38-46: Second argument to NewType cannot be a protocol [invalid-argument]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:54:38-48: Second argument to NewType is invalid [invalid-argument]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:61:38-41: Second argument to NewType is invalid [invalid-argument]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:63:43-46: Expected 2 positional arguments, got 3 in function `typing.NewType.__init__` [bad-argument-count]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_newtype.py:65:38-41: Second argument to NewType is invalid [invalid-argument]
"""
33 changes: 33 additions & 0 deletions conformance/results/pyrefly/aliases_recursive.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
conformant = "Unsupported"
conformance_automated = "Fail"
errors_diff = """
Line 19: Expected 1 errors
Line 20: Expected 1 errors
Line 38: Expected 1 errors
Line 39: Expected 1 errors
Line 50: Expected 1 errors
Line 51: Expected 1 errors
Line 52: Expected 1 errors
Line 72: Expected 1 errors
Line 75: Expected 1 errors
Line 30: Unexpected errors ["Expected a type form, got instance of `Literal['RecursiveTuple']` [not-a-type]"]
Line 42: Unexpected errors ["Expected a type form, got instance of `Literal['RecursiveMapping']` [not-a-type]"]
Line 58: Unexpected errors ['Expected a type form, got instance of `_SpecialForm` [not-a-type]']
Line 59: Unexpected errors ['`type[list[Unknown]]` is not subscriptable [unsupported-operation]']
Line 62: Unexpected errors ['`type[list[Unknown]]` is not subscriptable [unsupported-operation]']
Line 65: Unexpected errors ['Expected a type form, got instance of `_SpecialForm` [not-a-type]']
Line 67: Unexpected errors ['`type[list[Unknown]]` is not subscriptable [unsupported-operation]']
Line 68: Unexpected errors ['`type[list[Unknown]]` is not subscriptable [unsupported-operation]']
"""
output = """
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_recursive.py:30:36-52: Expected a type form, got instance of `Literal['RecursiveTuple']` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_recursive.py:42:45-63: Expected a type form, got instance of `Literal['RecursiveMapping']` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_recursive.py:58:26-54: Expected a type form, got instance of `_SpecialForm` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_recursive.py:59:25-47: `type[list[Unknown]]` is not subscriptable [unsupported-operation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_recursive.py:62:5-27: `type[list[Unknown]]` is not subscriptable [unsupported-operation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_recursive.py:63:5-27: `type[list[Unknown]]` is not subscriptable [unsupported-operation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_recursive.py:65:26-63: Expected a type form, got instance of `_SpecialForm` [not-a-type]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_recursive.py:67:5-32: `type[list[Unknown]]` is not subscriptable [unsupported-operation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_recursive.py:68:5-34: `type[list[Unknown]]` is not subscriptable [unsupported-operation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_recursive.py:69:5-32: `type[list[Unknown]]` is not subscriptable [unsupported-operation]
"""
40 changes: 40 additions & 0 deletions conformance/results/pyrefly/aliases_type_statement.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
conformant = "Partial"
notes = """
Does not reject type alias defined with type statement in base class list
Does not reject redeclaration of type alias with the same name.
Does not detect circular definitions.
"""
conformance_automated = "Fail"
errors_diff = """
Line 26: Expected 1 errors
Line 82: Expected 1 errors
Line 84: Expected 1 errors
Lines 51, 52: Expected error (tag 'TA14')
Lines 88, 89: Expected error (tag 'RTA6')
"""
output = """
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:17:1-21: Object of class `TypeAliasType` has no attribute `bit_count` [missing-attribute]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:19:1-11: Expected a callable, got TypeAlias[GoodAlias1, type[int]] [not-callable]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:23:7-30: Object of class `TypeAliasType` has no attribute `other_attrib` [missing-attribute]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:31:22-32: Expected class object, got `TypeAliasType` [invalid-argument]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:37:22-62: function call cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:38:22-32: list literal cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:39:22-35: tuple literal cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:40:22-45: list comprehension cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:41:22-32: dict literal cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:41:24-25: Could not find name `a` [unknown-name]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:41:29-30: Could not find name `b` [unknown-name]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:42:22-37: function call cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:43:22-30: invalid subscript expression cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:44:22-43: if expression cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:45:22-26: Expected `BadTypeAlias9` to be a type alias, got `Literal[1]` [type-alias-error]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:46:23-27: bool literal cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:47:23-24: number literal cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:48:23-34: boolean operation cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:49:23-33: f-string cannot be used in annotations [invalid-annotation]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:56:5-30: `type` statement is not allowed in this context [invalid-syntax]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:62:15-25: Type parameters used in `TA1` but not declared [invalid-type-var]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:67:12-20: Type parameters used in `TA2` but not declared [invalid-type-var]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:77:7-41: Type `str` is not assignable to upper bound `int` of type variable `S` [bad-specialization]
ERROR /Users/dannyyang/Documents/Documents/GitHub/typing/conformance/tests/aliases_type_statement.py:79:7-41: Type `int` is not assignable to upper bound `str` of type variable `T` [bad-specialization]
"""
Loading