Skip to content

conformance suite: do not assert questionable behavior with enums #2223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JelleZijlstra merged 2 commits into from
Mar 23, 2026
Merged

conformance suite: do not assert questionable behavior with enums #2223

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6a551ce
conformance suite: do not assert questionable behavior with enums
JelleZijlstra Mar 21, 2026
241ece1
regen
JelleZijlstra Mar 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 17 additions & 25 deletions conformance/results/mypy/enums_members.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,31 +9,23 @@ Does not treat attributes with private names as non-members.
"""
conformance_automated = "Fail"
errors_diff = """
Line 82: Expected 1 errors
Line 83: Expected 1 errors
Line 129: Expected 1 errors
Line 27: Unexpected errors ['enums_members.py:27: error: Expression is of type "Literal[Pet.genus]", not "str" [assert-type]']
Line 28: Unexpected errors ['enums_members.py:28: error: Expression is of type "Literal[Pet.species]", not "str" [assert-type]']
Line 35: Unexpected errors ['enums_members.py:35: error: Expression is of type "Literal[Pet2.genus]", not "str" [assert-type]']
Line 36: Unexpected errors ['enums_members.py:36: error: Expression is of type "Literal[Pet2.species]", not "str" [assert-type]']
Line 100: Unexpected errors ['enums_members.py:100: error: Expression is of type "Literal[TrafficLight.AMBER]", not "Literal[TrafficLight.YELLOW]" [assert-type]']
Line 117: Unexpected errors ['enums_members.py:117: error: Expression is of type "member[Callable[[Example], None]]", not "Any" [assert-type]', 'enums_members.py:117: error: Parameter 1 of Literal[...] is invalid [valid-type]']
Line 86: Expected 1 errors
Line 87: Expected 1 errors
Line 133: Expected 1 errors
Line 104: Unexpected errors ['enums_members.py:104: error: Expression is of type "Literal[TrafficLight.AMBER]", not "Literal[TrafficLight.YELLOW]" [assert-type]']
Line 121: Unexpected errors ['enums_members.py:121: error: Expression is of type "member[Callable[[Example], None]]", not "Any" [assert-type]', 'enums_members.py:121: error: Parameter 1 of Literal[...] is invalid [valid-type]']
"""
output = """
enums_members.py:27: error: Expression is of type "Literal[Pet.genus]", not "str" [assert-type]
enums_members.py:28: error: Expression is of type "Literal[Pet.species]", not "str" [assert-type]
enums_members.py:35: error: Expression is of type "Literal[Pet2.genus]", not "str" [assert-type]
enums_members.py:36: error: Expression is of type "Literal[Pet2.species]", not "str" [assert-type]
enums_members.py:50: error: Enum members must be left unannotated [misc]
enums_members.py:50: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
enums_members.py:84: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:85: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:100: error: Expression is of type "Literal[TrafficLight.AMBER]", not "Literal[TrafficLight.YELLOW]" [assert-type]
enums_members.py:116: error: Expression is of type "int", not "Literal[Example.b]" [assert-type]
enums_members.py:117: error: Expression is of type "member[Callable[[Example], None]]", not "Any" [assert-type]
enums_members.py:117: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:128: note: Revealed type is "Any"
enums_members.py:128: note: 'reveal_type' always outputs 'Any' in unchecked functions
enums_members.py:146: error: Expression is of type "Literal[Pet5.DOG]", not "int" [assert-type]
enums_members.py:147: error: Expression is of type "Literal[Pet5.FISH]", not "int" [assert-type]
enums_members.py:54: error: Enum members must be left unannotated [misc]
enums_members.py:54: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
enums_members.py:88: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:89: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:104: error: Expression is of type "Literal[TrafficLight.AMBER]", not "Literal[TrafficLight.YELLOW]" [assert-type]
enums_members.py:120: error: Expression is of type "int", not "Literal[Example.b]" [assert-type]
enums_members.py:121: error: Expression is of type "member[Callable[[Example], None]]", not "Any" [assert-type]
enums_members.py:121: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:132: note: Revealed type is "Any"
enums_members.py:132: note: 'reveal_type' always outputs 'Any' in unchecked functions
enums_members.py:150: error: Expression is of type "Literal[Pet5.DOG]", not "int" [assert-type]
enums_members.py:151: error: Expression is of type "Literal[Pet5.FISH]", not "int" [assert-type]
"""
22 changes: 11 additions & 11 deletions conformance/results/pyrefly/enums_members.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@ conformance_automated = "Pass"
errors_diff = """
"""
output = """
ERROR enums_members.py:50:5-8: Enum member `DOG` may not be annotated directly. Instead, annotate the `_value_` attribute. [invalid-annotation]
ERROR enums_members.py:82:20-34: `Pet4.converter` is not a valid enum member [invalid-literal]
ERROR enums_members.py:83:20-34: `Pet4.transform` is not a valid enum member [invalid-literal]
ERROR enums_members.py:84:18-30: `Pet4.species` is not a valid enum member [invalid-literal]
ERROR enums_members.py:85:16-26: `Pet4.speak` is not a valid enum member [invalid-literal]
ERROR enums_members.py:116:12-43: assert_type(int, Unknown) failed [assert-type]
ERROR enums_members.py:116:32-41: `Example.b` is not a valid enum member [invalid-literal]
ERROR enums_members.py:129:20-57: assert_type(int, Unknown) failed [assert-type]
ERROR enums_members.py:129:43-55: `Example2.__B` is not a valid enum member [invalid-literal]
ERROR enums_members.py:146:12-27: assert_type(Literal[Pet5.DOG], int) failed [assert-type]
ERROR enums_members.py:147:12-28: assert_type(Literal[Pet5.FISH], int) failed [assert-type]
ERROR enums_members.py:54:5-8: Enum member `DOG` may not be annotated directly. Instead, annotate the `_value_` attribute. [invalid-annotation]
ERROR enums_members.py:86:20-34: `Pet4.converter` is not a valid enum member [invalid-literal]
ERROR enums_members.py:87:20-34: `Pet4.transform` is not a valid enum member [invalid-literal]
ERROR enums_members.py:88:18-30: `Pet4.species` is not a valid enum member [invalid-literal]
ERROR enums_members.py:89:16-26: `Pet4.speak` is not a valid enum member [invalid-literal]
ERROR enums_members.py:120:12-43: assert_type(int, Unknown) failed [assert-type]
ERROR enums_members.py:120:32-41: `Example.b` is not a valid enum member [invalid-literal]
ERROR enums_members.py:133:20-57: assert_type(int, Unknown) failed [assert-type]
ERROR enums_members.py:133:43-55: `Example2.__B` is not a valid enum member [invalid-literal]
ERROR enums_members.py:150:12-27: assert_type(Literal[Pet5.DOG], int) failed [assert-type]
ERROR enums_members.py:151:12-28: assert_type(Literal[Pet5.FISH], int) failed [assert-type]
"""
30 changes: 15 additions & 15 deletions conformance/results/pyright/enums_members.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,19 @@ conformance_automated = "Pass"
errors_diff = """
"""
output = """
enums_members.py:50:5 - error: Type annotations are not allowed for enum members (reportGeneralTypeIssues)
enums_members.py:82:20 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:82:25 - error: Variable not allowed in type expression (reportInvalidTypeForm)
enums_members.py:83:20 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:83:25 - error: Variable not allowed in type expression (reportInvalidTypeForm)
enums_members.py:84:18 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:85:16 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:116:13 - error: "assert_type" mismatch: expected "Unknown" but received "int" (reportAssertTypeFailure)
enums_members.py:116:32 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:128:21 - information: Type of "Example2.__B" is "Literal[2]"
enums_members.py:129:21 - error: "assert_type" mismatch: expected "Unknown" but received "Literal[2]" (reportAssertTypeFailure)
enums_members.py:129:43 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:129:52 - error: Variable not allowed in type expression (reportInvalidTypeForm)
enums_members.py:146:13 - error: "assert_type" mismatch: expected "int" but received "Literal[Pet5.DOG]" (reportAssertTypeFailure)
enums_members.py:147:13 - error: "assert_type" mismatch: expected "int" but received "Literal[Pet5.FISH]" (reportAssertTypeFailure)
enums_members.py:54:5 - error: Type annotations are not allowed for enum members (reportGeneralTypeIssues)
enums_members.py:86:20 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:86:25 - error: Variable not allowed in type expression (reportInvalidTypeForm)
enums_members.py:87:20 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:87:25 - error: Variable not allowed in type expression (reportInvalidTypeForm)
enums_members.py:88:18 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:89:16 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:120:13 - error: "assert_type" mismatch: expected "Unknown" but received "int" (reportAssertTypeFailure)
enums_members.py:120:32 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:132:21 - information: Type of "Example2.__B" is "Literal[2]"
enums_members.py:133:21 - error: "assert_type" mismatch: expected "Unknown" but received "Literal[2]" (reportAssertTypeFailure)
enums_members.py:133:43 - error: Type arguments for "Literal" must be None, a literal value (int, bool, str, or bytes), or an enum value (reportInvalidTypeForm)
enums_members.py:133:52 - error: Variable not allowed in type expression (reportInvalidTypeForm)
enums_members.py:150:13 - error: "assert_type" mismatch: expected "int" but received "Literal[Pet5.DOG]" (reportAssertTypeFailure)
enums_members.py:151:13 - error: "assert_type" mismatch: expected "int" but received "Literal[Pet5.FISH]" (reportAssertTypeFailure)
"""
24 changes: 12 additions & 12 deletions conformance/results/ty/enums_members.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@ conformance_automated = "Pass"
errors_diff = """
"""
output = """
enums_members.py:50:10: error[invalid-enum-member-annotation] Type annotation on enum member `DOG` is not allowed
enums_members.py:82:20: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:83:20: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:84:18: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:85:16: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:116:1: error[type-assertion-failure] Type `int` does not match asserted type `Unknown`
enums_members.py:116:32: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:128:21: info[revealed-type] Revealed type: `Unknown | Literal[2]`
enums_members.py:129:9: error[type-assertion-failure] Type `Unknown | Literal[2]` does not match asserted type `Unknown`
enums_members.py:129:43: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:146:1: error[type-assertion-failure] Type `Unknown | Literal[2]` does not match asserted type `int`
enums_members.py:147:1: error[type-assertion-failure] Type `Unknown | Literal[3]` does not match asserted type `int`
enums_members.py:54:10: error[invalid-enum-member-annotation] Type annotation on enum member `DOG` is not allowed
enums_members.py:86:20: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:87:20: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:88:18: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:89:16: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:120:1: error[type-assertion-failure] Type `int` does not match asserted type `Unknown`
enums_members.py:120:32: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:132:21: info[revealed-type] Revealed type: `Unknown | Literal[2]`
enums_members.py:133:9: error[type-assertion-failure] Type `Unknown | Literal[2]` does not match asserted type `Unknown`
enums_members.py:133:43: error[invalid-type-form] Type arguments for `Literal` must be `None`, a literal value (int, bool, str, or bytes), or an enum member
enums_members.py:150:1: error[type-assertion-failure] Type `Unknown | Literal[2]` does not match asserted type `int`
enums_members.py:151:1: error[type-assertion-failure] Type `Unknown | Literal[3]` does not match asserted type `int`
"""
22 changes: 11 additions & 11 deletions conformance/results/zuban/enums_members.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ conformance_automated = "Pass"
errors_diff = """
"""
output = """
enums_members.py:50: error: Enum members must be left unannotated [misc]
enums_members.py:50: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
enums_members.py:82: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:83: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:84: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:85: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:116: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:128: note: Revealed type is "builtins.int"
enums_members.py:129: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:146: error: Expression is of type "Literal[Pet5.DOG]", not "int" [misc]
enums_members.py:147: error: Expression is of type "Literal[Pet5.FISH]", not "int" [misc]
enums_members.py:54: error: Enum members must be left unannotated [misc]
enums_members.py:54: note: See https://typing.readthedocs.io/en/latest/spec/enums.html#defining-members
enums_members.py:86: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:87: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:88: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:89: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:120: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:132: note: Revealed type is "builtins.int"
enums_members.py:133: error: Parameter 1 of Literal[...] is invalid [valid-type]
enums_members.py:150: error: Expression is of type "Literal[Pet5.DOG]", not "int" [misc]
enums_members.py:151: error: Expression is of type "Literal[Pet5.FISH]", not "int" [misc]
"""
12 changes: 8 additions & 4 deletions conformance/tests/enums_members.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,20 @@ def __init__(self, genus: str, species: str) -> None:
self.species = species


assert_type(Pet.genus, str)
assert_type(Pet.species, str)
def func1(pet: Pet) -> None:
assert_type(pet.genus, str)
assert_type(pet.species, str)

assert_type(Pet.CAT, Literal[Pet.CAT])
assert_type(Pet.DOG, Literal[Pet.DOG])


from _enums_members import Pet2

assert_type(Pet2.genus, str)
assert_type(Pet2.species, str)
def func2(pet: Pet2) -> None:
assert_type(pet.genus, str)
assert_type(pet.species, str)

assert_type(Pet2.CAT, Literal[Pet2.CAT])
assert_type(Pet2.DOG, Literal[Pet2.DOG])

Expand Down