Irregular inference with ternary based on attribute.

[randolf-scholz]

Consider the following test cases, which all work just fine:

from typing import TypeVar, Union, Callable, reveal_type

NOOP = lambda: None
class A: pass

def test_static_attr(x: Union[A, None]) -> None:
    def foo(t: A) -> None: ...

    l1: Callable[[], None] = (lambda: foo(x)) if x is not None else NOOP   # ✅
    r1: Callable[[], None] = NOOP if x is None else (lambda: foo(x))   # ✅
    l2 = (lambda: foo(x)) if x is not None else NOOP   # ✅
    r2 = NOOP if x is None else (lambda: foo(x))   # ✅
    reveal_type(l2)  # N: Revealed type is "def ()" ✅
    reveal_type(r2)  # N: Revealed type is "def ()" ✅

def test_generic_attr(x: Union[A, None]) -> None:
    T = TypeVar("T")
    def bar(t: T) -> T: return t

    l1: Callable[[], None] = (lambda: bar(x)) if x is None else NOOP   # ✅
    r1: Callable[[], None] = NOOP if x is not None else (lambda: bar(x))   # ✅
    l2 = (lambda: bar(x)) if x is None else NOOP   # ✅
    r2 = NOOP if x is not None else (lambda: bar(x))   # ✅
    reveal_type(l2)  # N: Revealed type is "def ()" ✅
    reveal_type(r2)  # N: Revealed type is "def ()" ✅

However, when we add a level of indirection by introducing a class B with B.attr: A | None, and basing the decision on this attribute, it sometimes works and sometimes doesn't:

from typing import TypeVar, Union, Callable, reveal_type

NOOP = lambda: None
class A: pass
class B:
    attr: Union[A, None]

def test_static_with_attr(x: B) -> None:
    def foo(t: A) -> None: ...

    l1: Callable[[], None] = (lambda: foo(x.attr)) if x.attr is not None else NOOP   # ❌
    r1: Callable[[], None] = NOOP if x.attr is None else (lambda: foo(x.attr))   # ❌
    l2 = (lambda: foo(x.attr)) if x.attr is not None else NOOP   # ✅
    r2 = NOOP if x.attr is None else (lambda: foo(x.attr))   # ❌
    reveal_type(l2)  # N: Revealed type is "def ()" ✅
    reveal_type(r2)  # N: Revealed type is "def ()" ✅

def test_generic_with_attr(x: B) -> None:
    T = TypeVar("T")
    def bar(t: T) -> T: return t

    l1: Callable[[], None] = (lambda: bar(x.attr)) if x.attr is None else NOOP  # ❌
    r1: Callable[[], None] = NOOP if x.attr is not None else (lambda: bar(x.attr))  # ❌
    l2 = (lambda: bar(x.attr)) if x.attr is None else NOOP  # ✅
    r2 = NOOP if x.attr is not None else (lambda: bar(x.attr))  # ✅
    reveal_type(l2)  # N: Revealed type is "def ()" ✅
    reveal_type(r2)  # N: Revealed type is "def () -> __main__.A | None"  ❌

https://mypy-play.net/?mypy=1.17.0&python=3.12&gist=106fbfcd39eee8f3a2c51d45b154b5fb

[randolf-scholz]

I believe one issue is that here the relevant entries are not copied from the old_binder:

mypy/mypy/checker.py

Lines 1403 to 1417 in 07d4a1b

	# Copy some type narrowings from an outer function when it seems safe enough
	# (i.e. we can't find an assignment that might change the type of the
	# variable afterwards).
	new_frame: Frame | None = None
	for frame in old_binder.frames:
	for key, narrowed_type in frame.types.items():
	key_var = extract_var_from_literal_hash(key)
	if key_var is not None and not self.is_var_redefined_in_outer_context(
	key_var, defn.line
	):
	# It seems safe to propagate the type narrowing to a nested scope.
	if new_frame is None:
	new_frame = self.binder.push_frame()
	new_frame.types[key] = narrowed_type
	self.binder.declarations[key] = old_binder.declarations[key]

[randolf-scholz]

Although, according to a comment in this unit test, attribute binding is not supported at all to begin with:

mypy/test-data/unit/check-optional.test

Lines 1103 to 1114 in 07d4a1b

	[case testNarrowedVariableInNestedFunctionMore1]
	from typing import Optional, overload
	class C:
	a: Optional[str]
	def attribute_narrowing(c: C) -> None:
	# This case is not supported, since we can't keep track of assignments to attributes.
	c.a = "x"
	def nested() -> str:
	return c.a # E: Incompatible return value type (got "Optional[str]", expected "str")
	nested()

[randolf-scholz]

The thing is, a similar piece of code appears in mypy's own codebase:

mypy/mypyc/irbuild/statement.py

Line 568 in 07d4a1b

else_body = (lambda: builder.accept(t.else_body)) if t.else_body else None

According to the comment in testNarrowedVariableInNestedFunctionMore1.attribute_narrowing, this code should be rejected, but it is essentially the one case l2 = (lambda: foo(x.attr)) if x.attr is not None else NOOP # ✅ that gets currently accepted in test_static_with_attr in my OP.

I think there are two possible resolutions:

1. All assignments in test_static_with_attr and test_generic_with_attr are to be rejected, as well as else_body code.
2. All assignments in test_static_with_attr and test_generic_with_attr are to be accepted, as well as the else_body code and the testNarrowedVariableInNestedFunctionMore1.attribute_narrowing code.

[randolf-scholz]

I made one PR for each option.

Option ② triggers quite a few additional unused-ignore and a few redundant-cast. Not sure about the unreachable alarms in core.

Fundamentally, I doubt option narrowing based on a member is any more unsafe than narrowing on a variable, though this example shows that in the current version option 2 does not even work with variables sometimes?

class C:
    a: str | None

def variable_narrowing(c: str | None) -> None:
    if isinstance(c, str):
        def nested() -> str:
            return c  # option1: ❌, option2: ❌
    else:
        return
    c = None
    nested()  # kaboom

def attribute_narrowing(c: C) -> None:
    if isinstance(c.a, str):
        def nested() -> str:
            return c.a  # option1: ❌, option2: ✅
    else:
        return
    c.a = None
    nested()  # kaboom

EDIT: Oh it's likely because is_var_redefined_in_outer_context triggers for the variable assignment, but not the attribute assignment.