Skip to content

Instantly share code, notes, and snippets.

@metatoaster
Last active June 13, 2025 05:15
Show Gist options
  • Save metatoaster/e8619d588b6a8bac2db0ebfaf83d4809 to your computer and use it in GitHub Desktop.
Save metatoaster/e8619d588b6a8bac2db0ebfaf83d4809 to your computer and use it in GitHub Desktop.
Python type hinting woes
from typing import Protocol
class IFoo(Protocol):
value: int
class Foo:
_value: int
@property
def value(self) -> int:
return self._value
@value.setter
def value(self, value: int):
self._value = value
foo = Foo()
foo.value = 3
print(foo.value)
# $ mypy cls_int_prop_get.py
# Success: no issues found in 1 source file
from typing import Protocol
class IFoo(Protocol):
value: int
class Foo:
@property
def value(self) -> int:
return self._value
_value: int # moved cls_int_prop_get.py:7 to here; all required declarations present in a different viable order
@value.setter # false postive in mypy
def value(self, value: int):
self._value = value
foo = Foo()
foo.value = 3
print(foo.value)
# $ mypy cls_prop_int_get.py
# cls_prop_int_get.py:13: error: Name "value" already defined on line 7
# cls_prop_int_get.py:13: error: "Callable[[Foo], int]" has no attribute "setter"
# cls_prop_int_get.py:19: error: Property "value" defined in "Foo" is read-only

Match exhaustiveness check is punted (e.g. to return), will not work universally

From this Stackoverflow Q&A. While PEP-0622 has a section on exhaustiveness checks on how it should be done, mypy currently (as of 1.9.0) fully relies on the return statements in all the arms to function correctly in the following example (derived using OP's self-answer):

from dataclasses import dataclass

@dataclass
class Mult:
    left: 'Expr'
    right: 'Expr'

@dataclass
class Add:
    left: 'Expr'
    right: 'Expr'

@dataclass
class Const:
    val: int

Expr = Const | Add | Mult

def my_eval(e : Expr) -> int:
    match e:
        case Const(val):
            return val
        case Add(left, right):
            return my_eval(left) + my_eval(right)
        case Mult(left, right):
            return my_eval(left) * my_eval(right)

def main():
    print(my_eval(Const(42)))
    print(my_eval(Add(Const(42), Const(45))))

if __name__ == '__main__':
    main()

If the Multi branch is omitted (e.g. by commenting the lines out), mypy will flag this following error:

exhaust.py:19: error: Missing return statement  [return]

Likewise, pyright will do something similar:

/tmp/exhaust.py
  /tmp/exhaust.py:19:26 - error: Function with declared return type "int" must return value on all code paths
    "None" is incompatible with "int" (reportReturnType)
1 error, 0 warnings, 0 informations 

Which is rather curious because they all imply return being involved. What if the code was restructured to not follow that? Consider the following my_eval:

def my_eval(e : Expr) -> int:
    result: int
    match e:
        case Const(val):
            result = val
        case Add(left, right):
            result = my_eval(left) + my_eval(right)
        case Mult(left, right):
            result = my_eval(left) * my_eval(right)
    return result

Both mypy and pyright will not complain, and neither should they as all branches are covered. But what if Multi was commented out?

$ mypy exhaust.py 
Success: no issues found in 1 source file
$ pyright exhaust.py 
/tmp/exhaust.py
  /tmp/exhaust.py:28:12 - error: "result" is possibly unbound (reportPossiblyUnboundVariable)
1 error, 0 warnings, 0 informations 

Welp, mypy fails, and pyright at least has a fallback, but given that pyright also punted this to an unbound variable, what if result was provided with a default value (e.g. result: int = 0)?

0 errors, 0 warnings, 0 informations 

So effectively, downstream users of Expr can have no warning about whether or not they properly handle all cases that might be provided by Expr.

In languages that have more strict definitions of types, they will flag missing arms as a failure unless there is a explicit default branch, the following is roughly equivalent to the failing example in Rust:

enum Expr {
    Multi {
        left: Box<Expr>,
        right: Box<Expr>,
    },
    Add {
        left: Box<Expr>,
        right: Box<Expr>,
    },
    Const {
        val: i64,
    }
}

fn my_eval(e: Expr) -> i64 {
    let result;
    match e {
        Expr::Const {val} => result = val,
        Expr::Add {left, right} => result = my_eval(*left) + my_eval(*right),
        Expr::Multi {left, right} => result = my_eval(*left) * my_eval(*right),
    }
    result
}

fn main() {
    let c = my_eval(Expr::Const { val: 42 });
    let a = my_eval(Expr::Add {
        left: Box::new(Expr::Const { val: 42 }),
        right: Box::new(Expr::Const { val: 45 }),
    });
    println!("{c}");
    println!("{a}");
}

Commenting out the Multi arm will result in the compilation error.

error[E0004]: non-exhaustive patterns: `Expr::Multi { .. }` not covered
  --> exhaust.rs:17:11
   |
17 |     match e {
   |           ^ pattern `Expr::Multi { .. }` not covered

This demonstrates that in Rust it is possible to rely on the compiler to catch fresh unhandled match branches, but this use case is (currently) not possible under Python, using mypy-1.9.0 or pyright-1.1.359 or earlier. This issue may be fixed for mypy if python/mypy#17141 is resolved (maybe only for these variants and not others, I personally don't have high hopes). Nevermind that! PEP 634 supposedly the replacement to that earlier PEP 622 and now there is no exhaustiveness requirements, as per the comment that closed that issue as duplicate of python/mypy#13597. (Maybe a fix will be provided via that other issue, eventually?)

Lack of true optional type

From this Stackoverflow question. In short, the following demonstrates how this problem can crop up:

from typing import Optional

myvar: Optional[int] = None
reveal_type(myvar)

myvar = 42
reveal_type(myvar)

if myvar < 1:
    print("Error")

Running the above through Mypy (mypy 1.8.0)

so78038503_1.py:4: note: Revealed type is "Union[builtins.int, None]"
so78038503_1.py:7: note: Revealed type is "builtins.int"
Success: no issues found in 1 source file

The type revealed is implicitly changed, due to lack of true optional type (e.g. Option in Rust or Maybe in Haskell/Elm). This will result in code that was working (the myvar < 1) failing later when the assignment is refactored, e.g.:

from typing import Optional

def answer() -> Optional[int]:
    return 42

myvar: Optional[int] = None
reveal_type(myvar)

myvar = answer()
reveal_type(myvar)

if myvar < 1:
    print("Error")

When the above is checked with mypy:

so78038503_3.py:7: note: Revealed type is "Union[builtins.int, None]"
so78038503_3.py:10: note: Revealed type is "Union[builtins.int, None]"
so78038503_3.py:12: error: Unsupported operand types for > ("int" and "None")  [operator]
so78038503_3.py:12: note: Left operand is of type "int | None"
Found 1 error in 1 file (checked 1 source file)

The answer function can legally return None as per that definition which would also trigger the runtime error that wasn't caught earlier, despite the correct annotation being added in the first place.

This particular issue seem to be a mypy (and pyright) specific issue as the underlying __annotation__ is unchanged. For completeness, equivalent code in Rust does not have this issue, given the abiilty to compare different Optional values:

let mut myvar: Option<i64> = None;
myvar = Some(42);

if myvar < Some(1) {
    println!("Error");
}
from typing import Protocol, Tuple
class Moveable(Protocol):
position: Tuple[int, int]
class Player:
_x: int
_y: int
@property
def position(self) -> Tuple[int, int]:
return (self._x, self._y)
@position.setter
def position(self, value: Tuple[int, int]):
self._x = value[0]
self._y = value[1]
entity: Moveable = Player() # false positive in pyright
entity.position = (1, 2)
print(entity.position)
# $ pyright protocol.py
# pyright 1.1.275
# /tmp/protocol.py:19:20 - error: Expression of type "Player" cannot be assigned to declared type "Moveable"
#   "Player" is incompatible with protocol "Moveable"
#     "position" is invariant because it is mutable
#     "position" is an incompatible type
#       "property" is incompatible with "Tuple[int, int]" (reportGeneralTypeIssues)
# $ mypy --version
# mypy 0.971 (compiled: yes)
# $ mypy protocol.py
# Success: no issues found in 1 source file

Poor integration between interpreter and the type hinting system

This is a clash of what was "traditionally" considered Pythonic vs. code considered legal type under hinting. Note the following examples:

def illegal(value: str, some_items: list[str] | None) -> bool:
    return some_items and value in some_items

def legal(value: str, some_items: list[str] | None) -> bool:
    return some_items is not None and value in some_items

def legal_conditional_expr(value: str, some_items: list[str] | None) -> bool:
    return value in some_items if some_items else False

def legal_but_more_awkward(value: str, some_items: list[str] | None) -> bool:
    return bool(some_items) and value in some_items

The first, now illegal example, results in the most concise and is understood by the interpreter to be a truthy value and the evaluation of optional_items by the interpreter effectively drop the value and evaluate the second expression and ultimately will return some bool value. However, the type hinting checkers thus far will report the third line in the illegal function as an error.

$ mypy short_circuit_illegality.py 
short_circuit_illegality.py:2: error: Incompatible return value type (got "list[str] | bool | None", expected "bool")  [return-value]
short_circuit_illegality.py:11: error: Unsupported right operand type for in ("list[str] | None")  [operator]
Found 2 errors in 1 file (checked 1 source file)
$ pyright short_circuit_illegality.py 
/tmp/short_circuit_illegality.py
  /tmp/short_circuit_illegality.py:2:12 - error: Expression of type "list[str] | bool | None" cannot be assigned to return type "bool"
    Type "list[str] | bool | None" cannot be assigned to type "bool"
      "list[str]" is incompatible with "bool" (reportGeneralTypeIssues)
1 error, 0 warnings, 0 informations 
$ pyright --version
pyright 1.1.316
$ mypy -V
mypy 1.4.1 (compiled: yes)
$ python -V
Python 3.11.3

Essentially, the bytecode generated for the legal examples will inevitably be longer and thus forcing the interpreter to do unnecessary evaluation.

That being said, using conditional expression is probably the least offensive (as much as the former BDFL deign its design as "intentionally ugly"), but it still requires an extra JUMP_FORWARD if match and an additional LOAD_CONST if not match.

Disassembly of illegal:
  1           0 RESUME                   0

  2           2 LOAD_FAST                1 (some_items)
              4 JUMP_IF_FALSE_OR_POP     3 (to 12)
              6 LOAD_FAST                0 (value)
              8 LOAD_FAST                1 (some_items)
             10 CONTAINS_OP              0
        >>   12 RETURN_VALUE

Disassembly of legal:
  4           0 RESUME                   0

  5           2 LOAD_FAST                1 (some_items)
              4 LOAD_CONST               0 (None)
              6 IS_OP                    1
              8 JUMP_IF_FALSE_OR_POP     3 (to 16)
             10 LOAD_FAST                0 (value)
             12 LOAD_FAST                1 (some_items)
             14 CONTAINS_OP              0
        >>   16 RETURN_VALUE

Disassembly of legal_conditional_expr:
  7           0 RESUME                   0

  8           2 LOAD_FAST                1 (some_items)
              4 POP_JUMP_FORWARD_IF_FALSE     4 (to 14)
              6 LOAD_FAST                0 (value)
              8 LOAD_FAST                1 (some_items)
             10 CONTAINS_OP              0
             12 JUMP_FORWARD             1 (to 16)
        >>   14 LOAD_CONST               1 (False)
        >>   16 RETURN_VALUE

Disassembly of legal_but_more_awkward:
 10           0 RESUME                   0

 11           2 LOAD_GLOBAL              1 (NULL + bool)
             14 LOAD_FAST                1 (some_items)
             16 PRECALL                  1
             20 CALL                     1
             30 JUMP_IF_FALSE_OR_POP     3 (to 38)
             32 LOAD_FAST                0 (value)
             34 LOAD_FAST                1 (some_items)
             36 CONTAINS_OP              0
        >>   38 RETURN_VALUE

In other statically-typed languages, these type of issues generally can be optimized away in the resulting machine readable code.

one_tuple: tuple[int] = tuple((1,)) # false positive in mypy
answer1: tuple[int] = (1,)
print(one_tuple == answer1)
two_tuple: tuple[int] = tuple((1, 2,)) # false negative in pyright
answer2: tuple[int] = (1, 2,)
print(two_tuple == answer2)
# $ mypy tuples.py
# tuples.py:1: error: Incompatible types in assignment (expression has type "Tuple[int, ...]", variable has type "Tuple[int]")
# tuples.py:4: error: Incompatible types in assignment (expression has type "Tuple[int, ...]", variable has type "Tuple[int]")
# tuples.py:5: error: Incompatible types in assignment (expression has type "Tuple[int, int]", variable has type "Tuple[int]")
# $ pyright tuples.py
# /tmp/tuples.py
# /tmp/tuples.py:5:23 - error: Expression of type "tuple[Literal[1], Literal[2]]" cannot be assigned to declared type "tuple[int]"
#   "tuple[Literal[1], Literal[2]]" is incompatible with "tuple[int]"
#     Element size mismatch; expected 1 but received 2 (reportGeneralTypeIssues)
# 1 error, 0 warnings, 0 informations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment