Can I use `CanAddSelf` or `CanAddSame` to implement `matmul` as a convolve operator involving zero or more additions of `_T`? #574

posita · 2026-04-12T02:17:44Z

posita
Apr 12, 2026

Pyright playground of code in this post: https://pyright-play.net/?code=JYWwDg9g...

Let's say I have a container capable of addition:

>>> H(2) + 2  # H[int]
H(4)
>>> H(2) + 2.0  # H[float]
H(4.0)
>>> H(2) + H(2)  # H[int]
H(4)
>>> 2 + H(2)  # H[int]
H(4)
>>> 2.0 + H(2)  # H[float]
H(4.0)
>>> H(None)  # H[None], which is like an "empty" container
H(None)
>>> H(None) + 1  # H[None]
H(None)
>>> None + H(2)  # H[None]
H(None)

I might try to define that as something like this:

import operator
from typing import Any, Generic, Literal, TypeVar, overload

import optype

_T = TypeVar("_T", covariant=True)
_T_co = TypeVar("_T_co", covariant=True)
_OtherT = TypeVar("_OtherT")
_ResultT = TypeVar("_ResultT")


class H(Generic[_T_co]):
    def __init__(self, init_val: _T_co, /) -> None:
        self._val = init_val

    def __repr__(self) -> str:
        return f"{type(self).__qualname__}({self._val!r})"

    @overload
    def __add__(self: "H[None]", rhs: Any) -> "H[None]": ...
    @overload
    def __add__(self: "H[Any]", rhs: "H[None] | None") -> "H[None]": ...
    @overload
    def __add__(
        self: "H[optype.CanAdd[_OtherT, _ResultT]]", rhs: "H[_OtherT]"
    ) -> "H[_ResultT]": ...
    @overload
    def __add__(
        self: "H[optype.CanAdd[_OtherT, _ResultT]]", rhs: _OtherT
    ) -> "H[_ResultT]": ...
    @overload
    def __add__(
        self: "H[_T]", rhs: "H[optype.CanAdd[_T, _ResultT]]"
    ) -> "H[_ResultT]": ...
    @overload
    def __add__(self: "H[_T]", rhs: optype.CanAdd[_T, _ResultT]) -> "H[_ResultT]": ...
    def __add__(self, rhs: object | None) -> "H":
        if self._val is None:
            return H(None)
        elif isinstance(rhs, H):
            if rhs._val is None:
                return H(None)
            result = operator.add(self._val, rhs._val)
        else:
            if rhs is None:
                return H(None)
            result = operator.add(self._val, rhs)
        return NotImplemented if result is NotImplemented else H(result)

    @overload
    def __radd__(self: "H[None]", lhs: Any) -> "H[None]": ...
    @overload
    def __radd__(self: "H[Any]", lhs: None) -> "H[None]": ...
    @overload
    def __radd__(
        self: "H[optype.CanRAdd[_OtherT, _ResultT]]", lhs: _OtherT
    ) -> "H[_ResultT]": ...
    @overload
    def __radd__(self: "H[_T]", lhs: optype.CanRAdd[_T, _ResultT]) -> "H[_ResultT]": ...
    def __radd__(self: "H", lhs: object | None) -> "H":
        if self._val is None:
            return H(None)
        if isinstance(lhs, H):
            result = operator.add(lhs._val, self._val)
        else:
            result = operator.add(lhs, self._val)
        return NotImplemented if result is NotImplemented else H(result)

I'd like to add handlers for __matmul__ and __rmatmul__ that do a kind of convolution, but whose implementation is limited to zero or more additions across _T. In our toy example, that would be akin to:

    @overload
    def __matmul__(self: "H[None]", rhs: Any) -> "H[None]": ...
    @overload
    def __matmul__(self: "H[_T]", rhs: Literal[0]) -> "H[None]": ...
    @overload
    def __matmul__(self: "H[_T]", rhs: int) -> "H[_T]": ...
    def __matmul__(self, rhs: int) -> "H":
        if rhs < 0:
            raise ValueError
        if self._val is None or rhs == 0:
            return H(None)
        return sum((self._val for _ in range(rhs - 1)), start=self._val)  # type: ignore[call-overload] # ty: ignore[no-matching-overload]

    @overload
    def __rmatmul__(self: "H[None]", lhs: Any) -> "H[None]": ...
    @overload
    def __rmatmul__(self: "H[_T]", lhs: Literal[0]) -> "H[None]": ...
    @overload
    def __rmatmul__(self: "H[_T]", lhs: int) -> "H[_T]": ...
    def __rmatmul__(self, lhs: int) -> "H":
        return self.__matmul__(lhs)

Note that the argument to __matmul__ and __rmatmul__ is purely an iterative count. It is not involved in the computation that gives rise to the return value. Only _T is. I see that CanAddSelf and CanAddSame exist. Could one use either of those to properly type those methods? If so, how?

from decimal import Decimal
from fractions import Fraction
from typing import reveal_type

# should be H[None]
reveal_type(None + H(42))

# should be H[None]
reveal_type(H(None) + 42)

# should be H[None]
reveal_type(42 + H(None))

# should be H[None]
reveal_type(H(42) + None)

# should be H[None]
reveal_type(H(None) + H(42))

# should be H[None]
reveal_type(H(42) + H(None))

# should be H[int]
reveal_type(H(42) + H(42))

# should be H[float]; ty thinks this is H[Unknown]
reveal_type(H(-273.15) + H(-273.15))

# should be H[Decimal]
reveal_type(H(Decimal("98.1")) + H(Decimal("98.1")))

# should be H[Fraction]; ty thinks this is H[Fraction | int | float | complex]
reveal_type(H(Fraction(10, 6)) + H(Fraction(10, 6)))

# should be H[None]
reveal_type(2 @ H(None))

# should be H[None]
reveal_type(0 @ H(42))

# should be H[int]; ty thinks this is H[Literal[42]]
reveal_type(2 @ H(42))

# should be H[float]; ty thinks this is H[int | float]
reveal_type(2 @ H(-273.15))

# should be H[Decimal]
reveal_type(2 @ H(Decimal("98.1")))

# should be H[Fraction]
reveal_type(2 @ H(Fraction(10, 6)))

# pyright and mypy properly flag this next one as not supported. ty thinks it is fine,
# but the result is H[Unknown]. pyrefly also thinks it's fine and H[H[Unknown]] for some
# reason.
reveal_type(H(frozenset({"hello", "world"})) + H(frozenset({"hello", "world"})))

# Can we get checkers to spot this one, too?
reveal_type(2 @ H(frozenset({"hello", "world"})))

Answered by jorenham

Apr 15, 2026

What I'd love is a story for this:

# [ ] (ideally) should all be H[int], since bool + bool is int
reveal_type(H(True) + H(False))  # properly typed as H[int]
reveal_type(H(True) @ 2)  # improperly flagged, but I would have at least expected H[bool]?
reveal_type(2 @ H(True))  # improperly flagged, but I would have at least expected H[bool]?

I'm guessing that this is rejected because bool + bool -> int, and CanAddSame requires some T of the form T + T -> T. So what you could do is use CanAddSame[int, int], so that this restriction is relaxed to T + T | int -> T | int, which I think should accept bool.

View full answer

jorenham · 2026-04-14T22:18:27Z

jorenham
Apr 14, 2026
Maintainer

# should be H[float]; ty thinks this is H[Unknown]
reveal_type(H(-273.15) + H(-273.15))

this one sounds like astral-sh/ty#3234

# should be H[Fraction]; ty thinks this is H[Fraction | int | float | complex]
reveal_type(H(Fraction(10, 6)) + H(Fraction(10, 6)))

This looks like it might be an issue with ty's constraint solver. The float and complex overloads of frozenset (typeshed src) should not apply here, but judging by the float | complex, it looks like ty used those regardless.

# should be H[int]; ty thinks this is H[Literal[42]]
reveal_type(2 @ H(42))

H(42): H[Literal[42]], so this calls H[Literal[42]].__rmatmul__. Only the third __rmatmul__ overload applies, which returns H[_T], where _T resolves to Literal[42]. So it's justifiable that ty infers this as H[Literal[42]].

# should be H[float]; ty thinks this is H[int | float]
reveal_type(2 @ H(-273.15))

int | float is equivalent to float according to the typing spec (which is a lie, but it looks like we're stuck with it), so H[float] is equivalent to H[int | float] (according to the typing spec). So ty isn't wrong here in that sense.

# pyright and mypy properly flag this next one as not supported. ty thinks it is fine,
# but the result is H[Unknown]. pyrefly also thinks it's fine and H[H[Unknown]] for some
# reason.
reveal_type(H(frozenset({"hello", "world"})) + H(frozenset({"hello", "world"})))

Ty tends to add a bunch of | Unknown to inferred types, so maybe it has something to do with that?

But in case of Pyrefly I have no idea what's going on. It's probably worth reporting though (after checking that this hasn't been solved in the aeons that it took me to answer this, of course).

# Can we get checkers to spot this one, too?
reveal_type(2 @ H(frozenset({"hello", "world"})))

I'd indeed try to narrow the _T in the third __rmatmul__ overload to _T: optype.CanAddSame here. Because, if I understand correctly, you want to prevent sum((self._val for _ in range(rhs - 1)), start=self._val) from being called in case self._val is a type that'll lead to a runtime error here. So the requirement is that self._val can be added multiple times to itself, i.e. that it has an self._val.__add__(other: Self) -> Self. And that's exactly what the (unsubscripted) optype.CanAddSame type enforces.

4 replies

posita Apr 15, 2026
Author

I'd indeed try to narrow the _T in the third __rmatmul__ overload to _T: optype.CanAddSame here. Because, if I understand correctly, you want to prevent sum((self._val for _ in range(rhs - 1)), start=self._val) from being called in case self._val is a type that'll lead to a runtime error here. So the requirement is that self._val can be added multiple times to itself, i.e. that it has an self._val.__add__(other: Self) -> Self. And that's exactly what the (unsubscripted) optype.CanAddSame type enforces.

Thanks so much for the detailed response!

Yes, this is exactly right. (By the way, I'm sorry for all the type-checker-specific distractions above. You identified my main question with the above paragraph. I didn't mean to obscure it, but I'm grateful for the thorough treatment! 😅)

I might still be a little confused, though. I changed the following (pyright playground):

# ...
_ConvolvableT = TypeVar("_ConvolvableT", bound=CanAddSame)
# ...
class H(Generic[_T_co]):
    # ...
    @overload
    def __matmul__(self: "H[None]", rhs: int) -> "H[None]": ...
    @overload
    def __matmul__(self: "H[_T]", rhs: Literal[0]) -> "H[None]": ...
    @overload
    def __matmul__(self: "H[_ConvolvableT]", rhs: int) -> "H[_ConvolvableT]": ...
    def __matmul__(self, rhs: int) -> "H":
        if rhs < 0:
            raise ValueError
        if self._val is None or rhs == 0:
            return H(None)
        return sum((self._val for _ in range(rhs - 1)), start=self._val)  # type: ignore[call-overload] # ty: ignore[no-matching-overload]

    @overload
    def __rmatmul__(self: "H[None]", lhs: int) -> "H[None]": ...
    @overload
    def __rmatmul__(self: "H[_T]", lhs: Literal[0]) -> "H[None]": ...
    @overload
    def __rmatmul__(self: "H[_ConvolvableT]", lhs: int) -> "H[_ConvolvableT]": ...
    def __rmatmul__(self, lhs: int) -> "H":
        return self.__matmul__(lhs)

I think I got that right because it (mostly) worked!

# [x] should all be H[float] (pyright gets this right)
reveal_type(H(-273.15) + H(-273.15))
reveal_type(H(-273.15) @ 2)  # poor ty gets confused, but that's okay because it's working
reveal_type(2 @ H(-273.15))  # through its issues on its own time, and we can respect that

# [x] flag these as not supported
reveal_type(H(frozenset({"hello", "world"})) + H(frozenset({"hello", "world"})))
reveal_type(H(frozenset({"hello", "world"})) @ 2)  # properly flagged now! w00t!
reveal_type(2 @ H(frozenset({"hello", "world"})))  # properly flagged now! w00t!

What I'd love is a story for this:

# [ ] (ideally) should all be H[int], since bool + bool is int
reveal_type(H(True) + H(False))  # properly typed as H[int]
reveal_type(H(True) @ 2)  # improperly flagged, but I would have at least expected H[bool]?
reveal_type(2 @ H(True))  # improperly flagged, but I would have at least expected H[bool]?

jorenham Apr 15, 2026
Maintainer

What I'd love is a story for this:

# [ ] (ideally) should all be H[int], since bool + bool is int
reveal_type(H(True) + H(False))  # properly typed as H[int]
reveal_type(H(True) @ 2)  # improperly flagged, but I would have at least expected H[bool]?
reveal_type(2 @ H(True))  # improperly flagged, but I would have at least expected H[bool]?

I'm guessing that this is rejected because bool + bool -> int, and CanAddSame requires some T of the form T + T -> T. So what you could do is use CanAddSame[int, int], so that this restriction is relaxed to T + T | int -> T | int, which I think should accept bool.

Answer selected by posita

posita Apr 24, 2026
Author

Is there a version of CanAdd... that resolves _T + _T = _ResultT where _ResultT can be different from _T (as is the case with bools, IntEnums, etc.)?

jorenham Apr 24, 2026
Maintainer

Hmm, not exactly, but what comes closest is CanAddSame[Never, _T], which'd give you

def __add__(self, rhs: Self, /) -> Self | _T: ...

which will accept bool when you use _T: int. For a demo, see https://basedpyright.com/?typeCheckingMode=all&reportUnannotatedClassAttribute=false&code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAK4MYAxmADYA0UAygKYXABQzpFAhgM5dQCCAbQAqAXQAURMCXIUAlAC5mUZVAAm9YFAD6WjqtU6xXRsBogAFl3l0TNAPSyoAWgB8NplAA%2BUIdYB0AazqmsBiWtaCqDAijq5QAHJgKPSKKlABfqyhQiAArvSyzEA

So for input positions it should be fine if you also accept int, and not only bool. For output positions it's probably not what you want though (returning protocols is generally speaking not what you want either way, but there are some valid use-cases for that I suppose).

But if you can think of a good naming scheme for such a protocol family, then I'd be happy to add it to optype.

EDIT: maybe something like CanSelfAdd instead of CanAddSelf could work 🤔. It'd mess up the free "grouping by alphabetical sort" stuff we've got going on though, so I'm not sure if that's worth it.

EDIT EDIT: Some other options that come to mind: CanAddFromSelf, CanAddWithSelf, or CanAddRSelf. But maybe there's something better than that, cus I'm not sure about these ones either...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Can I use `CanAddSelf` or `CanAddSame` to implement `matmul` as a convolve operator involving zero or more additions of `_T`? #574

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment 4 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Uh oh!

Can I use CanAddSelf or CanAddSame to implement __matmul__ as a convolve operator involving zero or more additions of _T? #574

Uh oh!

Uh oh!

posita Apr 12, 2026

Replies: 1 comment · 4 replies

Uh oh!

jorenham Apr 14, 2026 Maintainer

Uh oh!

Uh oh!

posita Apr 15, 2026 Author

Uh oh!

jorenham Apr 15, 2026 Maintainer

Uh oh!

posita Apr 24, 2026 Author

Uh oh!

Uh oh!

jorenham Apr 24, 2026 Maintainer

Can I use `CanAddSelf` or `CanAddSame` to implement `matmul` as a convolve operator involving zero or more additions of `_T`? #574

posita
Apr 12, 2026

Replies: 1 comment 4 replies

jorenham
Apr 14, 2026
Maintainer

posita Apr 15, 2026
Author

jorenham Apr 15, 2026
Maintainer

posita Apr 24, 2026
Author

jorenham Apr 24, 2026
Maintainer