Strongly typed helper functions for object store __getitem__ and Tree.lookup_path

[castedo]

Folks that use mypy (or similar) for type checking (especially in strict mode) might find either of these two helper functions useful. Or perhaps just some of the type-checking techniques demonstrated. This code will run on Python 3.9 (but you need much higher to run with mypy). I found them useful when porting from GitPython to Dulwich.

I've organized them into this small file:

https://gitlab.com/perm.pub/hidos/-/blob/912e65b32e872db48267d08105a2f4b119479853/hidos/dulwish.py

The contents are also copied below:

# dulwish.py -- wishful high-level functions for Dulwich
# (c) 2024 E. Castedo Ellerman <[email protected]>
# Released under the MIT License (https://spdx.org/licenses/MIT)

from __future__ import annotations

from pathlib import Path
from typing import Callable, TYPE_CHECKING, TypeVar

from dulwich.errors import NotTreeError
from dulwich.objects import Blob, ObjectID, ShaFile, SubmoduleEncountered, Tree


if TYPE_CHECKING:
    from typing import TypeAlias

    PathLike: TypeAlias = Path | str
    ObjectGetter: TypeAlias = Callable[[ObjectID], ShaFile]
    ShaFileType = TypeVar("ShaFileType", bound=ShaFile)


def cast_lookup(
    lookup: ObjectGetter, sha: ObjectID, sha_file_type: type[ShaFileType]
) -> ShaFileType:
    """Strongly typed helper function for object store __getitem__.

    Example:
        lookup = repo.object_store.__getitem__
        tree = cast_lookup(lookup, sha, Tree)
        # mypy knows that tree is of type Tree
    """
    obj = lookup(sha)
    if not isinstance(obj, sha_file_type):
        raise ValueError(f"ShaFile object {sha!r} is not of type {sha_file_type}")
    return obj


def tree_lookup_path(
    lookup: ObjectGetter, tree_id: ObjectID, path: PathLike
) -> Blob | Tree | None:
    """Strongly typed helper function for Tree.lookup_path.

    Example:
        lookup = repo.object_store.__getitem__
        baz = tree_lookup_path(lookup , tree_sha, "foo/bar/baz.txt")
        # mypy knows baz is None or of type Blob or Tree

    Raises:
        Unexpected exceptions due to reasons other than the path
        not leading to a Blob or Tree within the same repo.
    """
    try:
        tree = cast_lookup(lookup, tree_id, Tree)
        mode, sha = tree.lookup_path(lookup, str(path).encode("ascii"))
        ret = lookup(sha)
        assert isinstance(ret, (Blob, Tree))
        return ret
    except (KeyError, NotTreeError, SubmoduleEncountered):
        return None

[garyvdm]

A simpler way to achieve what you want is to use typing.cast:

from typing import cast

from dulwich.objects import Tree

tree = cast(Tree, repo[tree_id])
mode, sha = tree.lookup_path(lookup, str(path).encode("ascii"))
...

[castedo]

typing.cast is the right choice if one does NOT want a runtime check on the type. But cast_lookup does do a runtime check and will raise ValueError in the event the path `"foo/bar" (in the example above) does not go to a Git tree.

Using typing.cast, if the "foo/bar" path (in the example above) does not go to a Git Tree, then I believe the following will be raised:

AttributeError: 'Blob' object has no attribute 'lookup_path'

and possibly a KeyError... I suspect, if the path doesn't even go to anything.

In short, `typing.cast' does not achieve the same thing in general.

[jelmer]

Isn't it sufficient to just do this:

tree = repo[treeid]
assert isinstance(tree, Tree)
...

mypy also understands that tree would have to be a Tree object after that.

[castedo]

Yes, sufficient and simpler. Using assert seems a better option (instead of the entire cast_lookup function).

On a related note, my earlier claim about path "foo/bar" and ValueError is not correct. ValueError results from a tree_hash that is not a tree hash. Raising an AssertError instead of ValueError in this error situation seems equally good. if not better.