Skip to content

feat(bindings/python)!: Generate stubs for Operator overloads and Scheme #6729

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Xuanwo merged 16 commits into from
Oct 27, 2025
Merged

feat(bindings/python)!: Generate stubs for Operator overloads and Scheme #6729

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
1b69f2f
add services module with PyScheme
chitralverma Oct 22, 2025
b8207af
ensure python 3.10 compatibility file.pyi
chitralverma Oct 22, 2025
769a4e2
Add new service base
chitralverma Oct 22, 2025
7dae5df
add config to docstrings
chitralverma Oct 22, 2025
3ea5ed7
Merge branch 'main' into gen-services
chitralverma Oct 22, 2025
968e491
fix
chitralverma Oct 22, 2025
8f8010c
Add overloads to operators
chitralverma Oct 22, 2025
cf2d114
remove and replace files
chitralverma Oct 22, 2025
ababffe
overloads to use pyscheme directly
chitralverma Oct 23, 2025
613c3a0
overloads to use pyscheme directly
chitralverma Oct 23, 2025
3d736a6
Add Scheme to overloads
chitralverma Oct 23, 2025
fd97c21
Add returns section to overloads
chitralverma Oct 23, 2025
2259881
update doc style
chitralverma Oct 23, 2025
4226720
Merge branch 'main' into gen-services
chitralverma Oct 23, 2025
7f66a28
Merge branch 'main' into gen-services
chitralverma Oct 24, 2025
a9c2ef2
Merge branch 'main' into gen-services
chitralverma Oct 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions bindings/python/justfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ clean:
[group('build')]
stub-gen: setup
@echo "{{ BOLD }}--- Generating Python type stubs ---{{ NORMAL }}"
@cargo run --quiet --manifest-path=../../dev/Cargo.toml -- generate -l python
Copy link
Copy Markdown
Contributor Author

@chitralverma chitralverma Oct 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overloads and Scheme generates as part of the stub-gen process

Copy link
Copy Markdown
Member

@Xuanwo Xuanwo Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This made me think that can we move stub_gen to odev instead.

Copy link
Copy Markdown
Contributor Author

@chitralverma chitralverma Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This made me think that can we move stub_gen to odev instead.

actually these are 2 separate processes that cannot be merged into the binding or in odev from what i understood.

  1. pyo3-stub-gen requires.... well, pyo3 which is available in python bindings. this step is responsible for generating most of the python stubs from rust code.
  2. the stuff in odev uses jinja templating because it has parsing logic of services code used for java as well. if odev had lib target then it could be used as a build dependency but still would be separate from pyo3 stuff.

@cargo run --quiet --bin stub_gen
@echo "{{ BOLD }}--- Formatting and fixing generated stubs ---{{ NORMAL }}"
-@bash -c 'shopt -s globstar; uv run ruff check **/*.pyi --fix --unsafe-fixes --silent || true'
Expand Down
21 changes: 18 additions & 3 deletions bindings/python/python/opendal/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,25 @@
# ruff: noqa: D104
import builtins

from opendal._opendal import * # noqa: F403 # pyright:ignore
from opendal._opendal import ( # noqa: F403
capability,
exceptions,
file,
layers,
services,
types,
)
from opendal.operator import AsyncOperator, Operator # pyright:ignore

__version__: builtins.str

__all__ = _opendal.__all__ # noqa: F405 # pyright:ignore
__all__ += ["AsyncOperator", "Operator"] # pyright:ignore
__all__ = [
"capability",
"exceptions",
"file",
"layers",
"services",
"types",
"AsyncOperator",
"Operator",
Comment on lines +21 to +41
Copy link
Copy Markdown
Contributor Author

@chitralverma chitralverma Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes things explicit

]
4 changes: 3 additions & 1 deletion bindings/python/python/opendal/file.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import collections.abc
import types
import typing

import typing_extensions

@typing.final
class AsyncFile:
r"""
Expand Down Expand Up @@ -111,7 +113,7 @@ class AsyncFile:
-----
A closed file cannot be used for further I/O operations.
"""
def __aenter__(self) -> typing.Self: ...
def __aenter__(self) -> typing_extensions.Self: ...
Copy link
Copy Markdown
Contributor Author

@chitralverma chitralverma Oct 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typing.Self was not available in python 3.10 which opendal supports, so fixed this

def __aexit__(
self,
exc_type: type[builtins.BaseException] | None,
Expand Down
Loading
Loading