Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Process generic #13385

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Make Process generic #13385

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4b6b95e
Make Process generic
Dreamsorcerer Jan 10, 2025
d07c977
Update subprocess.pyi
Dreamsorcerer Jan 10, 2025
4c54bd5
Update stdlib/asyncio/subprocess.pyi
Dreamsorcerer Jan 10, 2025
5bed68e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jan 10, 2025
0839de3
Update subprocess.pyi
Dreamsorcerer Jan 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 57 additions & 30 deletions stdlib/asyncio/subprocess.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@
from _typeshed import StrOrBytesPath
from asyncio import events, protocols, streams, transports
from collections.abc import Callable, Collection
from typing import IO, Any, Literal
from typing import IO, Any, Literal, overload

# Keep asyncio.__all__ updated with any changes to __all__ here
__all__ = ("create_subprocess_exec", "create_subprocess_shell")

PIPE: int
STDOUT: int
DEVNULL: int
PIPE = subprocess.PIPE
STDOUT = subprocess.STDOUT
DEVNULL = subprocess.DEVNULL

class SubprocessStreamProtocol(streams.FlowControlMixin, protocols.SubprocessProtocol):
stdin: streams.StreamWriter | None
Expand All @@ -19,7 +19,7 @@
def __init__(self, limit: int, loop: events.AbstractEventLoop) -> None: ...
def pipe_data_received(self, fd: int, data: bytes | str) -> None: ...

class Process:
class Process[CommOut: (PIPE, int, IO[Any], None), CommErr: (PIPE, int, IO[Any], None)]:

Check failure on line 22 in stdlib/asyncio/subprocess.pyi

View workflow job for this annotation

GitHub Actions / Test typeshed with pyright (Linux, 3.12) 

Variable not allowed in type expression (reportInvalidTypeForm)

Check failure on line 22 in stdlib/asyncio/subprocess.pyi

View workflow job for this annotation

GitHub Actions / Test typeshed with pyright (Linux, 3.12) 

Variable not allowed in type expression (reportInvalidTypeForm)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like PIPE is a value of type int, not a type alias. I don't think it's a valid constraint for your type variable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the type to a Literal. My assumption then is that mypy etc. can detect if the PIPE itself is used as opposed to another (unknown) int.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem isn't int vs literal, it's that PIPE Is a value with that type, not a type, so it can't be used in the constraints.

stdin: streams.StreamWriter | None
stdout: streams.StreamReader | None
stderr: streams.StreamReader | None
Expand All @@ -33,14 +33,31 @@
def send_signal(self, signal: int) -> None: ...
def terminate(self) -> None: ...
def kill(self) -> None: ...
async def communicate(self, input: bytes | bytearray | memoryview | None = None) -> tuple[bytes, bytes]: ...
@overload
async def communicate(
self: Process[PIPE, PIPE], input: bytes | bytearray | memoryview | None = None

Check failure on line 38 in stdlib/asyncio/subprocess.pyi

View workflow job for this annotation

GitHub Actions / Test typeshed with pyright (Linux, 3.12) 

Variable not allowed in type expression (reportInvalidTypeForm)

Check failure on line 38 in stdlib/asyncio/subprocess.pyi

View workflow job for this annotation

GitHub Actions / Test typeshed with pyright (Linux, 3.12) 

Variable not allowed in type expression (reportInvalidTypeForm)
) -> tuple[bytes, bytes]: ...
@overload
async def communicate(
self: Process[PIPE, int | IO[Any] | None], input: bytes | bytearray | memoryview | None = None

Check failure on line 42 in stdlib/asyncio/subprocess.pyi

View workflow job for this annotation

GitHub Actions / Test typeshed with pyright (Linux, 3.12) 

Variable not allowed in type expression (reportInvalidTypeForm)
) -> tuple[bytes, None]: ...
@overload
async def communicate(
self: Process[int | IO[Any] | None, PIPE], input: bytes | bytearray | memoryview | None = None

Check failure on line 46 in stdlib/asyncio/subprocess.pyi

View workflow job for this annotation

GitHub Actions / Test typeshed with pyright (Linux, 3.12) 

Variable not allowed in type expression (reportInvalidTypeForm)
) -> tuple[None, bytes]: ...
@overload
async def communicate(
self: Process[int | IO[Any] | None, int | IO[Any] | None], input: bytes | bytearray | memoryview | None = None
) -> tuple[None, None]: ...

if sys.version_info >= (3, 11):
async def create_subprocess_shell(
async def create_subprocess_shell[
Out: (PIPE, int, IO[Any], None), Err: (PIPE, int, IO[Any], None)

Check failure on line 55 in stdlib/asyncio/subprocess.pyi

View workflow job for this annotation

GitHub Actions / Test typeshed with pyright (Linux, 3.12) 

Variable not allowed in type expression (reportInvalidTypeForm)

Check failure on line 55 in stdlib/asyncio/subprocess.pyi

View workflow job for this annotation

GitHub Actions / Test typeshed with pyright (Linux, 3.12) 

Variable not allowed in type expression (reportInvalidTypeForm)
](
cmd: str | bytes,
stdin: int | IO[Any] | None = None,
stdout: int | IO[Any] | None = None,
stderr: int | IO[Any] | None = None,
stdout: Out = None,
stderr: Err = None,
limit: int = 65536,
*,
# These parameters are forced to these values by BaseEventLoop.subprocess_shell
Expand All @@ -67,13 +84,15 @@
umask: int = -1,
process_group: int | None = None,
pipesize: int = -1,
) -> Process: ...
async def create_subprocess_exec(
) -> Process[Out, Err]: ...
async def create_subprocess_exec[
Out: (PIPE, int, IO[Any], None), Err: (PIPE, int, IO[Any], None)

Check failure on line 89 in stdlib/asyncio/subprocess.pyi

View workflow job for this annotation

GitHub Actions / Test typeshed with pyright (Linux, 3.12) 

Variable not allowed in type expression (reportInvalidTypeForm)

Check failure on line 89 in stdlib/asyncio/subprocess.pyi

View workflow job for this annotation

GitHub Actions / Test typeshed with pyright (Linux, 3.12) 

Variable not allowed in type expression (reportInvalidTypeForm)
](
program: StrOrBytesPath,
*args: StrOrBytesPath,
stdin: int | IO[Any] | None = None,
stdout: int | IO[Any] | None = None,
stderr: int | IO[Any] | None = None,
stdout: Out = None,
stderr: Err = None,
limit: int = 65536,
# These parameters are forced to these values by BaseEventLoop.subprocess_exec
universal_newlines: Literal[False] = False,
Expand All @@ -99,14 +118,16 @@
umask: int = -1,
process_group: int | None = None,
pipesize: int = -1,
) -> Process: ...
) -> Process[Out, Err]: ...

elif sys.version_info >= (3, 10):
async def create_subprocess_shell(
async def create_subprocess_shell[
Out: (PIPE, int, IO[Any], None), Err: (PIPE, int, IO[Any], None)
](
cmd: str | bytes,
stdin: int | IO[Any] | None = None,
stdout: int | IO[Any] | None = None,
stderr: int | IO[Any] | None = None,
stdout: Out = None,
stderr: Err = None,
limit: int = 65536,
*,
# These parameters are forced to these values by BaseEventLoop.subprocess_shell
Expand All @@ -132,13 +153,15 @@
user: None | str | int = None,
umask: int = -1,
pipesize: int = -1,
) -> Process: ...
async def create_subprocess_exec(
) -> Process[Out, Err]: ...
async def create_subprocess_exec[
Out: (PIPE, int, IO[Any], None), Err: (PIPE, int, IO[Any], None)
](
program: StrOrBytesPath,
*args: StrOrBytesPath,
stdin: int | IO[Any] | None = None,
stdout: int | IO[Any] | None = None,
stderr: int | IO[Any] | None = None,
stdout: Out = None,
stderr: Err = None,
limit: int = 65536,
# These parameters are forced to these values by BaseEventLoop.subprocess_exec
universal_newlines: Literal[False] = False,
Expand All @@ -163,14 +186,16 @@
user: None | str | int = None,
umask: int = -1,
pipesize: int = -1,
) -> Process: ...
) -> Process[Out, Err]: ...

else: # >= 3.9
async def create_subprocess_shell(
async def create_subprocess_shell[
Out: (PIPE, int, IO[Any], None), Err: (PIPE, int, IO[Any], None)
](
cmd: str | bytes,
stdin: int | IO[Any] | None = None,
stdout: int | IO[Any] | None = None,
stderr: int | IO[Any] | None = None,
stdout: Out = None,
stderr: Err = None,
loop: events.AbstractEventLoop | None = None,
limit: int = 65536,
*,
Expand All @@ -196,13 +221,15 @@
extra_groups: None | Collection[str | int] = None,
user: None | str | int = None,
umask: int = -1,
) -> Process: ...
async def create_subprocess_exec(
) -> Process[Out, Err]: ...
async def create_subprocess_exec[
Out: (PIPE, int, IO[Any], None), Err: (PIPE, int, IO[Any], None)
](
program: StrOrBytesPath,
*args: StrOrBytesPath,
stdin: int | IO[Any] | None = None,
stdout: int | IO[Any] | None = None,
stderr: int | IO[Any] | None = None,
stdout: Out = None,
stderr: Err = None,
loop: events.AbstractEventLoop | None = None,
limit: int = 65536,
# These parameters are forced to these values by BaseEventLoop.subprocess_exec
Expand All @@ -227,4 +254,4 @@
extra_groups: None | Collection[str | int] = None,
user: None | str | int = None,
umask: int = -1,
) -> Process: ...
) -> Process[Out, Err]: ...
6 changes: 3 additions & 3 deletions stdlib/subprocess.pyi
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1810,9 +1810,9 @@ else:
text: bool | None = None,
) -> Any: ... # morally: -> str | bytes

PIPE: Final[int]
STDOUT: Final[int]
DEVNULL: Final[int]
PIPE: Final[Literal[-1]]
STDOUT: Final[Literal[-2]]
DEVNULL: Final[Literal[-3]]

class SubprocessError(Exception): ...

Expand Down
Loading