-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
PEP 776: Emscripten support #4307
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Thanks for the PEP and work on Emscripten! I've edited the top post with the new PEP checklist, please could you work through it and check things off? You can use PEP number 776. |
What form does this formal confirmation take? Comment here? cc @ambv @freakboy3742. |
The preview link shows pep-0776 as a 404 for me: https://pep-previews--4307.org.readthedocs.build/pep-0776/ |
Lowercase name seemed to be the problem. |
Revert 871a223 but keep the newline before the bullet list: WebAssembly does not have native support for signals. Furthermore, on a
non-pthreads build, the address space of the WebAssembly module is not shared,
so it is impossible for any thread capable of seeing an interrupt to write to
the eval breaker while the Python interpreter is running code. To work around
this, there are two possible solutions:
* If Emscripten is run in a webworker and served with the shared memory headers,
it is possible to use shared memory outside of the WebAssembly address space
as a signal buffer. A signal handling UI thread can write the desired signal
into the signal buffer. The interpreter can periodically check the state of
this signal buffer in the eval breaker code. Checking the signal buffer is
slow compared to checking the eval breaker in native platforms, so we do only
do it once every 50 times through the eval breaker. See
`Python/emscripten_signal.c <https://github.com/python/cpython/blob/2bef8ea8ea045d20394f0daec7a5c5b1046a4e22/Python/emscripten_signal.c>`__
* Using stack switching, we can occasionally switch the stack and allow the
JavaScript event loop to go around, then check the state of a signal buffer.
This requires the experimental JavaScript Promise Integration API, and would
be best used with the techniques for optimizing long tasks described
`in this article <https://web.dev/articles/optimize-long-tasks>`__
Emscripten Python has already implemented the solution based on shared memory,
and it is in use in Pyodide. |
This reverts commit 0a8d64e.
@hugovk I think this can be merged now. I will address the remaining feedback in followup PRs. Thanks everyone! |
Co-authored-by: Brett Cannon <[email protected]>
@AA-Turner I saw you are intending to review, but I'd rather merge this now since Hugo and I already approved and I prefer for PEPs not to get stuck in the ready-but-not-yet-merged state. If you have feedback, we can always update the PEP text later. |
I'm about halfway through (mainly editorial points, but nothing major so far). Happy for you to merge now though. I agree we should avoid PEPs being "ready but not merged" (several have been as drafts in PRs for days/weeks(!)), but this PR was only opened this morning -- I had an unexpectedly busy day, so was late on a full read-through. A |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @hoodmane. Very well written and happy to see this continue to move forward. I remember the early days of pyodide and iodide at Mozilla.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry again for the delay, almost entirely editorial points.
This PEP formalizes the addition of Tier 3 for Emscripten support in Python 3.14 | ||
which `was approved by the steering council on October 25, 2024 | ||
<https://github.com/python/steering-council/issues/256>`__. The goal is to allow | ||
Pyodide wheels to be uploaded to PyPI. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pyodide (or the relationship to Emscripten) isn't defined here, perhaps a link?
peps/pep-0776.rst
Outdated
`Emscripten <https://emscripten.org/>`__ is a complete open source compiler | ||
toolchain which compiles C/C++ code into WebAssembly/JavaScript executables that | ||
run in JavaScript runtimes including browsers and Node.js. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sentence read a little breathlessly. I also would suggest removing the paragraph break afterwards.
Also, later on the PEP mentions the "Rust emscripten target" -- is this part of the emscripten project? Should the support for Rust/non-C native languages be mentioned here?
`Emscripten <https://emscripten.org/>`__ is a complete open source compiler | |
toolchain which compiles C/C++ code into WebAssembly/JavaScript executables that | |
run in JavaScript runtimes including browsers and Node.js. | |
`Emscripten <https://emscripten.org/>`__ is a complete open source compiler | |
toolchain. It compiles C/C++ code into WebAssembly/JavaScript executables, | |
for use in JavaScript runtimes, including browsers and Node.js. |
peps/pep-0776.rst
Outdated
Linux, and every smartphone. | ||
|
||
`The Pyodide project <https://pyodide.org/>`__ has for years supported | ||
Emscripten Python. Hundreds of thousands of students have learned Python through |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've read a couple of times and I'm unclear on the distinction between "Emscripten Python" and "Pyodide". Is there a reference for a disambiguation? The only mention of Python on the emscripten intro page references Pyodide.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work @hoodmane!
Formally confirming that I'm OK with the places where I've been referred to here (as PEP 11 contact, and as buildbot maintainer on Anaconda-provided hardware).
A couple of other comments inline; plus a high-level comment on the discussion forum about the formal relationship between Pyodide and CPython as projects.
|
||
There is also ``sys._emscripten_info`` which includes the Emscripten version and | ||
the runtime (either ``navigator.userAgent`` in a browser or ``"Node js" + | ||
process.version`` in Node.js). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The discussion around PEP 730 and PEP 738 concluded that platform
was a better place for a public API for this sort of data (recognising that there may be a need to have it embedded in sys
as a compiled property). There's probably space for an emscripten_ver()
analog to mac_ver()
, ios_ver()
and android_ver()
.
``pyodide_<abi>`` and ``emscripten_<version>`` is intended to be the same as the | ||
relationship between ``manylinux<version>`` and ``linux``. | ||
|
||
The specification of the ``pyodide_<abi>`` ABI includes: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've commented on this on the discussion thread; but tl;dr, I've got concerns about using pyodide_<abi>
as the naming here, even if in practice we end up using Pyodide's specification.
------ | ||
|
||
:pep:`11` will be updated to indicate that Emscripten is supported, specifically | ||
the triples ``wasm32-unknown-emscripten_xx_xx_xx``. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To confirm - are the wildcards going to be part of the PEP 11 spec, or will be updating PEP11 each time a new specific emscripten version is added to the test pool?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was hoping to keep the wild cards. Maybe I should give a concrete example here or otherwise clarify that they are standing in for a version number. Also, we don't support all emscripten versions, so we should probably add where we plan to indicate this.
Emscripten, but it is very complicated. Ideally, the Python packaging ecosystem | ||
would have standards for cross builds. This is a difficult long term project, | ||
particularly because the packaging system is complex and was designed from the | ||
ground up with the assumption that cross compilation would not happen. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not a blocker on this PEP, but for public visibility - I'm hoping to kick start a discussion about this very shortly.
Thanks for the copy edits and suggestions @AA-Turner! |
Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
Thanks all, let's merge! (Edit: annoyingly, we need to resolve all the review comments to do that.) |
The PEP is now published at https://peps.python.org/pep-0776/ |
Basic requirements (all PEP Types)
pep-NNNN.rst
), PR title (PEP 123: <Title of PEP>
) andPEP
headerAuthor
orSponsor
, and formally confirmed their approvalAuthor
,Status
(Draft
),Type
andCreated
headers filled out correctlyPEP-Delegate
,Topic
,Requires
andReplaces
headers completed if appropriate.github/CODEOWNERS
for the PEPStandards Track requirements
Python-Version
set to valid (pre-beta) future Python version, if relevantDiscussions-To
andPost-History
📚 Documentation preview 📚: https://pep-previews--4307.org.readthedocs.build/pep-0776/