Skip to content

Conversation

@Thinbungee
Copy link
Owner

No description provided.

dpgeorge and others added 30 commits March 27, 2025 11:51
This is a pretty fundamental built-in and having CPython-compatible
behaviour is beneficial.  The code size increase is not much, and
ports/boards can still disable it if needed to save space.

Addresses issue micropython#5384.

Signed-off-by: Damien George <[email protected]>
Because 2-arg `next()` is implemented, and now enabled at the basic feature
level.

Signed-off-by: Damien George <[email protected]>
Addresses some TODOs in this file.

Signed-off-by: Damien George <[email protected]>
Allocation of a large compression window may fail, and in that case keep
the `DeflateIO` state consistent so its other methods (such as `close()`)
still work.  Consistency is kept by only updating the `self->write` member
if the window allocation succeeds.

Thanks to @jimmo for finding the bug.

Signed-off-by: Damien George <[email protected]>
This extends the existing `vfs.mount()` function to accept zero arguments,
in which case it returns a list of tuples of mounted filesystem objects
and their mount location.

Signed-off-by: Anson Mansfield <[email protected]>
This is needed if you chdir to a ROMFS and want to query your current
directory.

Prior to this change, using `os.getcwd()` when in a ROMFS would raise:

    AttributeError: 'VfsRom' object has no attribute 'getcwd'

Signed-off-by: Damien George <[email protected]>
If a ROMFS is mounted then "/rom/lib" is usually in `sys.path` before the
writable filesystem's "lib" entry.  The ROMFS directory cannot be installed
to, so skip it if found.

Signed-off-by: Damien George <[email protected]>
Rather than having Make calling CMake to generate a list of submodules and
then run a Make target (which is complex and prone to masking other
errors), implement the submodule update logic in CMake itself.

Internal CMake-side changes are that GIT_SUBMODULES is now a CMake list,
and the trigger variable name is changed from ECHO_SUBMODULES to
UPDATE_SUBMODULES.

The run is otherwise 100% a normal CMake run now, so most of the other
special casing can be removed.

Signed-off-by: Angus Gratton <[email protected]>
This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
This prevents printing the lengthy command and makes the build output a
little cleaner.

Signed-off-by: Damien George <[email protected]>
This prevents printing the lengthy command and makes the build output a
little cleaner.

Signed-off-by: Damien George <[email protected]>
The rp2 port has an incomplete mDNS implementation.  The code in `main.c`
calls `mdns_resp_init()` which opens the UDP socket for mDNS.  However, no
code in the cyw43 driver makes the proper calls to `mdns_resp_add_netif()`
and `mdns_resp_remove_netif()` to send the announce packets.  The wiznet5k
driver does make these calls and was used as a model for these changes.

This commit attempts to address this by very small changes to the
`ports/rp2/cyw43_configport.h` file.  The change uses new cyw43 driver
hooks to map the driver macros `CYW43_CB_TCPIP_INIT_EXTRA` and
`CYW43_CB_TCPIP_DEINIT_EXTRA` to the appropriate lwIP mDNS calls.

Fixes issue micropython#15297.

Signed-off-by: Mark Seminatore <[email protected]>
On stm32, the startup code attempts to mount the configured filesystem.  If
there is an existing littlefs filesystem that's suitable corrupted it's
possible for the reported blocksize to be incorrect here:

     uint32_t block_size = lfs2_fromle32(superblock->block_size);

This `block_size` (which is read from the filesystem iteself) is used to
create the len argument passed to `pyb_flash_make_new()`.  In that function
the len arg is validated to be a mutliple of the underlying hardware block
size, as well as not bigger than the physical flash.  Any failure is raised
as a ValueError.  This exception is not caught currently in main, it flows
up to the high level assert / startup failure.

As this occurs before `boot.py` is run, the users (potentially frozen)
application code doesn't have any opportunity to detect and handle the
issue.

This commit adds a helper function which attempts to create a block device,
and on error returns `None` instead of raising an exception.  Using this in
main means that a potentially corrupt filesystem will simply remain
unmounted, and the application can handle the issue safely.

The fix here also handles the case where the littlefs filesystem is valid
but the autodetection code (which detects the filesystem size) does not
work correctly.  In that case it will retry mounting the filesystem using
the whole size of the block device.

Signed-off-by: Andrew Leech <[email protected]>
The PYBD boards use an F7xx which has an errata 2.4.3:

    Memory-mapped read operations may fail when timeout counter is enabled

This is unfortunate because it means that once QSPI memory-mapped flash is
accessed the QSPI peripheral will leave the CS pin active (low) forever,
which increases power consumption of the SPI flash chip (because it's
active and waiting for commands).  The exact amount of power increase
depends on the flash, but the PYBD_SFx increase by about 2.5mA.

Previously this increase in power only happened when QSPI flash was needed,
eg on PYBD_SF2 when mbedtls or nimble libraries were used.  On PYBD_SF6
it's actually never used.

But with the introduction of ROMFS which lives in the QSPI flash, the
memory is always access on start up to see if the ROMFS contains a valid
image (it must read the memory to find out).  That means these boards
always consume about 2.5mA more after starting up (compared to when ROMFS
is disabled).

The fix in this commit is to explicitly restart the QSPI memory mapped mode
during the start up process.  More precisely, the restart is done after
querying the ROMFS and just before trying to execute `boot.py`.  That's the
right location to keep power consumption permanently down if the QSPI is
never used (eg ROMFS image doesn't exist).

Signed-off-by: Damien George <[email protected]>
This commit changes the gchelper implementation in use for RV32-based
targets (ESP32C3, ESP32C6) from the generic one written in C to the one
written in assembler that is specific to the CPU in question.

The native implementation is already exercised on most CI builds as it
is used by the QEMU port to compile and test the RV32 target.

Signed-off-by: Alessandro Gatti <[email protected]>
This commit cleans up a couple of RISC-V specific directives in the
build script.  Namely, removes the forced inclusion of the "riscv"
component and introduces proper mpy-cross flags.

The "riscv" component is already included by the ESP-IDF build
framework, as certain low-level components would not build otherwise, so
there is no need to add it to the required components list.

The architecture flag for mpy-cross is now set for RISC-V targets, as it
was previously set only for Xtensa targets (and it relied on a string
comparison rather than using the appropriate configuration variable).

Signed-off-by: Alessandro Gatti <[email protected]>
This commit removes a definition used back when ESP-IDF v4 was supported
by MicroPython.  Those times are now long gone, and so is the need for
that particular definition to be set in the first place.

Signed-off-by: Alessandro Gatti <[email protected]>
This is a known limitation, so better to give a clear warning than a
catch-all AssertionError.  Happens for example when trying to use
soft-float on ARCH=armv6m

Also give more details on the assertion for unknown relocations, such that
one can see which symbol it affects etc, to aid in debugging.

References issue micropython#14430.

Signed-off-by: Jon Nordby <[email protected]>
If picotool is not installed, it's fetched and built when compiling each
rp2 board.  And the "develop" branch of picotool is used instead of a
release.  Installing it manually using the "master" branch means the latest
released version is used (instead of a possibly unstable development
version), and also makes building each rp2 board a little faster.

Signed-off-by: Damien George <[email protected]>
Since 7c1584a MicroPython matches CPython in most cases, aside from
nested comprehensions.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
MicroPython support for this behaviour was added in eb45d97.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
Previously this information was recorded in a "status" field of the result,
but nothing ever parsed this result which led to non-differences not being
removed.

This work was funded through GitHub Sponsors.

Signed-off-by: Angus Gratton <[email protected]>
jepler and others added 29 commits May 13, 2025 12:16
This groups non-decimal values by fours, such as bbb_bbbb_bbbb.  It also
supports `{:_d}` to use underscore for decimal numbers (grouped in threes).

Use of incorrect ":,b" is not diagnosed.

Thanks to @dpgeorge for the suggestion to reduce code size.

Signed-off-by: Jeff Epler <[email protected]>
This commit removes the explicit dependency on the vendored tinyusb
version for the ESP32S2 and ESP32S3 boards.

Tinyusb is still available to MicroPython through a dependency on the
`espressif/esp_tinyusb` ESP-IDF component, which in turn depends on
the `espressif/tinyusb` component itself.

Signed-off-by: Alessandro Gatti <[email protected]>
Raises a value error in that case, which happens after a timer was created
but not initialized, or after calling `timer.deinit()`.

Fixes issue micropython#17033.

Signed-off-by: robert-hh <[email protected]>
This commit introduces two extra CMake variables, MICROPY_DEF_COMPONENT
and MICROPY_COMPILE_COMPONENT, that make it easier to integrate
MicroPython as a custom ESP-IDF component.

Whilst there is no official MicroPython component available for ESP-IDF,
integration can be achieved with some minor CMake scripting outside the
MicroPython tree - except for customisation of compilation defines and
build flags, which is what this commit tries to provide.

Compilation defines customisation is especially important for
MicroPython configuration, as it is not possible to inject a value for
MP_CONFIGFILE otherwise.  This means that unless MicroPython itself is
forked first to edit ports/esp32/mpconfigport.h, it is not possible to
perform any meaningful configuration of the interpreter/runtime when
included as a component.

Signed-off-by: Alessandro Gatti <[email protected]>
This commit introduces a new port configuration entry allowing the entry
point function name to be changed, from "app_main" to a custom name.

This is needed when MicroPython is embedded as an ESP-IDF component,
since the "app_main" symbol is already provided elsewhere, making
compilation not possible.  Marking MicroPython's symbol as weak would
make it compile and make it possible to create and start the MicroPython
task anyway with the right FreeRTOS task creation incantation, but it is
probably easier to just rename the initialisation function into
something else that can be accessed from outside.

When MicroPython is embedded as an ESP-IDF component, the
MICROPY_ESP_IDF_ENTRY definition can be set to indicate the new entry
point function name.  The new function name prototype should still be
defined in external code to let linking succeed.

Also, the NLR failure callback is marked as weak to give the chance of
handling such error in a more controlled fashion rather than trigger an
unconditional board restart.

Signed-off-by: Alessandro Gatti <[email protected]>
If the driver was reinitialised while there was
already an event task running the queue that task
is trying to receive from would be deleted,
causing it to try to take a lock that no longer
existed and deadlocking the CPU.

This change ensures the task is always shut down
before recreating the queue and recreates the task
afterwards.

It also allows setting an IRQ handler before the
UART is initialized (like other ports allow),
removes the task when the UART is deinitialized
(which was previously missing), adds a check that
no event task can be started when no queue exists,
and adds a check to prevent reinitialising the
UART driver unnecessarily.

Signed-off-by: Daniël van de Giessen <[email protected]>
This commit changes the error handler for WiFi operations to recognise
out of memory conditions reported by ESP-IDF functions, and report them
as more descriptive exceptions rather than a generic "error 0x101".

The error handler only provided a human-readable error description for
WiFi-specific error codes (codes in the ESP_ERR_WIFI_BASE range), but
WiFi functions are known to return other codes.  Now ESP_ERR_NO_MEM is
covered with a specific error message, making it easier to debug issues
related to running out of ESP-IDF heap.

Signed-off-by: Alessandro Gatti <[email protected]>
The ESP32 PPP implementation predates the generic
implementation in extmod. The new extmod
implementation has a few advantages such as a
better deinitialisation procedure (the ESP32
implemementation would not clean up properly and
cause crashes if recreated) and using the UART IRQ
functionality instead of running a task to read
data from the UART.

This change restructures the ESP implementation to
be much closer to the new extmod version, while
also bringing a few tiny improvements from the
ESP32 version to the extmod version. The diff
between extmod/network_ppp_lwip.c and
ports/esp32/network_ppp.c is now a small set of
easy to review ESP32 port-specific changes.

Signed-off-by: Daniël van de Giessen <[email protected]>
OSError errno values should be positive.

Signed-off-by: Damien George <[email protected]>
To make sure they have the correct value.

Signed-off-by: Damien George <[email protected]>
In the syntax_space cpydiff, all the warnings were shown after the other
output.  This is because the output always showed all of stdout first and
all of stdout second.

By running Python in unbuffered mode and using `stderr=STDOUT`, the two
streams are interleaved in exactly the order they're printed, so the
SyntaxWarnings are interleaved with the other output.

By using the `encoding=` argument of Popen, the need to explicitly convert
to utf-8 is avoided.  The encoding of the input also becomes utf-8 in this
case, which all the test cases are (well, they're all ASCII, I think).  As
in `run-tests.py`, setting PYTHONIOENCODING ensures the Python
interpreter's input and output are in utf-8, which is not always the case,
especially on Windows systems.

I spot-checked the generated doc pages and they all seemed to make sense
still.

Signed-off-by: Jeff Epler <[email protected]>
The heading character for the difference title was always "~", but items
had been added which had just a single heading level.  This made the
generated table of contents confused about heading levels, because heading
levels are not fixed in rst, but are inferred from the order they appear in
the document.

Signed-off-by: Jeff Epler <[email protected]>
This improves the TOC display of the generated differences section.

Signed-off-by: Jeff Epler <[email protected]>
Previously, the navigation ended up messy when the (long) description of
the item became used as a 2nd level header, meaning that it was placed in
the navigation.  Check for this when generating cpydiff so that new cases
don't sneak in unnoticed.

Signed-off-by: Jeff Epler <[email protected]>
This reduces inconsitencies between esp32 and other ports.

According to the discussion in micropython#10817.

Signed-off-by: Ihor Nehrutsa <[email protected]>
This reduce inconsistencies between esp32 PWM and other ports:
1. duty_u16() high value is 2**16-1 == 65535
2. Invert PWM wave with invert=1 parameter
3. Enable PWM in light sleep mode
4. Allow PWM output and read pulse input simultaneously on the same Pin()
5. Code refactoring

Co-Authored-By: Angus Gratton <[email protected]>
Co-Authored-By: robert-hh <[email protected]>
Co-Authored-By: Andrew Leech <[email protected]>
Co-Authored-By: Yoann Darche <[email protected]>

Signed-off-by: Ihor Nehrutsa <[email protected]>
Python threads (created via the `_thread` module) are backed by a FreeRTOS
task.  Managing the deletion of the task can be tricky, and there are
currently some bugs with this in the esp32 port.

The actual crash seen was in FreeRTOS' `uxListRemove()`, and that's because
of two calls to `vTaskDelete()` for the same task: one in
`freertos_entry()` when the task ran to completion, and the other in
`mp_thread_deinit()`.  The latter tried to delete the task a second time
because it was still in the linked list, because `vTaskPreDeletionHook()`
had not yet been called.  And the reason `vTaskPreDeletionHook()` was yet
to be called is because the FreeRTOS idle task was starved.

This commit fixes that.

There are three things done by this commit:
- remove the `vTaskPreDeletionHook`, it's not needed anymore because task
  stack memory is allocated by the IDF, not on the MicroPython heap
- when a task finishes it now removes itself from the linked list, just
  before it deletes itself
- on soft reset, all tasks are deleted and removed from the linked list in
  one swoop (while the mutex is held)

Signed-off-by: Damien George <[email protected]>
This version of the IDF uses about 1KB more IRAM and 1KB more DRAM on most
boards, but 6.5KB more DRAM usage on the S3.  It seems that's due to a lot
of small increases in many components.

Signed-off-by: Ihor Nehrutsa <[email protected]>
This adds support for LAN8670 to the esp32 port.  Enabled conditionally for
the esp32 target, if ESP-IDF version is new enough (v5.3 or newer).

Fixes issue micropython#15731.

Signed-off-by: Damien George <[email protected]>
Signed-off-by: Angus Gratton <[email protected]>
This adds a new function, `esp32.idf_task_info()`, that can be used to
retrieve task statistics which is useful for diagnosing issues where some
tasks are using up a lot of CPU time.

It's best used in conjunction with the `utop` module from micropython-lib.

Signed-off-by: Daniël van de Giessen <[email protected]>
The default I2C init does not require setting SCL or SDA but the default
I2C0 pins for C3, S3 conflict with the espressif GPIO usage.

For the C3, pins 18/19 are for USB/JTAG.  If used for I2C() they will cause
the REPL to hang on initialization of the I2C.

For the S3 pin 19 is allocated for USB/JTAG also but the defaults do not
seem to affect the REPL.

See related micropython#16956.

Fixes issue micropython#17103.

Signed-off-by: Rick Sorensen <[email protected]>
Signed-off-by: Angus Gratton <[email protected]>
@Thinbungee Thinbungee merged commit 7ce2850 into Thinbungee:master May 19, 2025
62 of 66 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.