STM32H743 Micropython V1.23 Coroutine Error: 'SingletonGenerator' object has no attribute 'exc' after long runtime

[chiyu-Charon]

I'm using an STM32H743 with firmware I compiled myself, based on Micropython V1.23. After running normally for a long time, all of a sudden, all my coroutines start throwing an error. The error message is: "'SingletonGenerator' object has no attribute 'exc'". The error occurs in the asyncio/core.py file.

# "Yield" once, then raise StopIteration
class SingletonGenerator:
    def __init__(self):
        self.state = None
        self.exc = StopIteration()

    def __iter__(self):
        return self

    def __next__(self):
        if not hasattr(self, "exc") or self.exc is None:
            self.exc = StopIteration()
        if self.state is not None:
            _task_queue.push(cur_task, self.state)
            self.state = None
            return None
        else:
            self.exc.__traceback__ = None
            raise self.exc

Since I can't save the REPL print in real-time when the device is running and connected to the USB, when I connect the USB later, the buffer only has the content shown in the image. Apologies, my device doesn't allow me to export a screenshot, so I can only capture it with my phone.

Does anyone know what might cause this error? Thanks

[peterhinch]

Given that the error is occurring in asyncio.core.py we can probably rule out a bug - the strong likelihood is RAM fragmentation. I have seen similar "missing attribute" errors in these circumstances. In large applications expected to run for long durations it can help to run gc.collect() periodically. Also check out the official docs on optimisation.

[chiyu-Charon]

Thanks! I have a dedicated coroutine that calls gc.collect every 5 seconds or so and prints the free memory. The output shows that there’s always about 50MB of free memory (I’ve set up an external RAM for extra memory).
Could it be that the gc.collect is being called too frequently?
Would it be more appropriate to switch to automatic cleanup using gc.threshold?

[peterhinch]

Calling gc every 5s is fine. In asynchronous applications this easily done and is the approach I use. If you have 50MB of free RAM it seems unlikely that fragmentation is the problem unless you're creating and destroying large (>1MB) objects.

I'm stumped for answers here, except to re-iterate that missing attributes do suggest a RAM issue. Could there be a hardware problem with your extra RAM?

[chiyu-Charon]

Hello,We have refactored the code to minimize memory overhead and temporarily disabled the server communication module to help diagnose the issue. Currently, the device uses only about 0.2MB of memory during runtime, compared to around 9MB previously, with frequent fluctuations between 1MB and 9MB. However, the coroutine crash still occurs after running for some time. Could you please suggest any other potential causes for this issue? Thanks for you help.

[peterhinch]

I'm running out of ideas here.

We have run asynchronous WiFi enabled code for long periods (months) without issue, notably when developing mqtt-as. Is it possible there's a device driver bug somewhere? Although such bugs usually lead to a complete crash rather than your missing attribute symptom.

How does your board achieve WiFi connectivity? Is there any possibility of trying your code on alternative hardware - for example an ESP32-S3 with SPIRAM? I'm expecting the answer "no" here if you're using an H7.

Otherwise it's the laborious approach of cutting out chunks of code to try to narrow down the cause, to see if it's associated with a particular interface. Or seek someone with experience of using an H7 with SPIRAM in case there's an issue with the specific configuration.

[andrewleech]

I suspect the stack trace is from

micropython/extmod/asyncio/stream.py

Line 80 in 938e2c0

return (yield from core.sleep_ms(0))

as SingletonGenerator is used to create asyncio sleep.

In your it might help to temporarily change the opt field in https://github.com/micropython/micropython/blob/master/extmod/asyncio/manifest.py to zero to retain line numbers to be sure.

It's usually rare but ram corruption could be the issue for some reason, maybe check your sdram timing configuration, perhaps slow it down a little to test?

[chiyu-Charon]

@peterhinch
Thank you for your reply.
We are using a PHY chip LAN8720A to establish a wired local area network, and we are not using Wi-Fi. However, we have temporarily disabled the TCP communication to help locate the issue.
The SPIRAM firmware configuration was modified based on the 'MicroPython/ports/stm32/boards/ARDUINO_GIGA' configuration as a reference. We will review the configuration to ensure its correctness.
Currently, we don't have any other solutions and can only gradually reduce the code to try to pinpoint the problem. What is particularly frustrating is the uncertainty of when the error occurs, as we are unable to capture it in a timely and accurate manner. The error stack from the REPL output also doesn't provide more information.Hope we can capture and resolve the issue as soon as possible.
Once again, thank you.