'timer.init()' fails with ENOMEM

[mungewell]

I use a couple of timers to create a PWM system, running around 1s period, to modulate between 2 settings for the PIO clock dividers. This works OK, but it gets restarted everytime I restart the PIO machines.

After around 6 restarts, I get a EOMEM during the 'timer.init()'.

OSError: [Errno 12] ENOMEM

I previously thought it was because I was discarding the timer (timer1=None) and making a new one with timer1=timer(), but after refactoring the code (to prevent this) I still get the EOMEM. I even added a garbage collection...

Bit that fails is

        try:
            eng.timer3.init(period=eng.period + 2000, mode=Timer.ONE_SHOT, callback=timer_sched)
            eng.timer2.init(period=eng.period,        mode=Timer.ONE_SHOT, callback=timer_sched)
            eng.timer1.init(period=eng.period - part, mode=Timer.ONE_SHOT, callback=timer_sched)
        except:
            # restart whole timer system

Timer3 is a safety net, Timer 2 is the period (which can change from time to time), Timer1 is the first part/phase of the PWM.

[GitHubsSilverBullet]

Works perfectly fine with up to 16 Timers.

#!/micropython
# vim: fileencoding=utf-8: ts=4: sw=4: expandtab:
from machine import Timer
from random import randrange
from time import sleep_ms

def timer_cb(t):
    global new
    print(t)
    t.deinit()
    new += 1

new = 16        # OK
#new = 17        # OSError: [Errno 12] ENOMEM

while True:
    sleep_ms(0)
    if new:
        new -= 1
        Timer().init(mode=Timer.ONE_SHOT, period=randrange(50,100), callback=timer_cb)

[peterhinch]

Maybe that's because you're issuing t.deinit()? I think it's wise to do this.

[GitHubsSilverBullet]

Though it works without it as well.
But I like to clean-up before I leave the building. ;-)

[mungewell]

After doing a bit of validation to make sure I didn't completely break my project, I checked in the revised code.

The full call back is here:
https://github.com/mungewell/pico-timecode/blob/main/pico_timecode.py#L382

I do have .deinit()s so suspect that maybe the leak is somewhere else, just showing up when the timers is actioned.

[GitHubsSilverBullet]

Your problem is most likely that you're not tracking the creation and deletion of Timer objects.
Once sixteen of'em are in use (or at least instantiated),creating the seventeenth instance leads to your original problem.
Make sure that you're freeing'em up once done with'em. ›deinit()‹ doesn't do that. One way to do that is to keep a list with Timer objects and their ids .

#!/micropython
# vim: fileencoding=utf-8: ts=4: sw=4: expandtab:
from machine import Timer
from random import randrange
from time import sleep_ms

def timer_cb(t):
    idx = tids.index(id(t))                     # determine idx of the fired Timer
    timers[idx].deinit()                        # cleanup
    timers[idx] = None                          # make this one eligible for gc
    tids[idx] = None                            # cleanup slot, allow a new assignment
    visual[idx] = ord('-')                      # adjust visual
    print(f'fin: {idx:2}', str(visual,'utf-8'), id(t), t)

N = 16
timers = [None for _ in range(N)]               # store for the Timer objects
tids = [None for _ in range(N)]                 # and their ids
visual = memoryview(bytearray('-'*N, 'ascii'))  # visual indication
while True:
    if None in tids:                            # space for a new one?
        idx = tids.index(None)                  # determine idx of the empty space
        t = Timer()                             # instantiate new Timer
        t.init(mode=Timer.ONE_SHOT, period=randrange(50,100), callback=timer_cb)
        timers[idx] = t                         # store the new Timer object
        tids[idx] = id(t)                       # and its id
        visual[idx] = ord('O')                  # adjust visual
        print(f'new: {idx:2}', str(visual,'ascii'), tids[idx], timers[idx])
    else:
        sleep_ms(1)

[mungewell]

Agree. I believe I am tracking the 3x Timers that I use in main code, and do use .deinit() on them.

There is a library I pull into code (in my UI) to implement button debouncing, so I'll check if that's messing with me.

[mungewell]

Further interesting find...

I placed a print() either side of the collect()
https://github.com/mungewell/pico-timecode/blob/main/pico_timecode.py#L1030

        if self.timer1 == None and self.timer2 == None:
            self.calval = calval

            # Force Garbage collection
            print("Available memory (bytes):", mem_free())
            collect()
            print("     ... and now (bytes):", mem_free())

And in order to see this I disable my 'per frame' (30 times per second) prints of TX. With this I was able to cycle through 20 reset/jam cycles without issue.
20_jams_no_crash.txt

I then re-introduced the 'per frame' prints, and got the error

TX: 20:32:28.06
TX: 20:32:28.07
stopping
Jamming: 64
Available memory (bytes): 190416
     ... and now (bytes): 192032
Unhandled exception in thread started by <function pico_timecode_thread at 0x20013520>
Traceback (most recent call last):
  File "pico_timecode.py", line 1128, in pico_timecode_thread
  File "pico_timecode.py", line 1045, in micro_adjust
  File "pico_timecode.py", line 422, in timer_re_init
OSError: [Errno 12] ENOMEM

As you can see the reported 'free' memory is around the same as the working condition, so what else (other than lack of memory) might trigger the ENOMEM? Fragmentation???

[shariltumin]

As mentioned above, you can have a maximum of 16 timers. Because:

In micropython/lib/pico-sdk/src/common/pico_time/include/pico

#define PICO_TIME_DEFAULT_ALARM_POOL_MAX_TIMERS 16

Therefore, we can have a maximum of 16 concurrent timers. Unfortunately, the machine_timer_init_helper() function in machine_timer.c does:

    if (self->alarm_id == -1) {
        mp_raise_OSError(MP_ENOMEM);
    }

The error message is misleading. There is nothing to do with memory here.

Your code calls timer3 = Timer() within a loop, which eventually hit this max value.

I suggest creating your three timers as global variables (i.e once only) and not setting eng.timer1 = None (for example). Instead, use a flag, such as eng.timer1_done = True, when you denit().

It is okay to use the _thread module. Micropython will determine which core triggers the timer events.

[mungewell]

Unless I missing something, the timer3 = Timer() call is gated if timer3 is already assigned.

        # safety timer - triggers 2s after timer2, if timer2 fails
        if eng.timer3:
            eng.timer3.deinit()
        else:
            eng.timer3 = Timer()

That said, I take your point about having a global for each timer, and generated them out the scope of the 'engine' class. I'll give this a go before tackling asyncio.

I saw (via experimentation) that the same core is always used, please note that error is not systematic. Sometime it's 12 cycles, sometime 32... I also noticed that RP2350 immediately errors for some reason.

[mungewell]

I re-wrote the code with global timer1 timer2 and timer3, and a few more changes to make this work... and (for the test task) it does - succeeding over 100cycles. The rp2350 also work.

BUT, when used with the UI code it fails in the same way.

rp2040 after 6 (or so) re-Jams

Jamming: 64
Unhandled exception in thread started by <function pico_timecode_thread at 0x20013810>
Traceback (most recent call last):
  File "pico_timecode.py", line 1113, in pico_timecode_thread
  File "pico_timecode.py", line 1030, in micro_adjust
  File "pico_timecode.py", line 424, in timer_re_init
OSError: [Errno 12] ENOMEM

rp2350, immediate failure on re-jam. I'm exhausted... :-(

[mungewell]

Therefore, we can have a maximum of 16 concurrent timers.

Might be a bit pedantic, but the SDK only defines 16 if it is not already set.... the actual maximum limit is 255.
https://github.com/raspberrypi/pico-sdk/blob/master/src/common/pico_time/include/pico/time.h#L340

// PICO_CONFIG: PICO_TIME_DEFAULT_ALARM_POOL_MAX_TIMERS, Selects the maximum number of concurrent timers in the default alarm pool, min=0, max=255, default=16, advanced=true, group=pico_time
#ifndef PICO_TIME_DEFAULT_ALARM_POOL_MAX_TIMERS
/*!
 * \brief Selects the maximum number of concurrent timers in the default alarm pool
 * \ingroup alarm
 *
 * \note For implementation reasons this is limited to PICO_PHEAP_MAX_ENTRIES which defaults to 255
 * \sa #PICO_TIME_DEFAULT_ALARM_POOL_HARDWARE_ALARM_NUM
 * \sa alarm_pool_get_default()
 */
#define PICO_TIME_DEFAULT_ALARM_POOL_MAX_TIMERS 16
#endif

Trying to understand why rp2350 behaves different to rp2050.

[Josverl]

machine_timer.c does:

    if (self->alarm_id == -1) {
        mp_raise_OSError(MP_ENOMEM);
    }

The error message is misleading. There is nothing to do with memory here.

Good point, I suggest MP_EIO (Input/Output error), or MP_EBUSY (Resource busy) might be a little clearer and should not increase FW size.

[peterhinch]

As a general point there are some subtle issues with multi-core coding which this doc attempts to address. In particular the lack of a shared GIL has implications relating to objects shared between the two contexts.

I don't claim to understand the constraints of your application, but unless there are specific counter-indications it's worth considering using asyncio to achieve concurrency and timing. This avoids the tricky issues associated with hard ISR's and threading. See official docs and tutorial.

[mungewell]

I'll admit that the current 'threading' code came to be in a somewhat (cough-cough) organic process. It would probably be well worth a proper re-write to use asyncio. That tutorial is well written/understandable...

There's nothing complicated;

• 'UI' code runs in initial thread, starts (and restarts) thread for Core Code.
• 'Core' core runs in own thread. Sets up PIOs, then passes data to/from with FIFOs and monitors/processes IRQs

I don't think I even care where (ie which CPU core) stuff runs, both of these threads are non-time critical - providing TX FIFO to PIO does not empty...