Scheduler drifts during system suspend

[soumya92]

Forked from #150 to track the general case.

If a scheduler is set to trigger after 10 minutes, and the system is suspended for 9 minutes, it will actually trigger after 19 minutes.

There seems to be no way in go to trigger a scheduler consistently when accounting for time spend suspended (or otherwise not in the process, e.g. under extreme load).

There are open go issues for this, and at least two projects are patching the go language to work around this.

[WGH-]

There seems to be no way in go to trigger a scheduler consistently when accounting for time spend suspended (or otherwise not in the process, e.g. under extreme load).

I don't think any clock in the kernel drifts under load, though. Alternatively, the challenge to torture the kernel into skipping timer interrupts is now live.

[WGH-]

I think the current plan is to wait for #152, and then implement CLOCK_BOOTTIME timerfd support, right?

[soumya92]

Yep

[WGH-]

FWIW, for some time I've been running a naive timer implementation that creates a new CLOCK_BOOTTIME timerfd for every Scheduler instance. It creates (relatively) lots of fds, but works fine otherwise. If that's worth it, I might clean it up and send it as a PR.

   FD TYPE     DEVICE       SIZE       NODE NAME
    0    FIFO        0xd                638035 pipe
    1    FIFO        0xd                638034 pipe
    2     CHR        0x6                    44 /dev/tty1
    3 a_inode        0xe          0	  7725 [timerfd]
    4 a_inode        0xe          0	  7725 [eventpoll]
    5 a_inode        0xe          0	  7725 [timerfd]
    6 a_inode        0xe          0	  7725 [timerfd]
    7 a_inode        0xe          0	  7725 [timerfd]
    8    FIFO        0xd                638034 pipe
    9    FIFO        0xd                638035 pipe
   10 a_inode        0xe          0	  7725 [timerfd]
   11 a_inode        0xe          0	  7725 [timerfd]
   12 a_inode        0xe          0	  7725 [timerfd]
   13 a_inode        0xe          0	  7725 inotify
   14 a_inode        0xe          0	  7725 [eventpoll]
   15    FIFO        0xd                637032 pipe
   16    FIFO        0xd                637032 pipe
   17 a_inode        0xe          0	  7725 [timerfd]
   18 a_inode        0xe          0	  7725 [timerfd]
   19 a_inode        0xe          0	  7725 [timerfd]
   20 a_inode        0xe          0	  7725 [timerfd]
   21 a_inode        0xe          0	  7725 [timerfd]
   22 netlink                           637036 ROUTE
   23 a_inode        0xe          0	  7725 [timerfd]
   25 a_inode        0xe          0	  7725 [timerfd]
   30 a_inode        0xe          0	  7725 [timerfd]
   31 a_inode        0xe          0	  7725 [timerfd]
   32 a_inode        0xe          0	  7725 [timerfd]
   33 a_inode        0xe          0	  7725 [timerfd]
   34 a_inode        0xe          0	  7725 [timerfd]
   35    unix                           638048 type=STREAM

[soumya92]

Yeah, it's probably worth switching to CLOCK_BOOTTIME. I was holding out hope for native improvements in golang's time package, but we can always switch back to it later if it improves.