Skip to content

GroveEventController: SSE connections drop after ~60s due to nginx proxy_read_timeout (no heartbeat) #192

Description

@scuba10steve

Problem

SSE connections from GroveEventController are dropped after roughly 60 seconds when proxied through nginx, because nginx's default proxy_read_timeout (60s) closes idle upstream connections and the emitter sends no traffic in that window to keep it alive.

Root cause

GroveEventController opens SseEmitters but never writes anything to them unless a real grove event occurs. If no event fires within nginx's proxy_read_timeout for the /api/ location, nginx tears down the connection from under the still-open emitter.

Recommended fix

Have GroveEventController send a periodic SSE comment/heartbeat (e.g. every 15-20s) on each open emitter, well under nginx's 60s timeout. This is the standard, robust fix — it works regardless of any specific proxy's timeout configuration.

As defense-in-depth (not a substitute for the heartbeat), also consider bumping nginx's proxy_read_timeout for the /api/ location — that's a docker/Ansible-managed setting in the arboretum role, separate from the backend change.

Scope

  • Primary fix: trellis backend, GroveEventController (heartbeat on open emitters)
  • Optional defense-in-depth: arboretum Ansible role, nginx proxy_read_timeout for /api/

Metadata

Metadata

Assignees

No one assigned

    Labels

    apiREST API changesbugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions