Skip to content

Faust with Django connection closed problem #666

@MedAziz11

Description

@MedAziz11

I'm using Faust in combination with a Django backend. Faust is integrated as a Django app but runs in a separate container, similar to how Celery workers operate — except in this case, it's Faust with Kafka.

Inside my Faust agents, I'm performing asynchronous operations using Django's ORM. After the system runs for a while, I begin encountering connection closed errors during ORM operations. This seems related to how database connections are managed in long-lived async processes.
any idea how to handle this?
logs:

920 | psycopg2.InterfaceError: connection already closed |  
-- | -- | --
  |   | 2025-09-12 09:56:15.920 | django.db.utils.InterfaceError: connection already closed |  
  |   | 2025-09-12 09:56:15.920 | psycopg2.InterfaceError: connection already closed |  
  | 
 | Fields |   | container_name | processor_agent.1.dic96f1ikb3zgicm8nl9l7ih7 |   | filename | /var/log/docker/9cd7bc50abb53e57aabebe6f7f7b71df007ae4be86155ac89313e551c0fda5e3/json.log |   | host | XXXXXXX |   | source | stderr |   | swarm_service | processor_agent |   | swarm_stack | alertify
Fields
  | container_name | processor_agent.1.dic96f1ikb3zgicm8nl9l7ih7
  | filename | /var/log/docker/9cd7bc50abb53e57aabebe6f7f7b71df007ae4be86155ac89313e551c0fda5e3/json.log
  | host | XXXXXXXXX
  | source | stderr
  | swarm_service | processor_agent
  | swarm_stack | xxxx
  |   | 2025-09-12 09:56:15.920 | django.db.utils.OperationalError: server closed the connection unexpectedly |  
  |   | 2025-09-12 09:56:15.920 | psycopg2.OperationalError: server closed the connection unexpectedly |  
  |   | 2025-09-12 09:40:40.671 | elastic_transport.ConnectionError: Connection error caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) |  
  |   | 2025-09-12 09:40:40.671 | urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) |  
  |   | 2025-09-12 09:40:40.671 | raise RemoteDisconnected("Remote end closed connection without" |  
  |   | 2025-09-12 09:40:40.671 | http.client.RemoteDisconnected: Remote end closed connection without response |  
  |   | 2025-09-12 09:40:40.671 | raise RemoteDisconnected("Remote end closed connection without" |  
  |   | 2025-09-12 09:00:02.668 | elastic_transport.ConnectionError: Connection error caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) |  
  |   | 2025-09-12 09:00:02.668 | urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) |  
  |   | 2025-09-12 09:00:02.668 | raise RemoteDisconnected("Remote end closed connection without" |  
  |   | 2025-09-12 09:00:02.668 | http.client.RemoteDisconnected: Remote end closed connection without response |  
  |   | 2025-09-12 09:00:02.668 | raise RemoteDisconnected("Remote end closed connection without" |  
  |   | 2025-09-12 09:00:02.428 | django.db.utils.OperationalError: server closed the connection unexpectedly |  
  |   | 2025-09-12 09:00:02.428 | psycopg2.OperationalError: server closed the connection unexpectedly |  
  |   | 2025-09-12 09:00:02.428 | [2025-09-12 10:00:02,374] [7] [ERROR] 🔥 Error processing alert 'XXXXXXXXX': server closed the connection unexpectedly |  
  |   | 2025-09-12 09:00:02.346 | psycopg2.InterfaceError: connection already closed |  
  |   | 2025-09-12 09:00:02.346 | django.db.utils.InterfaceError: connection already closed

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions