Skip to content

OOM in update join #984

@kzsanam

Description

@kzsanam

hey!
I have some problem with joins and hope someone can help me.

I am doing update join of 2 streams as described in the documentation. Keys in each stream appear only one time.
I am setting updating_ttl to 10 seconds. After ~10 min a worker dies, I could not find any proper logs, but after looking at metrics I concluded that it can be an OOM error, because memory is linearly increasing until it reaches my memory limit. When I did this join on my PC it just froze, which also suggests an OOM.

So my question is the following. What does updating_ttl set? Is it time after which the record will be deleted from the memory? Or is it just setting ttl and some background process running not so often will eventually delete it? If it is the second option it would explain an OOM, and if so what can I do?

I am asking because window join is not the best choice for my case, I am loosing some information due to edge effects with tumble join.

below is the memory metric.
Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions