Skip to content

Conversation

@jsvd
Copy link
Member

@jsvd jsvd commented Jul 23, 2025

fixes #103

When loading a YAML dictionary the YamlFile class will store a YAML visitor and a parser.

This means that after "read_file_into_dictionary" is called the Psych:Parser will keep the object tree that was used to parse and navigate the YAML document, even after the resulting Hash dictionary is created.

While the memory required to parse the YAML dictionary is still required, we can change the YamlFile class to allow the Psych::Parser garbage collection, freeing a significant amount of memory from the heap.
@jsvd jsvd force-pushed the reduce_constant_memory_usage branch from 3d8c54f to 10c09d0 Compare July 24, 2025 06:15
@jsvd jsvd requested a review from kaisecheng July 24, 2025 06:32
Copy link
Contributor

@kaisecheng kaisecheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jsvd jsvd merged commit d9d5bbb into logstash-plugins:main Jul 24, 2025
2 of 3 checks passed
@jsvd jsvd deleted the reduce_constant_memory_usage branch July 24, 2025 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Visitor/Parser for YAML dictionary leads to excessive continuous memory consumption

2 participants