allow yaml parser and visitor to be garbage collected

When loading a YAML dictionary the YamlFile class will store a YAML visitor and a parser.

This means that after "read_file_into_dictionary" is called the Psych:Parser will keep the object tree that was used to parse and navigate the YAML document, even after the resulting Hash dictionary is created.

While the memory required to parse the YAML dictionary is still required, we can change the YamlFile class to allow the Psych::Parser garbage collection, freeing a significant amount of memory from the heap.

Author: jsvd

CHANGELOG.md

@@ -1,3 +1,6 @@
+## 3.4.3
+  - Allow YamlFile's Psych::Parser and Visitor instances to be garbage collected [#104](https://github.com/logstash-plugins/logstash-filter-translate/pull/104)
+
 ## 3.4.2
   - Fixed JRuby 9.4 compatibility issue[#98](https://github.com/logstash-plugins/logstash-filter-translate/pull/98)
 

lib/logstash/filters/dictionary/yaml_file.rb

@@ -8,19 +8,19 @@ class YamlFile < File
     protected
 
     def initialize_for_file_type(**file_type_args)
-      @visitor = YamlVisitor.create
-
-      @parser = Psych::Parser.new(Psych::TreeBuilder.new)
-      @parser.code_point_limit = file_type_args[:yaml_code_point_limit]
+      @yaml_code_point_limit = file_type_args[:yaml_code_point_limit]
     end
 
     def read_file_into_dictionary
+      visitor = YamlVisitor.create
+      parser = Psych::Parser.new(Psych::TreeBuilder.new)
+      parser.code_point_limit = @yaml_code_point_limit
       # low level YAML read that tries to create as
       # few intermediate objects as possible
       # this overwrites the value at key
       yaml_string = IO.read(@dictionary_path, :mode => 'r:bom|utf-8')
-      @parser.parse(yaml_string, @dictionary_path)
-      @visitor.accept_with_dictionary(@dictionary, @parser.handler.root)
+      parser.parse(yaml_string, @dictionary_path)
+      visitor.accept_with_dictionary(@dictionary, parser.handler.root)
     end
   end
 end end end

logstash-filter-translate.gemspec

@@ -1,7 +1,7 @@
 Gem::Specification.new do |s|
 
   s.name            = 'logstash-filter-translate'
-  s.version         = '3.4.2'
+  s.version         = '3.4.3'
   s.licenses        = ['Apache License (2.0)']
   s.summary         = "Replaces field contents based on a hash or YAML file"
   s.description     = "This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline using $LS_HOME/bin/logstash-plugin install gemname. This gem is not a stand-alone program"