Quoted string parser reports end of file when encountering an empty line

[Alainx277]

Describe the bug

When quoting a string it is possible to have text that goes on multiple lines. If the multiline string contains an empty line however, the parser fails even if the string is properly closed.

To Reproduce

Create a fluentd config like this:

<filter test>
  @type record_transformer
  <record>
    hello "world

"
  </record>
</filter>

Run fluentd in dry-run mode:

fluentd --dry-run -c /tmp/fluent.conf
Traceback (most recent call last):
	18: from /Users/me/.rbenv/versions/2.5.1/bin/fluentd:23:in `<main>'
	17: from /Users/me/.rbenv/versions/2.5.1/bin/fluentd:23:in `load'
	16: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/bin/fluentd:15:in `<top (required)>'
	15: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'
	14: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'
	13: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/command/fluentd.rb:351:in `<top (required)>'
	12: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/supervisor.rb:624:in `configure'
	11: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/supervisor.rb:682:in `setup_global_logger'
	10: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config.rb:52:in `build'
	 9: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config.rb:71:in `parse'
	 8: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/v1_parser.rb:33:in `parse'
	 7: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/v1_parser.rb:44:in `parse!'
	 6: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/v1_parser.rb:96:in `parse_element'
	 5: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/v1_parser.rb:96:in `parse_element'
	 4: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/v1_parser.rb:131:in `parse_element'
	 3: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/literal_parser.rb:75:in `parse_literal'
	 2: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/literal_parser.rb:82:in `scan_string'
	 1: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/literal_parser.rb:114:in `scan_double_quoted_string'
/Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/basic_parser.rb:92:in `parse_error!': unexpected end of file in a double quoted string at fluent.conf line 6,17 (Fluent::ConfigParseError)
  5:   <record>
  6:     hello "world

     -----------------^
  7: 

Expected behavior

The string should be parsed like any other multiline string. In this case I'd expect hello to be set to "world\n\n".

Your Environment

- Fluentd version: 1.16.6

Your Configuration

<filter test>
  @type record_transformer
  <record>
    hello "world

"
  </record>
</filter>

Your Error Log

Traceback (most recent call last):
	18: from /Users/me/.rbenv/versions/2.5.1/bin/fluentd:23:in `<main>'
	17: from /Users/me/.rbenv/versions/2.5.1/bin/fluentd:23:in `load'
	16: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/bin/fluentd:15:in `<top (required)>'
	15: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'
	14: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/site_ruby/2.5.0/rubygems/core_ext/kernel_require.rb:85:in `require'
	13: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/command/fluentd.rb:351:in `<top (required)>'
	12: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/supervisor.rb:624:in `configure'
	11: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/supervisor.rb:682:in `setup_global_logger'
	10: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config.rb:52:in `build'
	 9: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config.rb:71:in `parse'
	 8: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/v1_parser.rb:33:in `parse'
	 7: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/v1_parser.rb:44:in `parse!'
	 6: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/v1_parser.rb:96:in `parse_element'
	 5: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/v1_parser.rb:96:in `parse_element'
	 4: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/v1_parser.rb:131:in `parse_element'
	 3: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/literal_parser.rb:75:in `parse_literal'
	 2: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/literal_parser.rb:82:in `scan_string'
	 1: from /Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/literal_parser.rb:114:in `scan_double_quoted_string'
/Users/me/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/fluentd-1.16.6/lib/fluent/config/basic_parser.rb:92:in `parse_error!': unexpected end of file in a double quoted string at fluent.conf line 6,17 (Fluent::ConfigParseError)
  5:   <record>
  6:     hello "world

     -----------------^
  7:

Additional context

No response

[Alainx277]

This also happens if a line is terminated with a literal backslash:

<filter test>
  @type record_transformer
  <record>
    hello "world \\
test"
  </record>
</filter>

[kenhys]

It seems that the limitation of current implementation.
As a workaround, use "\n" explicitly.

[daipom]

Thanks for your report.
It appears this is a bug of Fluentd.

This works.

<filter test.**>
  @type record_transformer
  <record>
    hello "world
"
  </record>
</filter>

However, the following don't work.

<filter test.**>
  @type record_transformer
  <record>
    hello "world

"
  </record>
</filter>

<filter test.**>
  @type record_transformer
  <record>
    hello 'world
'
  </record>
</filter>

<filter test.**>
  @type record_transformer
  <record>
    hello 'world

'
  </record>
</filter>

[daipom]

It looks like single-quoted strings don't support multiline.
https://docs.fluentd.org/configuration/config-file#multiline-support-for-quoted-string-array-and-hash-values

So, we need to check double quoted strings behavior, as @Alainx277 says.