Possible bug with Lua time_as_table option and timestamp nanoseconds as integer

[juan-domenech]

Bug Report

Describe the bug
When using time_as_table to call Lua intending to have access to the nanoseconds part of the timestamp, the value obtained from timestamp['nsec]' is incorrect when the original nanoseconds value begins with zero.
i.e.:
Given this timestamp 1742647673.080140070, the value of timestamp['nsec'] inside Lua will be 80140070 (the first zero is missing) instead of 080140070.
This causes this timestamp to be incorrect.

To Reproduce
In Fluent-bit 3.2.9 call Lua with time_as_table true and return the nanoseconds part of the timestamp.
Example:

    [FILTER]
        Name          lua
        Match         *
        script        timestamp_time_as_table.lua
        call          timestamp_time_as_table
        time_as_table true

    [SERVICE]
        Log_Level     info
        Daemon        off
        Parsers_File  parsers.conf
        HTTP_Server   On
        HTTP_Listen   0.0.0.0
        HTTP_Port     2020
        scheduler.base   3
        scheduler.cap    10

    [INPUT]
        Name   dummy
        Dummy  {"message": "test"}

    [OUTPUT]
        Name        stdout
        Match       *

Lua script timestamp_time_as_table.lua

    function timestamp_time_as_table(tag, timestamp, record)
      record.timestamp_seconds          = timestamp['sec']
      record.timestamp_nanoseconds_part = timestamp['nsec']
      return 1, timestamp, record
    end

• Output:

[0] dummy.0: [[1742647673.080140070, {}], {"timestamp_seconds"=>1742647673, "message"=>"test", "timestamp_nanoseconds_part"=>80140070}]
[0] dummy.0: [[1742647674.080235692, {}], {"timestamp_seconds"=>1742647674, "message"=>"test", "timestamp_nanoseconds_part"=>80235692}]
[0] dummy.0: [[1742647675.080127813, {}], {"timestamp_seconds"=>1742647675, "message"=>"test", "timestamp_nanoseconds_part"=>80127813}]
[0] dummy.0: [[1742647676.080107643, {}], {"timestamp_seconds"=>1742647676, "message"=>"test", "timestamp_nanoseconds_part"=>80107643}]
[0] dummy.0: [[1742647677.080166557, {}], {"timestamp_seconds"=>1742647677, "message"=>"test", "timestamp_nanoseconds_part"=>80166557}]
[0] dummy.0: [[1742647678.080585972, {}], {"timestamp_seconds"=>1742647678, "message"=>"test", "timestamp_nanoseconds_part"=>80585972}]

Expected behavior
As you can see above, the values of the output variable timestamp_nanoseconds_part are missing the leading zero.
This is obviously expected for an integer number and could be the explanation of this issue.
I've tried to improve it using tostring() but no joy:

    function timestamp_time_as_table(tag, timestamp, record)
      record.timestamp_seconds          = tostring(timestamp['sec'])
      record.timestamp_nanoseconds_part = tostring(timestamp['nsec'])
      return 1, timestamp, record
    end

Output:

[0] dummy.0: [[1742649430.080068686, {}], {"message"=>"test", "timestamp_nanoseconds_part"=>"80068686", "timestamp_seconds"=>"1742649430"}]
[0] dummy.0: [[1742649431.080065796, {}], {"message"=>"test", "timestamp_nanoseconds_part"=>"80065796", "timestamp_seconds"=>"1742649431"}]
[0] dummy.0: [[1742649432.080047865, {}], {"message"=>"test", "timestamp_nanoseconds_part"=>"80047865", "timestamp_seconds"=>"1742649432"}]
[0] dummy.0: [[1742649433.080076516, {}], {"message"=>"test", "timestamp_nanoseconds_part"=>"80076516", "timestamp_seconds"=>"1742649433"}]
[0] dummy.0: [[1742649434.080134543, {}], {"message"=>"test", "timestamp_nanoseconds_part"=>"80134543", "timestamp_seconds"=>"1742649434"}]
[0] dummy.0: [[1742649435.080066693, {}], {"message"=>"test", "timestamp_nanoseconds_part"=>"80066693", "timestamp_seconds"=>"1742649435"}]
[0] dummy.0: [[1742649436.080060593, {}], {"message"=>"test", "timestamp_nanoseconds_part"=>"80060593", "timestamp_seconds"=>"1742649436"}]

Workaround
As a way to work around the issue I've expanded the Lua function to format timestamp['nsec'] and complement the missing zero(s).
Workaround Lua script timestamp_time_as_table.lua:

    function timestamp_time_as_table(tag, timestamp, record)
      record.timestamp = os.date("%Y-%m-%dT%H:%M:%S",timestamp['sec']) .. "." .. string.format("%09d", timestamp['nsec']) .. "Z"
      return 1, timestamp, record
    end

Workaround result with timestamp in ISO8601 format with fixed zeroes:

[0] dummy.0: [[1742649658.080160594, {}], {"timestamp"=>"2025-03-22T13:20:58.080160594Z", "message"=>"test"}]
[0] dummy.0: [[1742649659.080112130, {}], {"timestamp"=>"2025-03-22T13:20:59.080112130Z", "message"=>"test"}]
[0] dummy.0: [[1742649660.080123833, {}], {"timestamp"=>"2025-03-22T13:21:00.080123833Z", "message"=>"test"}]
[0] dummy.0: [[1742649661.080098202, {}], {"timestamp"=>"2025-03-22T13:21:01.080098202Z", "message"=>"test"}]
[0] dummy.0: [[1742649662.080267696, {}], {"timestamp"=>"2025-03-22T13:21:02.080267696Z", "message"=>"test"}]
[0] dummy.0: [[1742649663.080199898, {}], {"timestamp"=>"2025-03-22T13:21:03.080199898Z", "message"=>"test"}]

Your Environment

• Version used: 3.2.9
• Configuration: See above
• Environment: Kubernetes v.1.29

Additional context

• Doc page https://docs.fluentbit.io/manual/pipeline/filters/lua
• Feature introduction in 1.6 Lua filter either truncates or corrupts Fluent Bit's nanosecond-resolution timestamp #2519
• Related code in the Lua plugin