Syslog rfc5424 parsing does not accept NILVALUE timestamp

[jinnatar]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Syslog parsing for rfc5424 rejects a message that does not hold a timestamp and instead uses a NILVALUE, i.e. - for that field with the error:

unable to parse input as valid syslog message

As per the RFC:

TIMESTAMP = NILVALUE / FULL-DATE "T" FULL-TIME
NILVALUE = "-"

The TIMESTAMP field guidance 6.2.3 says:

A syslog application MUST use the NILVALUE as TIMESTAMP if the syslog
application is incapable of obtaining system time.

My case is exactly this, an embedded device incapable of determining time.

Configuration

N/A

Version

vector 0.48.0 (x86_64-unknown-linux-gnu a67e4e2 2025-06-30 18:25:45.272082383)

Debug Output

%> vector vrl --input test-timestamp.json "parse_syslog!(.message)"
2025-08-10T13:53:11.676549Z DEBUG vector::app: Internal log rate limit configured.
2025-08-10T13:53:11.676604Z  INFO vector::app: Log level is enabled. level="trace"
2025-08-10T13:53:11.676717Z DEBUG vector::app: messaged="Building runtime." worker_threads=48
2025-08-10T13:53:11.676828Z TRACE mio::poll: registering event source with poller: token=Token(1), interests=READABLE
{ "appname": "Serial-Debugger", "facility": "user", "hostname": "10.0.4.87", "message": "Serializer started!", "severity": "info", "timestamp": t'2003-08-24T12:14:15.000003Z', "version": 1 }
function call error for "parse_syslog" at (0:23): unable to parse input as valid syslog message

Example Data

{"message":"<14>1 2003-08-24T05:14:15.000003-07:00 10.0.4.87 Serial-Debugger - - - Serializer started!" }
{"message":"<14>1 - 10.0.4.87 Serial-Debugger - - - Serializer started!" }

Additional Context

No response

References

No response

[jinnatar]

Digging into the code a bit, the syslog_loose crate is the source of the trouble, i.e. that already forces a timestamp to be mandatory. I've logged the upstream issue: StephenWakely/syslog-loose#44

Even if that is fixed though, Vector would still need to substitute time of receipt (of which the details depend entirely on how/if this is fixed upstream).

[jinnatar]

For any poor soul who needs to deal with nil timestamped logs, here's what I did to work around this:

[sources.syslog-udp]
type = "socket"
address = "0.0.0.0:514"
mode = "udp"
[sources.syslog-udp.decoding]
codec = "vrl"
[sources.syslog-udp.decoding.vrl]
source = """
parsed, err = parse_syslog(.message)
if err == null {
  . |= parsed
} else {
  # attempt to "fix" RFC5424 nil timestamp
  timeful, err = replace(.message, "-", format_timestamp!(now(), "%+"), count: 1)
  if err == null {
    parsed, err = parse_syslog(timeful)
    if err == null {
      . |= parsed
    } 
  }
}
"""

[pront]

Hi @jinnatar, thank you for the taking the time to provide all the details and also for providing a workaround. Much appreciated.

[jinnatar]

The upstream library now has a new release that returns a null on a nil timestamp. Didn't get to test it yet, just mentioning in case someone can get to it first.