Skip to content

Support testing timestamping that is applied using an eval during indexing #402

Open
@ibilling-splunk

Description

@ibilling-splunk

It seems to us that the framework is not really testing timestamp extraction as the _time test will always pass regardless of the sample and props&transforms.

Our sample:
{"RoleLocation":"South Central US","time":"##Timestamp##"}

Our transform:

INGEST_EVAL = _time=coalesce(strptime(spath(_raw, "time"), "%Y-%m-%dT%H:%M:%S.%QZ"),strptime(spath(_raw, "time"), "%m/%d/%Y %l:%M:%S %p"),_time)

Our test(token.0.replacement was intentionally set to a bad value to prove the test will always pass):

[azure_activity_log.sample]
host_type = plugin

input_type = modinput
index = main
sourcetype = azure:activity
sourcetype_to_search = azure:activity
sample_count = 1
expected_event_count = 1

timestamp_type = event

token.0.token = ##Timestamp##
token.0.replacementType = timestamp
token.0.replacement = a
token.0.field = _time

earliest = -30m
latest = -30m

See picture for Ingested event with correct timestamp(now-30min) specified by test but neither is in sample nor ingest time.
image
No props&transforms were added when testing.
All tests passed

@rfaircloth-splunk 's suggestion is to have a new timestamp_type that:

this would need an option for "none" or "random" so that the even level ts is either not sent or sent invalid to confirm the eval itself works

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions