Skip to content

Prepend necessary conditions to query #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Conversation

arblade
Copy link
Contributor

@arblade arblade commented Apr 4, 2025

As tackled in #52 and #51, this PR is a proposal for fixing errors encountered in regex oring and in correlations rules.
For details see these issues.

Description

To summarize, we have two problems:

  • correlation rules delete deferred expressions (ex: if you put a |re in a correlation rule, the field targeted will disappear in the result query)
  • oring regex (and cidr regex) need to compute custom fields using | eval parts, and then handle the logic using a final | search. However, splunk need to get something at the beginning of the query (just before the first |)

for instance, currently, regex oring is giving use something like | rex .. | eval ... | rex ... | eval ... | search ..., and we need to get some expressions in front of the | rex to get some logs to work on, without that this will always return nothing.

Proposal

So, I implemented a way to detect necessary conditions (understand conditions which doesn't have an OR in their parents), and as AND operator is commutative, deferred them and add them at the start of the query in the finalize_query process.

This will :

  • fix correlations rules with deferred parts
  • fix oring regex and cidr
  • make them more efficient (by for instance computing regexes only on a smaller batch of logs)

Caution

This PR need pysigma to reactivate the finalize_query for rules that are part of correlation rules, as described in #52.

@arblade
Copy link
Contributor Author

arblade commented Apr 4, 2025

All tests are passing if we reactivate finalyze_query for all rules in pysigma by deleting these lines.

@thomaspatzke
Copy link
Member

Before merge and changing pySigma I have to check if this was something required by another backend or for some other reason or if this is just a bug. Possibly this will end up as switch between both behaviors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants