Skip to content

Problem with import of commented rules #62

Description

@regit

Rulezet is importing commented rules from ET Open and there is no way to tell they are commented in the user interface.

ET Open ships thousands of # alert … lines — signatures the ET team wants distributed for reference or opt-in enablement, but not enabled by default. Common reasons documented in the ruleset:

  • Cost: The rule is known to be expensive. Enabling it against high-volume traffic hurts sensor throughput. Prominent examples: ET INFO SSN Detected in Clear Text (dashed) (sid 2001328), the ET INFO Credit Card Number Detected in Clear family (sids 2001375/2001378/2001384) — payload-content pcre detectors that scan every packet.
  • Locality / policy sensitivity: The rule detects behavior that is only interesting in specific environments (compliance policies, insider-threat scenarios). Not appropriate for a default-enabled global ruleset.
  • False-positive prone: The rule matches a broad substring or a heuristic that fires on both benign and malicious traffic.

By shipping these signatures disabled, ET lets consumers opt in with knowledge of the trade-offs. Rulezet's re-enabling erases that opt-in.

Runtime consequence

Comparing PCAP replay time (same 310 MB / 733,721-packet PCAP, Section 1 methodology) with and without the uncommented ET slice:

Ruleset variant Sigs loaded Load-free replay time
rulezet-as-shipped (rulezet-native + uncommented ET) ~26,289 5.789 s
rulezet-native only (ET-numbered removed) 8,031 2.473 s
ET Open (for reference) 51,465 2.452 s
ET Pro (for reference) 94,066 2.476 s

The uncommented ET slice roughly triples rulezet's replay cost. Once removed, rulezet-native runs at the same speed as ET Open and ET Pro — the runtime penalty of "adopting rulezet" is entirely the ET-uncomment issue, not the community-authored content.

Runtime consequence, sharpened: top-100 offenders

The top-100 most CPU-expensive rules under rulezet-as-shipped:

  • 97.6% of top-100 ticks (18.5 B of 18.9 B) come from ET-numbered SIDs — nearly all of them from the uncommented slice.
  • 89.8% of top-100 ticks are wasted (rules that scan traffic without producing a single alert on this PCAP). By comparison, ET Open's top-100 wastes 69.9%.
  • The top 4 are ET INFO SSN / ET INFO Credit Card Number — the exact rules ET Open ships commented out for performance reasons.

Conclusion

We need a way to keep the opt-in when user build a ruleset as putting them in production can result in a severe performance issue. We could think about a explicit tag set on the commented signatures at import but it would not be really opt-in.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions