Backport 8.0.x: firewall: add default-policy to policy config v1 - #15951
Draft
lukashino wants to merge 69 commits into
Draft
Backport 8.0.x: firewall: add default-policy to policy config v1#15951lukashino wants to merge 69 commits into
lukashino wants to merge 69 commits into
Conversation
Add minimal SMTP progress states to support envelope validation before moving to data. Update SMTP, file and email keywords to hook into the appropriate states. Purposefully kept minimal for now as to not break the current idea of an SMTP transaction, which is probably not ideal for firewall mode. Ticket: OISF#8393 (cherry picked from commit c2728ee)
Add directionality to completion states, and replace tx->done by checking for both directions being complete. This means that the transaction is now not complete until the server responds to the clients of data marker, previously the tx was completed when the client send end of data without waiting for the server response. This keeps smtp:response_complete from being exposed before the server response is parsed. Ticket: OISF#8393 (cherry picked from commit 7b31f41)
(cherry picked from commit d81be73)
(cherry picked from commit 0e3c946)
(cherry picked from commit 0123cfd)
Don't allow trailing : (cherry picked from commit 7fe8f63)
It used the alstate name where it meant tx. (cherry picked from commit 280a1e5)
Since pstate can't be NULL, remove the conditional logic. (cherry picked from commit f4f5212)
(cherry picked from commit c1392d9)
(cherry picked from commit 695c9d8)
(cherry picked from commit 3b351e2)
Extend the app-layer-protocol keyword to accept a pipe-separated list of
protocol values, so a single rule can match any of several protocols:
app-layer-protocol:[!]<proto1>|<proto2>[|...][,<qualifier>]...;
A non-negated list matches when the flow's protocol equals any listed value
(OR); a negated list matches when it equals none of them (NOR). The
single-value form and the trailing mode qualifier are unchanged.
Matching keeps the historical AppProtoEquals() equivalences by default
(dns/doh2, http/http1/http2, dcerpc/smb, ...). An `exact` qualifier selects
strict identity matching with no equivalences and no http umbrella; it
combines with a direction mode in any order. Because a flow is never the
generic ALPROTO_HTTP, `http,exact` is rejected at load.
Values are expanded once at rule load into an effective match-set bitmask, so
the per-packet match is a single bitmask test. Single-value rules remain
prefilterable; multi-value rules are excluded from prefiltering and an
explicit prefilter on them is rejected. Conflicting keyword combinations
(duplicate or overlapping negations, mixed positive/negated) are rejected at
load. Engine-analysis reports the effective match set.
Ticket: 7705
(cherry picked from commit 43bc2db)
Ticket: 8518 Keywords that work for HTTP2 headers match now as soon as possible A push promise is now considered like a headers frame with regards to the progress (no dedicated "reserved" progress/state) http.protocol and http.stat_msg keywords are now registered at earliest progress, since these are synthetic like "HTTP/2" and not really seen on the wire. http.request_line and http.response_line match only on data, and not on headers, since we must wait the end of headers to be sure to have the full line http2.size_update now matches at headers progress as it should http2.frametype, http2.errorcode, http2.priority now match like http2.window, when the tx is complete from both sides, as a half-closed client may still send priority, rst_stream or window_update frames (cherry picked from commit daf68dc)
Ticket: 8518 Meaning they will now match only once per tx instead of twice: once for each direction (cherry picked from commit 8eed90c)
Ticket: 8518 (cherry picked from commit 26bb18c)
Allow registration of per substate progress name mappings. Implement logic for getting sub-state names, id's. Add transaction type and end of progress values to AppLayerTxData. Introduce helpers to keep logic clean. (cherry picked from commit b6bc025)
Support for per transaction sub states: different state machines per transaction type. Skip engines not belonging to our substate. Store tx_type in DetectTransaction. Each protocol supporting sub states will register states from 1 and up. To support: Ticket: OISF#8386. (cherry picked from commit 95365ce)
(cherry picked from commit 59e2c5a)
(cherry picked from commit 67e2cb5)
(cherry picked from commit 2f41e0e)
Split into 2 sub-states: - stream, which has the "HTTP" requests and responses, including DOH2 - global, which has the settings and other global or control handling Introduce a simpler progress tracking for the global sub state: - HTTP2ProgGlobalStart and HTTP2ProgGlobalComplete. The stream sub state uses the same state machine as before. Ticket: OISF#8386. (cherry picked from commit 7ebc699)
Hard coded for now. (cherry picked from commit c943de4)
(cherry picked from commit 416b9c1)
Each keyword supporting DOH2 must register explicitly (cherry picked from commit 1bb843d)
Now that DNS keywords are no longer registered for DOH2, the keywords need to manually registered for DOH2. (cherry picked from commit b36ec14)
Now that HTTP keywords are no longer automatically registered for HTTP/2, register them manually. (cherry picked from commit 17953d2)
Check that sub state handling is correct. (cherry picked from commit 2e1003a)
(cherry picked from commit 17a3e0d)
In preparation of being able to log it. (cherry picked from commit 7e60c32)
(cherry picked from commit e1e24d6)
(cherry picked from commit 2511070)
(cherry picked from commit e4176d2)
http2 states were not yet supported even if the built-in states could already work. Clearly error out on a hook w/o substate. (cherry picked from commit 7517ba9)
(cherry picked from commit cfc98ae)
(cherry picked from commit 110e746)
(cherry picked from commit f02a427)
(cherry picked from commit 56d9670)
In firewall mode the alproto logic when building prefilter and rule engines is strict, but was out of sync between prefilter and rule app inspect engines. For SIGNATURE_HOOK_TYPE_APP rules the logic is strict, with an exception for HTTP/2 and DOH2. DOH2 is not a full protocol implementation, but rather HTTP/2 with a different alproto. Fixes: d64954a ("detect: don't register unrelated inspect engines") (cherry picked from commit f8f28c6)
This is to make the hook use names consistent with the default hooks and other protocols. (cherry picked from commit cf216b6)
DOH2 depends on HTTP/2, so it makes no sense to enable it separately. It would also put the sub state handling in a weird state, as the DOH2 side reuses the registered HTTP/2 callbacks. (cherry picked from commit f545f06)
(cherry picked from commit 9a294bf)
(cherry picked from commit c5d4a25)
When building the per signature app_inspect list, only add engines that match the sub state specified in the hook. (cherry picked from commit a824d97)
Register HTTP/2 and DOH2 this way. (cherry picked from commit 89baff0)
Make sure that every buffer in a signature is actually used in the setup of the engines. Reject sigs that are locked to a certain substate and use buffers that require another. (cherry picked from commit 8a5e8c6)
Add substates to the --list-app-layer-hooks option. (cherry picked from commit 8455efd)
(cherry picked from commit f8945e7)
AppProtoToString(ALPROTO_HTTP1) returns "http", so an HTTP/1 policy had to be written as `http:` while its rule hooks were already spelled `http1:`. Use the same name in both places. Ticket: 8770 (cherry picked from commit 5391c44)
The policy config was a flat map mixing packet hooks and app-layer
protocols: `packet-filter` next to `dns`. There was no node that meant
"the packet hooks" or "the app-layer hooks", so a setting could not be
scoped to one group.
Move each group under its own node:
packet-filter -> packet.filter
packet-pre-flow -> packet.pre-flow
packet-pre-stream -> packet.pre-stream
<proto>.<hook> -> app.<proto>.<hook>
Ticket: 8770
(cherry picked from commit 2b17b7e)
Every hook has a built-in default policy, but expressing anything other
than the built-in meant naming each hook explicitly. Add a
`default-policy` setting that covers all hooks below it, so unlisted
hooks still get a policy. For any hook the most specific setting present
wins:
app.<proto>.<sub state>.<hook>
app.<proto>.<sub state>.default-policy
app.<proto>.default-policy
app.default-policy
default-policy
built-in
The packet hooks follow the same pattern under `packet`.
Resolution moves into ResolveFirewallPolicy(), which walks the candidate
paths most-specific-first and stops at the first one that is configured.
A path that is present but empty is now a startup error rather than being
treated as unset.
DoParseAppSubStatePolicy() collapses into DoParseAppPolicy() as a sub state
hook only differs by an extra path segment. Path assembly and hook-name
normalisation move to helpers now that both are needed in more places.
Ticket: 8770
(cherry picked from commit b5ced77)
Validate the resolved scope against the class of hook it is being applied to and fail at startup if it does not fit, naming the config path and the scopes that would be accepted there. A global `accept:tx` is now a startup error. Ticket: 8770 (cherry picked from commit ba26069)
|
Information: ERROR: QA failed on SURI_TLPW2_autofp_suri_time.
Pipeline = 32735 |
This was referenced Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up of #15948
Add a
default-policysetting that applies to all hooks below it. For any hook the most specific setting present wins.The flat layout had no way to express a default covering only the packet hooks or only the app hooks, so both move under their own node:
packet-filterbecomespacket.filterand<proto>becomesapp.<proto>.Action scopes are now validated against the hook they reach. A global
accept:txarriving at a packet hook is a startup error instead of being applied silently.Link to ticket: https://redmine.openinfosecfoundation.org/issues/8770
Describe changes:
SV_BRANCH=OISF/suricata-verify#3255