pfsense flushs crowdsec_blacklists table during applying rules

[ionoci]

What happened?

When I add a new rule or change a rule and apply this change the tables crowdsec_blacklists and crowdsec6_blacklists get flushed and seem to be refilled only by restarting crowdsec_firewall service.

pfsense version: 2.8.1 CE
crowdsec packages: crowdsec-1.7.4, crowdsec-firewall-bouncer-0.0.33, pfSense-pkg-crowdsec-0.1.6
remediation and log processor enabled, remote lapi ist used.

What did you expect to happen?

No change to crowdsec tables.

How can we reproduce it (as minimally and precisely as possible)?

1. check crowdsec_blacklist table are filled pfctl -T show -t crowdsec_blacklists
2. change a rule in ruleset
3. check crowdsec_blacklist table are still filled pfctl -T show -t crowdsec_blacklists

Anything else we need to know?

I'm not quite sure, which logs will be helpfull for this issue.

Crowdsec version

Details

version: v1.7.4-469b374e Codename: alphaga BuildDate: 2025-12-05_17:38:18 GoVersion: 1.25.5 Platform: freebsd libre2: WebAssembly User-Agent: crowdsec/v1.7.4-469b374e-freebsd Constraint_parser: >= 1.0, <= 3.0 Constraint_scenario: >= 1.0, <= 3.0 Constraint_api: v1 Constraint_acquis: >= 1.0, < 2.0 Built-in optional components: cscli_setup, datasource_appsec, datasource_cloudwatch, datasource_docker, datasource_file, datasource_http, datasource_journalctl, datasource_k8s-audit, datasource_kafka, datasource_kinesis, datasource_loki, datasource_s3, datasource_syslog, datasource_victorialogs, datasource_wineventlog, db_mysql, db_postgres, db_sqlite

OS version

Details

FreeBSD xxx 15.0-CURRENT FreeBSD 15.0-CURRENT #21 RELENG_2_8_1-n256095-47c932dcc0e9: Thu Aug 28 16:27:48 UTC 2025     root@pfsense-build-release-amd64-1.eng.atx.netgate.com:/var/jenkins/workspace/pfSense-CE-snapshots-2_8_1-main/obj/amd64/AupY3aTL/var/jenkins/workspace/pfSense-CE-snapshots-2_8_1-main/sources/FreeBSD-src-RELENG_2_8_1/amd64.amd64/sys/pfSense amd64

Enabled collections and parsers

Details

Loaded: 157 parsers, 11 postoverflows, 771 scenarios, 9 contexts, 5 appsec-configs, 166 appsec-rules, 157 collections
Unmanaged items: 0 local, 1 tainted
name,status,version,description,type
crowdsecurity/dateparse-enrich,enabled,0.2,,parsers
crowdsecurity/geoip-enrich,enabled,0.5,"Populate event with geoloc info : as, country, coords, source range.",parsers
crowdsecurity/haproxy-logs,enabled,0.8,Parse haproxy http logs,parsers
crowdsecurity/http-logs,enabled,1.3,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers
crowdsecurity/nginx-logs,enabled,2.0,Parse nginx access and error logs,parsers
crowdsecurity/pfsense-gui-logs,enabled,0.1,Parse pfSense web auth logs,parsers
crowdsecurity/public-dns-allowlist,enabled,0.1,Allow events from public DNS servers,parsers
crowdsecurity/sshd-logs,enabled,3.1,Parse openSSH logs,parsers
crowdsecurity/syslog-logs,enabled,1.0,,parsers
crowdsecurity/whitelists,"enabled,tainted",?,Whitelist events from private ipv4 addresses,parsers
firewallservices/pf-logs,enabled,0.7,Parse packet filter logs,parsers
crowdsecurity/cdn-whitelist,enabled,0.4,Whitelist CDN providers,postoverflows
crowdsecurity/rdns,enabled,0.3,Lookup the DNS associated to the source IP only for overflows,postoverflows
crowdsecurity/seo-bots-whitelist,enabled,0.5,Whitelist good search engine crawlers,postoverflows
crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.7,Detect cve-2021-44228 exploitation attempts,scenarios
crowdsecurity/CVE-2017-9841,enabled,0.2,Detect CVE-2017-9841 exploits,scenarios
crowdsecurity/CVE-2019-18935,enabled,0.2,Detect Telerik CVE-2019-18935 exploitation attempts,scenarios
crowdsecurity/CVE-2022-26134,enabled,0.4,Confluence - RCE (CVE-2022-26134),scenarios
crowdsecurity/CVE-2022-35914,enabled,0.2,Detect CVE-2022-35914 exploits,scenarios
crowdsecurity/CVE-2022-37042,enabled,0.2,Detect CVE-2022-37042 exploits,scenarios
crowdsecurity/CVE-2022-40684,enabled,0.3,Detect cve-2022-40684 exploitation attempts,scenarios
crowdsecurity/CVE-2022-41082,enabled,0.4,Detect CVE-2022-41082 exploits,scenarios
crowdsecurity/CVE-2022-41697,enabled,0.2,Detect CVE-2022-41697 enumeration,scenarios
crowdsecurity/CVE-2022-42889,enabled,0.3,Detect CVE-2022-42889 exploits (Text4Shell),scenarios
crowdsecurity/CVE-2022-44877,enabled,0.3,Detect CVE-2022-44877 exploits,scenarios
crowdsecurity/CVE-2022-46169,enabled,0.2,Detect CVE-2022-46169 brute forcing,scenarios
crowdsecurity/CVE-2023-22515,enabled,0.1,Detect CVE-2023-22515 exploitation,scenarios
crowdsecurity/CVE-2023-22518,enabled,0.3,Detect CVE-2023-22518 exploits,scenarios
crowdsecurity/CVE-2023-49103,enabled,0.3,Detect owncloud CVE-2023-49103 exploitation attempts,scenarios
crowdsecurity/CVE-2024-0012,enabled,0.1,Detect CVE-2024-0012 exploitation attempts,scenarios
crowdsecurity/CVE-2024-38475,enabled,0.1,Detect CVE-2024-38475 exploitation attempts,scenarios
crowdsecurity/CVE-2024-9474,enabled,0.1,Detect CVE-2024-9474 exploitation attempts,scenarios
crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.3,F5 BIG-IP TMUI - RCE (CVE-2020-5902),scenarios
crowdsecurity/fortinet-cve-2018-13379,enabled,0.4,Detect cve-2018-13379 exploitation attempts,scenarios
crowdsecurity/grafana-cve-2021-43798,enabled,0.3,Grafana - Arbitrary File Read (CVE-2021-43798),scenarios
crowdsecurity/http-admin-interface-probing,enabled,0.5,Detect generic HTTP admin interface probing,scenarios
crowdsecurity/http-backdoors-attempts,enabled,0.6,Detect attempt to common backdoors,scenarios
crowdsecurity/http-bad-user-agent,enabled,1.2,Detect usage of bad User Agent,scenarios
crowdsecurity/http-crawl-non_statics,enabled,0.7,Detect aggressive crawl on non static resources,scenarios
crowdsecurity/http-cve-2021-41773,enabled,0.3,Apache - Path Traversal (CVE-2021-41773),scenarios
crowdsecurity/http-cve-2021-42013,enabled,0.3,Apache - Path Traversal (CVE-2021-42013),scenarios
crowdsecurity/http-cve-probing,enabled,0.6,Detect generic HTTP cve probing,scenarios
crowdsecurity/http-generic-bf,enabled,0.9,Detect generic http brute force,scenarios
crowdsecurity/http-generic-test,enabled,0.2,Crowdsec Generic Test Scenario: basic HTTP trigger,scenarios
crowdsecurity/http-open-proxy,enabled,0.5,Detect scan for open proxy,scenarios
crowdsecurity/http-path-traversal-probing,enabled,0.4,Detect path traversal attempt,scenarios
crowdsecurity/http-probing,enabled,0.4,Detect site scanning/probing from a single ip,scenarios
crowdsecurity/http-sap-interface-probing,enabled,0.1,Detect generic HTTP SAP interface probing,scenarios
crowdsecurity/http-sensitive-files,enabled,0.4,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios
crowdsecurity/http-sqli-probing,enabled,0.4,A scenario that detects SQL injection probing with minimal false positives,scenarios
crowdsecurity/http-wordpress-scan,enabled,0.4,Detect exploitation attempts against common WordPress endpoints,scenarios
crowdsecurity/http-xss-probing,enabled,0.4,A scenario that detects XSS probing with minimal false positives,scenarios
crowdsecurity/jira_cve-2021-26086,enabled,0.4,Detect Atlassian Jira CVE-2021-26086 exploitation attempts,scenarios
crowdsecurity/netgear_rce,enabled,0.4,Detect Netgear RCE DGN1000/DGN220 exploitation attempts,scenarios
crowdsecurity/nginx-req-limit-exceeded,enabled,0.3,Detects IPs which violate nginx's user set request limit.,scenarios
crowdsecurity/pfsense-gui-bf,enabled,0.2,Detect bruteforce on pfsense web interface,scenarios
crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.4,Detect cve-2019-11510 exploitation attempts,scenarios
crowdsecurity/spring4shell_cve-2022-22965,enabled,0.3,Detect cve-2022-22965 probing,scenarios
crowdsecurity/ssh-bf,enabled,0.3,Detect ssh bruteforce,scenarios
crowdsecurity/ssh-cve-2024-6387,enabled,0.2,Detect exploitation attempt of CVE-2024-6387,scenarios
crowdsecurity/ssh-generic-test,enabled,0.2,Crowdsec Generic Test Scenario: SSH brute force trigger,scenarios
crowdsecurity/ssh-refused-conn,enabled,0.1,Detect sshd refused connections,scenarios
crowdsecurity/ssh-slow-bf,enabled,0.4,Detect slow ssh bruteforce,scenarios
crowdsecurity/thinkphp-cve-2018-20062,enabled,0.7,Detect ThinkPHP CVE-2018-20062 exploitation attempts,scenarios
crowdsecurity/vmware-cve-2022-22954,enabled,0.3,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios
crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.3,Detect VMSA-2021-0027 exploitation attempts,scenarios
firewallservices/pf-scan-multi_ports,enabled,0.5,Detect aggressive portscans (pf),scenarios
ltsich/http-w00tw00t,enabled,0.3,detect w00tw00t,scenarios
crowdsecurity/bf_base,enabled,0.1,,contexts
crowdsecurity/firewall_base,enabled,0.2,,contexts
crowdsecurity/http_base,enabled,0.3,,contexts
crowdsecurity/base-http-scenarios,enabled,1.2,http common : scanners detection,collections
crowdsecurity/freebsd,enabled,0.4,core freebsd support : syslog+geoip+ssh,collections
crowdsecurity/haproxy,enabled,0.1,haproxy support : parser and generic http scenarios,collections
crowdsecurity/http-cve,enabled,2.9,Detect CVE exploitation in http logs,collections
crowdsecurity/nginx,enabled,0.2,nginx support : parser and generic http scenarios,collections
crowdsecurity/pfsense,enabled,0.2,core pfsense support,collections
crowdsecurity/pfsense-gui,enabled,0.1,pfSense web authentication support,collections
crowdsecurity/sshd,enabled,0.7,sshd support : parser and brute-force detection,collections
crowdsecurity/whitelist-good-actors,enabled,0.2,Good actors whitelists,collections
firewallservices/pf,enabled,0.2,Parser and scenario for Packet Filter logs,collections

Acquisition config

Details

```console filenames: - /var/log/nginx/*.log - ./tests/nginx/nginx.log #this is not a syslog log, indicate which kind of logs it is labels: type: nginx --- filenames: - /var/log/auth.log - /var/log/syslog labels: type: syslog --- filenames: - /var/log/httpd-access.log - /var/log/httpd-error.log labels: type: apache2 filenames: - /var/log/haproxy.log force_inotify: true poll_without_inotify: true labels: type: haproxy # DO NOT EDIT - to add new datasources (log locations), # create new files in /usr/local/etc/crowdsec/acquis.d/

filenames:

This should cover crowdsecurity/sshd and crowdsecurity/pfsense-gui,

and is already defined in acquis.yaml

- /var/log/auth.log

collection: firewallservices/pf

• /var/log/filter.log

collection: crowdsecurity/nginx

• /var/log/nginx.log

If /var/log is in RAM, the log directories are created after crowdsec is run.

We force crowdsec to watch over directory creation as well

as file creation. FreeBSD has kqueue instead of inotify

but the option works with both.

force_inotify: true

This option is required from crowdsec v1.5.0 to follow

changes in symlinks. We usually don't have them in pfSense,

but let's stay on the safe side.

poll_without_inotify: true

labels:
type: syslog

</details>


### Config show

<details>

```console
Global:
   - Configuration Folder   : /usr/local/etc/crowdsec
   - Data Folder            : /var/db/crowdsec/data
   - Hub Folder             : /usr/local/etc/crowdsec/hub
   - Notification Folder    : /usr/local/etc/crowdsec/notifications
   - Simulation File        : /usr/local/etc/crowdsec/simulation.yaml
   - Log Folder             : /var/log/crowdsec
   - Log level              : info
   - Log Media              : file
Crowdsec:
  - Acquisition File        : /usr/local/etc/crowdsec/acquis.yaml
  - Parsers routines        : 1
  - Acquisition Folder      : /usr/local/etc/crowdsec/acquis.d
cscli:
  - Output                  : human
  - Hub Branch              : 
API Client:
  - URL                     : http://192.168.178.77:8080/
  - Login                   : pfsense
  - Credentials File        : /usr/local/etc/crowdsec/local_api_credentials.yaml
Local API Server (disabled):
  - Listen URL              : 127.0.0.1:8088
  - Listen Socket           : 
  - Profile File            : /usr/local/etc/crowdsec/profiles.yaml

  - Trusted IPs:
      - 127.0.0.1
      - ::1
  - Database:
      - Type                : sqlite
      - Path                : /var/db/crowdsec/data/crowdsec.db
      - Flush age           : 168h0m0s
      - Flush size          : 5000

Prometheus metrics

Details

│ Acquisition Metrics                                                                                                 │
├───────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────┤
│ Source                    │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├───────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
│ file:/var/log/filter.log  │ 1.55k      │ 411          │ 1.14k          │ 96                     │ 26                │
│ file:/var/log/haproxy.log │ 314        │ -            │ 314            │ -                      │ -                 │
│ file:/var/log/nginx.log   │ 1.35k      │ 1.35k        │ 1              │ -                      │ 1.35k             │
╰───────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────┴───────────────────╯
╭────────────────────────────────────────────────────────────────╮
│ Parser Metrics                                                 │
├────────────────────────────────────┬───────┬────────┬──────────┤
│ Parsers                            │ Hits  │ Parsed │ Unparsed │
├────────────────────────────────────┼───────┼────────┼──────────┤
│ child-crowdsecurity/haproxy-logs   │ 628   │ -      │ 628      │
│ child-crowdsecurity/http-logs      │ 4.04k │ 2.76k  │ 1.28k    │
│ child-crowdsecurity/nginx-logs     │ 1.35k │ 1.35k  │ -        │
│ child-crowdsecurity/syslog-logs    │ 2.90k │ 2.90k  │ -        │
│ crowdsecurity/dateparse-enrich     │ 1.76k │ 1.76k  │ -        │
│ crowdsecurity/geoip-enrich         │ 385   │ 385    │ -        │
│ crowdsecurity/haproxy-logs         │ 314   │ -      │ 314      │
│ crowdsecurity/http-logs            │ 1.35k │ 1.35k  │ -        │
│ crowdsecurity/nginx-logs           │ 1.35k │ 1.35k  │ -        │
│ crowdsecurity/non-syslog           │ 314   │ 314    │ -        │
│ crowdsecurity/public-dns-allowlist │ 1.76k │ 1.76k  │ -        │
│ crowdsecurity/syslog-logs          │ 2.90k │ 2.90k  │ -        │
│ crowdsecurity/whitelists           │ 1.76k │ 1.76k  │ -        │
│ firewallservices/pf-logs           │ 1.55k │ 464    │ 1.08k    │
│ firewallservices/pf-logs-drop      │ 411   │ 411    │ -        │
╰────────────────────────────────────┴───────┴────────┴──────────╯
╭────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Scenario Metrics                                                                                   │
├──────────────────────────────────────┬───────────────┬───────────┬──────────────┬────────┬─────────┤
│ Scenario                             │ Current Count │ Overflows │ Instantiated │ Poured │ Expired │
├──────────────────────────────────────┼───────────────┼───────────┼──────────────┼────────┼─────────┤
│ firewallservices/pf-scan-multi_ports │ 8             │ -         │ 89           │ 96     │ 81      │
╰──────────────────────────────────────┴───────────────┴───────────┴──────────────┴────────┴─────────╯
╭───────────────────────────────────────────────────────────────────────────────────────╮
│ Whitelist Metrics                                                                     │
├────────────────────────────────────┬─────────────────────────────┬──────┬─────────────┤
│ Whitelist                          │ Reason                      │ Hits │ Whitelisted │
├────────────────────────────────────┼─────────────────────────────┼──────┼─────────────┤
│ crowdsecurity/public-dns-allowlist │ public DNS server           │ 1758 │ -           │
│ crowdsecurity/whitelists           │ private ipv4/ipv6 ip/ranges │ 1758 │ 1373        │
╰────────────────────────────────────┴─────────────────────────────┴──────┴─────────────╯

[github-actions]

@ionoci: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.