feat: first implementation of honeypot logic (#1342)

* feat: first implementation of honeypot logic

This is a bit of an experiment, stick with me.

The core idea here is that badly written crawlers are that: badly
written. They look for anything that contains `<a href="whatever" />`
tags and will blindly use those values to recurse. This takes advantage
of that by hiding a link in a `<script>` tag like this:

```html
<script type="ignore"><a href="/bots-only">Don't click</a></script>
```

Browsers will ignore it because they have no handler for the "ignore"
script type.

This current draft is very unoptimized (it takes like 7 seconds to
generate a page on my tower), however switching spintax libraries will
make this much faster.

The hope is to make this pluggable with WebAssembly such that we force
administrators to choose a storage method. First we crawl before we
walk.

The AI involvement in this commit is limited to the spintax in
affirmations.txt, spintext.txt, and titles.txt. This generates a bunch
of "pseudoprofound bullshit" like the following:

> This Restoration to Balance & Alignment
>
> There's a moment when creators are being called to realize that the work
> can't be reduced to results, but about energy. We don't innovate products
> by pushing harder, we do it by holding the vision. Because momentum can't
> be forced, it unfolds over time when culture are moving in the same
> direction. We're being invited into a paradigm shift in how we think
> about innovation. [...]

This is intended to "look" like normal article text. As this is a first
draft, this sucks and will be improved upon.

Assisted-by: GLM 4.6, ChatGPT, GPT-OSS 120b
Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(honeypot/naive): optimize hilariously

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(honeypot/naive): attempt to automatically filter out based on crawling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(lib): use mazeGen instead of bsGen

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: add honeypot docs

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(test): go mod tidy

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: fix spelling metadata

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>

Author: Xe

.github/actions/spelling/allow.txt

@@ -12,3 +12,9 @@ maintnotifications
 azurediamond
 cooldown
 verifyfcrdns
+Spintax
+spintax
+clampip
+pseudoprofound
+reimagining
+iocaine

.github/actions/spelling/expect.txt

@@ -1,4 +1,3 @@
-
 acs
 Actorified
 actorifiedstore
@@ -398,3 +397,13 @@ Zenos
 zizmor
 zombocom
 zos
+GLM
+iocaine
+nikandfor
+pagegen
+pseudoprofound
+reimagining
+Rhul
+shoneypot
+spammer
+Y'shtola

docs/docs/CHANGELOG.md

@@ -28,6 +28,12 @@ Anubis is back and better than ever! Lots of minor fixes with some big ones inte
 - Open Graph passthrough now reuses the configured target Host/SNI/TLS settings, so metadata fetches succeed when the upstream certificate differs from the public domain. ([1283](https://github.com/TecharoHQ/anubis/pull/1283))
 - Stabilize the CVE-2025-24369 regression test by always submitting an invalid proof instead of relying on random POW failures.
 
+### Dataset poisoning
+
+Anubis has the ability to engage in [dataset poisoning attacks](https://www.anthropic.com/research/small-samples-poison) using the [dataset poisoning subsystem](./admin/honeypot/overview.mdx). This allows every Anubis instance to be a honeypot to attract and flag abusive scrapers so that no administrator action is required to ban them.
+
+There is much more information about this feature in [the dataset poisoning subsystem documentation](./admin/honeypot/overview.mdx). Administrators that are interested in learning how this feature works should consult that documentation.
+
 ### Deprecate `report_as` in challenge configuration
 
 Previously Anubis let you lie to users about the difficulty of a challenge to interfere with operators of malicious scrapers as a psychological attack:

docs/docs/admin/honeypot/_category_.json

@@ -0,0 +1,8 @@
+{
+  "label": "Honeypot",
+  "position": 40,
+  "link": {
+    "type": "generated-index",
+    "description": "Honeypot features in Anubis, allowing Anubis to passively detect malicious crawlers."
+  }
+}

docs/docs/admin/honeypot/overview.mdx

@@ -0,0 +1,40 @@
+---
+title: Dataset poisoning
+---
+
+Anubis offers the ability to participate in [dataset poisoning](https://www.anthropic.com/research/small-samples-poison) attacks similar to what [iocaine](https://iocaine.madhouse-project.org/) and other similar tools offer. Currently this is in a preview state where a lot of details are hard-coded in order to test the viability of this approach.
+
+In essence, when Anubis challenge and error pages are rendered they include a small bit of HTML code that browsers will ignore but scrapers will interpret as a link to ingest. This will then create a small forest of recursive nothing pages that are designed according to the following principles:
+
+- These pages are _cheap_ to render, rendering in at most ten milliseconds on decently specced hardware.
+- These pages are _vacuous_, meaning that they essentially are devoid of content such that a human would find it odd and click away, but a scraper would not be able to know that and would continue through the forest.
+- These pages are _fairly large_ so that scrapers don't think that the pages are error pages or are otherwise devoid of content.
+- These pages are _fully self-contained_ so that they load fast without incurring additional load from resource fetches.
+
+In this limited preview state, Anubis generates pages using [spintax](https://outboundly.ai/blogs/what-is-spintax-and-how-to-use-it/). Spintax is a syntax that is used to create different variants of utterances for use in marketing messages and email spam that evades word filtering. In its current form, Anubis' dataset poisoning has AI generated spintax that generates vapid LinkedIn posts with some western occultism thrown in for good measure. This results in utterances like the following:
+
+> There's a moment when visionaries are being called to realize that the work can't be reduced to optimization, but about resonance. We don't transform products by grinding endlessly, we do it by holding the vision. Because meaning can't be forced, it unfolds over time when culture are in integrity. This moment represents a fundamental reimagining in how we think about work. This isn't a framework, it's a lived truth that requires courage. When we get honest, we activate nonlinear growth that don't show up in dashboards, but redefine success anyway.
+
+This should be fairly transparent to humans that this is pseudoprofound anti-content and is a signal to click away.
+
+## Plans
+
+Future versions of this feature will allow for more customization. In the near future this will be configurable via the following mechanisms:
+
+- WebAssembly logic for customizing how the poisoning data is generated (with examples including the existing spintax method).
+- Weight thresholds and logic for how they are interpreted by Anubis.
+- Other configuration settings as facts and circumstances dictate.
+
+## Implementation notes
+
+In its current implementation, the Anubis dataset poisoning feature has the following flaws that may hinder production deployments:
+
+- All Anubis instances use the same method for generating dataset poisoning information. This may be easy for malicious actors to detect and ignore.
+- Anubis dataset poisoning routes are under the `/.within.website/x/cmd/anubis` URL hierarchy. This may be easy for malicious actors to detect and ignore.
+
+Right now Anubis assigns 30 weight points if the following criteria are met:
+
+- A client's User-Agent has been observed in the dataset poisoning maze at least 25 times.
+- The network-clamped IP address (/24 for IPv4 and /48 for IPv6) has been observed in the dataset poisoning maze at least 25 times.
+
+Additionally, when any given client by both User-Agent and network-clamped IP address has been observed, Anubis will emit log lines warning about it so that administrative action can be taken up to and including [filing abuse reports with the network owner](/blog/2025/file-abuse-reports).

go.mod

@@ -20,6 +20,7 @@ require (
 	github.com/joho/godotenv v1.5.1
 	github.com/lum8rjack/go-ja4h v0.0.0-20250828030157-fa5266d50650
 	github.com/nicksnyder/go-i18n/v2 v2.6.0
+	github.com/nikandfor/spintax v0.0.0-20181023094358-fc346b245bb3
 	github.com/playwright-community/playwright-go v0.5200.1
 	github.com/prometheus/client_golang v1.23.2
 	github.com/redis/go-redis/v9 v9.17.2

go.sum

@@ -320,6 +320,8 @@ github.com/natefinch/atomic v1.0.1 h1:ZPYKxkqQOx3KZ+RsbnP/YsgvxWQPGxjC0oBt2AhwV0
 github.com/natefinch/atomic v1.0.1/go.mod h1:N/D/ELrljoqDyT3rZrsUmtsuzvHkeB/wWjHV22AZRbM=
 github.com/nicksnyder/go-i18n/v2 v2.6.0 h1:C/m2NNWNiTB6SK4Ao8df5EWm3JETSTIGNXBpMJTxzxQ=
 github.com/nicksnyder/go-i18n/v2 v2.6.0/go.mod h1:88sRqr0C6OPyJn0/KRNaEz1uWorjxIKP7rUUcvycecE=
+github.com/nikandfor/spintax v0.0.0-20181023094358-fc346b245bb3 h1:foZ9X1bz2KmW7b8Yx5V0LAQKhTazdllv5rnGUe6iGTY=
+github.com/nikandfor/spintax v0.0.0-20181023094358-fc346b245bb3/go.mod h1:wwDYKfVF3WHdY0rugsAZoIpyQjDA3bn9wEzo/QXPx1Y=
 github.com/onsi/gomega v1.35.1 h1:Cwbd75ZBPxFSuZ6T+rN/WCb/gOc6YgFBXLlZLhC7Ds4=
 github.com/onsi/gomega v1.35.1/go.mod h1:PvZbdDc8J6XJEpDK4HCuRBm8a6Fzp9/DmhC9C7yFlog=
 github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=

internal/clampip.go

@@ -0,0 +1,33 @@
+package internal
+
+import "net/netip"
+
+func ClampIP(addr netip.Addr) (netip.Prefix, bool) {
+	switch {
+	case addr.Is4():
+		result, err := addr.Prefix(24)
+		if err != nil {
+			return netip.Prefix{}, false
+		}
+		return result, true
+
+	case addr.Is4In6():
+		// Extract the IPv4 address from IPv4-mapped IPv6 and clamp it
+		ipv4 := addr.Unmap()
+		result, err := ipv4.Prefix(24)
+		if err != nil {
+			return netip.Prefix{}, false
+		}
+		return result, true
+
+	case addr.Is6():
+		result, err := addr.Prefix(48)
+		if err != nil {
+			return netip.Prefix{}, false
+		}
+		return result, true
+
+	default:
+		return netip.Prefix{}, false
+	}
+}