Captcha support

[Simbiat]

My config has custom error pages, but looks like they are not being applied if access is blocked by CrowdSec (they do get applied if it's done by Coraza, though). Instead I am seeing generic Caddy's page (not the one from CrowdSec either). Furthermore if I block IP with "captcha" instead of "ban" I am still getting 403 error, instead of expected 401.
Is this some limitation of the plugin or am I doing something wrong?

[LaurenceJJones]

Seems captcha is a TODO:

caddy-crowdsec-bouncer/http/http.go

Lines 127 to 130 in 6022901

	func writeCaptchaResponse(w http.ResponseWriter) error {
	// TODO: implement showing a captcha in some way. How? hCaptcha? And how to handle afterwards?
	return writeBanResponse(w)
	}

There is no custom pages, so this needs to be implemented also as since crowdsec directive acts as a middleware it stops the chaining input. I dont know how the coraza implementation works to get around this.

[hslatman]

That's quite the config, @Simbiat 😛

Yes, captcha has been on my list for a while now. Also see my not-so-strict TODO list. Main thing to decide is what technology to use, as I don't want to force people to use reCaptcha, for example. Ideally I would like to make it pluggable.

Without looking much deeper into the code, I think this might be what makes the error pages work with Coraza: https://github.com/corazawaf/coraza-caddy/blob/main/coraza.go#L116-L131.

[LaurenceJJones]

Without looking much deeper into the code, I think this might be what makes the error pages work with Coraza: https://github.com/corazawaf/coraza-caddy/blob/main/coraza.go#L116-L131.

Ahhh so returning that type / struct informs caddy to trigger the pages it has in it directives interesting.

[Simbiat]

If you strip forced MIME types from my config it will be much more tame 😅

I can confirm, that Coraza seems to return something about "Interruption", which I can get from error messages in Caddy, which matches errInterruptionTriggered. So, I guess it's just required to return caddyhttp.HandlerError then. If that's the case, then it could solve the "pluggable" CAPTCHA, too: return 401 for CAPTCHA and 403 for everything else (optionally allow customization), and let users handle 401 with whatever CAPTCHA thing they want.

The only other thing that would be useful here is to have some variable to indicate that 403/401 was caused by CrowdSec, and ideally some details on what triggered the interruption (same as what I've asked for for Coraza module here).

[hslatman]

I can confirm, that Coraza seems to return something about "Interruption", which I can get from error messages in Caddy, which matches errInterruptionTriggered. So, I guess it's just required to return caddyhttp.HandlerError then. If that's the case, then it could solve the "pluggable" CAPTCHA, too: return 401 for CAPTCHA and 403 for everything else (optionally allow customization), and let users handle 401 with whatever CAPTCHA thing they want.

I can see that working, but I think you'll still want that to be connected to the fact that it was CrowdSec that decided a specific action was needed, vs. some other reason a 401 was returned.

The only other thing that would be useful here is to have some variable to indicate that 403/401 was caused by CrowdSec, and ideally some details on what triggered the interruption (same as what I've asked for for Coraza module here).

Yes, I think this is useful, and also ties in with my previous line. I'll see what I can do 🙂

[LaurenceJJones]

I can see that working, but I think you'll still want that to be connected to the fact that it was CrowdSec that decided a specific action was needed, vs. some other reason a 401 was returned.

I agree with @hslatman Captcha is alot more involved, you need to track the request / responses from the Captcha itself rather than looking just for 401 since you need to know when to validate them with the provider. It's something similar I have done on HAProxy SPOA which is using a session cookie which has an internal state.

[Aetylus]

I don't know how idiomatic or kosher this would be in regards to implementing CrowdSec's captcha decision, but would implementing Anubis or something similar be feasible?

At a high level, without looking into it in more depth myself, it feels like it could be simpler because you don't need to track a captcha response back. And I would think that, in general, if the idea of a captcha is to distinguish between a possible bot and a human, then something designed to prevent bots from scraping content would at some level fulfill a similar enough purpose?

This is more food for thought that might be worth looking into rather than a suggestion. I did come across an blog post here that reports Anubis as successfully preventing a DDoS as well, which feels like at least part of the reason one might want to have a captcha to begin with.

[LaurenceJJones]

I don't know how idiomatic or kosher this would be in regards to implementing CrowdSec's captcha decision, but would implementing Anubis or something similar be feasible?

At a high level, without looking into it in more depth myself, it feels like it could be simpler because you don't need to track a captcha response back. And I would think that, in general, if the idea of a captcha is to distinguish between a possible bot and a human, then something designed to prevent bots from scraping content would at some level fulfill a similar enough purpose?

This is more food for thought that might be worth looking into rather than a suggestion. I did come across an blog post here that reports Anubis as successfully preventing a DDoS as well, which feels like at least part of the reason one might want to have a captcha to begin with.

Yes if your aim is to filter between bots and humans then Anubis is probably the better candidate as it screens all users. However, if your aim is to only screen malicious IPs then Anubis cant check our decisions and only impose on those requests / users.

I havent looked much into Anubis but AFAIK you would have to spin up one per application as it meant to be a proxy in between your reverse proxy and the application so little more overhead but I have seen it quite performant and easy to use.

[Simbiat]

@LaurenceJJones, I think @Aetylus meant to suggest, that if Crowdsec captcha is supposed to be triggered - pass it to Anubis. Which , probably, could make sense, it's like another layer of protection, after all. But that would require either a separate Caddy module (which was requested in TecharoHQ/anubis#16 and is being worked in https://github.com/daegalus/caddy-anubis) or an integration with Anubis on Crowdsec side. I think neither is relevant to the bouncer module, though, at least not directly.

[LaurenceJJones]

@LaurenceJJones, I think @Aetylus meant to suggest, that if Crowdsec captcha is supposed to be triggered - pass it to Anubis. Which , probably, could make sense, it's like another layer of protection, after all. But that would require either a separate Caddy module (which was requested in TecharoHQ/anubis#16 and is being worked in daegalus/caddy-anubis) or an integration with Anubis on Crowdsec side. I think neither is relevant to the bouncer module, though, at least not directly.

Those issues are asking to use Anubis as a module so you dont have to run an external anubis as a proxy as I understand. The goal of Anubis is you send all traffic to it so it can screen every user, as then you would have the most chance to catch bots as only picking and choosing which traffic to send means you could get false negatives. However, its just another layer in the security onion so if you do get false negatives you could then pick them up on behavior from CrowdSec.

[Aetylus]

@Simbiat is correct in what I was thinking about. While I think the idea of Anubis is to have it on your proxy for all requests, the request flow I'm more envisioning is:

1. Request comes into Caddy
2. Caddy CrowdSec module parses the request and sends it to the LAPI to get a decision
3. Caddy bouncer handles decision
   3a. If ban decision, Caddy handles the ban request as the module already functions
   3b. If captcha decision, Caddy forwards request to some proof-of-work, whether that's Anubis or something like the Cerberus module mentioned in A module for Caddy TecharoHQ/anubis#16
   4b. Anubis/Cerberus/some PoW module returns request to the rest of the route flow(?) or handles forwarding the request onwards to its destination itself

This is overly simplified, and this does in fact circumvent in some ways the purpose of Anubis by only directing some traffic to it rather than all. In this case, the PoW is acting as a faux captcha in the sense that it's functioning as a system to deter bots but isn't actually a captcha itself.

[Simbiat]

Personally, I think it should be fine to have Anubis here as a layer to analyze only part of the requests. If I understand correctly, Crowdsec bans based on known bad IPs and known patterns, "known" being the keyword. But PoW (and Captcha) is meant more for suspicious behavior which may be malicious or may be not. If Crowdsec blocks the request we already know that it's a bad one, if not, then we may want to check if through PoW logic to determine if it is legitimate, and if not - block it. That way we should waste less resources on PoW logic, since we are not covering everything.

But anyway, I do not think this is relevant to this issue or to Crowdsec/Appse implementation in a Caddy module. I think it's more of a question of what we do with failed requests, and that should be handled in #77. This issue is more about native captcha support, since Crowdsec does have that functionality in it, if I understand correctly. Perhaps I should rename the issue to make it a bit clearer.

On the other hand, it's possible that implementing proper bubbling and sharing of Crowdsec decision details would make this issue obsolete entirely, because you will be able to then process the request using whatever other module, and you can watch for ban/captcha in the decision, I guess.