Add Spam Confidence Score to Detections

[ssejal1508]

Problem / Gap
Currently, SmokeDetector flags posts as spam/non-spam in a binary manner. Moderators and metasmoke reviewers sometimes have to manually assess how “strong” a detection is (e.g., was it caught by 1 weak rule or 5 strong ones?).

Proposed Solution

• Introduce a confidence score (0–100) alongside spam detections.

• Score can be derived from:

  • Number of detection rules triggered
  • Strength/weight of the rules (e.g., hard blacklist = high weight, fuzzy match = low weight)
  • Presence of multiple spam signals (links, repetition, Unicode obfuscation, etc.)

• Display this score in chat reports and in the metasmoke interface.

Benefits

• Helps moderators quickly decide which spam needs urgent review.
• Reduces noise by highlighting high-confidence spam.
• Provides a foundation for future automation (e.g., auto-deletion if score > 90).

Difficulty: Medium
Labels: enhancement, feature-request

Checklist:

• Define rule weights and scoring system
• Implement score calculation in detection pipeline
• Display score in chat reports
• Add support in metasmoke for showing scores
• Document scoring system in README/wiki

[makyen]

It's unclear to me what you're looking for here, but it sounds like "let people not have to think/evaluate the post and be able to just automatically perform an action based on a single number", which doesn't sound like a good idea to me. If we are looking for humans to be able to perform actions without evaluating the post themselves, then the more appropriate action is to just automatically perform the action without the human involved, but that is not permitted (per discussions on MSE).

The total weight of all detections, which is effectively a confidence score (but confidence of it varies by site), is already posted with every report, as are a list of the detections. Those also already displayed on MS nearly everywhere a post is listed (at least for weight).

Actually having a real confidence score, based on the criteria we use for autoflagging, is a non-trivial database lookup within MS. Prior to my disabling it, such calculations, which used to be done on every post in order to determine the number of flags to raise (when we were doing 3 or 4 autoflags out of 6), cost about 70 seconds per post, IIRC, which resulted in our delaying autoflagging by that much on every post.

Now, quite a bit could be done to speed that up. I have some ideas for doing so, and at least some of them would be generally useful, but they would require some database changes, pre-computing things, keeping a Redis cache of values, etc.

I don't really see what we gain from adding this, other than making user/mod actions more automatic, when the user/mod is specifically in the loop in order to have human judgement, as was vociferously argued for on MSE when SD/MS was developed and when we moved from only ever auto-casting 3 (out of 6) red flags to larger number (even though human flagging is less accurate). Moving humans out of the loop, even to the extent of using 5 (out of 6) autoflags was resoundingly rejected.

[vannu07]

Thanks for the detailed feedback, @makyen, and for providing the context on past MSE discussions—that's crucial. I absolutely agree that the goal should not be to replace human judgment or encourage blind automation by mods.

I think the value of this feature lies purely in triage and prioritization, not automation.

1. Triage/Priority Metric: The confidence score wouldn't replace the need for human judgment; rather, it would act as a visual flag for posts where the sheer volume and strength of signals (which are sometimes hard to calculate quickly in the heat of the moment) suggest urgent review is required. The existing "total weight" is a good technical metric, but a 0-100 Confidence Score is a more intuitive triage signal.

2. Performance: The 70-second delay for a real-time database lookup is definitely unacceptable. My proposal is to structure the scoring so that the confidence value is calculated and stored once at the time of detection—similar to how the current rule list is generated—to avoid any heavy database lookups in metasmoke during review. This should have minimal impact on post-processing speed.

3. Refined Scope: The initial problem statement still holds: reviewers currently have to manually mentally combine the signals. This feature just surfaces that combined value to aid in quick decision-making, keeping the human firmly in the loop.

Would a refined scope focusing purely on triage and low-latency pre-computation be a more palatable path forward? I'm happy to explore performance-optimized ways to define and store this value.

[talflon]

As Makyen explained, this feature request seems to misunderstand how the project works. I think that might be because it's from people not involved in Charcoal. If you are active on Stack Overflow / Stack Exchange, you could watch or become active in Charcoal and learn more about it. There's also a website with a wiki.