Commit ecf668e
committed
analyzer: Improve detection algorithms and refactor HTML rendering
Why:
The analyzer previously treated all sentence differences as binary
(missing or added), failing to capture how Grokipedia often rewords
Wikipedia content rather than omitting or fabricating it entirely.
This made it impossible to distinguish benign paraphrasing from
misleading alterations. Additionally, HTML rendering logic was tightly
coupled with command code, hindering maintainability, and the content
hash calculation relied on a deprecated field.
What:
Algorithmic improvements:
- Add detectRewordedSentences() with 65% similarity threshold
- Add detectAgreedSentences() for exact sentence matching
- Add detectFactualErrors() analyzing numeric/entity discrepancies
- Enhance detectHallucinationEvents() with speculative keywords
- Add reconstructTextFromStructured() to rebuild article text
- Update confidence scoring to account for new metrics
- Fix content hash to use reconstructed text instead of field
UI/rendering improvements:
- Extract HTML generation to html-renderer.ts
- Create template-renderer.ts for variable substitution
- Add report.html template with card-based layout
- Refactor show.ts to use new rendering system
- Update build script to copy templates to dist
Data structure updates:
- Add reworded_sentences, truly_missing_sentences, agreed_sentences
- Add factual_errors to discrepancies
- Remove deprecated text and media fields
- Add analysis_window metadata1 parent c5f0a3b commit ecf668e
15 files changed
Lines changed: 1547 additions & 571 deletions
Large diffs are not rendered by default.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
93 | | - | |
| 93 | + | |
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | | - | |
| 58 | + | |
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| |||
0 commit comments