Buzzword Inflation Detection

[avifenesh]

Overview

Detect claims in documentation that are not supported by evidence in the code.

The Buzzword-Evidence Matrix

Buzzword	Required Evidence
production-ready	Tests, error handling, logging
enterprise-grade	Auth, audit logs, scalability
secure	Input validation, encryption, auth
scalable	Async, caching, load handling
battle-tested	Usage metrics, issue history
comprehensive	Edge case handling

Detection Approach

1. Scan documentation (README, docs/, comments) for buzzwords
2. Search codebase for evidence patterns
3. Flag claims without supporting code

Example

Claim: production-ready authentication system

Evidence search:

• No test files matching auth.test.*
• No error handling in auth modules
• No logging statements
• No rate limiting

Result: BUZZWORD INFLATION DETECTED

4-Layer Test Detection (for production-ready claims)

Layer	What It Checks	Example
1. Paths	Test directory structure	tests/, tests/, spec/
2. Filenames	Test file patterns	*.test.ts, *.spec.js, *_test.go
3. Markers	Test framework markers	@pytest.mark.e2e, describe(), #[test]
4. Runtime	Actual test execution	Test commands in CI, coverage reports

production-ready claims specifically require layer 4 evidence.

Certainty

MEDIUM - Agent should verify context and whether claims are reasonable.

Reference

See docs/AI-SLOP-RESEARCH.md Category 4: Buzzword Inflation

Priority

P3 - Advanced (Hard, Critical Impact)

[avifenesh]

Implemented in PR #113. Added buzzword_inflation pattern with multi-pass analyzer analyzeBuzzwordInflation() - detects quality claims (production-ready, secure, scalable, etc.) without supporting code evidence.