Growth: AI Agent Documentation Freshness Score — Track Docs-Code Sync, Staleness & Community Update Velocity to Help Developers Choose Well-Maintained Frameworks

[sykp241095]

Problem/Opportunity

When evaluating AI agent frameworks, developers heavily rely on documentation quality. However, there's no systematic way to assess whether docs are kept up-to-date with code changes. A framework might have beautiful docs, but if they're 6 months behind the latest release, users will hit frustrating gaps.

Currently, OSSInsight tracks stars, contributors, releases, and issues — but documentation health is a blind spot. This is a critical signal for AI builders choosing infrastructure they'll depend on.

Implementation Plan

Phase 1: Documentation Freshness Metrics

1. Docs-Code Sync Lag: Measure time between code release (GitHub release tag) and corresponding docs update
2. Stale Docs Detector: Identify docs pages not updated in >90 days while code actively develops
3. Version Coverage Score: What % of released versions have matching docs?
4. Community Docs Contribution Ratio: Ratio of community PRs vs maintainer-only docs updates (indicates healthy docs culture)

Phase 2: Data Collection

• Parse , , directories in repos
• Track commit timestamps in docs paths vs src paths
• Cross-reference release tags with docs changelog updates
• Analyze PRs touching docs files (community vs maintainer authors)

Phase 3: Visualization & Scoring

• Add "Docs Health" badge to AI framework collection pages
• 0-100 freshness score with breakdown (sync lag, staleness, coverage, community)
• Trend chart showing docs update velocity over time
• Alert when docs lag exceeds threshold (e.g., >30 days behind latest release)

Phase 4: Integration

• Surface in AI Agent Framework comparison view
• Include in "AI Founder's Morning Briefing" for tracked projects
• Optional: GitHub App comment on releases when docs update lag detected

Why AI Builders Would Care

1. Risk Mitigation: Building on a framework with stale docs means more time debugging, more support tickets, slower development
2. Maintainer Signal: Fresh docs = active, attentive maintainers who care about DX
3. Enterprise Procurement: Teams evaluating frameworks for production need to know docs won't become a liability
4. Competitive Intelligence: Founders can benchmark their docs velocity against competitors

Estimated Impact

• Traffic: High — "documentation quality" is a top-3 search criterion for developers evaluating frameworks (per Stack Overflow Developer Survey)
• Engagement: High — comparison feature drives repeat visits when evaluating multiple frameworks
• Retention: Medium-High — teams will bookmark and track docs health of frameworks they're using
• Differentiation: Unique — no existing tool (DB-Engines, State of JS, etc.) offers docs freshness as a first-class metric

Technical Considerations

• GitHub API rate limits: batch docs path analysis across collections
• Repo structure varies: need heuristics to identify docs directories (docs/, /docs, .md files in root, Docusaurus/Next.js patterns)
• False positives: some projects use external docs sites (GitBook, Notion) — need manual override or detection

Success Metrics

• % of AI framework collections with Docs Freshness score populated
• Click-through rate on docs health badges
• User feedback: "This saved me from choosing a framework with terrible docs"