Trending Storyline Monitor

A GitHub Pages-ready trend monitor for reviewed URL clusters, topic momentum, source spread, and audio briefings. The static site now reads from data/stories.json, and the repo includes a small Outlook-to-review-queue pipeline so the feed can become operational instead of staying hand-seeded.

What is included

• Public-facing topic feed with filters, tag pages, trends, charts, and audio briefings
• data/stories.json as the published source consumed by the site
• Outlook ingestion script that pulls messages from Microsoft Graph into data/review-queue.json
• Review script for approving or rejecting queued items
• Publish script that turns approved review items into live site stories
• GitHub Pages workflow for static deployment

Files

• index.html - page structure
• styles.css - layout and presentation
• app.js - client-side rendering from data/stories.json
• data/stories.json - published topic input used by the site
• data/review-queue.json - editorial review queue populated from Outlook
• scripts/sync-stories.js - pulls candidate stories from Outlook via Microsoft Graph
• scripts/review-story.js - marks queued items as approved, pending, or rejected
• scripts/publish-stories.js - publishes approved stories to the live feed
• scripts/publish-airtable.js - rebuilds the live feed from approved Airtable records
• scripts/generate-podcast-briefing.js - generates an MP3 briefing from approved Airtable stories using Gemini plus Google TTS
• docs/airtable-schema.md - Airtable field layout for the review table
• docs/audio-briefing-setup.md - Gemini plus Google TTS setup and briefing workflow
• prompts/story-extraction.md - OpenAI prompt for Zapier extraction
• .github/workflows/pages.yml - GitHub Pages deployment workflow
• .github/workflows/publish-from-airtable.yml - on-demand Airtable publish workflow
• .github/workflows/generate-podcast-briefing.yml - on-demand audio briefing workflow

Setup

1. Copy .env.example to .env or export the variables in your shell.
2. Register a Microsoft Entra application with application permission to read mail for the target mailbox.
3. Grant admin consent for the Graph mail permission.
4. Point OUTLOOK_USER_EMAIL at the mailbox that receives story emails.
5. Add Airtable secrets in GitHub Actions for publish automation.
6. Commit and push after publishing data updates so GitHub Pages picks them up.

The Outlook sync script uses Microsoft Graph application auth and expects:

• MS_TENANT_ID
• MS_CLIENT_ID
• MS_CLIENT_SECRET
• OUTLOOK_USER_EMAIL
• OUTLOOK_MAIL_FOLDER such as inbox or a folder id prefixed with id:
• OUTLOOK_MAX_MESSAGES
• AIRTABLE_BASE_ID
• AIRTABLE_TABLE_NAME
• AIRTABLE_TOKEN
• AIRTABLE_VIEW_NAME
• SEMRUSH_API_KEY
• SEMRUSH_DATABASE
• GOOGLE_CLOUD_PROJECT_ID
• GOOGLE_OAUTH_ACCESS_TOKEN
• GEMINI_LOCATION
• GEMINI_MODEL
• GOOGLE_TTS_USE_MULTISPEAKER
• GOOGLE_TTS_MULTISPEAKER_VOICE
• GOOGLE_TTS_FALLBACK_VOICE_NAME
• GOOGLE_TTS_FALLBACK_GENDER
• AUDIO_BRIEFING_STORY_LIMIT
• AUDIO_BRIEFING_MINUTES

Workflow

1. Pull messages from Outlook into the review queue:

npm run sync:outlook

2. See the queued items:

npm run review:list

3. Approve one with an editorial patch:

node scripts/review-story.js approve story-id '{"topic":"Water access","tags":["drought","policy"],"whyItMatters":"Why this should be in the public feed.","momentum":72}'

4. Publish approved stories into the live site feed:

npm run publish:stories

5. Push the updated JSON to GitHub so Pages republishes.

Airtable Publish Path

To rebuild the site directly from Airtable:

npm run publish:airtable

For GitHub automation, add these repository secrets:

• AIRTABLE_BASE_ID
• AIRTABLE_TABLE_NAME
• AIRTABLE_TOKEN
• AIRTABLE_VIEW_NAME (optional)
• SEMRUSH_API_KEY (optional but recommended for trend direction)
• SEMRUSH_DATABASE such as us (optional)

Then trigger .github/workflows/publish-from-airtable.yml manually, or from Zapier using GitHub repository_dispatch with event type publish-stories.

When SEMRUSH_API_KEY is present, the publish pipeline enriches each clustered story with Semrush daily traffic for its source domain and uses that traffic series as the preferred directional signal for topic trend movement.

Audio Briefings

To generate an audio briefing from approved Airtable stories:

npm run generate:podcast

See docs/audio-briefing-setup.md for Google Cloud setup, secrets, and the GitHub workflow path.

Recommended Zapier Chain

1. Outlook new email in Story Intake
2. OpenAI extraction using prompts/story-extraction.md
3. Airtable create record using docs/airtable-schema.md
4. Airtable status changed to Approved
5. Zapier webhook to GitHub repository_dispatch

Published Story Shape

The site consumes data/stories.json in this form:

{
  "stories": [
    {
      "id": "string",
      "headline": "string",
      "source": "string",
      "date": "YYYY-MM-DD",
      "slug": "string",
      "tags": ["string"],
      "topic": "string",
      "summary": "string",
      "whyItMatters": "string",
      "relevance": "string",
      "momentum": 0,
      "recentMovement": [1, 2, 3],
      "related": ["story-id"]
    }
  ],
  "audioBriefings": []
}

Notes

• The frontend falls back to embedded sample data if data/stories.json is missing or unreadable.
• The review queue is intentionally JSON-first for speed. If you want multi-editor workflow next, Airtable is the obvious next source of truth.