Skip to content

CLAUDE.md

Latest commit

 

History

History
972 lines (783 loc) · 52.2 KB

CLAUDE.md

File metadata and controls

972 lines (783 loc) · 52.2 KB

DECUR Project Guidelines

Project Overview

DECUR (Data Exceeding Current Understanding of Reality) is a Next.js application for exploring UAP/NHI phenomena data with a component-based architecture using Tailwind CSS. The platform focuses on cataloging and analyzing insider testimony, documented cases, and associated research while maintaining scientific rigor and methodological transparency.

Development Commands

npm run dev        # Start development server
npm run build      # Build production version
npm run start      # Start production server
npm run lint       # Run ESLint
npm run typecheck  # Run TypeScript type checking
npm run check      # Run both lint and typecheck

UFOSINT Phase 3 - Self-Hosting Checklist

Phase 3 moves the UFOSINT sightings database off the external ufosint.com API into our own Supabase table (ufosint_sightings).

Status (as of 2026-04-20): FULLY LIVE IN PRODUCTION. All migrations applied, ~618,000+ records imported from ufo_public_v2.db (DuelingGroks export - v2 supersedes the original ufo_public.db), UFOSINT_USE_SUPABASE=true active in Vercel. The /sightings page is deployed and publicly accessible.

Testing note: Test /sightings against production (Vercel URL), NOT localhost. The local dev server does not have UFOSINT_USE_SUPABASE=true set in .env.local by default, so the viewport and hexbin APIs fall back to the legacy ufosint.com proxy (which 503s for viewport calls). All sightings map work should be verified on the deployed Vercel instance.

Phase 3 is fully applied to production. Record of what was done:

  1. Migrations applied to prod Supabase (iyvngosoyzptliytlcov):

    • 004_ufosint_sightings.sql - base table
    • 005_widen_sightings_state_country.sql - widen state/country columns
    • 006_add_enriched_sightings_fields.sql - hynek, vallee, quality_score, standardized_shape, vader_compound, has_media, etc.

    To re-apply future migrations:

    npx supabase link --project-ref iyvngosoyzptliytlcov
echo "Y" | npx supabase db push

  2. Import data from ufo_public_v2.db (~618,000+ records; DONE as of 2026-04-20):

    # Requires ufo_public_v2.db at .plans/ufo_public_v2.db (from DuelingGroks export)
# NOTE: ufo_public_v2.db supersedes the original ufo_public.db - always use v2.
# Set IMPORT_SUPABASE_URL + IMPORT_SERVICE_KEY in .env.local, then:
node --env-file=.env.local scripts/import-from-sqlite.mjs
# Resume if interrupted:
node --env-file=.env.local scripts/import-from-sqlite.mjs --resume
# Retry a specific id range (for fixing transient errors):
node --env-file=.env.local scripts/import-from-sqlite.mjs --retry-range 1 430000

    After any import or re-import, refresh the static chart JSON files (shape/country/yearly distributions used by the sightings charts):

    # Run against the prod Supabase (decur project: iyvngosoyzptliytlcov)
UFOSINT_SUPABASE_URL=https://iyvngosoyzptliytlcov.supabase.co \
UFOSINT_SERVICE_KEY=sb_secret_<prod_key> \
node scripts/refresh-ufosint-static.mjs
# Then commit and redeploy:
git add data/ufosint/*.json && git commit -m "Refresh ufosint static chart data"

    No automated cron job is needed - the sightings data changes only when a new DuelingGroks export is imported. Run this script manually each time that happens.

  3. Activate in prod env (Vercel dashboard → Environment Variables):

    UFOSINT_USE_SUPABASE=true
UFOSINT_SUPABASE_URL=https://iyvngosoyzptliytlcov.supabase.co
UFOSINT_SERVICE_KEY=<prod-sb_secret_key>

    The prod service key is in Supabase dashboard → iyvngosoyzptliytlcov → Settings → API Keys.

  4. Re-link CLI back to decur-dev after:

    npx supabase link --project-ref bosszjlkhglatuashtbd

To activate locally (optional - prod is already live):

Add to .env.local to test sightings APIs against the cloud decur-dev project:

UFOSINT_USE_SUPABASE=true
UFOSINT_SUPABASE_URL=https://bosszjlkhglatuashtbd.supabase.co
UFOSINT_SERVICE_KEY=sb_secret_...  # decur-dev secret key from Settings → API Keys

Without these vars set locally: The /sightings page loads and the heatmap renders (hexbin JSON is served from static files), but the viewport pin API (/api/sightings/viewport) will 503 and no cyan sighting dots will appear. Use the production Vercel URL to verify full sightings map behavior.

Why separate env vars? The main Supabase env vars (SUPABASE_INTERNAL_URL, SUPABASE_SERVICE_ROLE_KEY) point to the local Docker instance for the rest of the app. The sightings data lives in decur-dev (cloud), so lib/supabase/sightings.ts needs its own connection vars.

CRITICAL - Never hardcode Supabase keys in scripts

The import-ufosint-to-supabase.mjs and backfill-ufosint-by-source.mjs scripts read credentials from environment variables only. Never paste a service role key directly into a script file - GitHub secret scanning will detect it and the key will be publicly exposed.

Always pass keys at runtime:

IMPORT_SUPABASE_URL=https://... IMPORT_SERVICE_KEY=sb_secret_... node scripts/import-ufosint-to-supabase.mjs

Or store in .env.local (which is gitignored) for repeated local use:

IMPORT_SERVICE_KEY=sb_secret_your_key_here

Key locations:

  • decur-dev secret key: Supabase dashboard → bosszjlkhglatuashtbd → Settings → API Keys → Secret keys
  • prod secret key: Supabase dashboard → iyvngosoyzptliytlcov → Settings → API Keys → Secret keys
  • If a key is ever accidentally committed: immediately generate a new secret key in the Supabase dashboard to invalidate the exposed one. The legacy JWT-format service_role key cannot be selectively revoked - use the newer sb_secret_... format keys instead.

Data Audit Script

Before any gap analysis or "what do we have?" research, always run the audit script first to get a deterministic, exact inventory. Never rely on agent summarization of large JSON files for counts - it will produce inaccurate numbers.

node scripts/audit-data.js              # Summary table + counts for all categories
node scripts/audit-data.js --ids        # Same + full ID list for every category
node scripts/audit-data.js --category figures   # Single category only
node scripts/audit-data.js --ids --category cases

The script reads directly from the source JSON files and prints exact counts. Use the output as the ground-truth baseline when determining what is missing.

Project Structure

components/
  data/
    key-figures/         - Generic + bespoke profile components
      GenericInsiderProfile.tsx  - Renders all standard JSON-schema profiles
      burisch/           - Bespoke tabs for Dan Burisch (unique schema)
    shared/
      profileStyles.ts   - Centralized Tailwind class constants (ps.*)
      ProfileShell.tsx   - Shared tab-nav shell for all profiles
      CredibilityBalance.tsx, ArgumentsSection.tsx, etc.
    DocumentsList.tsx, InsidersList.tsx, CasesList.tsx ...
  explore/
    NetworkGraph.tsx             - Force-directed relationship graph (react-force-graph-2d)
    ClaimsCorroborationGraph.tsx - Bipartite claims/figures graph (react-force-graph-2d)
    TimelineOverlay.tsx          - Swimlane timeline overlay (Recharts)
    EventFrequencyChart.tsx      - Event frequency by decade (Recharts)
    ProgramLineageFlow.tsx       - Program succession DAG (@xyflow/react)
    OversightHierarchyFlow.tsx   - Org authority hierarchy (@xyflow/react)
    EvidenceTierFlow.tsx         - Cases by evidence tier (@xyflow/react)
    CongressionalDisclosureFlow.tsx - Congressional disclosure timeline (@xyflow/react)
    CaseMap.tsx                  - Geographic case map
  resources/
pages/
  data.tsx, figures/[id].tsx, cases/[id].tsx, explore.tsx ...
  timeline.tsx, search.tsx, blue-book.tsx, sources.tsx ...
data/
  key-figures/
    index.json           - Searchable index of all figures
    registry.ts          - Maps id -> JSON data for GenericInsiderProfile
    [id].json            - One file per profile (144 profiles as of 2026-06-11)
  cases.json, timeline.json, programs.json, glossary.json
  network-graph.ts       - Relationship network nodes and edges
  claims-network.ts      - Bipartite claims graph data (aggregated from profile JSONs)
  org-hierarchy.json     - Oversight hierarchy nodes and edges
  contractors.json       - Private defense contractor profiles
  documents.json, resources.json
public/                  - Static assets
styles/                  - Global CSS
.claude/                 - Claude AI context files
ytdl/                    - YouTube transcript processing utilities

Key Files to Reference

When working on this project, prioritize these files for context:

  1. CLAUDE.md - Development commands, schema rules, and code style
  2. components/data/shared/profileStyles.ts - Shared Tailwind constants (ps.*)
  3. components/data/key-figures/GenericInsiderProfile.tsx - Standard profile renderer
  4. data/key-figures/index.json - All registered figures
  5. data/key-figures/registry.ts - id -> JSON mapping
  6. data/network-graph.ts - Relationship network node/edge definitions
  7. data/claims-network.ts - Claims corroboration graph data; aggregates claims[] from all profiles with category fields; defines CATEGORY_MAP and canonical category normalization
  8. .claude/ folder:
    • code-structure.json - Component relationships and data flows
    • architecture-overview.md - System design documentation

Dark Mode & Shared Styles

Always use ps.* constants for profile components

components/data/shared/profileStyles.ts exports a ps object with pre-built Tailwind strings that include both light and dark variants. Never hardcode bare Tailwind classes in profile components when a ps.* constant exists - this keeps dark mode consistent globally and ensures a single edit propagates everywhere.

import { ps } from '../shared/profileStyles';

// Correct - uses shared constant
<div className={ps.infoCard}>
<p className={ps.body}>
<span className={ps.label}>

// Wrong - hardcoded without dark variant
<div className="bg-gray-50 rounded-lg p-4">
<p className="text-sm text-gray-700">

Available constants:

Constant Purpose
ps.infoCard Gray-tinted card (service, clearance, orgs)
ps.infoCardSm Same, smaller padding
ps.borderCard Outlined card, no tint
ps.borderCardLg Outlined card, larger padding
ps.borderCardNoP Outlined card, no padding (use for tables)
ps.accentBox Primary-color tinted accent box
ps.accentBoxLg Same, larger padding
ps.h3 Section heading (lg, semibold)
ps.h4 Sub-section label (uppercase, tracking)
ps.h4Inline Inline label (same weight, no uppercase)
ps.body Standard body text
ps.bodyMuted Slightly muted body text
ps.value Data value text (slightly brighter)
ps.meta xs meta / secondary text
ps.muted xs muted text (labels, dates)
ps.label xs uppercase tracking label
ps.listItem Flex list item row
ps.divider Horizontal rule
ps.timelineLine Timeline container with left border
ps.timelineDot Filled circle dot on timeline
ps.filterPill Inactive filter pill button
ps.filterPillActive Active filter pill button

To extend: Add new constants to profileStyles.ts before introducing new ad-hoc class strings in components.

Two-Tier Profile System

Profiles use one of two rendering approaches:

Tier 1 - Generic (standard schema, no TSX file needed)

Most profiles. GenericInsiderProfile.tsx reads from data/key-figures/[id].json and renders Overview, Timeline, optional Feature tab, People, Disclosures, and Sources tabs automatically.

Tier 2 - Bespoke (custom TSX component)

Only for profiles with fundamentally different tab structures (e.g., Dan Burisch, which has 10 specialized tabs). These require:

  • A dedicated component directory under components/data/
  • An entry in BESPOKE_REGISTRY in pages/figures/[id].tsx

Default to Tier 1. Only create a Tier 2 component if a figure's data cannot be reasonably expressed in the standard schema.

IMPORTANT - Before editing any key figure profile JSON: Check BESPOKE_REGISTRY in pages/figures/[id].tsx to determine if the figure uses a bespoke component. If they do, the GenericInsiderProfile.tsx logic (auto-tabs for sources, disclosures, etc.) does NOT apply - you must inspect and potentially update the bespoke component directly.

Current Tier 2 bespoke profiles (as of 2026-03):

Figure ID Component file
dan-burisch components/data/InsiderProfile.tsx (+ key-figures/burisch/ dir)
bob-lazar components/data/LazarProfile.tsx
david-grusch components/data/GruschProfile.tsx
luis-elizondo components/data/ElizondoProfile.tsx
david-fravor components/data/FravorProfile.tsx
hal-puthoff components/data/PuthoffProfile.tsx
garry-nolan components/data/NolanProfile.tsx
karl-nell components/data/NellProfile.tsx
chris-mellon components/data/MellonProfile.tsx
eric-davis components/data/DavisProfile.tsx
robert-bigelow components/data/BigelowProfile.tsx
jacques-vallee components/data/ValleeProfile.tsx
nick-pope components/data/PopeProfile.tsx
jake-barber components/data/BarberProfile.tsx
tim-gallaudet components/data/GallaudetProfile.tsx

Update this table when adding new Tier 2 components.

Adding a New Key Figure

1. Create the profile JSON

Create data/key-figures/[id].json following the canonical schema:

{
  "profile": {
    "id": "jane-smith",
    "name": "Dr. Jane Smith",
    "aliases": ["Jane Smith", "J. Smith"],
    "born": "January 1, 1960 - Washington D.C.",
    "died": null,
    "roles": [
      "Senior Intelligence Officer, DIA",
      "Author - The Phenomenon (2021)"
    ],
    "service_period": "1985-2010 (government); 2010-present (independent research)",
    "organizations": ["Defense Intelligence Agency", "AATIP"],
    "clearance": "TS/SCI (former)",
    "summary": "Two to three sentence summary of who this person is and why they matter.",
    "education": ["M.D., Johns Hopkins, 1984", "B.S., MIT, 1980"],
    "early_career": ["Brief career note 1", "Brief career note 2"],
    "key_events": [
      { "year": "1985", "event": "Joined the DIA as a senior analyst." },
      { "year": "Jan 2017", "event": "Resigned from AATIP and began public disclosure efforts." }
    ]
  },
  "associated_people": [
    {
      "id": "luis-elizondo",
      "name": "Luis Elizondo",
      "role": "Former AATIP director",
      "relationship": "Collaborated on AATIP research from 2008-2017; both resigned in the same period citing institutional resistance."
    }
  ],
  "disclosures": [
    {
      "date": "2021",
      "type": "written",
      "title": "The Phenomenon",
      "outlet": "Publisher Name (New York)",
      "notes": "Primary written disclosure describing 20 years of government UAP research."
    },
    {
      "date": "June 2023",
      "type": "congressional-testimony",
      "title": "UAP Disclosure Act Hearing",
      "outlet": "U.S. Senate Armed Services Committee",
      "interviewer": "Sen. Kirsten Gillibrand",
      "notes": "Testified under oath that non-human intelligence has been confirmed by multiple classified programs."
    }
  ],
  "sources": [
    {
      "title": "The Phenomenon (book)",
      "url": "https://www.amazon.com/...",
      "type": "Book",
      "notes": "Primary source - first published account (Publisher, 2021)"
    }
  ]
}

Canonical field rules:

  • key_events[].year - use "YYYY" or "Mon YYYY" (e.g. "1994" or "Sep 1994"). Never use ISO date strings like "1994-09-16".
  • associated_people[] - must include id (slug), name, role (one-line title), and relationship (narrative sentence).
  • disclosures[].type - must be one of: article, written, print, television, film, documentary, podcast, radio, interview, speech, congressional-testimony, congressional-briefing, formal-complaint, declassification, academic-paper, conference, symposium, preprint. Adding a new type requires a corresponding entry in components/data/shared/disclosureTypes.ts.
  • disclosures[].notes - use notes, not description.
  • disclosures[].title - required; give the disclosure a short name.

Optional feature section: Add a top-level key for specialized data (e.g., "aawsap": {...}, "major_investigations": [...], "claims": [...]). GenericInsiderProfile auto-detects and renders it as an extra tab. Register the key in detectFeature() in GenericInsiderProfile.tsx.

2. Register the profile

Add one line to data/key-figures/registry.ts:

import janeSmithData from './jane-smith.json';
// ...
'jane-smith': janeSmithData,

3. Add the index entry

Add an entry to data/key-figures/index.json:

{
  "id": "jane-smith",
  "name": "Dr. Jane Smith",
  "aliases": ["Jane Smith"],
  "role": "Senior Intelligence Officer, DIA",
  "period": "1985-2010",
  "affiliation": "Defense Intelligence Agency",
  "summary": "2-3 sentence summary shown in the listing card.",
  "status": "detailed",
  "tags": ["intelligence", "dod", "aatip"],
  "data_file": "jane-smith.json",
  "type": "insider",
  "includeInExplore": true
}

Valid type values: insider, journalist, pilot, scientist, official, executive.

4. Add Explore overlay color (optional)

If includeInExplore: true, you may add an entry to SOURCE_CONFIG in components/explore/TimelineOverlay.tsx to assign a custom display label and swimlane color:

'jane-smith': { label: 'Dr. Jane Smith', color: '#hexcolor' },

If omitted: the display name resolves automatically from data/key-figures/index.json (the figure's name field), and the swimlane color defaults to gray (#6b7280). You only need a SOURCE_CONFIG entry if you want a shortened label (e.g. 'Sen. Smith') or a distinct color.

Do not hardcode display names in SOURCE_CONFIG that already match the index.json name field exactly - that's redundant. SOURCE_CONFIG entries are for abbreviated/role-prefixed labels only.

5. Add Relationship Network edges (required)

All profiled figures are auto-derived as nodes in the Relationship Network from index.json - no manual node entry needed. However, you must add link edges to data/network-graph.ts or the figure will appear as an isolated island with no connections.

Append a comment block with links to the links: [] array at the bottom of network-graph.ts:

// Jane Smith connections
{ source: 'jane-smith', target: 'luis-elizondo', label: 'collaborated on AATIP research 2008-2017' },
{ source: 'jane-smith', target: 'aatip',         label: 'program director' },
  • source and target must match node id values exactly (registry ids for profiled figures; explicit node ids for organizations/cases/documents)
  • Every new figure should have at least 2-3 edges to existing nodes
  • Use factual, specific relationship labels - not generic descriptions

6. Add career_connections for a non-linear Career Network tab (required)

Without career_connections, the Career Network tab renders as a pure linear chain of key_events - identical in content to the Timeline tab. Always add a career_connections array to give the graph lateral branching nodes.

Add a top-level career_connections key to the profile JSON:

"career_connections": [
  {
    "event_index": 3,
    "node_type": "person",
    "node_id": "luis-elizondo",
    "node_label": "Luis Elizondo",
    "relationship": "Collaborated on AATIP research from 2008-2017.",
    "connection_type": "professional"
  },
  {
    "event_index": 5,
    "node_type": "program",
    "node_id": "aatip",
    "node_label": "AATIP",
    "relationship": "Served as program director.",
    "connection_type": "institutional"
  }
]
  • event_index is 0-based into the key_events array - the node branches off that point in the timeline chain
  • node_type must be one of: person, case, program
  • connection_type must be one of: investigative, professional, institutional
  • Aim for at least 2-3 lateral nodes across different event_index values so the graph has visible branching at multiple points
  • Spread nodes across different timeline points rather than clustering them at one index

7. Update the Data Sources page (required)

Every new key figure must have its primary research sources added to pages/sources.tsx. This is a mandatory step - the sources page is the platform's research attribution record.

Where to insert: Find the {/* Key Figure Profile Sources */} section comment and add <SourceCard> entries inside it.

Component interface:

<SourceCard
  name="Fire in the Sky: The Walton Experience (Walton, 1993)"
  url="https://www.amazon.com/dp/1569800855"
  type="Published Book"
  typeColor="bg-purple-100 text-purple-700"
  description="Travis Walton's first-person memoir of the 1975 Snowflake, Arizona abduction. Primary source for all biographical data and event sequence."
  notes="Used for: Travis Walton."
/>

Props reference:

Prop Required Notes
name Yes Source title, author, and year - e.g. "The Interrupted Journey - John Fuller (1966)"
url Yes Direct URL (Amazon, YouTube, archive.org, official site). Never a ufotimeline.com link.
type Yes e.g. "Published Book", "Television Interview", "Research Archive", "Congressional Record", "Academic Paper"
typeColor Yes Tailwind badge classes. Use "bg-purple-100 text-purple-700" for books, "bg-blue-100 text-blue-700" for interviews/video, "bg-green-100 text-green-700" for archives/databases, "bg-yellow-100 text-yellow-700" for government/official
description Yes 1-2 sentences: what the source is and what it contributed to the profile
notes No Attribution string: "Used for: [Figure Name]."

Rule: Add one SourceCard per distinct source used. If the same source was used for multiple figures, add it once with notes="Used for: Figure A, Figure B.".

8. Update the What's New changelog (required)

Add an entry to the top of data/changelog.json:

{
  "date": "YYYY-MM-DD",
  "category": "figure",
  "id": "jane-smith",
  "name": "Dr. Jane Smith",
  "action": "added",
  "note": "One sentence describing what was added or what changed."
}

Field rules:

  • date - today's date in YYYY-MM-DD format
  • category - one of: figure, case, document, program, timeline, quote
  • id - the registry slug (must match the profile/case/document/program ID exactly)
  • action - "added" for new entries, "updated" for significant updates to existing ones
  • note - one sentence; focus on what's distinctive about this person/event (not generic "profile added")

This feeds the /whats-new full feed page and the "Recently Added" widget on the homepage. Skipping this step means returning users get no signal that new content exists.

9. Sync the Supabase search index (required)

The global search (/search) runs against a Supabase search_index table — it does NOT auto-populate from the JSON files. After adding any new figure, case, document, or program, you must re-run the sync script or the new entry will not appear in search results:

node --env-file=.env.local scripts/populate-search-index.mjs

This script is idempotent (safe to re-run at any time). It upserts all categories in one pass. Required env vars are IMPORT_SUPABASE_URL and IMPORT_SERVICE_KEY (or the UFOSINT_* equivalents) — these should already be in .env.local.

Symptom if skipped: The new figure's profile page loads correctly (JSON-driven), but searching by name on /search returns no results.

That's all

No component file is needed. No if check in InsidersList.tsx is needed.

Adding a New Case

Cases live in data/cases/ as individual JSON files plus a lightweight index. After adding a case, complete all of the following steps or the case will be invisible in search and missing from the changelog.

1. Create data/cases/[id].json and add an entry to data/cases/index.json

Step 1a - Create the full case file at data/cases/[id].json following the existing case schema. Required top-level fields:

  • id - kebab-case slug (e.g. "1978-tarija-bolivia")
  • name - human-readable title
  • date - display date string (e.g. "May 6, 1978")
  • location - full location description
  • country - primary country
  • category - one of: crash-recovery, military-aviation, close-encounter, military-ground, mass-witness, mass-sighting, ground-visual, civilian-mass-sighting, aviation, transmedium, military-nuclear, military-maritime, military-aerial, ground-observation, government-nuclear, commercial-aviation
  • evidence_tier - one of: tier-1, tier-2, tier-3
  • classification_status - typically "unresolved", "explained", or "disputed"
  • summary - 3-5 sentence overview shown in the listing card
  • tags - array of keyword strings
  • insider_connections - typed array (see insider_connections schema section below); use [] only if genuinely no connected figures exist - always cross-check the key figures registry first
  • overview.key_facts - bullet array of the most important facts
  • evidence - video_audio, documentation, physical sub-arrays
  • sensor_context - object with a systems array; each system entry has name (string, e.g. "AN/TPY-2 radar", "NVG-equipped helicopter camera") and optionally operator, resolution, notes. Populate this field for any case with sensor/instrument data — it drives EQI Component 1 (25% of score, the highest single weight). Omitting it zeros out that component even if strong sensor evidence is documented elsewhere in the file.
  • witnesses - array with name, role, type, testimony per witness
  • official_response - agencies array and statements array
  • credibility - supporting and contradicting arrays
  • coordinates - { "lat": float, "lng": float } for map pin
  • timeline - chronological event array with local (date string) and event (description)
  • competing_hypotheses - array with name, assessment ("possible", "disputed", "ruled-out"), summary
  • claims_taxonomy - object with verified, probable, disputed, speculative sub-arrays; each item is { "claim": "...", "type": "<ClaimType>" }. Valid ClaimType values: kinematic, physical-effect, witness-account, official-record, sensor-data, psychological-effect, institutional-acknowledgment
  • sources - array with title, url, type, date, notes

EQI/BAI scoring signal fields — populate these carefully:

The case detail page displays an Evidence Quality Index (EQI) and Behavioral Anomalousness Index (BAI) computed by scripts/compute-case-scores.mjs. These scores are determined entirely by fields in the case JSON. The highest-impact fields per component are:

EQI Component Weight Driven by
Capture Technology 25% sensor_context.systems[].name — radar, FLIR, sonar, satellite, etc.
Witness Credential 20% witnesses[].type — "military", "government", "scientist", "pilot", "law-enforcement" score highest
Corroboration 15% Multi-platform sensor_context.systems + multi-category witnesses[].type
Physical Evidence 15% evidence.physical array entries
Official Response 15% official_response.statements entries
Competing Hypotheses 10% competing_hypotheses entries with "ruled-out" assessments

BAI is primarily driven by tags (e.g. "fission-behavior", "transmedium", "acceleration", "structured-craft") and claims_taxonomy entries with kinematic or physical-effect type in the verified/probable tiers.

After authoring the case file, run the scorer to check for zero-score components (indicating missing fields) before committing.

Step 1b - Add an index entry to data/cases/index.json. This is a lightweight summary used by the listing page, explore map, and figure cross-references. Required fields:

{
  "id": "your-case-id",
  "name": "Case Display Name",
  "date": "Month DD, YYYY",
  "location": "City, Region, Country",
  "country": "Country",
  "category": "military-aviation",
  "evidence_tier": "tier-1",
  "classification_status": "unresolved",
  "summary": "3-5 sentence overview shown in listing cards.",
  "tags": ["tag1", "tag2"],
  "coordinates": { "lat": 0.0, "lng": 0.0 },
  "insider_connections": []
}

Both files must be created together. The index entry mirrors the top-level fields of the full case file - coordinates and insider_connections are duplicated intentionally so the map and figure cross-reference pages can work without loading every full case file.

CRITICAL - sources[].type must be one of these exact values (any other value renders a typeless pill with no background color):

type value Badge color Use for
official Green Formal government/military briefings, sworn testimony documents, congressional records
foia Teal Documents released via FOIA requests, CIA/State Dept. reading room releases
media Blue News articles, blogs, podcasts, YouTube interviews, databases, research websites
testimony Purple Firsthand witness statements, affidavits (when the source IS the testimony)
academic Amber Peer-reviewed papers, university publications, scholarly journals
book Gray Published books

Do not use: "government", "research", "interview", or any other value - these are not in the CaseSourceType union and will render without a pill badge.

2. Check insider_connections (required - do not skip)

Before leaving insider_connections as [], search the key figures registry for any figure whose documented work, programs, or testimony directly connects to the case. Common vectors:

  • Named figures who investigated or witnessed the case
  • Researchers whose published works cover the case
  • Officials whose programs (e.g. Moon Dust, AATIP) responded to the case
  • Congressional figures who cited or investigated the case

Run: grep -ri "[keyword]" data/key-figures/*.json to check.

3. Add a changelog entry (required)

Add to the top of data/changelog.json:

{
  "date": "YYYY-MM-DD",
  "category": "case",
  "id": "your-case-id",
  "name": "Case Display Name",
  "action": "added",
  "note": "One sentence describing what makes this case distinctive."
}

4. Sync the Supabase search index (required - do not skip)

CRITICAL: The global search runs against Supabase, NOT the JSON files. Adding a case to data/cases/ does NOT make it searchable. You MUST run:

node --env-file=.env.local scripts/populate-search-index.mjs

Symptom if skipped: The case detail page loads correctly, but searching by name on /search returns no results.

This script is idempotent - safe to re-run at any time.

5. Add a map pin to data/ufosint/case-pins.json (required)

Every case must have an entry in data/ufosint/case-pins.json to appear as an amber pin on the /sightings map. Without this entry the case is invisible on the map regardless of whether coordinates is set in the case file.

{
  "id": "your-case-id",
  "name": "Case Display Name",
  "lat": 34.05,
  "lng": -106.91,
  "total": 18000
}
  • id must match the case id in data/cases/[id].json and data/cases/index.json exactly
  • lat/lng must match the coordinates in the case file
  • total is the approximate number of UFOSINT community sightings in the geographic area around the case location. Use a regional estimate based on nearby existing pins as a reference - this is a display-only count, not an exact figure. US domestic cases typically range 10,000-40,000; international cases range 1,000-15,000; remote ocean/polar areas range 500-2,000.
  • Keep the array sorted alphabetically by id after inserting

The coordinates field in the case file and the pin entry in case-pins.json are not linked automatically - both must be set manually. If a case location is known but coordinates were not initially included, infer them from the documented location (city, landmark, or region centroid).

6. Add the case to the Relationship Network (required)

Every case must be added as a node to data/network-graph.ts and connected via edges. Without this step the case is invisible in the /explore Relationship Network visualization — it will not appear as a node even if it has insider_connections.

Add a node in the cases section of the nodes: [] array:

{ id: 'your-case-id', name: 'Case Display Name (YYYY)', type: 'case', val: N },
  • id must match the id in data/cases/[id].json exactly — a mismatch creates a ghost node that won't link to the case detail page and will show as "missing" in the audit
  • name should include the year in parentheses for readability in the graph
  • val controls node size: use 4-5 for tier-1, 3 for tier-2, 2 for tier-3

Add edges connecting the case to its insider_connections figures and relevant program/org nodes. Append a comment block to the links: [] array:

// Case Display Name (YYYY)
{ source: 'your-case-id', target: 'figure-id',   label: 'one sentence describing the specific connection' },
{ source: 'your-case-id', target: 'program-id',  label: 'one sentence describing the specific connection' },
  • Every figure listed in insider_connections[] in the case file should have a corresponding edge — these must be kept in sync
  • Add at least 2-3 edges total; connect to program/org nodes (e.g. project-blue-book, nicap, pursue, aatip, disclosure-project, project-moon-dust) where applicable
  • Edge labels are shown on hover in the graph — write them as factual, specific sentences (not generic descriptions)

Verify with the audit scripts after adding:

node scripts/audit-cases-in-graph.mjs          # must show 0 missing cases
node scripts/audit-insider-network-gaps.mjs    # must show 0 items in Table 2

Run both scripts before committing any relationship network change. They are the ground-truth check for network completeness. Table 1 = items with insider_connections but no graph node; Table 2 = items in graph missing edges to their insiders.

7. Compute EQI/BAI scores (required)

Run the scoring engine against the new case to generate its Evidence Quality Index and Behavioral Anomalousness Index:

node scripts/compute-case-scores.mjs your-case-id

Review the output for any zero-score components — a zero on Capture Technology almost always means sensor_context.systems is empty or missing; a zero on Witness Credential means witnesses[].type values aren't recognized by the scorer. Fix the case file and re-run until all applicable components have non-zero scores before committing.

If the script does not accept a single case ID argument and re-scores all cases, check the output row for your new case and verify its EQI and BAI values are non-zero and plausible relative to the evidence tier. Tier-1 cases should generally score EQI > 0.6; tier-2 cases EQI 0.3-0.6; tier-3 cases EQI < 0.3.

8. Add sources to pages/sources.tsx (required)

Find the {/* Documented Cases */} section and add <SourceCard> entries for the primary research sources used. See the Data Sources Page section below for the <SourceCard> interface.

Adding a New Research Paper

Papers live in data/research/papers.json. After adding a paper, complete all of the following steps.

1. Add the paper entry to data/research/papers.json

Required fields:

  • id - kebab-case slug, typically [first-author-lastname]-[year] or [first-author]-[coauthor]-[year]
  • title - full paper title
  • authors - display name array (e.g. ["J. Vallée", "G. Nolan"])
  • author_ids - DECUR key figure slugs for any registered figures; [] if none
  • year - four-digit integer
  • journal - journal/book name or null
  • volume / issue - string or null
  • doi - DOI string without https://doi.org/ prefix, or null
  • url - canonical link (DOI resolver, PubMed, arXiv, publisher page)
  • open_access - true if freely available, false otherwise
  • source_type - one of: peer-reviewed, preprint, book-chapter, report, conference, government-report, thesis
  • tags - keyword array (reuse existing tags from other papers where applicable for cross-linking)
  • case_ids - array of DECUR case slugs directly discussed; [] if none
  • organization_ids - array of DECUR org slugs from data/research/organizations.json; [] if none
  • summary - 2-4 sentence plain-language DECUR-context summary
  • abstract - verbatim abstract text (or "" if not available)

2. Update linked organization notable_paper_ids (required)

For every org listed in the paper's organization_ids, open data/research/organizations.json and add the paper's id to that org's notable_paper_ids array.

3. Add a changelog entry (required)

Add to the top of data/changelog.json:

{
  "date": "YYYY-MM-DD",
  "category": "paper",
  "id": "your-paper-id",
  "name": "Paper Title",
  "action": "added",
  "note": "One sentence describing what makes this paper distinctive."
}

4. Sync the Supabase search index (required)

node --env-file=.env.local scripts/populate-search-index.mjs

5. Back-navigation pattern for Related Papers links (required - do not break)

pages/research/papers/[id].tsx uses a backState system so the back button reflects the actual navigation context. The Related Papers <Link> must include ?ref=paper&paperId=${paper.id} so the destination paper's back button reads "← Back to [source paper title]" instead of the generic "← Back to Papers".

This is already implemented in the component. Do not remove or simplify the href on Related Papers links. The pattern is:

href={`/research/papers/${rp.id}?ref=paper&paperId=${paper.id}`}

The useEffect in the page handles three ref values:

  • ref=search → "← Back to Search Results" (uses router.back())
  • ref=org&orgId=X → "← Back to [Org name]" (links to org page)
  • ref=paper&paperId=X → "← Back to [Paper title]" (links back to source paper)

Both the top back button and the footer back button use backState dynamically - never hardcode href="/research?tab=papers" on either.

IMPORTANT — use router.query, not window.location.search: The useEffect reads params via router.query with [router.isReady, router.query] as dependencies. Do NOT revert to window.location.search — when navigating between two instances of the same dynamic page component (paper A → paper B), Next.js may reuse the component without a full unmount, so useEffect([]) won't re-run. router.query in the dependency array ensures the effect re-runs on every route change and reads the correct params after hydration.

Adding a New Research Organization

Research organizations live in data/research/organizations.json. After adding an org, complete all of the following steps or it will be invisible in search and disconnected from the network graph.

1. Add the org entry to data/research/organizations.json

Required fields:

  • id - kebab-case slug (e.g. "society-for-scientific-exploration")
  • name - full human-readable name
  • abbreviation - common abbreviation or null
  • type - one of: research-institute, advocacy, archive, citizen-science, media, funding, government-body
  • status - "active" or "inactive"
  • founded - four-digit year string (e.g. "1982") or null
  • location - city/country or distributed note
  • website - canonical URL
  • description - 2-4 sentence description. Include what the org does, why it is relevant to DECUR's scope, and any notable figure connections.
  • focus_areas - array of keyword strings
  • key_member_ids - array of DECUR key figure IDs (must match data/key-figures/index.json slugs exactly); use [] if none
  • notable_paper_ids - array of paper IDs from data/research/papers.json; use [] initially
  • decur_url - optional; set only if the org has a dedicated DECUR program page (e.g. MUFON → /programs/mufon); otherwise omit

2. Add network graph node and edges (required)

Add the org as a node in data/network-graph.ts (in the Organizations section) and add at least 2-3 edges to existing nodes.

{ id: 'your-org-id', name: 'Display Name', type: 'organization', group: 'shared', val: 2 },

Then add edges:

{ source: 'your-org-id', target: 'hal-puthoff', label: 'SSE fellow' },

3. Sync the Supabase search index (required - do not skip)

node --env-file=.env.local scripts/populate-search-index.mjs

Symptom if skipped: The org's detail page loads correctly, but searching by name on /search returns no results.

4. Add events and opportunities (if applicable)

If the org runs a recurring annual conference or offers grants/fellowships, add entries to data/research/events.json and data/research/opportunities.json.

That's all for orgs

No component file is needed. The /research/organizations/[id] page is generic and renders from the JSON automatically.

CRITICAL: Sync the Supabase Search Index After Any Content Addition

The global search (/search) queries a Supabase search_index table. It does NOT read JSON files directly. Editing any of the data files below does NOT make new content searchable. You MUST run the sync script after adding or updating entries in any of these files:

Data file Content type Run sync after editing?
data/key-figures/*.json + index.json Key figures / insiders Yes
data/cases.json Documented cases Yes
data/documents.json Primary documents Yes
data/programs.json (or equivalent) Government programs Yes
data/glossary.json Glossary terms Yes (lower priority)
data/timeline.json Timeline events Yes (usually bulk-imported)
data/research/organizations.json Research organizations Yes
data/research/papers.json Research papers Yes

The sync command (idempotent - safe to run at any time):

node --env-file=.env.local scripts/populate-search-index.mjs

Symptom if skipped: The detail page or listing loads correctly (JSON-driven), but searching by name on /search returns no results.

This script upserts all categories in a single pass. Required env vars (IMPORT_SUPABASE_URL and IMPORT_SERVICE_KEY) should already be in .env.local.

Data Sources Page (pages/sources.tsx)

The sources page is the platform's research attribution record. It must be kept current whenever new data is added to any category.

When to update

  • Key figures - Add to the {/* Key Figure Profile Sources */} section
  • Cases - Add to the {/* Documented Cases */} section
  • Programs - Add to the {/* Government Programs */} section
  • Changelog - Also add an entry to data/changelog.json (see "Adding a New Key Figure" Step 8 for schema)
  • Documents - Add to the {/* Primary Documents */} section
  • Glossary - Add to the {/* Glossary Sources */} section

Section structure

Each section uses <SourceCard> components. Find the relevant section comment in pages/sources.tsx and append new cards inside it. Do not create new top-level sections unless the source genuinely belongs to a new category that doesn't exist yet.

FeaturedSource vs SourceCard

  • <FeaturedSource> - reserved for the most significant institutional sources (NICAP, Project Blue Book, Congressional records, etc.). Use <SourceCard> for all individual profile/case/program sources.

typeColor conventions

Source type typeColor
Published books "bg-purple-100 text-purple-700"
TV/video/interviews "bg-blue-100 text-blue-700"
Archives/databases "bg-green-100 text-green-700"
Government/official "bg-yellow-100 text-yellow-700"
Academic papers "bg-indigo-100 text-indigo-700"
News/journalism "bg-orange-100 text-orange-700"

SharedAssessmentTab Usage

SharedAssessmentTab has two variants. Always use the baseline variant (the default) when wiring a new bespoke profile's Assessment tab.

// Correct - baseline variant (CredibilityBalance bar + bordered green/red argument cards)
<SharedAssessmentTab credibility={data.credibility} />

// Wrong - compact variant renders a plain 2-column +/- list, no balance bar, no cards
<SharedAssessmentTab credibility={data.credibility} variant="compact" />

The compact variant exists but is visually inconsistent with the rest of the platform. Do not use it in new profiles. If an existing profile is using variant="compact", remove it so it falls back to baseline.

Optional props (baseline only):

  • methodologyNote - amber box shown above the arguments (e.g. Elizondo, Mellon)
  • sources - source cards rendered below the arguments

Dynamic Architecture Conventions

Several components derive display data dynamically from data/key-figures/index.json and the data JSON files. Do not bypass these patterns by adding new hardcoded lookup tables.

insider_connections schema (documents.json and cases.json)

Both data/documents.json and data/cases.json use a typed InsiderConnection[] array - not bare string arrays. When adding or editing connections:

"insider_connections": [
  {
    "id": "luis-elizondo",
    "role": "AATIP director whose advocacy led directly to the UAPTF assessment",
    "note": "The UAPTF assessment is the institutionalized result of Elizondo's disclosure work."
  }
]
  • id - required; must match the key figures registry slug exactly
  • role - required; one sentence describing the figure's relationship to this specific document or case
  • note - optional; additional context or caveat

The component (InsiderLinksTab) reads role/note directly from the data. Never add new hardcoded per-ID label dicts to the component - put the content in the JSON.

Figure display name resolution

TimelineOverlay.tsx and DocumentDetail.tsx both auto-resolve figure display names from data/key-figures/index.json as a fallback. When a new figure is added to the registry, their name appears automatically in these components without any code change. A SOURCE_CONFIG entry is only needed for a custom abbreviated label or non-default color.

Disclosure type configuration

All disclosure type labels, badge colors, and timeline dot colors are centralized in:

components/data/shared/disclosureTypes.ts

Exports: DISCLOSURE_TYPE_LABELS, DISCLOSURE_TYPE_COLORS, DISCLOSURE_TYPE_DOT, disclosureLabel(type).

Never define local TYPE_COLORS, TYPE_DOT, or label maps in individual components. Both SharedDisclosuresTab.tsx and GenericInsiderProfile.tsx import from this file. If a new disclosure type is needed, add it once to disclosureTypes.ts and it propagates everywhere automatically.

Writing & Copy Rules

  • No em dashes - do not use — (U+2014) anywhere in UI copy, component text, or documentation. Use a regular hyphen-minus - or rewrite the sentence.
  • Document name fields - common name first - the site search (Fuse.js, threshold: 0.35, ignoreLocation: false) scores matches by position in the string. If a document's common/identifying name appears late in a long academic-style title, searches for that name will score above the threshold and return no results. Always lead with the identifying common name: "Condon Report: Scientific Study of Unidentified Flying Objects (1969)" not "Scientific Study of Unidentified Flying Objects (Condon Report)". This applies to any string field used as a Fuse.js search key (name, title, role, etc.).

timeline.json Data Rules

  • Never use a ufotimeline.com URL as article_url in timeline.json entries. article_url must point to the final destination (Amazon, YouTube, FBI Vault, archive.org, etc.).
  • source_url may remain as the ufotimeline.com permalink for reference/attribution.
  • When adding new entries that were sourced from ufotimeline, always populate article_url directly from the ufotimeline page's outbound CTA link ("On Amazon", "View More", "Read Article", etc.).
  • If no outbound link exists, leave article_url as null -- the component falls back to source_url.
  • To backfill missing article_url values for media entries, run: node scripts/scrape-media-article-urls.js

Code Style Guidelines

  • Component Structure: Functional components with TypeScript and hooks
  • Naming: PascalCase for components, camelCase for functions/variables
  • Imports: Group imports: React, Next.js, components, styles
  • Component Props: Define TypeScript interfaces for all props
  • TypeScript: Use strict type checking, avoid any type when possible
  • Styling: Use Tailwind CSS with color variables from tailwind.config.js; prefer ps.* constants in profile components
  • Error Handling: Try/catch for async operations, fallback UI for errors
  • File Organization: Follow existing patterns in the components/ directory
  • State Management: React hooks for local state, page-level state for shared data
  • DRY: Before hardcoding Tailwind strings in a new component, check profileStyles.ts and existing shared components in components/data/shared/

TypeScript Guidelines

  • Use interfaces for object shapes (props, state, etc.)
  • Define explicit return types for functions
  • Use React component types (FC, FunctionComponent) for components
  • Use type aliases for union types and complex types
  • Use generics for reusable components and functions
  • Avoid any; use unknown with type guards or define a proper interface
  • Make use of TypeScript utility types (Partial, Omit, etc.)

Adding a New PURSUE Release

Always reference the pipeline document before starting any work on a new PURSUE release:

.plans/pursue-new-release-pipeline.md

This file contains the complete 16-phase pipeline derived from Release 1 and Release 2 implementations, including: ZIP download/extraction, Playwright crawl, pre-flight overlap audit, document/case entry authoring, network graph, media backfill, collection seed script, and production QA checklist.

Key rules (from the pipeline):

  • Run the Phase 2 overlap audit before writing any data — R02 had blocking TypeScript failures from an unregistered document_type value
  • All PDF content must be extracted via Playwright/Chromium (AES-256 encrypted; programmatic extraction fails)
  • Every new case needs a case-pins.json entry for the sightings map
  • Run populate-search-index.mjs before seeding the collection
  • Seed script template: scripts/seed-pursue-release-2-collection.mjs

PURSUE Release history:

  • R01 — May 8, 2026 — collection: pursue-release-1-2026
  • R02 — May 22, 2026 — collection: pursue-release-2-2026
  • R03 — Jun 12, 2026 — 826MB documents + 4.6GB videos — collection: pursue-release-3-2026 (pending)

Domain-Specific Context

This platform organizes research into several key areas:

  • Key Figures: Firsthand testimony from government insiders, scientists, pilots, journalists, and officials
  • Documented Cases: High-evidence UAP incidents with structured evidence tiers, witness accounts, and competing hypotheses
  • Government Programs: Official and private programs involved in UAP investigation, research, and disclosure (Blue Book, AAWSAP, AARO, TTSA, NICAP, and others)
  • Primary Documents: Declassified government reports, memos, and legislative records
  • Historical Timeline: 1,800+ events spanning 1561 to present, sourced from NICAP, UFO Timeline, OpenMinds, and Papoose Lake Archive

Note: Dan Burisch has a bespoke Tier 2 profile component (components/data/key-figures/burisch/) due to his unique data schema. He is one figure among many and should not be treated as the platform's primary subject.

Platform Architecture Priorities

  1. Home: Clean landing page with mission statement and navigation
  2. Data: Core focus - Key Figures, Cases, Documents, Programs, Timeline
  3. Explore: Interactive network graph and timeline overlay visualizations
  4. Resources: Curated materials, transcripts, and glossary
  5. About/Contact: Simple contact form and project information

When exploring code or creating new features, start by understanding related components using the .claude documentation to navigate the codebase efficiently.