Cognitive Bias Detection

Week 12 Implementation Status: ✅ Complete

Overview

The Cognitive Bias Detection system identifies 25+ types of cognitive biases and logical fallacies in conversation nodes using AI-powered analysis. This reveals systematic errors in reasoning, helping teams improve decision quality and identify blind spots.

Key Features

25+ Bias Types: Comprehensive taxonomy across 6 categories
AI-Powered Detection: Uses Claude 3.5 Sonnet with specialized prompts
Severity & Confidence Scoring: Each detection includes 0-1 metrics
Category Distribution: Visualize bias patterns by category
Evidence Extraction: See specific quotes exemplifying each bias
Interactive Filtering: Focus on specific categories or bias types
Node-Level Analysis: Multiple biases detected per node

Bias Taxonomy

6 Categories, 25+ Bias Types

1. Confirmation Biases

Seeking information that confirms existing beliefs

Confirmation Bias: Favoring information that confirms pre-existing beliefs
Cherry Picking: Selecting only supporting data while ignoring contradictory evidence
Motivated Reasoning: Reasoning to reach a desired conclusion
Belief Perseverance: Maintaining beliefs despite contradictory evidence

2. Memory Biases

Distortions in how we recall information

Hindsight Bias: "I knew it all along" - seeing past events as predictable
Availability Heuristic: Overestimating likelihood based on memorability
Recency Bias: Giving undue weight to recent events
False Memory: Remembering events incorrectly

3. Social Biases

Influence of group dynamics and social pressure

Groupthink: Desire for harmony leading to poor decisions
Authority Bias: Overvaluing opinions of authority figures
Bandwagon Effect: Adopting beliefs because many others hold them
Halo Effect: Positive impression in one area influencing opinion in other areas
In-Group Bias: Favoring members of one's own group over outsiders

4. Decision-Making Biases

Systematic errors in judgment

Anchoring: Over-relying on first piece of information encountered
Sunk Cost Fallacy: Continuing investment based on past costs rather than future value
Status Quo Bias: Preferring current state over change
Optimism Bias: Overestimating likelihood of positive outcomes
Planning Fallacy: Underestimating time, costs, and risks

5. Attribution Biases

How we explain behavior and events

Fundamental Attribution Error: Overemphasizing personality while underemphasizing situational factors
Self-Serving Bias: Attributing successes to self and failures to external factors
Just World Hypothesis: Believing the world is fundamentally fair and people get what they deserve

6. Logical Fallacies

Errors in reasoning and argumentation

Slippery Slope: Assuming one action will lead to a chain of negative consequences
Straw Man: Misrepresenting someone's argument to make it easier to attack
False Dichotomy: Presenting only two options when more exist
Ad Hominem: Attacking the person rather than their argument
Appeal to Emotion: Manipulating emotions rather than using valid reasoning
Hasty Generalization: Drawing broad conclusions from limited evidence

Architecture

Backend Services

BiasDetector Service

class BiasDetector:
    async def analyze_conversation(conversation_id, force_reanalysis=False):
        """Returns: total_nodes, nodes_with_biases, bias_count, 
                   by_category, by_bias, nodes"""
        
    async def _analyze_node(node, conversation_id):
        """Returns: List of detected biases with severity, confidence, 
                    evidence"""
    
    async def get_conversation_results(conversation_id):
        """Get cached analysis results"""
    
    async def get_node_biases(node_id):
        """Get biases for specific node"""

Key Features:

Multiple biases per node (one node can have several biases)
Empty array for bias-free nodes
Severity: 0.0 (minor) to 1.0 (severe)
Confidence: 0.0 to 1.0 (only returns >0.6)
Evidence: Specific quotes exemplifying the bias

Database Schema

BiasAnalysis Table

CREATE TABLE bias_analysis (
    id UUID PRIMARY KEY,
    node_id UUID REFERENCES nodes(id),
    conversation_id UUID REFERENCES conversations(id),
    bias_type TEXT NOT NULL,
    category TEXT NOT NULL,
    severity FLOAT CHECK (severity >= 0.0 AND severity <= 1.0),
    confidence FLOAT CHECK (confidence >= 0.0 AND confidence <= 1.0),
    description TEXT,
    evidence JSONB,
    analyzed_at TIMESTAMP DEFAULT NOW()
);

CREATE INDEX idx_bias_node ON bias_analysis(node_id);
CREATE INDEX idx_bias_conversation ON bias_analysis(conversation_id);
CREATE INDEX idx_bias_type ON bias_analysis(bias_type);
CREATE INDEX idx_bias_category ON bias_analysis(category);

Design: One-to-many relationship (one node can have multiple bias analyses)

Frontend Components

BiasAnalysis Page

Route: /biases/:conversationId

Features:

Summary cards: Total nodes, nodes with biases, total biases, average per node
Category distribution: 6 cards with color coding and click-to-filter
Most common biases: Top 10 bias types with counts
Node-by-node results: Detailed bias cards with evidence
Dual filtering: By category AND by specific bias type

Color Scheme:

Confirmation: Purple
Memory: Blue
Social: Green
Decision: Orange
Attribution: Yellow
Logical: Red

API Endpoints

POST /api/conversations/{id}/biases/analyze

Analyze all nodes for cognitive biases.

Query Parameters:

force_reanalysis (boolean): Re-analyze even if results exist

Response:

{
  "total_nodes": 50,
  "analyzed": 50,
  "nodes_with_biases": 18,
  "bias_count": 27,
  "by_category": {
    "social": 10,
    "decision": 8,
    "confirmation": 5,
    "logical": 4
  },
  "by_bias": {
    "groupthink": 4,
    "anchoring": 3,
    "confirmation_bias": 3
  },
  "nodes": [...]
}

GET /api/conversations/{id}/biases

Get cached bias analysis results.

GET /api/nodes/{id}/biases

Get biases for a specific node.

Response:

{
  "biases": [
    {
      "bias_type": "groupthink",
      "category": "social",
      "severity": 0.75,
      "confidence": 0.88,
      "description": "Team showing strong conformity pressure without critical evaluation",
      "evidence": [
        "We all agree this is the right direction",
        "Nobody has raised any concerns"
      ],
      "analyzed_at": "2025-11-12T10:30:00Z"
    }
  ]
}

Usage Workflow

1. Run Analysis

Navigate to /biases/:conversationId and click "Run Analysis".

Processing:

Analyzes each node with Claude 3.5 Sonnet
Returns 0-N biases per node (many nodes have zero)
Only includes biases with confidence > 0.6
~2-3 seconds per node

2. Review Distribution

Category View: See which types of biases are most common

High social biases → groupthink, authority issues
High confirmation biases → echo chamber, not seeking contradictory evidence
High decision biases → systematic errors in planning/judgment
High logical fallacies → poor argumentation

Top Biases View: See most frequent specific biases

Identifies team's systematic blind spots
Reveals patterns across conversations

3. Examine Individual Nodes

Each detected bias shows:

Bias name and category
Severity level (Low/Medium/High)
Confidence score
Description of how it manifests
Evidence quotes from the node

4. Filter and Focus

By Category: Click category card to see only those biases By Bias Type: Click bias name to see all instances Combined: Filter by both category and specific type

Interpretation Guide

Healthy vs. Warning Distributions

Healthy Meeting:

60-70% of nodes have zero biases
Low severity (0.3-0.5) for detected biases
Diverse bias types (not clustered in one category)

Warning Signs:

50% of nodes have biases
High severity (>0.7) biases
Clustered in one category (e.g., all social → groupthink)
Many confirmation biases → echo chamber

Common Patterns

Startup/Innovation Meeting:

Optimism bias (overestimating success)
Planning fallacy (underestimating time)
Some acceptable for motivation

Corporate Strategy:

Status quo bias (resisting change)
Authority bias (deferring to executives)
May indicate risk aversion

Crisis Response:

Availability heuristic (overweighting recent events)
Hasty generalization (jumping to conclusions)
Natural under time pressure, but risky

Testing

Backend Tests

pytest tests/test_bias_detector.py -v

Results: 8 passed, 2 skipped

Coverage:

✅ Detector initialization
✅ Multi-bias detection
✅ Zero-bias handling
✅ Error handling
✅ Empty conversation results
✅ Bias info utilities
✅ Category structure validation

Performance

Cost Analysis

Per-Node: ~$0.006

Input: ~1000 tokens (prompt + node + bias descriptions)
Output: ~400 tokens (JSON with multiple biases)

Example Costs:

25-node conversation: ~$0.15
100-node conversation: ~$0.60
500-node conversation: ~$3.00

Optimization: Results cached, no re-analysis unless forced.

Processing Time

Single node: ~2-3 seconds
50 nodes: ~2-3 minutes (concurrent processing)
Results cached for instant re-display

Integration Points

Week 8: Speaker Analytics

Cross-reference bias detection with speaker analytics:

Which speakers exhibit more biases?
Do facilitators show different bias patterns?
Does speaking time correlate with bias rate?

Week 11: Simulacra Levels

Compare Simulacra levels with bias detection:

Do Level 4 (Simulacrum) nodes have more logical fallacies?
Do Level 1 (Reality) nodes have fewer biases?
Relationship between abstraction and reasoning quality

Decision Quality Metrics

Combine with other analyses:

High biases + High Simulacra Level 3-4 = Risky decisions
Low biases + High Level 1 = Grounded, factual discussions
Social biases + Low speaker diversity = Groupthink risk

File Structure

lct_python_backend/
├── services/
│   └── bias_detector.py           # Detection service (600+ lines)
├── models.py                       # BiasAnalysis model
├── backend.py                      # API endpoints
├── prompts.json                    # bias_detection prompt
└── tests/
    └── test_bias_detector.py      # Unit tests

lct_app/src/
├── pages/
│   └── BiasAnalysis.jsx            # Main UI (450+ lines)
├── services/
│   └── biasApi.js                  # API client (200+ lines)
└── routes/
    └── AppRoutes.jsx               # Route config

Example Analysis

Example 1: Echo Chamber (Warning)

{
  "nodes_with_biases": 35,
  "bias_count": 48,
  "by_category": {
    "confirmation": 25,
    "social": 15,
    "decision": 8
  },
  "by_bias": {
    "confirmation_bias": 12,
    "groupthink": 8,
    "cherry_picking": 7
  }
}

Interpretation: Team in echo chamber. Seeking confirming evidence, groupthink dynamics. Needs devil's advocate or external input.

Example 2: Healthy Discussion

{
  "nodes_with_biases": 8,
  "bias_count": 10,
  "by_category": {
    "decision": 4,
    "memory": 3,
    "social": 2,
    "confirmation": 1
  }
}

Interpretation: Low bias rate, diverse types. Likely productive, critical thinking present.

Troubleshooting

High False Positives

Symptom: Many biases detected that don't seem real

Solutions:

Check confidence scores (should be >0.6)
Review evidence quotes
May indicate unclear node summaries
Consider re-running analysis after improving summaries

No Biases Detected

Symptom: Zero biases across all nodes

Possibilities:

Genuinely factual, unbiased discussion (rare!)
Node summaries too generic/abstract
Prompt needs domain-specific tuning
Check backend logs for errors

Future Enhancements

Planned Features

Temporal Analysis: Track bias evolution over conversation time
Speaker Profiling: Identify each speaker's typical biases
Bias Severity Trends: Watch for increasing severity
Mitigation Suggestions: AI-generated debiasing strategies
Custom Bias Types: Allow users to define domain-specific biases

Research Directions

Inter-Bias Correlations: Which biases co-occur?
Outcome Prediction: Do high bias rates predict poor decisions?
Cultural Variations: Bias patterns across cultures/domains
Intervention Testing: A/B test debiasing techniques

Conclusion

Week 12's Cognitive Bias Detection provides:

Awareness: Identify systematic errors in reasoning
Quality Control: Flag risky decision-making patterns
Team Development: Understand group dynamics and blind spots
Decision Support: Make better-informed judgments

The system is production-ready with comprehensive testing and documentation.

Implementation Date: November 12, 2025 Status: ✅ Complete Lines of Code: ~1,900 (backend + frontend + tests) Test Coverage: 8 tests passing Model: Claude 3.5 Sonnet Cost: ~$0.006 per node

FilesExpand file tree

BIAS_DETECTION.md

Latest commit

History