BMAD Federated Knowledge System

A comprehensive knowledge management system for AI-driven development that enables distributed knowledge sharing across multiple sources including Git repositories, web pages, and databases.

You can read the associated article for this code here

📚 Knowledge Management with BMAD Federated Knowledge

BMAD Federated Knowledge System provides a centralized way to manage context, knowledge, and tasks for AI-driven development projects. This system:

• Centralizes Knowledge Sources: Aggregates content from Git repositories, web pages, and databases into a unified knowledge base
• Creates Unified Context: Generates a context.md file that maps all knowledge sources to their cached content
• Enables AI-Driven Development: Provides rich context for AI agents to understand your project's domain and requirements
• Supports Multi-Source Integration: Combines knowledge from disparate sources into a cohesive knowledge system

The Context.md File

The heart of the system is the context.md file generated by the build-context command. This file:

• Maps each knowledge source to its cached file location
• Provides descriptions and metadata about each source
• Groups sources by type (repository, web, database)
• Serves as the entry point for AI agents to navigate the knowledge system

All knowledge sources are cached locally:

• Git repositories in .bmad-fks-cache/[repo-name]
• Web pages as PDFs in .bmad-fks-cache/[source-name].pdf
• Database query results as PDF or JSON in bmad-cache/db-knowledge/[source-name].[pdf|json]

This approach ensures that your AI tools have access to all the necessary context without requiring constant network access.

🚀 Features

• Multi-Source Knowledge Management: Integrate Git repositories, web pages, and database queries
• Git-Based Architecture: Leverage familiar Git workflows for knowledge management
• Federated Repositories: Distribute knowledge across multiple repositories
• Priority-Based Conflict Resolution: Intelligent merging with configurable priorities
• Unified Context Generation: Create comprehensive context.md file mapping all knowledge sources
• Database Knowledge Integration: Query databases and cache results as PDF or JSON
• Web Page Knowledge Capture: Save web pages as PDF for offline reference
• Offline Support: Work offline with cached local copies
• Flexible Sync Policies: Daily, weekly, on-demand, or manual synchronization
• CLI Management: Comprehensive command-line interface for knowledge management
• Seamless Integration: Extends existing BMAD-METHOD functionality

🗺️ Future Roadmap

• Web UI for knowledge source management
• Advanced conflict resolution strategies
• Repository templates and scaffolding
• Integration with popular Git hosting services
• Performance optimizations for large repositories
• Backup and restore functionality
• Support for additional knowledge source types (e.g., APIs, documents)
• Enhanced filtering and selection of knowledge subsets
• Improved offline support and caching strategies
• AI-assisted knowledge summarization and indexing
• Bi-directional sync for database sources

📦 Installation

Note: For optional dependencies like PDFKit (needed for database sync to PDF), see Optional Dependencies.

npm install bmad-federated-knowledge

Or install globally for CLI usage:

npm install -g bmad-federated-knowledge

🏗️ Architecture

The system extends BMAD-METHOD's core configuration to support federated knowledge repositories, web sources, and database connections:

bmad_config:
  version: "2.0"
  
  # Local knowledge (existing)
  local_knowledge:
    core_data: "./.bmad-fks-core/core-data"
    templates: "./.bmad-fks-core/templates"
    workflows: "./.bmad-fks-core/workflows"
  
  # Federated knowledge repositories
  federated_knowledge:
    org_standards:
      repo: "git@github.com:company/bmad-org-standards.git"
      branch: "main"
      local_cache: "./.bmad-fks-cache/org-standards"
      sync_policy: "daily"
      priority: 1
    
    industry_templates:
      repo: "git@github.com:bmad-community/software-templates.git"
      branch: "stable"
      local_cache: "./.bmad-fks-cache/industry"
      sync_policy: "weekly"
      priority: 0
      
  # Database connections
  connections:
    mongodb_prod:
      type: "mongodb"
      connection_string: "mongodb://username:password@host:27017/dbname"
      metadata:
        description: "Production MongoDB database"
    
    mysql_analytics:
      type: "mysql"
      connection_string: "mysql://user:pass@host:3306/analytics"
      metadata:
        description: "Analytics MySQL database"
  
  # Knowledge sources (web pages, database queries)
  knowledge_sources:
    api_docs:
      type: "web"
      url: "https://api.example.com/documentation"
      sync_policy: "weekly"
      metadata:
        description: "API documentation"
    
    active_users:
      type: "database"
      connection_ref: "mongodb_prod"
      query: "SELECT * FROM users WHERE active = true"
      sync_policy: "daily"
      metadata:
        description: "Active users from MongoDB"

🚀 Quick Start

1. Initialize the System

# Initialize with default configuration
bmad-fed init

# Or specify custom config path
bmad-fed init --config ./my-config.yaml

Database Knowledge Sources

# Add a database connection
bmad-fed connection-add my-mongo -t mongodb -s "mongodb://localhost:27017/mydb"

# Add a database knowledge source
bmad-fed add-knowledge employee_data -t database --connection-ref my-mongo --query "SELECT * FROM employees"

# Sync database data to PDF (requires PDFKit)
bmad-fed sync-db employee_data

# Or sync to JSON format
bmad-fed sync-db employee_data --json

# Sync all database sources
bmad-fed sync-db --all

2. Add Federated Repositories

# Interactive mode
bmad-fed add my-repo --interactive

# Command line mode
bmad-fed add my-repo \
  --repo git@github.com:company/knowledge.git \
  --branch main \
  --priority 1 \
  --sync-policy daily

3. Sync Repositories

# Sync all repositories
bmad-fed sync

# Sync specific repository
bmad-fed sync my-repo

# Force sync (ignore policies)
bmad-fed sync --force

4. Check Status

# Basic status
bmad-fed status

# Detailed status
bmad-fed status --verbose

5 . Add a webpage or other knowledge source

npx bmad-fed add-knowledge global_warming

and then provide all the details

C:\work\comicsforall>npx bmad-fed add-knowledge global_warming
2025-09-14 09:31:02 [info]: Initializing BMAD Federated Knowledge System... {"service":"bmad-federated-knowledge"}
2025-09-14 09:31:02 [info]: Git Manager initialized with cache directory: ./.bmad-fks-cache {"service":"bmad-federated-knowledge"}
2025-09-14 09:31:02 [info]: Initializing Federated Dependency Resolver... {"service":"bmad-federated-knowledge"}
2025-09-14 09:31:02 [info]: Configuration loaded from ./.bmad-fks-core/fks-core-config.yaml {"service":"bmad-federated-knowledge"}
2025-09-14 09:31:02 [info]: Initialized 4 federated repositories {"service":"bmad-federated-knowledge"}
2025-09-14 09:31:02 [info]: Federated Dependency Resolver initialized successfully {"service":"bmad-federated-knowledge"}
2025-09-14 09:31:02 [info]: BMAD Federated Knowledge System initialized successfully {"service":"bmad-federated-knowledge"}
? Select knowledge source type: web
? Webpage URL: https://www.nrdc.org/stories/global-warming-101
? Priority (0-999): 1
? Description (optional): knowledge on global warming
- Adding knowledge source: global_warming2025-09-14 09:32:42 [info]: Added knowledge source: global_warming {"service":"bmad-federated-knowledge"}
Syncing webpage: https://www.nrdc.org/stories/global-warming-101
- Adding knowledge source: global_warmingWebpage https://www.nrdc.org/stories/global-warming-101 saved to C:\work\comicsforall\.bmad-fks-cache\global_warming.pdf
√ Knowledge source "global_warming" added successfully!

6. Resolve Knowledge

C:\work\comicsforall>npx bmad-fed sync-web global_warming
2025-09-14 09:33:47 [info]: Initializing BMAD Federated Knowledge System... {"service":"bmad-federated-knowledge"}
2025-09-14 09:33:47 [info]: Git Manager initialized with cache directory: ./.bmad-fks-cache {"service":"bmad-federated-knowledge"}
2025-09-14 09:33:47 [info]: Initializing Federated Dependency Resolver... {"service":"bmad-federated-knowledge"}
2025-09-14 09:33:47 [info]: Configuration loaded from ./.bmad-fks-core/fks-core-config.yaml {"service":"bmad-federated-knowledge"}
2025-09-14 09:33:47 [info]: Initialized 4 federated repositories {"service":"bmad-federated-knowledge"}
2025-09-14 09:33:47 [info]: Federated Dependency Resolver initialized successfully {"service":"bmad-federated-knowledge"}
2025-09-14 09:33:47 [info]: BMAD Federated Knowledge System initialized successfully {"service":"bmad-federated-knowledge"}
- Syncing repository: global_warmingSyncing webpage: https://www.nrdc.org/stories/global-warming-101
/ Syncing repository: global_warmingWebpage https://www.nrdc.org/stories/global-warming-101 saved to C:\work\comicsforall\.bmad-fks-cache\global_warming.pdf
√ Wep page "global_warming" synced successfully!

as you can see the webpage is saved as a pdf in the cache folder this you can use with you analyst agent as additional context

7. Build entire context

npx bmad-fed build-context

This command creates a comprehensive knowledge context from all sources:

1. Syncs Git repositories from your federated knowledge configuration
2. Syncs database sources and generates PDFs/JSON files from query results
3. Syncs web pages and saves them as PDFs
4. Flattens repositories into XML files for better AI consumption
5. Generates context.md with a complete map of all knowledge sources

Context is the single most essential element for AI-driven development. An AI model's ability to produce relevant, accurate code is directly tied to its understanding of the "why" and "how" behind a task. Context isn't just the prompt you type; it is a comprehensive ecosystem of knowledge.

After running the command:

• Repository content is flattened into XML files in the .bmad-fks-cache/[repo-name].xml folder
• Web pages are saved as PDFs in the .bmad-fks-cache/[source-name].pdf files
• Database query results are saved as PDFs/JSON in bmad-cache/db-knowledge/[source-name].[pdf|json]
• The context.md file is created in the root folder with links to all these resources

This unified context can be used by your agents to access all relevant knowledge across your project ecosystem.

📚 API Usage

Basic Usage

const { BmadFederatedKnowledge } = require('bmad-federated-knowledge');

const bmadFed = new BmadFederatedKnowledge({
  configPath: './.bmad-fks-core/core-config.yaml',
  cacheDir: './.bmad-fks-cache',
  logLevel: 'info'
});

// Initialize the system
await bmadFed.initialize();

// Resolve knowledge for an agent
const knowledge = await bmadFed.resolveKnowledge({
  dependencies: ['templates', 'workflows']
});

// Sync all repositories
const syncResults = await bmadFed.syncAll();

Advanced Configuration

const { FederatedDependencyResolver, GitManager, KnowledgeMerger } = require('bmad-federated-knowledge');

// Custom Git Manager
const gitManager = new GitManager({
  cacheDir: './custom-cache',
  timeout: 600000, // 10 minutes
  retryAttempts: 5
});

// Custom Knowledge Merger
const knowledgeMerger = new KnowledgeMerger({
  conflictResolution: 'priority',
  mergeStrategies: {
    templates: 'priority',
    workflows: 'merge',
    data: 'manual'
  }
});

// Custom Dependency Resolver
const resolver = new FederatedDependencyResolver({
  gitManager,
  knowledgeMerger,
  parallelSync: true
});

🔧 Configuration

Repository Configuration

Each federated repository can be configured with the following options:

repo_name:
  repo: "git@github.com:user/repo.git"     # Repository URL (required)
  branch: "main"                           # Branch to sync (default: main)
  local_cache: "./.bmad-fks-cache/repo_name"    # Local cache path (required)
  sync_policy: "weekly"                    # Sync policy (daily|weekly|on_demand|manual)
  priority: 1                              # Priority for conflict resolution (0-999)
  
  # Authentication (optional)
  auth:
    type: "ssh"                            # ssh|token|basic
    token: "${GITHUB_TOKEN}"               # For token auth
    username: "user"                       # For basic auth
    password: "pass"                       # For basic auth
  
  # Filters (optional)
  filters:
    include: ["templates/**", "workflows/**"]
    exclude: ["*.tmp", "*.log"]
  
  # Metadata (optional)
  metadata:
    description: "Repository description"
    maintainer: "team@company.com"
    tags: ["templates", "workflows"]

Global Settings

federated_settings:
  cache_root: "./.bmad-fks-cache"               # Root cache directory
  max_cache_size: "1GB"                   # Maximum cache size
  sync_timeout: 300                       # Sync timeout in seconds
  retry_attempts: 3                       # Number of retry attempts
  parallel_sync: true                     # Enable parallel syncing
  conflict_resolution: "priority"         # priority|manual|local_wins

🔄 Sync Policies

• daily: Sync once per day
• weekly: Sync once per week
• on_demand: Only sync when explicitly requested
• manual: Never auto-sync, manual sync only

⚡ Conflict Resolution

The system supports three conflict resolution strategies:

1. Priority: Higher priority sources override lower priority ones
2. Local Wins: Local knowledge always takes precedence
3. Manual: Conflicts require manual resolution

📋 CLI Commands

Repository Management

# Add repository
bmad-fed add <n> [options]

# Remove repository
bmad-fed remove <n> [--force]

# List repositories
bmad-fed list [--json]

Database Connection Management

# Add a database connection
bmad-fed connection-add <name> -t <type> -s <connection-string>

# List all connections
bmad-fed connection-list

# Remove a connection
bmad-fed connection-remove <name>

# Test a connection
bmad-fed connection-test <name>

Database Knowledge Management

# Sync database to PDF or JSON
bmad-fed sync-db <name>

# Sync all database sources
bmad-fed sync-db --all

# Use mock data for testing (no actual connection)
bmad-fed sync-db <name> --mock

# Force sync to JSON format
bmad-fed sync-db <name> --json

Synchronization

# Sync all repositories
bmad-fed sync [--force] [--parallel]

# Sync specific repository
bmad-fed sync <n> [--force]

Status and Maintenance

# Show status
bmad-fed status [--verbose]

# Clean cache
bmad-fed clean [name] [--force]

# Validate configuration
bmad-fed validate [config-path]

System Management

# Initialize system
bmad-fed init [--config path] [--force]

🧪 Testing

# Run tests
npm test

# Run tests with coverage
npm run test:coverage

# Run tests in watch mode
npm run test:watch

🔍 Debugging

Enable debug logging:

const bmadFed = new BmadFederatedKnowledge({
  logLevel: 'debug'
});

Or set environment variable:

export BMAD_LOG_LEVEL=debug
bmad-fed status

Multi-Repository Context Enhancement Workflow

When your logic spans multiple repositories, use this workflow:

Sync repositories:

  bmad-fed sync

Flatten each relevant repository:

   npx bmad-method flatten -i .\.bmad-fks-cache\backend-templates
   npx bmad-method flatten -i .\.bmad-fks-cache\frontend-components  
   npx bmad-method flatten -i .\.bmad-fks-cache\shared-workflows

Use XML files for context enhancement:

Load XML files into your AI context
Reference cross-repository knowledge
Maintain consistency across distributed logic
Enable intelligent code generation spanning multiple repos

🤝 Contributing

1. Fork the repository
2. Make sure you dont break anything
3. Add your features :-)
4. Submit a pull request