ENHANCEMENTS_DESIGN.md

CrucibleAdversary v0.3.0 Enhancement Design Document

Date: 2025-11-25 Version: 0.3.0 Author: Claude Code Analysis Status: Implementation Ready

---

Executive Summary

This document outlines enhancements to CrucibleAdversary for version 0.3.0, focusing on advanced attack types, improved metrics, and enhanced defense mechanisms. The enhancements address gaps identified in the current v0.2.0 implementation and align with the project roadmap.

Current State Analysis

Implemented Features (v0.2.0)

• ✅ 21 attack types across 5 categories
• ✅ Character, word, and semantic perturbations
• ✅ Prompt injection (4 types)
• ✅ Jailbreak techniques (4 types)
• ✅ Basic defense mechanisms (detection, filtering, sanitization)
• ✅ Core metrics (accuracy drop, ASR, consistency)
• ✅ 203 tests with 88.5% coverage

Identified Gaps

1. Missing Attack Categories (from roadmap Phase 3):

   • Data extraction attacks
   • Model inversion attacks
   • Membership inference attacks
   • Backdoor/poisoning attacks
   • Gradient-based attacks

2. Limited Metrics:

   • No certified robustness measurements
   • No adversarial training support metrics
   • Limited statistical rigor in consistency measurements

3. Defense Limitations:

   • Pattern-based detection only (no ML-based)
   • No adversarial training data generation
   • Limited sanitization strategies

4. Missing Capabilities:

   • No attack composition framework
   • No adaptive attack strategies
   • No report generation module

Enhancement Scope for v0.3.0

Based on roadmap Phase 3 priorities and feasibility analysis, v0.3.0 will focus on:

1. Data Extraction Attacks

Priority: HIGH Complexity: Medium Rationale: Critical security vulnerability testing capability

1.1 Prompt-Based Data Extraction

Extract training data through carefully crafted prompts.

defmodule CrucibleAdversary.Attacks.Extraction do
  @doc """
  Attempts to extract training data through repetition attacks.

  ## Examples
      {:ok, result} = Extraction.repetition_attack(
        prompt: "Repeat the following: ",
        target_length: 500
      )
  """
  def repetition_attack(prompt, opts)

  @doc """
  Probes for memorized patterns in training data.
  """
  def memorization_probe(prompt, opts)

  @doc """
  Tests for PII leakage through context manipulation.
  """
  def pii_extraction(prompt, opts)
end

1.2 Model Inversion

Reconstruct training examples from model outputs.

defmodule CrucibleAdversary.Attacks.Inversion do
  @doc """
  Attempts model inversion to recover training data characteristics.

  Uses output analysis to infer input properties.
  """
  def inversion_attack(model, opts)

  @doc """
  Membership inference: determine if specific data was in training set.
  """
  def membership_inference(model, candidate_data, opts)
end

2. Advanced Robustness Metrics

Priority: HIGH Complexity: Medium-High Rationale: Research-grade evaluation requires rigorous metrics

2.1 Certified Robustness

Provable robustness guarantees using randomized smoothing.

defmodule CrucibleAdversary.Metrics.Certified do
  @doc """
  Calculates certified robustness radius using randomized smoothing.

  ## Parameters
    * model - The model to evaluate
    * input - Input text to certify
    * num_samples - Number of random samples (default: 1000)
    * noise_std - Noise standard deviation (default: 0.1)

  ## Returns
    %{
      certified_radius: float(),
      confidence: float(),
      base_prediction: any(),
      certification_level: :high | :medium | :low | :none
    }
  """
  def randomized_smoothing(model, input, opts)

  @doc """
  Calculates worst-case robustness using interval bound propagation.
  """
  def interval_bound_propagation(model, input, perturbation_budget)
end

2.2 Attack Transferability

Measure how attacks transfer across models.

defmodule CrucibleAdversary.Metrics.Transferability do
  @doc """
  Evaluates attack transferability across multiple models.

  ## Returns
    %{
      source_model_success_rate: float(),
      transfer_success_rates: %{model_name => float()},
      average_transfer_rate: float(),
      transferability_score: float()
    }
  """
  def evaluate_transferability(source_model, target_models, attacks, test_set)
end

3. Enhanced Defense Mechanisms

Priority: MEDIUM Complexity: Medium Rationale: Complete defense-in-depth strategy

3.1 ML-Based Attack Detection

Augment pattern-based detection with learned classifiers.

defmodule CrucibleAdversary.Defenses.MLDetector do
  @doc """
  Trains a classifier on adversarial vs. benign examples.

  Uses character n-gram features and statistical properties.
  """
  def train_detector(training_data, opts)

  @doc """
  Detects adversarial inputs using trained model.

  ## Returns
    %{
      is_adversarial: boolean(),
      confidence: float(),
      detection_method: :pattern | :ml | :ensemble,
      feature_importance: map()
    }
  """
  def detect_with_ml(input, trained_model)
end

3.2 Adversarial Training Support

Generate adversarial training data.

defmodule CrucibleAdversary.Defenses.AdversarialTraining do
  @doc """
  Generates augmented training dataset with adversarial examples.

  ## Parameters
    * original_dataset - Clean training data
    * attack_types - List of attacks to use for augmentation
    * augmentation_ratio - Ratio of adversarial to clean (default: 0.3)

  ## Returns
    Augmented dataset with labeled adversarial examples
  """
  def generate_training_data(original_dataset, attack_types, opts)

  @doc """
  Creates curriculum for progressive adversarial training.
  """
  def create_curriculum(dataset, difficulty_schedule)
end

4. Attack Composition Framework

Priority: MEDIUM Complexity: Low-Medium Rationale: Enable sophisticated multi-stage attacks

defmodule CrucibleAdversary.Composition do
  @doc """
  Chains multiple attacks sequentially.

  ## Example
      pipeline = Composition.chain([
        {:character_swap, rate: 0.1},
        {:prompt_injection_basic, payload: "Override"},
        {:jailbreak_encode, encoding: :base64}
      ])

      {:ok, result} = Composition.execute(pipeline, input)
  """
  def chain(attack_specs)

  @doc """
  Executes attacks in parallel and selects best result.
  """
  def parallel_best(attack_specs, selection_criterion)

  @doc """
  Conditional attack execution based on intermediate results.
  """
  def conditional(attack_spec, condition_fn)
end

5. Report Generation

Priority: LOW-MEDIUM Complexity: Low Rationale: Professional evaluation reporting

defmodule CrucibleAdversary.Reports.Generator do
  @doc """
  Generates comprehensive robustness report.

  ## Formats
    * :markdown - Human-readable Markdown
    * :latex - Academic paper format
    * :html - Interactive visualization
    * :json - Machine-readable data
  """
  def generate_report(evaluation_results, format, opts)

  @doc """
  Creates vulnerability assessment report.
  """
  def vulnerability_report(scan_results, severity_threshold)
end

Architecture Changes

New Modules

lib/crucible_adversary/
├── attacks/
│   ├── extraction.ex          # NEW: Data extraction attacks
│   └── inversion.ex           # NEW: Model inversion attacks
├── metrics/
│   ├── certified.ex           # NEW: Certified robustness
│   └── transferability.ex     # NEW: Attack transferability
├── defenses/
│   ├── ml_detector.ex         # NEW: ML-based detection
│   └── adversarial_training.ex # NEW: Training data generation
├── composition.ex             # NEW: Attack composition
└── reports/
    └── generator.ex           # NEW: Report generation

Updated Modules

• lib/crucible_adversary.ex - Add new attack types to main API
• lib/crucible_adversary/evaluation/robustness.ex - Integrate new metrics
• lib/crucible_adversary/defenses/detection.ex - Add ML detection option

Implementation Plan

Phase 1: Core Attacks (Week 1)

1. ✅ Design document creation
2. Implement Attacks.Extraction module
   • repetition_attack/2
   • memorization_probe/2
   • pii_extraction/2
3. Implement Attacks.Inversion module
   • membership_inference/3
   • attribute_inference/3
4. Write comprehensive tests (TDD approach)
5. Update main API to expose new attacks

Phase 2: Advanced Metrics (Week 1)

1. Implement Metrics.Certified module
   • randomized_smoothing/3
   • Statistical certification
2. Implement Metrics.Transferability module
   • Cross-model evaluation
   • Transferability scoring
3. Add metric tests
4. Update evaluation framework

Phase 3: Enhanced Defenses (Week 2)

1. Implement Defenses.MLDetector module
   • Feature extraction
   • Classifier training
   • Detection interface
2. Implement Defenses.AdversarialTraining module
   • Data augmentation
   • Curriculum generation
3. Integration tests

Phase 4: Composition & Reporting (Week 2)

1. Implement Composition module
   • Pipeline DSL
   • Execution engine
2. Implement Reports.Generator module
   • Multi-format export
   • Visualization generation
3. End-to-end integration tests

Technical Specifications

Data Extraction Implementation

Repetition Attack

Mechanism: Exploit model's tendency to continue patterns.

# Algorithm
1. Construct prompt with repetition trigger: "Repeat: X X X..."
2. Query model with increasing repetition count
3. Analyze output for memorized continuations
4. Extract and validate leaked information

Success Criteria:

• Output contains data not in prompt
• Data appears to be from training set
• Data is coherent and structured

Membership Inference

Mechanism: Statistical analysis of model confidence.

# Algorithm
1. Query model with candidate training examples
2. Measure prediction confidence/perplexity
3. Compare to non-member baseline
4. Statistical test for significant difference
5. Classify as member/non-member

Metrics:

• True Positive Rate (sensitivity)
• False Positive Rate (1 - specificity)
• AUC-ROC for classification quality

Certified Robustness Implementation

Randomized Smoothing

Mechanism: Create smoothed classifier via random noise.

# Algorithm
1. Add random noise to input (character-level)
2. Sample model predictions (n=1000)
3. Calculate most frequent prediction (base class)
4. Compute certification radius using binomial proportion
5. Return certified radius and confidence

Mathematical Foundation:

• Based on Cohen et al. (2019) "Certified Adversarial Robustness via Randomized Smoothing"
• Provides provable L2 robustness guarantees
• Adapted for text domain using character-level noise

ML-Based Detection Implementation

Feature Engineering

# Features for adversarial detection
1. Character n-gram statistics (n=2,3,4)
2. Word frequency distribution
3. Delimiter density
4. Entropy measurements
5. Vocabulary diversity
6. Syntactic complexity
7. Pattern match counts

Classifier Architecture

# Logistic regression with feature engineering
- Input: Text features (50-100 dimensions)
- Model: Regularized logistic regression
- Training: Supervised on labeled adversarial/benign
- Output: Probability score + binary classification

Testing Strategy

Test Coverage Goals

• Unit tests: 95%+ coverage
• Integration tests: All attack pipelines
• Property-based tests: Invariants and edge cases
• Performance tests: Latency benchmarks

Test Categories

1. Attack Tests

defmodule CrucibleAdversary.Attacks.ExtractionTest do
  test "repetition attack generates valid attack result" do
    {:ok, result} = Extraction.repetition_attack("Test", target_length: 100)
    assert result.attack_type == :data_extraction_repetition
    assert String.length(result.attacked) >= 100
  end

  test "membership inference detects training examples" do
    model = TestModel
    training_example = "Known training text"
    {:ok, result} = Inversion.membership_inference(model, training_example)
    assert result.is_member == true
    assert result.confidence > 0.6
  end
end

2. Metric Tests

defmodule CrucibleAdversary.Metrics.CertifiedTest do
  test "randomized smoothing provides certification" do
    model = TestModel
    input = "Test input"
    {:ok, cert} = Certified.randomized_smoothing(model, input, num_samples: 100)

    assert is_float(cert.certified_radius)
    assert cert.certified_radius >= 0
    assert cert.confidence > 0.5
  end
end

3. Integration Tests

defmodule CrucibleAdversary.Integration.AdvancedAttackTest do
  test "composed attack pipeline executes successfully" do
    pipeline = Composition.chain([
      {:character_swap, rate: 0.1},
      {:data_extraction_repetition, target_length: 100}
    ])

    {:ok, result} = Composition.execute(pipeline, "Input text")
    assert length(result.intermediate_results) == 2
    assert result.final_attack.attack_type == :data_extraction_repetition
  end
end

Performance Considerations

Computational Complexity

Feature	Time Complexity	Space Complexity	Notes
Repetition Attack	O(n)	O(n)	Linear in output length
Membership Inference	O(k*m)	O(k)	k=queries, m=model inference
Randomized Smoothing	O(n*s)	O(s)	n=samples, s=sample size
ML Detection	O(f)	O(f)	f=feature dimension
Attack Composition	O(Σc_i)	O(n)	Sum of component complexities

Optimization Strategies

1. Caching: Cache model responses for repeated queries
2. Parallelization: Parallel execution of random samples
3. Lazy Evaluation: Compute metrics only when requested
4. Batch Processing: Group similar operations

Performance Targets

• Data extraction attack: < 100ms per attempt
• Membership inference: < 500ms per example
• Randomized smoothing (1000 samples): < 10s
• ML detection: < 10ms per input
• Report generation: < 5s for standard report

Security & Ethical Considerations

Responsible Disclosure

• All attacks for defensive research only
• No generation of actually harmful content
• Clear documentation of ethical usage

Safety Guardrails

defmodule CrucibleAdversary.Safety do
  @doc "Validates attack usage is within ethical bounds"
  def validate_usage(attack_type, target, opts) do
    # Check for production environment
    # Verify research/testing context
    # Ensure informed consent
    # Log all operations for audit
  end
end

Privacy Protection

• No storage of extracted PII
• Anonymization of training data samples
• Clear data retention policies
• GDPR compliance considerations

Migration & Backwards Compatibility

API Compatibility

• All v0.2.0 APIs remain unchanged
• New features are additive only
• Deprecation warnings for future changes
• Comprehensive migration guide

Breaking Changes

None - v0.3.0 is fully backwards compatible with v0.2.0

Deprecation Schedule

• No deprecations in v0.3.0
• Future v0.4.0 may deprecate old metric APIs in favor of unified interface

Documentation Updates

New Documentation Required

1. docs/20251125/DATA_EXTRACTION.md - Data extraction attack guide
2. docs/20251125/CERTIFIED_ROBUSTNESS.md - Certification methodology
3. docs/20251125/COMPOSITION.md - Attack composition patterns
4. docs/20251125/REPORT_GENERATION.md - Report format specifications

README Updates

• Add new attack types to features list
• Update attack count (21 → 27)
• Add certified robustness to metrics
• Update examples with new features

API Documentation

• Complete ExDoc coverage for all new modules
• Code examples for all public functions
• Performance characteristics documentation
• Security considerations for each attack

Success Criteria

Functional Requirements

• ✅ All 6 new attack types implemented and tested
• ✅ Certified robustness metric with >90% accuracy on test cases
• ✅ ML detector with >85% precision/recall on validation set
• ✅ Attack composition supporting 3+ chaining strategies
• ✅ Report generation in 3+ formats

Quality Requirements

• ✅ Test coverage >90%
• ✅ Zero compilation warnings
• ✅ Zero Dialyzer errors
• ✅ All tests passing
• ✅ Performance targets met

Documentation Requirements

• ✅ Complete API documentation
• ✅ Usage examples for all features
• ✅ Integration guide updated
• ✅ CHANGELOG.md updated

Risk Assessment

Technical Risks

Risk	Probability	Impact	Mitigation
Randomized smoothing too slow	Medium	Medium	Implement sampling optimization, caching
ML detector overfits	Medium	Low	Cross-validation, regularization
Attack composition complexity	Low	Medium	Start with simple chains, iterate
Memory usage with large reports	Low	Low	Streaming generation, pagination

Mitigation Strategies

1. Performance: Early benchmarking, profiling
2. Quality: Rigorous TDD, code review
3. Complexity: Incremental implementation, modular design
4. Compatibility: Comprehensive integration tests

Future Roadmap (Post v0.3.0)

v0.4.0 - Integration & Scale

• CrucibleBench integration
• Distributed evaluation
• Real-time monitoring
• Cloud deployment

v0.5.0 - Advanced Features

• Gradient-based attacks
• Adaptive attack strategies
• Multi-modal attacks
• Reinforcement learning

v1.0.0 - Production Hardening

• API stabilization
• Performance optimization
• Security audit
• Enterprise features

Conclusion

Version 0.3.0 represents a significant advancement in CrucibleAdversary's capabilities, adding critical data extraction attacks, certified robustness metrics, and enhanced defense mechanisms. The implementation follows strict TDD principles, maintains backwards compatibility, and positions the framework for future advanced features.

Estimated Implementation Time: 2-3 weeks Lines of Code (Estimated): ~2,500 new LOC, ~1,500 test LOC Test Count Increase: +85 tests (203 → 288) Coverage Target: 90%+

---

Approval Status: Ready for Implementation Next Steps: Begin Phase 1 implementation with TDD approach