nist-ai-cards

AI Risk Assessment & NIST AI RMF-Aligned Model Card Generator

nist-ai-cards is a production-quality Python library for creating, assessing, auditing, and exporting AI model cards aligned with the NIST AI Risk Management Framework (AI RMF).

Features

• Model Card Generator — produce structured model cards in Markdown, HTML, and JSON
• Risk Assessor — map model characteristics to NIST AI RMF categories (Govern, Map, Measure, Manage)
• Audit Trail — version-track card changes, produce diffs, and export compliance snapshots
• Pre-built Templates — jumpstart cards for classification, LLM deployment, recommendation systems, and anomaly detection
• Pydantic v2 schemas — validated, type-safe data models throughout

Installation

pip install nist-ai-cards
# or with uv
uv add nist-ai-cards

Quick Start

from nist_ai_cards.generator import ModelCardGenerator
from nist_ai_cards.risk import RiskAssessor
from nist_ai_cards.audit import AuditTrail
from nist_ai_cards.templates import create_from_template

# Build a model card
generator = ModelCardGenerator(
    model_name="Customer Churn Classifier",
    model_type="XGBoost",
    version="1.0.0",
    intended_use="Predict 30-day churn probability for B2C accounts.",
    training_data_description="12 months of anonymised CRM data.",
    evaluation_metrics={"auc": 0.93, "f1": 0.89},
    known_limitations=["Degrades on out-of-distribution data."],
    ethical_considerations=["Bias evaluation conducted across demographic groups."],
    author="ml-team@example.com",
)

card = generator.generate()
print(generator.to_markdown(card))

# Assess risk against NIST AI RMF
assessor = RiskAssessor(
    model_card=card,
    deployment_context={
        "decision_impact": "medium",
        "data_sensitivity": "pii",
        "regulatory_domain": ["financial_services"],
    },
)
report = assessor.assess()
print(f"Risk level: {report.overall_level.value}")

# Track versions
audit = AuditTrail("audit_logs/")
audit.record(card, version="1.0.0", author="ml-team@example.com")

# Use a template
gen = create_from_template("llm_deployment", model_name="Internal LLM v2")
card2 = gen.generate()

Available Templates

Template Name	Model Type
general_classification	Classification Model
llm_deployment	Large Language Model
recommendation_system	Recommendation System
anomaly_detection	Anomaly Detection Model

NIST AI RMF Alignment

The risk assessor evaluates model cards across all four RMF functions:

Function	Checks
Govern	Model ownership, author, version, license
Map	Intended use, known limitations, training data
Measure	Evaluation metrics, bias and fairness testing
Manage	Model type, rollback procedures, regulatory domain

Development

git clone <repo>
cd nist-ai-cards
uv sync --all-extras
uv run pytest tests/ -v --cov=src --cov-report=term-missing
uv run isort . && uv run black .

License

Apache 2.0 — see LICENSE.