Add Document Data Extraction Module

infra/modules/document-data-extraction/resources/README.md

@@ -0,0 +1,177 @@
+# Bedrock Data Automation Terraform Module
+
+This module provisions AWS Bedrock Data Automation resources, including the data automation project and blueprints.
+
+
+## Overview
+
+The module creates:
+- **Bedrock Data Automation Project** - Main project resource for data automation workflows
+- **Bedrock Blueprints** - Custom extraction blueprints configured via a list
+
+## Important Notes
+
+- **BDA uses its own internal service role** - This module does not create a custom IAM role for BDA. Bedrock Data Automation uses an AWS-managed internal service role for S3 access.
+- **S3 bucket encryption** - S3 buckets used with BDA should use the KMS encryption key provided by the storage module.
+- **Lambda permissions** - Any Lambda function invoking BDA must have S3 permissions for both input and output buckets (to upload documents and retrieve results) and KMS permissions (for the storage module's encryption key) directly attached to its execution role.
+
+## Features
+- Creates resources required for Bedrock Data Automation workflows
+- Uses a `name` variable to prefix all resource names for uniqueness and consistency
+- Supports both standard and custom output configurations
+- Flexible blueprint creation from file paths or ARNs
+- Complies with Checkov recommendations for security and compliance
+- Designed for cross-layer usage (see project module conventions)
+
+## Usage
+
+```hcl
+module "bedrock_data_automation" {
+  source = "../../modules/document-data-extraction/resources"
+
+  name  = "my-app-prod"
+
+  blueprints = [
+    "./schemas/invoice.json",
+    "arn:aws:bedrock:us-east-1::blueprint/aws-managed-blueprint-id"
+  ]
+
+  standard_output_configuration = {
+    document = {
+      extraction = {
+        granularity = {
+          types = ["PAGE", "ELEMENT"]
+        }
+      }
+    }
+  }
+
+  tags = {
+    Environment = "production"
+    ManagedBy   = "terraform"
+  }
+}
+```
+
+## Inputs
+
+### Required Variables
+
+| Name  | Description | Type | Required |
+|-------|-------------|------|----------|
+| `name` | Prefix to use for resource names (e.g., "my-app-prod") | `string` | yes |
+| `blueprints` | List of blueprint file paths or ARNs. File paths create custom blueprints, ARNs reference existing blueprints. | `list(string)` | [] |
+
+
+### Optional Variables
+
+| Name | Description | Type | Default |
+|------|-------------|------|---------|
+| `standard_output_configuration` | Standard output configuration for extraction | `object` | `null` |
+| `override_configuration` | Override configuration for standard BDA behavior | `string` | `null` |
+| `tags` | Resource tags as key-value pairs | `map(string)` | `{}` |
+
+
+#### `standard_output_configuration` Object Structure
+
+Complex nested object supporting extraction configuration for audio, document, image, and video content types. Each content type supports:
+- **extraction** - Category, bounding box, and granularity configuration
+- **generative_field** - State and types for generative AI fields
+- **output_format** (document only) - Additional file format and text format settings
+
+See `variables.tf` for complete structure details.
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| `bda_project_arn` | The ARN of the Bedrock Data Automation project |
+| `access_policy_arn` | The ARN of the IAM policy for accessing the Bedrock Data Automation project |
+| `bda_profile_arn` | The profile ARN for cross-region inference |
+| `bda_blueprint_arns` | List of created blueprint ARNs |
+| `bda_blueprint_names` | List of created blueprint names |
+| `bda_blueprint_arn_to_name` | Map of blueprint ARNs to names |
+
+## Resources Created
+
+- `awscc_bedrock_data_automation_project.bda_project` - Main BDA project
+- `awscc_bedrock_blueprint.bda_blueprint` - One or more blueprints (created from `blueprints` list)
+
+## Project Conventions
+
+- All resource names are prefixed with `var.name`
+- For cross-layer modules, use the interface/data/resources pattern as described in project documentation
+- Write code that complies with Checkov recommendations
+- Follow Terraform best practices for naming and organization
+
+## File Structure
+
+- `main.tf` - Resource definitions
+- `variables.tf` - Input variable definitions
+- `outputs.tf` - Output values
+- `providers.tf` - Provider configuration
+- `README.md` - This documentation
+
+## Examples
+
+### Minimal Configuration
+```hcl
+module "bedrock_data_automation" {
+  source = "../../modules/document-data-extraction/resources"
+
+  name = "my-app"
+
+  blueprints = [] # No custom blueprints
+}
+```
+
+### With Standard Output Configuration
+```hcl
+module "bedrock_data_automation" {
+  source = "../../modules/document-data-extraction/resources"
+
+  name       = "my-app"
+  blueprints = [] # Add file paths or ARNs here
+
+  standard_output_configuration = {
+    document = {
+      extraction = {
+        bounding_box = {
+          state = "ENABLED"
+        }
+        granularity = {
+          types = ["PAGE", "ELEMENT", "LINE"]
+        }
+      }
+      generative_field = {
+        state = "ENABLED"
+      }
+      output_format = {
+        text_format = {
+          types = ["MARKDOWN", "HTML"]
+        }
+      }
+    }
+    image = {
+      extraction = {
+        category = {
+          state = "ENABLED"
+          types = ["TABLES", "CHARTS"]
+        }
+      }
+    }
+  }
+}
+```
+
+## Prerequisites
+
+- AWS provider configured
+- AWS Cloud Control provider (awscc) configured
+- Appropriate AWS permissions to create Bedrock and IAM resources
+
+## References
+
+- [AWS Bedrock Data Automation](https://docs.aws.amazon.com/bedrock/latest/userguide/data-automation.html)
+- [Project Terraform Conventions](../../../../.github/copilot-instructions.md)
+- [Checkov Documentation](https://www.checkov.io/)

infra/modules/document-data-extraction/resources/access_control.tf

@@ -0,0 +1,24 @@
+resource "aws_iam_policy" "bedrock_access" {
+  name   = "${var.name}-access"
+  policy = data.aws_iam_policy_document.bedrock_access.json
+}
+
+data "aws_iam_policy_document" "bedrock_access" {
+  statement {
+    actions = [
+      "bedrock:InvokeDataAutomationAsync",
+      "bedrock:GetDataAutomationProject",
+      "bedrock:GetBlueprint",
+      "bedrock:StartDataAutomationJob",
+      "bedrock:GetDataAutomationJob",
+      "bedrock:ListDataAutomationJobs"
+    ]
+    effect = "Allow"
+    resources = [
+      awscc_bedrock_data_automation_project.bda_project.project_arn,
+      "${awscc_bedrock_data_automation_project.bda_project.project_arn}/*",
+      "arn:aws:bedrock:*:*:blueprint/*",
+      "arn:aws:bedrock:*:*:data-automation-profile/*"
+    ]
+  }
+}

infra/modules/document-data-extraction/resources/main.tf

@@ -0,0 +1,64 @@
+locals {
+  # convert standard terraform tags to bedrock data automation format
+  bda_tags = [
+    for key, value in var.tags : {
+      key   = key
+      value = value
+    }
+  ]
+
+  blueprint_arns = [
+    for bp in var.blueprints : bp
+    if startswith(bp, "arn:")
+  ]
+
+  blueprint_files = [
+    for bp in var.blueprints : bp
+    if !startswith(bp, "arn:")
+  ]
+
+  # create map of custom blueprints from files
+  custom_blueprints_map = {
+    for file_path in local.blueprint_files :
+    replace(basename(file_path), ".json", "") => {
+      schema = file(file_path)
+      type   = "DOCUMENT"
+      tags   = var.tags
+    }
+  }
+
+  all_blueprints = concat(
+    # custom blueprints created from json schemas
+    [for k, v in awscc_bedrock_blueprint.bda_blueprint : {
+      blueprint_arn   = v.blueprint_arn
+      blueprint_stage = v.blueprint_stage
+    }],
+    # aws managed blueprints referenced by arn
+    length(local.blueprint_arns) > 0 ? [
+      for arn in local.blueprint_arns : {
+        blueprint_arn   = arn
+        blueprint_stage = "LIVE"
+      }
+    ] : []
+  )
+}
+
+resource "awscc_bedrock_data_automation_project" "bda_project" {
+  project_name                  = "${var.name}-project"
+  project_description           = "Project for ${var.name}"
+  tags                          = local.bda_tags
+  standard_output_configuration = var.standard_output_configuration
+  custom_output_configuration = length(local.all_blueprints) > 0 ? {
+    blueprints = local.all_blueprints
+  } : null
+  override_configuration = var.override_configuration
+}
+
+resource "awscc_bedrock_blueprint" "bda_blueprint" {
+  for_each = local.custom_blueprints_map
+
+  blueprint_name = "${var.name}-${each.key}"
+  schema         = each.value.schema
+  type           = each.value.type
+  tags           = local.bda_tags
+}

infra/modules/document-data-extraction/resources/outputs.tf

@@ -0,0 +1,40 @@
+data "aws_region" "current" {}
+data "aws_caller_identity" "current" {}
+
+output "access_policy_arn" {
+  description = "The ARN of the IAM policy for accessing the Bedrock Data Automation project"
+  value       = aws_iam_policy.bedrock_access.arn
+}
+
+output "bda_project_arn" {
+  description = "The ARN of the Bedrock Data Automation project"
+  value       = awscc_bedrock_data_automation_project.bda_project.project_arn
+}
+
+# aws bedrock data automation requires users to use cross Region inference support 
+# when processing files. the following like the profile ARNs for different inference
+# profiles
+# https://docs.aws.amazon.com/bedrock/latest/userguide/bda-cris.html
+# TODO(https://github.com/navapbc/template-infra/issues/993) Add GovCloud Support
+output "bda_profile_arn" {
+  description = "The profile ARN associated with the BDA project"
+  value       = "arn:aws:bedrock:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:data-automation-profile/us.data-automation-v1"
+}
+
+output "bda_blueprint_arns" {
+  value = [
+    for key, bp in awscc_bedrock_blueprint.bda_blueprint : bp.blueprint_arn
+  ]
+}
+
+output "bda_blueprint_names" {
+  value = [
+    for key, bp in awscc_bedrock_blueprint.bda_blueprint : bp.blueprint_name
+  ]
+}
+
+output "bda_blueprint_arn_to_name" {
+  value = {
+    for key, bp in awscc_bedrock_blueprint.bda_blueprint : bp.blueprint_arn => bp.blueprint_name
+  }
+}

infra/modules/document-data-extraction/resources/providers.tf

@@ -0,0 +1,11 @@
+terraform {
+  required_providers {
+    aws = {
+      source = "hashicorp/aws"
+    }
+    awscc = {
+      source  = "hashicorp/awscc"
+      version = ">= 1.63.0"
+    }
+  }
+}