Skip to content

feat(fsx): add L2 construct for DataRepositoryAssociation - #38141

Open
gingeekrishna wants to merge 8 commits into
aws:mainfrom
gingeekrishna:feat/34649-fsx-data-repository-association
Open

feat(fsx): add L2 construct for DataRepositoryAssociation#38141
gingeekrishna wants to merge 8 commits into
aws:mainfrom
gingeekrishna:feat/34649-fsx-data-repository-association

Conversation

@gingeekrishna

Copy link
Copy Markdown

Summary

Fixes #34649

Adds DataRepositoryAssociation, an L2 construct for AWS::FSx::DataRepositoryAssociation. This links an S3 bucket to a path on an FSx for Lustre file system so data can be automatically imported from S3 into the file system and automatically exported from the file system back to S3.

Before this change, users had to reach for CfnDataRepositoryAssociation directly, with no type safety or validation on event types, path format, or chunk size ranges.

Changes

New file: data-repository-association.ts

  • DataRepositoryEventType enum — NEW | CHANGED | DELETED
  • S3AutoImportPolicy / S3AutoExportPolicy interfaces with typed events arrays
  • S3DataRepositoryConfiguration grouping both policies
  • DataRepositoryAssociationProps with fileSystem, bucket, fileSystemPath, optional bucketPrefix, s3, importedFileChunkSizeMiB, batchImportMetaDataOnCreate, removalPolicy
  • DataRepositoryAssociation L2 class with:
    • Validates fileSystemPath starts with /
    • Validates importedFileChunkSizeMiB is 1–512,000
    • Validates event arrays are non-empty
    • Builds dataRepositoryPath from bucket + optional bucketPrefix
    • Calls bucket.grantReadWrite(fsx.amazonaws.com) automatically

Updated: index.ts — exports new construct

New file: data-repository-association.test.ts — 12 unit tests covering happy path, prefix handling, policy configuration, bucket policy grant, and all validation errors

Example usage

const bucket = new s3.Bucket(this, 'DataBucket');
const fileSystem = new fsx.LustreFileSystem(this, 'FileSystem', {
  lustreConfiguration: { deploymentType: fsx.LustreDeploymentType.SCRATCH_2 },
  storageCapacityGiB: 1200,
  vpc,
  vpcSubnet,
  fileSystemTypeVersion: fsx.FileSystemTypeVersion.V_2_15,
});

new fsx.DataRepositoryAssociation(this, 'DRA', {
  fileSystem,
  fileSystemPath: '/data',
  bucket,
  s3: {
    autoImportPolicy: { events: [fsx.DataRepositoryEventType.NEW, fsx.DataRepositoryEventType.CHANGED] },
    autoExportPolicy: { events: [fsx.DataRepositoryEventType.NEW, fsx.DataRepositoryEventType.CHANGED, fsx.DataRepositoryEventType.DELETED] },
  },
});

Test plan

  • 12 new unit tests — all pass
  • Validates fileSystemPath format
  • Validates importedFileChunkSizeMiB range
  • Validates non-empty event arrays
  • Confirms bucket policy is created granting fsx.amazonaws.com access
  • Confirms DataRepositoryPath is built correctly with and without bucketPrefix

🤖 Generated with Claude Code

Adds `DataRepositoryAssociation`, an L2 construct that links an S3
bucket to a path on a Lustre file system, enabling automatic import
and export of data between S3 and FSx for Lustre.

The construct wraps `CfnDataRepositoryAssociation` and provides:
- Type-safe `S3AutoImportPolicy` and `S3AutoExportPolicy` with
  `DataRepositoryEventType` enum (NEW / CHANGED / DELETED)
- `S3DataRepositoryConfiguration` to configure both policies together
- Validation for `fileSystemPath` format, `importedFileChunkSizeMiB`
  range, and non-empty event arrays
- Automatic `grantReadWrite` to `fsx.amazonaws.com` on the bucket so
  the file system can fulfil import/export requests
- `bucketPrefix` shorthand to scope the association to a key prefix

Fixes aws#34649

Signed-off-by: RadhaKrishnan Pachyappan <gingeekrishna@gmail.com>
Copilot AI review requested due to automatic review settings June 13, 2026 19:21
@github-actions github-actions Bot added beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK feature-request A feature should be added or improved. p2 labels Jun 13, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a new FSx for Lustre L2 construct to model AWS::FSx::DataRepositoryAssociation, including validation logic and unit tests.

Changes:

  • Introduces DataRepositoryAssociation L2 with props for S3 association, auto import/export policies, and validation.
  • Exports the new construct from the FSx module index.
  • Adds unit tests covering required properties, S3 path behavior, policy rendering, and input validation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
packages/aws-cdk-lib/aws-fsx/lib/data-repository-association.ts Implements the new DataRepositoryAssociation L2 construct with validation and S3 bucket permissions.
packages/aws-cdk-lib/aws-fsx/lib/index.ts Re-exports data-repository-association from the FSx module.
packages/aws-cdk-lib/aws-fsx/test/data-repository-association.test.ts Adds unit tests validating CloudFormation output and constructor validation behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +175 to +177
const dataRepositoryPath = props.bucketPrefix
? `s3://${props.bucket.bucketName}/${props.bucketPrefix.replace(/^\//, '')}`
: `s3://${props.bucket.bucketName}/`;
Comment on lines +188 to +202
const resource = new CfnDataRepositoryAssociation(this, 'Resource', {
fileSystemId: props.fileSystem.fileSystemId,
fileSystemPath: props.fileSystemPath,
dataRepositoryPath,
importedFileChunkSize: props.importedFileChunkSizeMiB,
batchImportMetaDataOnCreate: props.batchImportMetaDataOnCreate,
s3: s3Config,
});
resource.applyRemovalPolicy(props.removalPolicy ?? RemovalPolicy.RETAIN);

this.associationId = resource.ref;

// Grant FSx service principal read/write access to the bucket so it can
// fulfil import and export requests on behalf of the file system.
props.bucket.grantReadWrite(new ServicePrincipal('fsx.amazonaws.com'));
Comment on lines +221 to +223
if (fileSystemPath.length > 4096) {
throw new ValidationError(lit`FileSystemPathExceedsMaxLength`, `fileSystemPath cannot exceed 4096 characters, got length: ${fileSystemPath.length}`, this);
}
Comment on lines +179 to +186
const s3Config = props.s3 ? {
autoImportPolicy: props.s3.autoImportPolicy
? { events: props.s3.autoImportPolicy.events }
: undefined,
autoExportPolicy: props.s3.autoExportPolicy
? { events: props.s3.autoExportPolicy.events }
: undefined,
} : undefined;

@aws-cdk-automation aws-cdk-automation left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter fails with the following errors:

❌ Features must contain a change to an integration test file and the resulting snapshot.

If you believe this pull request should receive an exemption, please comment and provide a justification. A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed, add Clarification Request to a comment.

✅ A exemption request has been requested. Please wait for a maintainer's review.

- Strip all leading slashes from bucketPrefix (was only stripping one)
- Pass props.s3 directly to CfnDataRepositoryAssociation instead of
  re-wrapping events arrays (the shapes are compatible)
- Add explicit node dependency from the DRA resource onto the bucket
  policy grant so CloudFormation orders them correctly
- Add test for fileSystemPath exceeding 4096 characters
- Update README with DataRepositoryAssociation usage, compatibility
  notes, and guidance on when to use DRA vs legacy importPath/exportPath

Signed-off-by: RadhaKrishnan Pachyappan <gingeekrishna@gmail.com>
@gingeekrishna

Copy link
Copy Markdown
Author

Addressed all four Copilot review comments:

  1. Leading-slash stripping — changed replace(/^\//, '') to replace(/^\/+/, '') so multiple leading slashes in bucketPrefix are all removed.

  2. CloudFormation ordering — captured the return value of grantReadWrite and called resource.node.addDependency(grant) to ensure the bucket policy exists before CloudFormation creates the DRA.

  3. Missing test for max path length — added a test asserting that a fileSystemPath longer than 4096 characters throws.

  4. S3 config simplification — removed the intermediate re-wrapping object; props.s3 is now passed directly to CfnDataRepositoryAssociation since S3AutoImportPolicy/S3AutoExportPolicy are structurally compatible with the L1 property types.

Also updated the README with a full DataRepositoryAssociation usage example, compatibility notes (supported deployment types / file system versions), and guidance on when to prefer DRA over the legacy importPath/exportPath approach.


Exemption Request

Requesting an exemption for the integration test requirement. FSx for Lustre file systems take 5–10 minutes to provision and incur significant cost per run, making them impractical for automated integration tests in CI. The existing LustreFileSystem L2 construct also has no integration test for the same reason. Unit tests cover all CloudFormation output, validation logic, and the bucket policy dependency.

@aws-cdk-automation aws-cdk-automation added the pr-linter/exemption-requested The contributor has requested an exemption to the PR Linter feedback. label Jun 13, 2026
@aws-cdk-automation aws-cdk-automation added the pr/needs-further-review PR requires additional review from our team specialists due to the scope or complexity of changes. label Jun 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK feature-request A feature should be added or improved. p2 pr/needs-further-review PR requires additional review from our team specialists due to the scope or complexity of changes. pr-linter/exemption-requested The contributor has requested an exemption to the PR Linter feedback.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aws-fsx: L2 Constructs for DataRepositoryAssociation

5 participants