feat(fsx): add L2 construct for DataRepositoryAssociation - #38141
feat(fsx): add L2 construct for DataRepositoryAssociation#38141gingeekrishna wants to merge 8 commits into
Conversation
Adds `DataRepositoryAssociation`, an L2 construct that links an S3 bucket to a path on a Lustre file system, enabling automatic import and export of data between S3 and FSx for Lustre. The construct wraps `CfnDataRepositoryAssociation` and provides: - Type-safe `S3AutoImportPolicy` and `S3AutoExportPolicy` with `DataRepositoryEventType` enum (NEW / CHANGED / DELETED) - `S3DataRepositoryConfiguration` to configure both policies together - Validation for `fileSystemPath` format, `importedFileChunkSizeMiB` range, and non-empty event arrays - Automatic `grantReadWrite` to `fsx.amazonaws.com` on the bucket so the file system can fulfil import/export requests - `bucketPrefix` shorthand to scope the association to a key prefix Fixes aws#34649 Signed-off-by: RadhaKrishnan Pachyappan <gingeekrishna@gmail.com>
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a new FSx for Lustre L2 construct to model AWS::FSx::DataRepositoryAssociation, including validation logic and unit tests.
Changes:
- Introduces
DataRepositoryAssociationL2 with props for S3 association, auto import/export policies, and validation. - Exports the new construct from the FSx module index.
- Adds unit tests covering required properties, S3 path behavior, policy rendering, and input validation.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| packages/aws-cdk-lib/aws-fsx/lib/data-repository-association.ts | Implements the new DataRepositoryAssociation L2 construct with validation and S3 bucket permissions. |
| packages/aws-cdk-lib/aws-fsx/lib/index.ts | Re-exports data-repository-association from the FSx module. |
| packages/aws-cdk-lib/aws-fsx/test/data-repository-association.test.ts | Adds unit tests validating CloudFormation output and constructor validation behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const dataRepositoryPath = props.bucketPrefix | ||
| ? `s3://${props.bucket.bucketName}/${props.bucketPrefix.replace(/^\//, '')}` | ||
| : `s3://${props.bucket.bucketName}/`; |
| const resource = new CfnDataRepositoryAssociation(this, 'Resource', { | ||
| fileSystemId: props.fileSystem.fileSystemId, | ||
| fileSystemPath: props.fileSystemPath, | ||
| dataRepositoryPath, | ||
| importedFileChunkSize: props.importedFileChunkSizeMiB, | ||
| batchImportMetaDataOnCreate: props.batchImportMetaDataOnCreate, | ||
| s3: s3Config, | ||
| }); | ||
| resource.applyRemovalPolicy(props.removalPolicy ?? RemovalPolicy.RETAIN); | ||
|
|
||
| this.associationId = resource.ref; | ||
|
|
||
| // Grant FSx service principal read/write access to the bucket so it can | ||
| // fulfil import and export requests on behalf of the file system. | ||
| props.bucket.grantReadWrite(new ServicePrincipal('fsx.amazonaws.com')); |
| if (fileSystemPath.length > 4096) { | ||
| throw new ValidationError(lit`FileSystemPathExceedsMaxLength`, `fileSystemPath cannot exceed 4096 characters, got length: ${fileSystemPath.length}`, this); | ||
| } |
| const s3Config = props.s3 ? { | ||
| autoImportPolicy: props.s3.autoImportPolicy | ||
| ? { events: props.s3.autoImportPolicy.events } | ||
| : undefined, | ||
| autoExportPolicy: props.s3.autoExportPolicy | ||
| ? { events: props.s3.autoExportPolicy.events } | ||
| : undefined, | ||
| } : undefined; |
There was a problem hiding this comment.
The pull request linter fails with the following errors:
❌ Features must contain a change to an integration test file and the resulting snapshot.
If you believe this pull request should receive an exemption, please comment and provide a justification. A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed, add Clarification Request to a comment.
✅ A exemption request has been requested. Please wait for a maintainer's review.
- Strip all leading slashes from bucketPrefix (was only stripping one) - Pass props.s3 directly to CfnDataRepositoryAssociation instead of re-wrapping events arrays (the shapes are compatible) - Add explicit node dependency from the DRA resource onto the bucket policy grant so CloudFormation orders them correctly - Add test for fileSystemPath exceeding 4096 characters - Update README with DataRepositoryAssociation usage, compatibility notes, and guidance on when to use DRA vs legacy importPath/exportPath Signed-off-by: RadhaKrishnan Pachyappan <gingeekrishna@gmail.com>
|
Addressed all four Copilot review comments:
Also updated the README with a full Exemption Request Requesting an exemption for the integration test requirement. FSx for Lustre file systems take 5–10 minutes to provision and incur significant cost per run, making them impractical for automated integration tests in CI. The existing |
Summary
Fixes #34649
Adds
DataRepositoryAssociation, an L2 construct forAWS::FSx::DataRepositoryAssociation. This links an S3 bucket to a path on an FSx for Lustre file system so data can be automatically imported from S3 into the file system and automatically exported from the file system back to S3.Before this change, users had to reach for
CfnDataRepositoryAssociationdirectly, with no type safety or validation on event types, path format, or chunk size ranges.Changes
New file: data-repository-association.ts
DataRepositoryEventTypeenum —NEW | CHANGED | DELETEDS3AutoImportPolicy/S3AutoExportPolicyinterfaces with typedeventsarraysS3DataRepositoryConfigurationgrouping both policiesDataRepositoryAssociationPropswithfileSystem,bucket,fileSystemPath, optionalbucketPrefix,s3,importedFileChunkSizeMiB,batchImportMetaDataOnCreate,removalPolicyDataRepositoryAssociationL2 class with:fileSystemPathstarts with/importedFileChunkSizeMiBis 1–512,000dataRepositoryPathfrombucket+ optionalbucketPrefixbucket.grantReadWrite(fsx.amazonaws.com)automaticallyUpdated: index.ts — exports new construct
New file: data-repository-association.test.ts — 12 unit tests covering happy path, prefix handling, policy configuration, bucket policy grant, and all validation errors
Example usage
Test plan
fileSystemPathformatimportedFileChunkSizeMiBrangefsx.amazonaws.comaccessDataRepositoryPathis built correctly with and withoutbucketPrefix🤖 Generated with Claude Code