aws-fsx: L2 Constructs for DataRepositoryAssociation

[AlJohri]

Describe the feature

Currently we have an L2 construct for LustreFileSystem but we are missing the L2 constructs for DataRepositoryAssociation (CfnDataRepositoryAssociation).

Use Case

Use case is to create an FSx backed by an S3 bucket using a DRA. Currently its confusing since the LustreFileSystem L2 construct accepts the LustreConfiguration which defines an importPath and exportPath but its not clear (or verified via the type system) that this cannot be used in conjunction with the DRA.

Docs: https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-fsx-filesystem-lustreconfiguration.html

This parameter is not supported for file systems with a data repository association.

Proposed Solution

Create an L2 construct for DataRepositoryAssociation which validates that importPath/exportPath is not set on the LustreFileSystem. And make the documentation clear on how to set up a LustreFileSystem backed by an S3 bucket.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

AWS CDK Library version (aws-cdk-lib)

2.191.0

AWS CDK CLI version

2.1011.0 (build cd4ec06)

Environment details (OS name and version, etc.)

Amazon Linux 2

[ykethan]

Hey @AlJohri , thank you for reaching out and filing this valuable feature request. The LustreFileSystem does lacks an L2 construct for DataRepositoryAssociation.

The existing Lustre file system construct may need the following changes

1. Create L2 DataRepositoryAssociation construct with:

   • Type-safe properties for common configurations
   • Validation for path conflicts and file system version compatibility
   • Integration with existing LustreFileSystem with a optional property
   • Support for auto import/export policies

2. Add validation to LustreFileSystem to prevent conflicting configurations:

   • Warn when mixing importPath/exportPath with DRA
   • Validate file system version compatibility
   • Check deployment type compatibility
     eg, Data repository associations are supported on all FSx for Lustre 2.12 and 2.15 file systems, excluding scratch_1 deployment type
     https://docs.aws.amazon.com/fsx/latest/LustreGuide/overview-dra-data-repo.html

3. Update the README documentation and examples explaining:

   • When to use each approach
   • Compatibility requirements

Additionally, we may need to validate options for folks looking to switch from importPath/exportPath to DRA as method.

Marking this as P2 feature request. We greatly appreciate community contributions! If you are interested in implementing this on a Pull Request. Please review the contribution guidelines. Meanwhile, welcome to add 👍 to this issue to help us prioritize.

Additionally, on a quick test I was able to deploy a DRA using the following

    const dataBucket = new s3.Bucket(this, 'DataRepositoryBucket', {
      bucketName: `fsx-data-repo-${this.account}-${this.region}`,
      versioned: true,
      blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
      removalPolicy: cdk.RemovalPolicy.DESTROY,
      autoDeleteObjects: true,
    });

  const myFileSystem = new fsx.LustreFileSystem(this, 'myFileSystem', {
      lustreConfiguration: {
        deploymentType: fsx.LustreDeploymentType.SCRATCH_2,
      },
      storageCapacityGiB: 1200,
      vpc: vpc,
      vpcSubnet: vpc.privateSubnets[1],
      removalPolicy: cdk.RemovalPolicy.DESTROY,
      fileSystemTypeVersion: fsx.FileSystemTypeVersion.V_2_15,
    });

    const primaryDra = new fsx.CfnDataRepositoryAssociation(
      this,
      'PrimaryDRA',
      {
        fileSystemId: myFileSystem.fileSystemId,
        fileSystemPath: '/data',
        dataRepositoryPath: `s3://${dataBucket.bucketName}/primary/`,
        s3: {
          autoImportPolicy: {
            events: ['NEW', 'CHANGED'],
          },
          autoExportPolicy: {
            events: ['NEW', 'CHANGED', 'DELETED'],
          },
        },
        importedFileChunkSize: 1024,
        batchImportMetaDataOnCreate: true,
      }
    );

    new fsx.CfnDataRepositoryAssociation(
      this,
      'SecondaryDRA',
      {
        fileSystemId: myFileSystem.fileSystemId,
        fileSystemPath: '/backup',
        dataRepositoryPath: `s3://${dataBucket.bucketName}/backup/`,
        s3: {
          autoImportPolicy: {
            events: ['NEW'],
          },
          autoExportPolicy: {
            events: ['NEW', 'CHANGED'],
          },
        },
        importedFileChunkSize: 2048,
      }
    );

    // Grant FSx permissions to access the S3 bucket
    dataBucket.grantReadWrite(
      new cdk.aws_iam.ServicePrincipal('fsx.amazonaws.com')
    );