Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions platform-cloud/docs/compute-envs/aws-batch.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,20 @@ The listing of secrets cannot be restricted, but the management actions can be r
}
```

If you specify a customer-managed KMS key (CMK) in the **Pipeline secrets KMS key** field under **Advanced options**, the compute environment credentials also require `kms:GenerateDataKey` and `kms:Decrypt` on that key. Grant these actions either by naming the compute environment principal in the key policy, or in the principal's own IAM policy if the key policy delegates to IAM. The default key policy created by `aws kms create-key` delegates to IAM.

```json
{
"Sid": "PipelineSecretsKmsKey",
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": "arn:aws:kms:<REGION>:<ACCOUNT_ID>:key/<KEY_ID>"
}
```

#### Additional steps required to use secrets in a pipeline

To successfully use pipeline secrets, the IAM roles manually created must follow the steps detailed in the [documentation](../secrets/overview#aws-secrets-manager-integration).
Expand Down Expand Up @@ -798,6 +812,7 @@ Seqera Platform compute environments for AWS Batch include advanced options to c

- Use **Head job role** and **Compute job role** to grant fine-grained IAM permissions to the **Head job** and **Compute jobs**.
- Add an execution role ARN to the **Batch execution role** field to grant permissions to make API calls on your behalf to the ECS container used by Batch. This is required if the pipeline launched with this compute environment needs access to the secrets stored in this workspace. This field can be ignored if you are not using secrets.
- Use **Pipeline secrets KMS key** to specify a customer-managed KMS key that encrypts the temporary AWS Secrets Manager secrets Seqera creates for runs that use pipeline secrets. This field accepts a key ARN or a key ID. A key ARN must be in the same region as the compute environment. Leave this field empty to use the default AWS-managed key. See [Pipeline secrets (optional)](#pipeline-secrets-optional) for the KMS permissions your compute environment credentials require.
- Specify an EBS block size (in GB) in the **EBS auto-expandable block size** field to control the initial size of the EBS auto-expandable volume. New blocks of this size are added when the volume begins to run out of free space. This feature is deprecated, and is not compatible with Fusion v2.
- Enter the **Boot disk size** (in GB) to specify the size of the boot disk in the VMs created by this compute environment.
- If you're using **Spot** instances, you can also specify the **Cost percentage**, which is the maximum allowed price of a **Spot** instance as a percentage of the **On-Demand** price for that instance type. Spot instances will not be launched until the current Spot price is below the specified cost percentage.
Expand Down Expand Up @@ -934,6 +949,7 @@ Seqera compute environments for AWS Batch include advanced options to configure
- Use **Head job CPUs** and **Head job memory** to specify the hardware resources allocated for the Nextflow head job. The default head job memory allocation is 4096 MiB.
- Use **Head job role** and **Compute job role** to grant fine-grained IAM permissions to the head job and compute jobs,
- Add an execution role ARN to the **Batch execution role** field to grant permissions to make API calls on your behalf to the ECS container used by Batch. This is required if the pipeline launched with this compute environment needs access to the secrets stored in this workspace. This field can be ignored if you are not using secrets.
- Use **Pipeline secrets KMS key** to specify a customer-managed KMS key that encrypts the temporary AWS Secrets Manager secrets Seqera creates for runs that use pipeline secrets. This field accepts a key ARN or a key ID. A key ARN must be in the same region as the compute environment. Leave this field empty to use the default AWS-managed key. See [Pipeline secrets (optional)](#pipeline-secrets-optional) for the KMS permissions your compute environment credentials require.
- Use **AWS CLI tool path** to specify the location of the `aws` CLI.
- Specify a **CloudWatch Log group** for the `awslogs` driver to stream the logs entry to an existing Log group in Cloudwatch.

Expand Down
17 changes: 17 additions & 0 deletions platform-cloud/docs/compute-envs/aws-cloud.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -443,6 +443,7 @@ For role-based AWS credentials in Seqera Cloud, allow the Seqera Cloud access ro
- **Subnets**: The list of VPC subnets where the EC2 instance will run. If unspecified, all the subnets of the VPC will be used.
- **Security groups**: The security groups the EC2 instance will be a part of. If unspecified, no security groups will be used.
- **Instance Profile**: The ARN of the `InstanceProfile` used by the EC2 instance to assume a role while running. If unspecified, Seqera will provision one with enough permissions to run. See [Custom instance profile](#custom-instance-profile) for the minimum permissions required if you provide your own.
- **Pipeline secrets KMS key**: A customer-managed KMS key (CMK) that encrypts the temporary AWS Secrets Manager secrets Seqera creates for runs that use pipeline secrets. This field accepts a key ARN or a key ID. A key ARN must be in the same region as the compute environment. If unspecified, Seqera uses the default AWS-managed key. See [Custom instance profile](#custom-instance-profile) for the KMS permissions the instance profile role requires.
- **Boot disk size**: The size of the EBS boot disk for the EC2 instance. If undefined, a default 50 GB `gp3` volume will be used.

### Custom instance profile
Expand Down Expand Up @@ -532,6 +533,22 @@ In addition to the managed policies, attach the following inline policies:
}
```

If the compute environment specifies a **Pipeline secrets KMS key**, the role also requires `kms:Decrypt` on that key to read secrets encrypted with the key at task start. Grant the action either by naming the role in the key policy, or in the role's own IAM policy if the key policy delegates to IAM. The default key policy created by `aws kms create-key` delegates to IAM:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PipelineSecretsKmsKey",
"Effect": "Allow",
"Action": ["kms:Decrypt"],
"Resource": "arn:aws:kms:<REGION>:<ACCOUNT_ID>:key/<KEY_ID>"
}
]
}
```

**KMS for S3** — required if any of the S3 buckets used by the compute environment are encrypted with a customer-managed KMS key (SSE-KMS):

```json
Expand Down
11 changes: 11 additions & 0 deletions platform-cloud/docs/secrets/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,17 @@ The ECS Agent uses the [Batch Execution role](https://docs.aws.amazon.com/batch/
Including `tower-*` in the Resource ARN above limits access to Platform secrets only (as opposed to all secrets in the given region).
:::

If the compute environment specifies a **Pipeline secrets KMS key**, the execution role also requires `kms:Decrypt` on that key to inject secrets encrypted with the key. Grant the action either by naming the role in the key policy, or in the role's own IAM policy if the key policy delegates to IAM. The default key policy created by `aws kms create-key` delegates to IAM:

```json
{
"Sid": "AllowECSAgentToDecryptSecrets",
"Effect": "Allow",
"Action": "kms:Decrypt",
"Resource": "arn:aws:kms:<YOUR_COMPUTE_REGION>:<ACCOUNT_ID>:key/<KEY_ID>"
}
```

**IAM trust relationship**

```json
Expand Down
16 changes: 16 additions & 0 deletions platform-enterprise_docs/compute-envs/aws-batch.md
Original file line number Diff line number Diff line change
Expand Up @@ -585,6 +585,20 @@ The listing of secrets cannot be restricted, but the management actions can be r
}
```

If you specify a customer-managed KMS key (CMK) in the **Pipeline secrets KMS key** field under **Advanced options**, or as the `TOWER_AWS_SECRETS_KMS_KEY_ID` installation default, the compute environment credentials also require `kms:GenerateDataKey` and `kms:Decrypt` on that key. Grant these actions either by naming the compute environment principal in the key policy, or in the principal's own IAM policy if the key policy delegates to IAM. The default key policy created by `aws kms create-key` delegates to IAM.

```json
{
"Sid": "PipelineSecretsKmsKey",
"Effect": "Allow",
"Action": [
"kms:GenerateDataKey",
"kms:Decrypt"
],
"Resource": "arn:aws:kms:<REGION>:<ACCOUNT_ID>:key/<KEY_ID>"
}
```

#### Additional steps required to use secrets in a pipeline

To successfully use pipeline secrets, the IAM roles manually created must follow the steps detailed in the [documentation](../secrets/overview#aws-secrets-manager-integration).
Expand Down Expand Up @@ -976,6 +990,7 @@ Seqera Platform compute environments for AWS Batch include advanced options to c

- Use **Head job role** and **Compute job role** to grant fine-grained IAM permissions to the **Head job** and **Compute jobs**.
- Add an execution role ARN to the **Batch execution role** field to grant permissions to make API calls on your behalf to the ECS container used by Batch. This is required if the pipeline launched with this compute environment needs access to the secrets stored in this workspace. This field can be ignored if you are not using secrets.
- Use **Pipeline secrets KMS key** to specify a customer-managed KMS key that encrypts the temporary AWS Secrets Manager secrets Seqera creates for runs that use pipeline secrets. This field accepts a key ARN or a key ID. A key ARN must be in the same region as the compute environment. Leave this field empty to use the installation default set with [`TOWER_AWS_SECRETS_KMS_KEY_ID`](../enterprise/configuration/overview#compute-environments), or the default AWS-managed key if neither is set. See [Pipeline secrets (optional)](#pipeline-secrets-optional) for the KMS permissions your compute environment credentials require.
- Specify an EBS block size (in GB) in the **EBS auto-expandable block size** field to control the initial size of the EBS auto-expandable volume. New blocks of this size are added when the volume begins to run out of free space. This feature is deprecated, and is not compatible with Fusion v2.
- Enter the **Boot disk size** (in GB) to specify the size of the boot disk in the VMs created by this compute environment.
- If you're using **Spot** instances, you can also specify the **Cost percentage**, which is the maximum allowed price of a **Spot** instance as a percentage of the **On-Demand** price for that instance type. Spot instances will not be launched until the current Spot price is below the specified cost percentage.
Expand Down Expand Up @@ -1112,6 +1127,7 @@ Seqera compute environments for AWS Batch include advanced options to configure
- Use **Head job CPUs** and **Head job memory** to specify the hardware resources allocated for the Nextflow head job. The default head job memory allocation is 4096 MiB.
- Use **Head job role** and **Compute job role** to grant fine-grained IAM permissions to the head job and compute jobs,
- Add an execution role ARN to the **Batch execution role** field to grant permissions to make API calls on your behalf to the ECS container used by Batch. This is required if the pipeline launched with this compute environment needs access to the secrets stored in this workspace. This field can be ignored if you are not using secrets.
- Use **Pipeline secrets KMS key** to specify a customer-managed KMS key that encrypts the temporary AWS Secrets Manager secrets Seqera creates for runs that use pipeline secrets. This field accepts a key ARN or a key ID. A key ARN must be in the same region as the compute environment. Leave this field empty to use the installation default set with [`TOWER_AWS_SECRETS_KMS_KEY_ID`](../enterprise/configuration/overview#compute-environments), or the default AWS-managed key if neither is set. See [Pipeline secrets (optional)](#pipeline-secrets-optional) for the KMS permissions your compute environment credentials require.
- Use **AWS CLI tool path** to specify the location of the `aws` CLI.
- Specify a **CloudWatch Log group** for the `awslogs` driver to stream the logs entry to an existing Log group in Cloudwatch.

Expand Down
17 changes: 17 additions & 0 deletions platform-enterprise_docs/compute-envs/aws-cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -472,6 +472,7 @@ The AWS Cloud compute environment uses an AMI maintained by Seqera, and the pipe
- **Subnets**: The list of VPC subnets where the EC2 instance will run. If unspecified, all the subnets of the VPC will be used.
- **Security groups**: The security groups the EC2 instance will be a part of. If unspecified, no security groups will be used.
- **Instance Profile**: The ARN of the `InstanceProfile` used by the EC2 instance to assume a role while running. If unspecified, Seqera will provision one with enough permissions to run. See [Custom instance profile](#custom-instance-profile) for the minimum permissions required if you provide your own.
- **Pipeline secrets KMS key**: A customer-managed KMS key (CMK) that encrypts the temporary AWS Secrets Manager secrets Seqera creates for runs that use pipeline secrets. This field accepts a key ARN or a key ID. A key ARN must be in the same region as the compute environment. If unspecified, Seqera uses the installation default set with [`TOWER_AWS_SECRETS_KMS_KEY_ID`](../enterprise/configuration/overview#compute-environments), or the default AWS-managed key if neither is set. See [Custom instance profile](#custom-instance-profile) for the KMS permissions the instance profile role requires.
- **Boot disk size**: The size of the EBS boot disk for the EC2 instance. If undefined, a default 50 GB `gp3` volume will be used.

### Custom instance profile
Expand Down Expand Up @@ -561,6 +562,22 @@ In addition to the managed policies, attach the following inline policies:
}
```

If the compute environment specifies a **Pipeline secrets KMS key**, the role also requires `kms:Decrypt` on that key to read secrets encrypted with the key at task start. Grant the action either by naming the role in the key policy, or in the role's own IAM policy if the key policy delegates to IAM. The default key policy created by `aws kms create-key` delegates to IAM:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PipelineSecretsKmsKey",
"Effect": "Allow",
"Action": ["kms:Decrypt"],
"Resource": "arn:aws:kms:<REGION>:<ACCOUNT_ID>:key/<KEY_ID>"
}
]
}
```

**KMS for S3** — required if any of the S3 buckets used by the compute environment are encrypted with a customer-managed KMS key (SSE-KMS):

```json
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,8 @@
Description: >
Enable legacy role-based AWS credentials. When true, users provide an IAM role ARN only when creating AWS credentials. Access keys, secret keys, and External ID are not used.
Value: 'Default: `false`'
-
Environment variable: '`TOWER_AWS_SECRETS_KMS_KEY_ID`'
Description: >
Installation-wide customer-managed KMS key that encrypts the temporary AWS Secrets Manager secrets created for runs that use pipeline secrets. Accepts a key ARN or a key ID. Applies to AWS Batch and AWS Cloud compute environments, and only when the compute environment does not define its own **Pipeline secrets KMS key**. A key is account- and region-specific. Set this default only when every AWS Batch and AWS Cloud compute environment in the installation targets the same AWS account and region. The compute environment credentials require `kms:GenerateDataKey` and `kms:Decrypt` on the key, granted either in the key policy or in the credentials' own IAM policy. A malformed value prevents backend startup.
Value: 'Default: none (AWS-managed key)'
11 changes: 11 additions & 0 deletions platform-enterprise_docs/secrets/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,17 @@ The ECS Agent uses the [Batch Execution role](https://docs.aws.amazon.com/batch/
Including `tower-*` in the Resource ARN above limits access to Platform secrets only (as opposed to all secrets in the given region).
:::

If the compute environment specifies a **Pipeline secrets KMS key**, the execution role also requires `kms:Decrypt` on that key to inject secrets encrypted with the key. Grant the action either by naming the role in the key policy, or in the role's own IAM policy if the key policy delegates to IAM. The default key policy created by `aws kms create-key` delegates to IAM:

```json
{
"Sid": "AllowECSAgentToDecryptSecrets",
"Effect": "Allow",
"Action": "kms:Decrypt",
"Resource": "arn:aws:kms:<YOUR_COMPUTE_REGION>:<ACCOUNT_ID>:key/<KEY_ID>"
}
```

**IAM trust relationship**

```json
Expand Down