Skip to content

Commit 51f6936

Browse files
committed
v4.0.0
This is a security-focused release of the AWS Deployment Framework (ADF) that aims to restrict the default access required and provided by ADF via the least-privilege principle. __Key security enhancements include:__ - Applying IAM best practices by restricting excessive permissions granted to IAM roles and policies used by ADF. - Leveraging new IAM features to further limit access privileges granted by default, reducing the potential attack surface. - Where privileged access is required for specific ADF use cases, the scope and duration of elevated privileges have been minimized to limit the associated risks. By implementing these security improvements, ADF now follows the principle of least privilege, reducing the risk of unauthorized access or privilege-escalation attacks. Please make sure to go through the list of changes breaking changes carefully. As with every release, it is strongly recommended to thoroughly review and test this version of ADF in a non-production environment first. ### Breaking changes #### Security: Confused Deputy Problem Addressed the [Confused Deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html) in IAM roles created by ADF to use by the AWS Services. Where supported, the roles are restricted to specific resources via an `aws:SourceArn` condition. If you were using the ADF roles for other resources or use cases not covered by ADF, you might need to patch the Assume Role policies accordingly. #### Security: Cross-Account Access Role and the new Jump Role ADF relies on the privileged Cross-Account Access Role to bootstrap accounts. In the past, ADF used this role for every update and deployment of the bootstrap stacks, as well as account management features. With the release of v4.0, a jump role is introduced to lock-down the usage of the privileged cross-account access role. Part of the bootstrap stack, the `adf-bootstrap-update-deployment-role` is created. This role grants access to perform restricted updates that are frequently performed via the `aws-deployment-framework-bootstrap` pipeline. By default, the jump role is granted access to assume into this update deployment role. A dedicated jump role manager is responsible to grant the jump role access to the cross-account access role for AWS accounts where ADF requires access and the `adf-bootstrap-update-deployment-role` is not available yet. For example, accounts that are newly created only have the cross-account access role to assume into. Same holds for ADF managed accounts that are not updated to the new v4.0 bootstrap stack yet. During the installation/update of ADF, a new parameter enables you to grant the jump role temporary access to the cross-account access role as an privileged escalation path. This parameter is called `GrantOrgWidePrivilegedBootstrapAccessUntil`. By setting this to a date/time in the future you will grant access to the cross-account access role until that date/time. This would be required if you modify ADF itself or the bootstrap stack templates. Changing permissions like the `adf-cloudformation-deployment-role` is possible without relying on the cross-account access role. For most changes deployed via the bootstrap pipeline it does not require elevated privileged access to update. With the above changes, the `aws-deployment-framework-bootstrap` CodeBuild project no longer has unrestricted access to the privileged cross-account role. Starting from version 4.0, access to assume the privileged cross-account access role is restricted and must be obtained through the Jump Role as described above. #### Security: Restricted account management access Account Management is able to access non-protected organization units. Prior to ADF v4.0, the account management process used the privileged cross-account assess role to operate. Hence it could move an account or update the properties of an account that is located in a protected organization unit too. With the release of v4.0, it is only able to move or manage accounts if they are accessible via the Jump Role. The Jump Role is restricted to non-protected organization units only. This enhances the security of ADF, as defining a organization unit as protected will block access to that via the Jump Role accordingly. #### Security: Restricted bootstrapping of management account The `adf-global-base-adf-build` stack in the management account was initially deployed to facilitate bootstrap access to the management account. It accomplished this by creating a cross-account access role with limited permissions in the management account ahead of the bootstrapping process. ADF created this role as it is not provisioned by AWS Organizations or AWS Control Tower in the management account itself. However, ADF required some level of access to deploy the necessary bootstrap stacks when needed. It is important to note that deploying this role and bootstrapping the management account introduces a potential risk. A pipeline created via a deployment map could target the management account and create resources within it, which may have unintended consequences. To mitigate the potential risk, it is recommended to implement strict least-privilege policies and apply permission boundaries to protect the management account. Additionally, thoroughly reviewing all deployment map changes is crucial to ensure no unintended access is granted to the management account. With the release of ADF v4.0, the `adf-global-base-adf-build` stack is removed and its resources are moved to the main ADF CloudFormation template. These resources will only get deployed if the new `AllowBootstrappingOfManagementAccount` parameter is set to `Yes`. By default it will not allow bootstrapping of the management account. #### Security: Restricted bootstrapping of deployment account Considering the sensitive workloads that run in the deployment account, it is important to limit the permissions granted for pipelines to deploy to the deployment account itself. You should consider the deployment account a production account. It is recommended to apply the least-privilege principle and only allow pipelines to deploy resources that are required in the deployment account. Follow these steps after the changes introduced by the ADF v4.0 release are applied in the main branch of the `aws-deployment-framework-bootstrap` repository. Please take this moment to review the following: * Navigate to the `adf-boostrap/deployment` folder in that repository. * Check if it contains a `global-iam.yml` file: * If it does __not__ contain a `global-iam.yml` file yet, please ensure you copy the `example-global-iam.yml` file in that directory. * If it does, please compare it against the `example-global-iam.yml` file in that directory. * Apply the least-privilege principle on the permissions you grant in the deployment account. #### Security: Shared Modules Bucket ADF uses the Shared Modules Bucket as hosted in the management account in the main deployment region to share artifacts from the `aws-deployment-framework-bootstrap` repository. The breaking change enforces all objects to be owned by the bucket owner from v4.0 onward. #### Security: ADF Role policy restrictions With the v4.0 release, all ADF roles and policies were reviewed, applying the latest best-practices and granting access to ADF resources only where required. This review also includes the roles that were used by the pipelines generated by ADF. Please be aware of the changes made to the following roles: ##### adf-codecommit-role The `adf-codecommit-role` no longer grants read/write access to all buckets. It only grants access to the buckets created and managed by ADF where it needed to. Please grant access accordingly if you use custom S3 buckets or need to copy from an S3 bucket in an ADF-generated pipeline. ##### adf-codebuild-role The `adf-codebuild-role` can only be used by CodeBuild projects in the main deployment region. ADF did not allow running CodeBuild projects in other regions before. But in case you manually configured the role in a project in a different region it will fail to launch. The `adf-codebuild-role` is no longer allowed to assume any IAM Role in the target accounts if those roles would grant access in the Assume Role Policy Document. The `adf-codebuild-role` is restricted to assume only the `adf-readonly-automation-role` roles in the target accounts. And, in the case that the Terraform ADF Extension is enabled, it is allowed to assume the `adf-terraform-role` too. It is therefore not allowed to assume the `adf-cloudformation-deployment-role` any longer. If you were deploying with `cdk deploy` into target accounts from an ADF pipeline you will need to specifically grant the `adf-codebuild-role` access to assume the `adf-cloudformation-deployment-role`. However, we strongly recommend you synthesize the templates instead and let AWS CloudFormation do the deployment for you. For Terraform support, CodeBuild was granted access to the `adf-tflocktable` table in release v3.2.0. This access is restricted to only grant read/write access to that table if the Terraform extension is enabled. Please bear in mind that if you enable Terraform access the first time, you will need to use the `GrantOrgWidePrivilegedBootstrapAccessUntil` parameter if ADF v4.0 bootstrapped to accounts before. As this operation requires privileged access. The `adf-codebuild-role` is allowed to assume into the `adf-terraform-role` if the Terraform extension is enabled. As written in the docs, the `adf-terraform-role` is configured in the `global-iam.yml` file. This role is commented out by default. When you define this role, it is important to make sure to grant it [least-privilege access](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege) only. ##### adf-cloudformation-role The `adf-cloudformation-role` is no longer assumable by CloudFormation. This role is used by CodePipeline to orchestrate various deployment actions across accounts. For example, CodeDeploy, S3, and obviously the CloudFormation actions. For CloudFormation, it would instruct the service to use the CloudFormation Deployment role for the actual deployment. The CloudFormation deployment role is the role that is assumed by the CloudFormation service. This change should not impact you, unless you use this role in relation with CloudFormation that is not managed by ADF. With v4.0, the `adf-cloudformation-role` is only allowed to pass the CloudFormation Deployment role to CloudFormation and no other roles to other services. If you were/want to make use of a custom CloudFormation deployment role for specific pipelines, you need to make sure that the `adf-cloudformation-role` is allowed to perform an `iam:PassRole` action with the given role. It is recommended to limit this to be passed to the CloudFormation service only. You can find an example of this in the `adf-bootstrap/deployment/global.yml` file where it allows the CloudFormation role to perform `iam:PassRole` with the `adf-cloudformation-deployment-role`. When required, please grant this access in the `adf-bootstrap/deployment/global-iam.yml` file in the `aws-deployment-framework-bootstrap` repository. Additionally, the `adf-cloudformation-role` is not allowed to access S3 buckets except the ADF buckets it needs to transfer pipeline assets to CloudFormation. ##### adf-codepipeline-role The `adf-codepipeline-role` is no longer assumable by CloudFormation, CodeDeploy, and S3. The role itself was not passed to any of these services by ADF. If you relied on the permissions that were removed, feel free to extend the role permissions via the `global-iam.yml` stack. #### Security: Restricted access to ADF-managed S3 buckets only With v4.0, access is restricted to ADF-managed S3 buckets only. If a pipeline used the S3 source or deployment provider, it will require the required access to those buckets. Please add the required access to the `global-iam.yml` bootstrap stack in the OU where it is hosted. Grant read access to the `adf-codecommit-role` for S3 source buckets. Grant write access to the `adf-cloudformation-role` for S3 buckets an ADF pipeline deploys to. #### Security: Bootstrap stack no longer named after organization unit The global and regional bootstrap stacks are renamed to `adf-global-base-bootstrap` and `adf-regional-base-bootstrap` respectively. In prior releases of ADF, the name ended with the organization unit name. As a result, an account could not move from one organization unit to another without first removing the bootstrap stacks. Additionally, it made writing IAM policies and SCPs harder in a least-privilege way. When ADF v4.0 is installed, the legacy stacks will get removed by the `aws-deployment-framework-bootstrap` pipeline automatically. Shortly after removal, it will deploy the new bootstrap stacks. With v4.0, accounts can move from one organization unit to another, without requiring the removal of the ADF bootstrap stacks. #### Security: KMS Encryption required on Deployment Account Pipeline Buckets The deployment account pipeline buckets only accepts KMS Encrypted objects from v4.0 onward. Ensuring that all objects are encrypted with the same KMS Key. Before, some objects used KMS encryption while others did not. The bucket policy now requires all objects to be encrypted via the KMS key. All ADF components have been adjusted to upload with this key. If, however, you copy files from systems that are not managed by ADF, you will need to adjust these to encrypt the objects with the KMS key as well. #### Security: TLS Encryption required on all ADF-managed buckets S3 Buckets created by ADF will require TLS 1.2 or later. All actions that occur on these buckets with older TLS versions will be denied via the bucket policies that these buckets received. #### New installer The dependencies that are bundled by the move to the AWS Cloud Development Kit (CDK) v2 increased the deployment size of ADF. Unfortunately it increased the deployment size beyond the limit that is supported by the Serverless Application Repository (SAR). Hence a new installation mechanism is required. Please read the [installation instructions](https://github.com/awslabs/aws-deployment-framework/blob/master/docs/installation-guide.md) carefully. In case you are upgrading an existing installation of ADF, please consider following the [upgrade steps as defined in the admin guide](https://github.com/awslabs/aws-deployment-framework/blob/master/docs/admin-guide.md#updating-between-versions). #### CDK v2 ADF v4.0 is built on the AWS Cloud Development Kit (CDK) v2. Which is an upgrade to CDK v1 that ADF relied on before. For most end-users, this change would not have an immediate impact. If, however, you made customizations to ADF it might require you to upgrade these customizations to CDK v2 as well. #### CodeBuild default image As written in the [CodeBuild provider docs](./docs/providers-guide.md#properties-3), it is a best-practice to define the exact CodeBuild container image you would like to use for each pipeline. However, in case you rely on the default, in prior ADF releases it would default to `UBUNTU_14_04_PYTHON_3_7_1`. This container image is no longer supported. With ADF v4.0, the new default is `STANDARD_7_0`. Also referred to as: `aws/codebuild/standard:7.0`. #### ADF Renaming of Roles ADF v4.0 changes most of the roles that it relies on. The reason for this change is to make it easier to secure ADF with Service Control Policies and IAM permission boundaries. Where applicable, the roles received a new prefix. This makes it easier to identify what part of ADF relies on those roles and whom should have access to assume the role or modify it. | Previous prefix | Previous name | New prefix | New name | |------------------|---------------------------------------------------------------------|----------------------------|---------------------------------------------------------------| | / | ${CrossAccountAccessRoleName}-readonly | /adf/organizations/ | adf-organizations-readonly | | / | adf-update-cross-account-access-role | /adf/bootstrap/ | adf-update-cross-account-access | | /adf-automation/ | adf-create-repository-role | /adf/pipeline-management/ | adf-pipeline-management-create-repository | | /adf-automation/ | adf-pipeline-provisioner-generate-inputs | /adf/pipeline-management/ | adf-pipeline-management-generate-inputs | | /adf-automation/ | adf-pipeline-create-update-rule | /adf/pipeline-management/ | adf-pipeline-management-create-update-rule | | / | adf-event-rule-${AWS::AccountId}-${DeploymentAccountId}-EventRole-* | /adf/cross-account-events/ | adf-cc-event-from-${AWS::AccountId}-to-${DeploymentAccountId} | |------------------|---------------------------------------------------------------------|----------------------------|---------------------------------------------------------------| #### ADF Renaming of Resources | Type | Previous name | New name | |--------------|-----------------------------------------------|--------------------------------------------------------| | StateMachine | EnableCrossAccountAccess | adf-bootstrap-enable-cross-account | | StateMachine | ADFPipelineManagementStateMachine | adf-pipeline-management | | StateMachine | PipelineDeletionStateMachine-* | adf-pipeline-management-delete-outdated | | Lambda | DeploymentMapProcessorFunction | adf-pipeline-management-deployment-map-processor | | Lambda | ADFPipelineCreateOrUpdateRuleFunction | adf-pipeline-management-create-update-rule | | Lambda | ADFPipelineCreateRepositoryFunction | adf-pipeline-management-create-repository | | Lambda | ADFPipelineGenerateInputsFunction | adf-pipeline-management-generate-pipeline-inputs | | Lambda | ADFPipelineStoreDefinitionFunction | adf-pipeline-management-store-pipeline-definition | | Lambda | ADFPipelineIdentifyOutOfDatePipelinesFunction | adf-pipeline-management-identify-out-of-date-pipelines | |--------------|-----------------------------------------------|--------------------------------------------------------| #### ADF Parameters in AWS Systems Manager Parameter Store Some of the parameters stored by ADF in AWS Systems Manager Parameter Store were located at the root of the Parameter Store. This made it hard to maintain and restrict access to the limited set of ADF specific parameters. With ADF v4.0, the parameters used by ADF are located under the `/adf/` prefix. For example, `/adf/deployment_account_id`. The `global-iam.yml` bootstrap stack templates get copied from their `example-global-iam.yml` counterparts. When this was copied in v3.2.0, the default path for the `deployment_account_id` parameter should be updated to `/adf/deployment_account_id`. Please apply this new default value to the CloudFormation templates accordingly. If you forget to do this, the stack deployment of the `adf-global-base-iam` stack might fail with a failure stating that it does not have permission to fetch the `deployment_account_id` parameter. The error you run into if the parameter path is not updated: > An error occurred (ValidationError) when calling the CreateChangeSet > operation: User: > arn:aws:sts::111111111111:assumed-role/${CrossAccountAccessRoleName}/base_update > is not authorized to perform: ssm:GetParameters on resource: > arn:aws:ssm:${deployment_region}:111111111111:parameter/deployment_account_id > because no identity-based policy allows the ssm:GetParameters action > (Service: AWSSimpleSystemsManagement; Status Code: 400; > Error Code: AccessDeniedException; Request ID: xxx). If an application or customization to ADF relies on one of these parameters they will need to be updated to include this prefix. Unless the application code relies on ADF's ParameterStore class, in that case it will automatically prefix the `/adf/` to all parameters read or written. With the changes in the IAM policies, ADF's access is restricted to the `/adf/` prefix. This, unfortunately implies that old parameters are not deleted when you update your installation of ADF. There is no cost associated to these parameters, so you can leave them as is. Feel free to delete the old parameters. The parameters that are managed by ADF that got their path changed are: For the __management account__, in the __AWS Organizations region__ (`us-east-1`, or `us-gov-west-1`): | Old Parameter Path | New Parameter Path | |------------------------------|----------------------------------| | `/adf_log_level` | `/adf/adf_log_level` | | `/adf_version` | `/adf/adf_version` | | `/bucket_name` | `/adf/bucket_name` | | `/confit` | `/adf/config` | | `/cross_account_access_role` | `/adf/cross_account_access_role` | | `/deployment_account_id` | `/adf/deployment_account_id` | | `/deployment_account_region` | `/adf/deployment_account_region` | | `/kms_arn` | `/adf/kms_arn` | | `/notification_channel` | `/adf/notification_channel` | | `/organization_id` | `/adf/organization_id` | | `/protected` | `/adf/protected` | | `/scp` | `/adf/scp` | | `/shared_modules_bucket` | `/adf/shared_modules_bucket` | | `/tagging-policy` | `/adf/tagging_policy` | | `/target_regions` | `/adf/target_regions` | For the __management account__, in __other ADF regions__: | Old Parameter Path | New Parameter Path | |------------------------------|----------------------------------| | `/adf_version` | `/adf/adf_version` | | `/bucket_name` | `/adf/bucket_name` | | `/cross_account_access_role` | `/adf/cross_account_access_role` | | `/deployment_account_id` | `/adf/deployment_account_id` | | `/kms_arn` | `/adf/kms_arn` | For the __deployment account__, in __the deployment region__: | Old Parameter Path | New Parameter Path | |------------------------------|-------------------------------------| | `/adf_log_level` | `/adf/adf_log_level` | | `/adf_version` | `/adf/adf_version` | | `/auto_create_repositories` | `/adf/scm/auto_create_repositories` | | `/cross_account_access_role` | `/adf/cross_account_access_role` | | `/default_scm_branch` | `/adf/scm//default_scm_branch` | | `/deployment_account_bucket` | `/adf/shared_modules_bucket` | | `/master_account_id` | `/adf/management_account_id` | | `/notification_endpoint` | `/adf/notification_endpoint` | | `/notification_type` | `/adf/notification_type` | | `/organization_id` | `/adf/organization_id` | For the __deployment account__, in __other ADF regions__: | Old Parameter Path | New Parameter Path | |------------------------------|----------------------------------| | `/adf_log_level` | `/adf/adf_log_level` | | `/adf_version` | `/adf/adf_version` | | `/cross_account_access_role` | `/adf/cross_account_access_role` | | `/deployment_account_bucket` | `/adf/shared_modules_bucket` | | `/master_account_id` | `/adf/management_account_id` | | `/notification_endpoint` | `/adf/notification_endpoint` | | `/notification_type` | `/adf/notification_type` | | `/organization_id` | `/adf/organization_id` | For a __target account__, in __each ADF region__: | Old Parameter Path | New Parameter Path | |--------------------------|------------------------------| | `/bucket_name` | `/adf/bucket_name` | | `/deployment_account_id` | `/adf/deployment_account_id` | | `/kms_arn` | `/adf/kms_arn` | #### AWS CodeStar Connections OAuth Token support dropped ADF v4.0 discontinued the support for the OAuth Token stored in SSM Parameter Store. As this method is not advised to be used by CodePipeline, and might leave the OAuth Token accessible to other users of the deployment account. As this is not a security best practice, ADF v4.0 no longer supports it. To upgrade, please read the [Administrator Guide on Using AWS CodeConnections for Bitbucket, GitHub, or GitLab](./docs/admin-guide.md#using-aws-codeconnections-for-bitbucket-github-github-enterprise-or-gitlab). #### AWS CodeStar Connections changed to AWS CodeConnections The AWS CodeStar Connection service [changed its name to AWS CodeConnections](https://docs.aws.amazon.com/dtconsole/latest/userguide/rename.html). If you configured a CodeStar Connection before, you can continue to use that. You do not need to update the CodeStar policy as defined in the `aws-deployment-framework-bootstrap/adf-bootstrap/deployment/global-iam.yml` stack. However, please update the pipeline definitions in your deployment map files. The changes you need to make are renaming the source provider from `codestar` to `codeconnections`. Also update the `codestar_connection_path` source property to `codeconnections_param_path`. Both of these changes can be seen in the following example: ```yaml pipelines: - name: sample-vpc default_providers: source: # provider: codestar provider: codeconnections properties: # codestar_connection_path: /adf/my_connection_arn_param codeconnections_param_path: /adf/my_connection_arn_param ``` If you are upgrading from the GitHub OAuth token or otherwise require a new source code connection, please proceed with the AWS CodeConnections configuration as defined in the [Admin Guide - Using AWS CodeConnections for Bitbucket, GitHub, or GitLab](./docs/admin-guide.md#using-aws-codeconnections-for-bitbucket-github-or-gitlab). ### Features - Update CDK from v1 to v2 (#619), by @pergardebrink, resolves #503, #614, and #617. - Account Management State Machine will now opt-in to target regions when creating an account (#604) by @StewartW. - Add support for nested organization unit targets (#538) by @StewartW, resolves #20. - Enable single ADF bootstrap and pipeline repositories to multi-AWS Organization setup, resolves #410: - Introduce the org-stage (#636) by @AndyEfaa. - Add support to allow empty targets in deployment maps (#634) by @AndyEfaa. - Add support to define the "default-scm-codecommit-account-id" in adfconfig.yml, no value in either falls back to deployment account id (#633) by @AndyEfaa. - Add multi AWS Organization support to adfconfig.yml (#668) by @alexevansigg. - Add multi AWS Organization support to generate_params.py (#672) by @AndyEfaa. - Terraform: add support for distinct variable files per region per account in Terraform pipelines (#662) by @igordust, resolves #661. - CodeBuild environment agnostic custom images references, allowing to specify the repository name or ARN of the ECR repository to use (#623) by @abhi1094. - Add kms_encryption_key_arn and cache_control parameters to S3 deploy provider (#669) by @alFReD-NSH. - Allow inter-ou move of accounts (#712) by @sbkok. ### Fixes - Fix Terraform terrascan failure due to incorrect curl call (#607), by @lasv-az. - Fix custom pipeline type configuration not loaded (#612), by @lydialim. - Fix Terraform module execution error (#600), by @stemons, resolves #599 and #602. - Fix resource untagging permissions (#635) by @sbkok. - Fix GitHub Pipeline secret token usage (#645) by @sbkok. - Fix Terraform error masking by tee (#643) by @igordust, resolves #642. - Fix create repository bug when in rollback complete state (#648) by @alexevansigg. - Fix cleanup of parameters upon pipeline retirement (#652) by @sbkok. - Fix wave calculation for non-default CloudFormation actions and multi-region deployments (#624 and #651), by @alexevansigg. - Fix ChatBot channel ref + add notification management permissions (#650) by @sbkok. - Improve docs and add CodeStar Connection policy (#649) by @sbkok. - Fix Terraform account variables were not copied correctly (#665) by @donnyDonowitz, resolves #664. - Fix pipeline management state machine error handling (#683) by @sbkok. - Fix target schema for tags (#667) by @AndyEfaa. - Fix avoid overwriting truncated pipeline definitions with pipelines that share the same start (#653) by @AndyEfaa. - Fix updating old global-iam stacks in the deployment account (#711) by @sbkok. - Remove default org-stage reference to dev (#717) by @alexevansigg. - Fix racing condition on first-usage of ADF pipelines leading to an auth error (#732) by @sbkok. - Fix support for custom S3 deployment roles (#732) by @sbkok, resolves #355. - Fix pipeline completion trigger description (#734) by @sbkok, resolves #654. ### Improvements - Sanitizing account names before using them in SFn Invocation (#598) by @StewartW, resolves #597. - Improve Terraform documentation sample (#605), by @lasv-az. - Fix CodeDeploy sample to work in gov-cloud (#609), by @sbkok. - Fix documentation error on CodeBuild custom image (#622), by @abhi1094. - Speedup bootstrap pipeline by removing unused SAM Build (#613), by @AlexMackechnie. - Upgrade CDK (v2.88), SAM (v1.93), and others to latest compatible version (#647) by @sbkok, resolves #644. - Update pip before installing dependencies (#606) by @lasv-az. - Fix: Adding hash to pipelines processing step function execution names to prevent collisions (#641) by @avolip, resolves #640. - Modify trust relations for roles to ease redeployment of roles (#526) by @AndreasAugustin, resolves #472. - Limit adf-state-machine-role to what is needed (#657) by @alFReD-NSH. - Upload SCP policies with spaces removed (#656) by @alFReD-NSH. - Move from ACL enforced bucket ownership to Ownership Controls + MegaLinter prettier fix (#666) by @sbkok. - Upgrade CDK (v2.119), SAM (v1.107), Jinja2 (v3.1.3), and others to latest compatible version (#676) by @sbkok. - Fix initial value type of allow-empty-targets (#678) by @sbkok. - Fix Shared ADF Lambda Layer builds and add move to ARM-64 Lambdas (#680) by @sbkok. - Add /adf params prefix and other SSM Parameter improvements (#695) by @sbkok, resolves #594 and #659. - Fix pipeline support for CodeBuild containers with Python < v3.10 (#705) by @sbkok. - Update CDK v2.136, SAM CLI 1.114, and others (#715) by @sbkok. - AWS CodeStar Connections name change to CodeConnections (#714) by @sbkok, resolves #616. - Adding retry logic for #655 and add tests for delete_default_vpc.py (#708) by @javydekoning, resolves #655. - Fix allow-empty-targets to match config boolean style (#725) by @sbkok. - Require previously optional CodeBuild image property in build/deploy from v4 onward (#731) by @sbkok, resolves #626 and #601. - YAML files are interpreted via `YAML.safe_load` instead of `YAML.load` (#732) by @sbkok. - Hardened all urlopen calls by checking the protocol (#732) by @sbkok. - Added check to ensure the CloudFormation deployment account id matches with the `/adf/deployment_account_id` if that exists (#732) by @sbkok. - Add automatic creation of the `/adf/deployment_account_id` and `/adf/management_account_id` if that does not exist (#732) by @sbkok. - Separate delete outdated state machine from pipeline creation state machines (#732) by @sbkok. - Review and restrict access provided by ADF managed IAM roles and permissions (#732) by @sbkok, resolves #608 and #390. - Add automatic clean-up of legacy bootstrap stacks, auto recreate if required (#732) by @sbkok. #### Installation improvements With the addition of CDK v2 support. The dependencies that go with it, unfortunately increased the deployment size beyond the limit that is supported by the Serverless Application Repository. Hence the SAR installer is replaced by a new installation process. Please read the [Installation Guide](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/installation-guide.md) how to install ADF. In case you are upgrading, please follow [the admin guide on updating ADF](https://github.com/awslabs/aws-deployment-framework/blob/make/latest/docs/admin-guide.md#updating-between-versions) instead. - New installation process (#677) by @sbkok. - Auto generate unique branch names on new version deployments (#682) by @sbkok. - Ensure tox fails at first pytest failure (#686) by @sbkok. - Install: Add checks to ensure installer dependencies are available (#702) by @sbkok. - Install: Add version checks and pre-deploy warnings (#726) by @sbkok. - Install: Add uncommitted changes check (#733) by @sbkok. #### Documentation, ADF GitHub, and code only improvements - Fixing broken Travis link and build badge (#625), by @javydekoning. - Temporarily disabled cfn-lint after for #619 (#630), by @javydekoning. - Upgrade MegaLinter to v7 and enable cfn-lint (#632), by @javydekoning. - Fix linter failures (#637) by @javydekoning. - Linter fixes (#646) by @javydekoning. - Add docs enhancement regarding ADF and AWS Control Tower (#638) by @AndyEfaa. - Fix include all tests in pytest.ini for bootstrap CodeBuild project (#621) by @AndyEfaa. - Remove CodeCommitRole from initial base stack (#663) by @alFReD-NSH. - Fix bootstrap pipeline tests (#679) by @sbkok. - Add AccessControl property on S3 Buckets (#681) by @sbkok. - Version bump GitHub actions (#704) by @javydekoning, resolves #698. - Bump express from 4.17.3 to 4.19.2 in /samples/sample-fargate-node-app (#697) by @dependabot. - Update copyright statements and license info (#713) by @sbkok. - Fix dead-link in docs (#707) by @javydekoning. - Add BASH_SHFMT linter + linter fixes (#709) by @javydekoning. - Fix sample expunge VPC, if-len, and process deployment maps (#716) by @sbkok. - Moving CDK example app to latest CDK version (#706) by @javydekoning, resolves #618. - Fix Markdown Anchor Link Check (#722) by @sbkok. - Improve samples (#718) by @sbkok. - Explain special purpose of adf-bootstrap/global.yml in docs (#730) by @sbkok, resolves #615. - Rename `deployment_account_bucket` to `shared_modules_bucket` (#732) by @sbkok. - Moved CodeCommit and EventBridge templates from lambda to the bootstrap repository to ease maintenance (#732) by @sbkok.
1 parent b2da600 commit 51f6936

File tree

105 files changed

+6331
-1494
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

105 files changed

+6331
-1494
lines changed

.cspell.json

+1
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
],
1515
"ignorePaths": [
1616
".pylintrc",
17+
"CHANGELOG.md",
1718
"requirements.txt",
1819
"requirements-dev.txt",
1920
"maven-wrapper.jar",

CHANGELOG.md

+569-3
Large diffs are not rendered by default.

docs/admin-guide.md

+31-38
Original file line numberDiff line numberDiff line change
@@ -1036,7 +1036,7 @@ In the management account in `us-east-1`:
10361036
2. There might be a pull request if the `aws-deployment-framework-bootstrap`
10371037
repository that you have has to be updated to apply recent changes of ADF.
10381038
This would show up with the version that you deployed recently, for example
1039-
`v3.2.0`.
1039+
`v4.0.0`.
10401040
3. If there is no pull request, nothing to worry about. In that case, no
10411041
changes were required in your repository for this update. Continue to
10421042
the next step. If there is a pull request, open it and review the
@@ -1091,7 +1091,7 @@ This process is managed in an AWS Step Function state machine.
10911091

10921092
1. Navigate to the AWS Step Functions service in the deployment account
10931093
in _your main region_.
1094-
2. Check the `ADFPipelineManagementStateMachine` state machine, all recent
1094+
2. Check the `adf-pipeline-management` state machine, all recent
10951095
invocations since we performed the update should succeed.
10961096

10971097
We need to confirm that the pipelines generated by ADF are fully functional
@@ -1138,45 +1138,38 @@ Alternatively, you can also perform the update using the AWS CLI.
11381138

11391139
If you wish to remove ADF you can delete the CloudFormation stack named
11401140
`serverlessrepo-aws-deployment-framework` in the management account in
1141-
the `us-east-1` region. This will move into a `DELETE_FAILED` at some stage because
1142-
there is an S3 Bucket that is created via a custom resource _(cross region)_.
1143-
After it moves into `DELETE_FAILED`, you can right-click on the stack and hit
1144-
delete again while selecting to skip the Bucket the stack will successfully
1145-
delete, you can then manually delete the bucket and its contents.
1146-
1147-
After the main stack has been removed you can remove the base stack in the
1148-
deployment account `adf-global-base-deployment` and any associated regional
1141+
the `us-east-1` region. This will remove most resources created by ADF
1142+
in the management account. With the exception of S3 buckets and SSM parameters.
1143+
If you bootstrapped ADF into the management account you need to manually remove
1144+
the bootstrap stacks as well.
1145+
1146+
Feel free to delete the S3 buckets, SSM parameters that start with the `/adf`
1147+
prefix, as well as other CloudFormation stacks such as:
1148+
1149+
- adf-global-base-bootstrap (in the main deployment region)
1150+
- adf-global-base-iam (in the main deployment region)
1151+
- adf-regional-base-bootstrap (in every other region configured for ADF)
1152+
1153+
When these stacks are removed, you can switch into the deployment
1154+
account. We need to remove the base stack in the deployment account
1155+
`adf-global-base-deployment` and any associated regional
11491156
deployment account base stacks. After you have deleted these stacks, you can
11501157
manually remove any base stacks from accounts that were bootstrapped.
1158+
11511159
Alternatively prior to removing the initial
11521160
`serverlessrepo-aws-deployment-framework` stack, you can set the _moves_ section
11531161
of the `adfconfig.yml` file to _remove-base_ which would automatically clean up
11541162
the base stack when the account is moved to the Root of the AWS Organization.
11551163

11561164
One thing to keep in mind if you are planning to re-install ADF is that you
1157-
will want to clean up the parameter from SSM Parameter Store named
1158-
_deployment_account_id_ in `us-east-1` on the management account. AWS Step
1159-
Functions uses this parameter to determine if ADF has already got a deployment
1160-
account setup. If you re-install ADF with this parameter set to a value,
1161-
ADF will attempt an assume role to the account to do some work, which will fail
1162-
since that role will not be on the account at that point.
1163-
1164-
There is also a CloudFormation stack named `adf-global-base-adf-build` which
1165-
lives on the management account in your main deployment region. This stack
1166-
creates two roles on the management account after the deployment account has
1167-
been setup. These roles allow the deployment accounts CodeBuild role to assume a
1168-
role back to the management account in order to query Organizations for AWS
1169-
Accounts. This stack must be deleted manually also. If you do not remove this
1170-
stack and then perform a fresh install of ADF, AWS CodeBuild on the deployment
1171-
account will not be able to assume a role to the management account to query
1172-
AWS Organizations. This is because this specific stack creates IAM roles with a
1173-
strict trust relationship to the CodeBuild role on the deployment account, if
1174-
that role gets deleted _(Which is will when you delete
1175-
`adf-global-base-deployment`)_ then this stack references invalid IAM roles that
1176-
no longer exist. If you forget to remove this stack and notice the trust
1177-
relationship of the IAM roles referenced in the stack are no longer valid,
1178-
you can delete the stack and re-run the main bootstrap pipeline which will
1179-
recreate it with valid roles and links to the correct roles.
1165+
will want to clean up the parameter from SSM Parameter Store. You can safely
1166+
remove all `/adf` prefixed SSM parameters. But most importantly, you need to
1167+
remove the `/adf/deployment_account_id` in `us-east-1` on the
1168+
management account.
1169+
As AWS Step Functions uses this parameter to determine if ADF has already got a
1170+
deployment account setup. If you re-install ADF with this parameter set to a
1171+
value, ADF will attempt an assume role to the account to configure it, which
1172+
will fail since that role will not be on the account at that point.
11801173

11811174
## Troubleshooting
11821175

@@ -1234,15 +1227,15 @@ The main components to look at are:
12341227
deployment region.
12351228
8. Navigate to the [AWS Step Functions service](https://eu-west-1.console.aws.amazon.com/states/home?region=eu-west-1#/statemachines)
12361229
in the deployment account in your main region. Please note, the link points
1237-
to the `eu-west-` region. Please update that to your own deployment region.
1238-
Check the state machines named `ADFPipelineManagementStateMachine`,
1239-
`EnableCrossAccountAccess`, and `PipelineDeletionStateMachine...`.
1240-
Look at recent executions only.
1230+
to the `eu-west-1` region. Please update that to your own deployment region.
1231+
Check the state machines named `adf-pipeline-management`,
1232+
`adf-bootstrap-enable-cross-account`, and
1233+
`adf-pipeline-management-delete-outdated`. Look at recent executions only.
12411234
- When you find one that has a failed execution, check the components that
12421235
are marked orange/red in the diagram.
12431236
- If one failed and you want to trigger it again, you can execute it with
12441237
the `New Execution` button in AWS Step Functions. Or even better in case
1245-
of the `ADFPipelineManagementStateMachine`, trigger all executions again,
1238+
of the `adf-pipeline-management`, trigger all executions again,
12461239
Release a Change in the
12471240
[ADF Pipeline generation CodePipeline - aws-deployment-framework-pipelines](https://console.aws.amazon.com/codesuite/codepipeline/pipelines/aws-deployment-framework-pipelines/view?region=eu-west-1).
12481241

docs/installation-guide.md

+29-2
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,9 @@ AWS Control Tower prior to installing ADF.**
3535

3636
---------------------------------
3737

38-
## 1. Enable CloudTrail
38+
## 1. Enable Services
39+
40+
### 1.1. Enable CloudTrail
3941

4042
Ensure you have setup [AWS CloudTrail](https://aws.amazon.com/cloudtrail/)
4143
*(Not the default trail)* in your Management Account that spans **all
@@ -49,6 +51,28 @@ instructions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtr
4951
to configure the CloudTrail in the `us-east-1` region within the AWS
5052
Organizations Management AWS Account.
5153

54+
### 1.2. Enable AWS Organizations API Access
55+
56+
ADF will setup and configure [AWS
57+
Organizations](https://us-east-1.console.aws.amazon.com/organizations/v2/home?region=us-east-1)
58+
automatically.
59+
60+
However, ADF requires, but does not configure AWS Account Management
61+
automatically.
62+
63+
Without configuring AWS Account Management, the `adf-account-management` Step
64+
Functions state machine will fail to configure the AWS accounts such as the
65+
deployment account for you. The error message that it would return would state:
66+
67+
> An error occurred (AccessDeniedException) when calling the ListRegions operation:
68+
> User: arn:[...assumed-sts-role-arn...]/adf-account-management-config-region
69+
> is not authorized to perform: account:ListRegions
70+
> (Your organization must first enable trusted access with AWS Account Management.)
71+
72+
To enable this, go to AWS Organizations service console after it is configured
73+
and [enable AWS Account Management via this
74+
link](https://us-east-1.console.aws.amazon.com/organizations/v2/home/services/AWS%20Account%20Management).
75+
5276
## 2. Setup Your Build Environment
5377

5478
### 2.1. Local Instructions
@@ -191,7 +215,7 @@ You can checkout a specific version by running:
191215
git checkout ${version_tag_goes_here}
192216

193217
# For example:
194-
git checkout v3.2.0
218+
git checkout v4.0.0
195219
```
196220

197221
### 3.3. Update Makefile
@@ -624,6 +648,9 @@ automatically in the background, to follow its progress:
624648
open AWS CodePipeline from within the management account in `us-east-1` and
625649
see that there is an initial pipeline execution that started.
626650

651+
Upon first installation, this pipeline might fail to fetch the source
652+
code from the repository. Click the retry failed action button to try again.
653+
627654
When ADF is deployed for the first-time, it will make the initial commit
628655
with the skeleton structure of the `aws-deployment-framework-bootstrap`
629656
CodeCommit repository.

docs/providers-guide.md

+50-16
Original file line numberDiff line numberDiff line change
@@ -87,10 +87,12 @@ Provider type: `codecommit`.
8787
information on the use of the owner attribute can be found in the
8888
[CodePipeline
8989
documentation](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_ActionTypeId.html).
90-
- *role* - *(String)* default ADF managed role.
91-
- The role to use to fetch the contents of the CodeCommit repository. Only
92-
specify when you need a specific role to access it. By default ADF will use
93-
its own role to access it instead.
90+
- *role* - *(String)* default: `adf-codecommit-role`.
91+
- The role name of the role to use to fetch the contents of the CodeCommit
92+
repository. Only specify when you need a specific role to access it.
93+
By default ADF will use its own role to access it instead.
94+
- Please read the [user guide](./user-guide.md#custom-roles-for-pipelines) to
95+
learn more about creating custom roles.
9496
- *trigger_on_changes* - *(Boolean)* default: `True`.
9597
- Whether CodePipeline should release a change and trigger the pipeline.
9698
- **When set to False**, you either need to trigger the pipeline manually,
@@ -114,9 +116,15 @@ S3 can be used as the source for a pipeline too. **Please note:** you can use
114116
S3 as a source and deployment provider. The properties that are available are
115117
slightly different.
116118

117-
The role used to fetch the object from the S3 bucket is:
119+
The default role used to fetch the object from the S3 bucket is:
118120
`arn:${partition}:iam::${source_account_id}:role/adf-codecommit-role`.
119121

122+
Please add the required S3 read permissions to the `adf-codecomit-role` via the
123+
`adf-bootstrap/deployment/global-iam.yml` file in the
124+
`aws-deployment-framework-bootstrap` repository. Or, allow
125+
the `adf-codecommit-role` S3 read permissions in the bucket policy of the
126+
source bucket.
127+
120128
Provider type: `s3`.
121129

122130
#### Properties
@@ -277,6 +285,8 @@ Provider type: `codebuild`.
277285
**Please note:** Since the CodeBuild environment runs in the deployment
278286
account, the role you specify will be assumed in and should be available
279287
in the deployment account too.
288+
- Please read the [user guide](./user-guide.md#custom-roles-for-pipelines) to
289+
learn more about creating custom roles.
280290
- *timeout* *(Number)* in minutes, default: `20`.
281291
- If you wish to define a custom timeout for the Build stage.
282292
- *privileged* *(Boolean)* default: `False`.
@@ -452,12 +462,15 @@ Provider type: `codedeploy`.
452462
- The name of the CodeDeploy Application you want to use for this deployment.
453463
- *deployment_group_name* *(String)* **(required)**
454464
- The name of the Deployment Group you want to use for this deployment.
455-
- *role* - *(String)* default
456-
`arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-role`.
465+
- *role* - *(String)* default `adf-cloudformation-role`
466+
- Automatically assumes into the given role in the target account, i.e.
467+
`arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-role`.
457468
- The role you would like to use on the target AWS account to execute the
458469
CodeDeploy action. The role should allow the CodeDeploy service to assume
459470
it. As is [documented in the CodeDeploy service role
460471
documentation](https://docs.aws.amazon.com/codedeploy/latest/userguide/getting-started-create-service-role.html).
472+
- Please read the [user guide](./user-guide.md#custom-roles-for-pipelines) to
473+
learn more about creating custom roles.
461474

462475
### CloudFormation
463476

@@ -513,11 +526,23 @@ Provider type: `cloudformation`.
513526
to `infra`.
514527
- **Defaults to empty string**, the root of the source repository or input
515528
artifact.
516-
- *role* - *(String)* default
517-
`arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-deployment-role`.
529+
- *role* - *(String)* default `adf-cloudformation-deployment-role`
530+
- Automatically assumes into the given role in the target account, i.e.
531+
`arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-deployment-role`.
518532
- The role you would like to use on the target AWS account to execute the
519-
CloudFormation action. Ensure that the CloudFormation service should be
520-
allowed to assume that role.
533+
CloudFormation action.
534+
- Ensure that the CloudFormation service should be allowed to assume that
535+
role.
536+
- Additionally, make sure that the `adf-cloudformation-role` is allowed to
537+
perform an `iam:PassRole` action with the given role. Restrict this action
538+
for the CloudFormation service only.
539+
You can find an example of this in the `adf-bootstrap/deployment/global.yml`
540+
file where it allows the CloudFormation Role to perform `iam:PassRole` with
541+
the `adf-cloudformation-deployment-role`.
542+
Please grant this access in the `adf-bootstrap/deployment/global-iam.yml`
543+
file in the `aws-deployment-framework-bootstrap` repository.
544+
- Please read the [user guide](./user-guide.md#custom-roles-for-pipelines) to
545+
learn more about creating custom roles.
521546
- *action* -
522547
(`CHANGE_SET_EXECUTE|CHANGE_SET_REPLACE|CREATE_UPDATE|DELETE_ONLY|REPLACE_ON_FAILURE`)
523548
default: `CHANGE_SET_EXECUTE`.
@@ -586,7 +611,7 @@ Provider type: `service_catalog`.
586611

587612
### S3
588613

589-
S3 can use used to deploy with too.
614+
S3 is available as a source and deployment provider.
590615

591616
S3 cannot be used to target multiple accounts or regions in one stage.
592617
As the `bucket_name` property needs to be defined and these are globally
@@ -597,9 +622,17 @@ instead. Where each will target the specific bucket in the target account.
597622
Please note: you can use S3 as a source and deployment provider. The properties
598623
that are available are slightly different.
599624

600-
The role used to upload the object(s) to the S3 bucket is:
625+
When S3 is used as the deployment provider, the default role used to upload
626+
the object(s) to the S3 bucket is the:
601627
`arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-role`.
602628

629+
The `adf-cloudformation-role` is not granted access to read S3 buckets yet.
630+
Please add the required S3 write permissions to the `adf-cloudformation-role`
631+
via the `adf-bootstrap/global-iam.yml` file in the
632+
`aws-deployment-framework-bootstrap` repository. Or, alternatively, allow
633+
the `adf-cloudformation-role` S3 write permissions in the bucket policy of the
634+
target bucket.
635+
603636
Provider type: `s3`.
604637

605638
#### Properties
@@ -611,9 +644,10 @@ Provider type: `s3`.
611644
- *extract* - *(Boolean)* default: `False`.
612645
- Whether CodePipeline should extract the contents of the object when it
613646
deploys it.
614-
- *role* - *(String)* default:
615-
`arn:${partition}:iam::${target_account_id}:role/adf-cloudformation-role`.
616-
- The role you would like to use for this action.
647+
- *role* - *(String)* default: `adf-cloudformation-role`.
648+
- The role name of the role you would like to use for this action.
649+
- Please read the [user guide](./user-guide.md#custom-roles-for-pipelines) to
650+
learn more about creating custom roles.
617651
- *kms_encryption_key_arn* - *(String)*
618652
- The ARN of the AWS KMS encryption key for the host bucket. The
619653
`kms_encryption_key_arn` parameter encrypts uploaded artifacts with the

0 commit comments

Comments
 (0)