Merge branch 'master' into filter-aws-groups

Author: agnes-gajda

.gitignore

@@ -34,3 +34,10 @@ ssosync
 # Noise from os/editors
 .DS_Store
 *.swp
+*/.DS_Store
+cicd/.DS_Store
+release.yaml
+staging.yaml
+*.orig
+*.rej
+cicd/.DS_Store

Makefile

@@ -20,6 +20,10 @@ go-build:
 clean:
 	rm -f $(OUTPUT) $(PACKAGED_TEMPLATE)
 
+build-SSOSyncFunction:
+	GOOS=linux GOARCH=arm64 go build -o bootstrap main.go
+	cp ./bootstrap $(ARTIFACTS_DIR)/.
+
 .PHONY: install
 install:
 	go get ./...

README.md

@@ -10,13 +10,25 @@
 
 SSO Sync will run on any platform that Go can build for. It is available in the [AWS Serverless Application Repository](https://console.aws.amazon.com/lambda/home#/create/app?applicationId=arn:aws:serverlessrepo:us-east-2:004480582608:applications/SSOSync)
 
-> :warning: there are breaking changes for versions `>= 0.02`
+> [!CAUTION]
+> When using ssosync with an instance or IAM Identity Center integrated with AWS Control Tower. AWS Control Tower creates a number of groups and users (directly via the Identity Store API), when an external identity provider is configured these users and groups are can not be used to log in. However it is important to remember that because ssosync implemements a uni-directional sync it will make the IAM Identity Store match the subset of your Google Workspaces directory you specify, including removing these groups and users created by AWS Control Tower. There is a PFR [#88 - ssosync deletes Control Tower groups](https://github.com/awslabs/ssosync/issues/88) to implement an option to ignore these users and groups, hopefully this will be implemented in version 3.x.
 
-> :warning: `>= 1.0.0-rc.5` groups to do not get deleted in AWS SSO when deleted in the Google Directory, and groups are synced by their email address
+> [!WARNING]
+> There are breaking changes for versions `>= 0.02`
 
-> :warning: `>= 2.0.0` this makes use of the **Identity Store API** which means:
-* if deploying the lambda from the [AWS Serverless Application Repository](https://console.aws.amazon.com/lambda/home#/create/app?applicationId=arn:aws:serverlessrepo:us-east-2:004480582608:applications/SSOSync) then it needs to be deployed into the [IAM Identity Center delegated administration](https://docs.aws.amazon.com/singlesignon/latest/userguide/delegated-admin.html) account. Technically you could deploy in the management account but we would recommend against this.
-* if you are running the project as a cli tool, then the environment will need to be using credentials of a user in the [IAM Identity Center delegated administration](https://docs.aws.amazon.com/singlesignon/latest/userguide/delegated-admin.html) account, with appropriate permissions.
+> [!WARNING]
+> `>= 1.0.0-rc.5` groups to do not get deleted in AWS SSO when deleted in the Google Directory, and groups are synced by their email address
+
+> [!WARNING]
+> `>= 2.0.0` this makes use of the **Identity Store API** which means:
+> * if deploying the lambda from the [AWS Serverless Application Repository](https://console.aws.amazon.com/lambda/home#/create/app?applicationId=arn:aws:serverlessrepo:us-east-2:004480582608:applications/SSOSync) then it needs to be deployed into the [IAM Identity Center delegated administration](https://docs.aws.amazon.com/singlesignon/latest/userguide/delegated-admin.html) account. Technically you could deploy in the management account but we would recommend against this.
+> * if you are running the project as a cli tool, then the environment will need to be using credentials of a user in the [IAM Identity Center delegated administration](https://docs.aws.amazon.com/singlesignon/latest/userguide/delegated-admin.html) account, with appropriate permissions.
+
+> [!WARNING]
+> `>= 2.1.0` make use of named IAM resources, so if deploying via CICD or IaC template will require **CAPABILITY_NAMED_IAM** to be specified.
+
+> [!IMPORTANT]
+> `>= 2.1.0` switched to using `provided.al2` powered by ARM64 instances.
 
 ## Why?
 
@@ -161,22 +173,49 @@ Flags Notes:
 * `--group-match` works for both `--sync-method` values and also in combination with `--ignore-groups` and `--ignore-users`.  This is the filter query passed to the [Google Workspace Directory API when search Groups](https://developers.google.com/admin-sdk/directory/v1/guides/search-groups), if the flag is not used, groups are not filtered.
 * `--user-match` works for both `--sync-method` values and also in combination with `--ignore-groups` and `--ignore-users`.  This is the filter query passed to the [Google Workspace Directory API when search Users](https://developers.google.com/admin-sdk/directory/v1/guides/search-users), if the flag is not used, users are not filtered.
 
-NOTES:
-
-1. Depending on the number of users and groups you have, maybe you can get `AWS SSO SCIM API rate limits errors`, and more frequently happens if you execute the sync many times in a short time.
-2. Depending on the number of users and groups you have, `--debug` flag generate too much logs lines in your AWS Lambda function.  So test it in locally with the `--debug` flag enabled and disable it when you use a AWS Lambda function.
+> [!NOTE]
+> 1. Depending on the number of users and groups you have, maybe you can get `AWS SSO SCIM API rate limits errors`, and more frequently happens if you execute the sync many times in a short time.
+> 2. Depending on the number of users and groups you have, `--debug` flag generate too much logs lines in your AWS Lambda function.  So test it in locally with the `--debug` flag enabled and disable it when you use a AWS Lambda function.
 
 ## AWS Lambda Usage
 
-NOTE: Using Lambda may incur costs in your AWS account. Please make sure you have checked
+> [!TIP]
+> Using Lambda may incur costs in your AWS account. Please make sure you have checked
 the pricing for AWS Lambda and CloudWatch before continuing.
 
 Additionally, before choosing to deploy with Lambda, please ensure that the [AWS Lambda SLAs](https://aws.amazon.com/lambda/sla/) are sufficient for your use cases.
 
 Running ssosync once means that any changes to your Google directory will not appear in
 AWS SSO. To sync regularly, you can run ssosync via AWS Lambda.
 
-:warning: You find it in the [AWS Serverless Application Repository](https://eu-west-1.console.aws.amazon.com/lambda/home#/create/app?applicationId=arn:aws:serverlessrepo:us-east-2:004480582608:applications/SSOSync).
+> [!WARNING]
+> You find it in the [AWS Serverless Application Repository](https://eu-west-1.console.aws.amazon.com/lambda/home#/create/app?applicationId=arn:aws:serverlessrepo:us-east-2:004480582608:applications/SSOSync).
+
+> [!TIP]
+> ### v2.1 Changes
+> * user and group selection fields in the Cloudformation template can now be left empty where not required and will not be added as environment variables to the Lambda function, this provides consistency with CLI use of ssosync.
+> * Stronger validation of parameters in the Cloudformation template, to improve likelhood of success for new users.
+> * Now supports multiple deployment patterns, defaults are consistent with previous versions.
+
+**App + secrets** This is the default mode and fully backwards compatible with previous versions
+
+**App only** This mode does not create the secrets but expects you to deployed a separate stack using the **Secrets only** mode within the same account
+> [!CAUTION]
+> If you want to use your own existing secrets then provide them as a comma separated list in the ##CrossStackConfigI## field in the following order:
+> __GoogleCredentials ARN__,__GoogleAdminEmail ARN__,__SCIMEndpoint ARN__,__SCIMAccessToken ARN__,__Region ARN__,__IdentityStoreID ARN__
+> 
+**App for cross-account** This mode is used where you have deployed the secrets in a separate account, the arns of the KMS key and secrets need to be passed into the __CrossStackConfig__ field, It is easiest to have created the secrets in the other account using the ** Secrest for cross-account** mode, as the output can simply copied and pasted into the above field.
+
+> [!CAUTION]
+> If you want to use your own existing secrets then provide them as a comma separated list in the __CrossStackConfig__ field in the following order:
+> __GoogleCredentials ARN__,__GoogleAdminEmail ARN__,__SCIMEndpoint ARN__,__SCIMAccessToken ARN__,__Region ARN__,__IdentityStoreID ARN__,__KMS Key ARN__
+
+> [!IMPORTANT]
+> Be sure to allow access to the key and secrets in their respective policies to the role __SSOSyncAppRole__ in the app account.
+
+**Secrets only** This mode creates a set of secrets but does not deploy the app itself, it requires the app is deployed in that same account using the **App only** mode. This allows for decoupling of the secrets and the app.
+
+**Secrets for cross-account** This mode creates a set of secrets and KMS key but does not deploy the app itself, this is for use with an app stack, deployed using the **App for cross-account** mode. This allows for a single set of secrets to be shared with multipl app instance for testing, and improve secrets security.
 
 ## SAM
 

cicd/.DS_Store

@@ -0,0 +1,57 @@
+version: 0.2
+
+phases:
+
+  build:
+    commands:
+      # Create parameters
+      - export AppVersion="${GitTag#v}-${GitVersionHash}"
+
+      # Copy in the executable
+      - cp ${CODEBUILD_SRC_DIR_Built}/dist/ssosync_linux_amd64_v1/ssosync ./
+
+      # Copy in the tests
+      - cp -r cicd/tests ./
+
+      # Copy in the stack and params templates
+      - mkdir deploy
+      - cp cicd/account_execution/staging/stack.yml ./deploy/
+
+      # Update params with the values for this run for a developer account
+      - |
+        jq -n \
+        --argjson Parameters "{\"AppArn\": \"$AppArn\", \"AppVersion\": \"$AppVersion\", \"GoogleAdminEmailArn\": \"$SecretGoogleAdminEmail\", \"GoogleCredentialsArn\": \"$SecretGoogleCredentials\", \"SCIMEndpointUrlArn\": \"$SecretSCIMEndpoint\", \"SCIMAccessTokenArn\": \"$SecretSCIMAccessToken\", \"RegionArn\": \"$SecretRegion\", \"IdentityStoreIdArn\": \"$SecretIdentityStoreID\", \"GroupMatch\": \"name:AWS*\"}" \
+        --argjson StackPolicy "{\"Statement\":[{\"Effect\": \"Allow\", \"NotAction\": \"Update:Delete\", \"Principal\": \"*\", \"Resource\": \"*\"}]}" \
+        '$ARGS.named' > ./deploy/developer.json
+      - cat ./deploy/developer.json
+
+      # Update params with the values for this run for the management account
+      - |
+        jq -n \
+        --argjson Parameters "{\"AppArn\": \"$AppArn\", \"AppVersion\": \"$AppVersion\", \"GoogleAdminEmailArn\": \"$SecretGoogleAdminEmail\", \"GoogleCredentialsArn\": \"$SecretGoogleCredentials\", \"SCIMEndpointUrlArn\": \"$SecretSCIMEndpoint\", \"SCIMAccessTokenArn\": \"$SecretSCIMAccessToken\", \"RegionArn\": \"$SecretRegion\", \"IdentityStoreIdArn\": \"$SecretIdentityStoreID\", \"GroupMatch\": \"name:Man*\"}" \
+        --argjson StackPolicy "{\"Statement\":[{\"Effect\": \"Allow\", \"NotAction\": \"Update:Delete\", \"Principal\": \"*\", \"Resource\": \"*\"}]}" \
+        '$ARGS.named' > ./deploy/management.json
+      - cat ./deploy/management.json
+
+      # Update params with the values for this run for the delegated account
+      - |
+        jq -n \
+        --argjson Parameters "{\"AppArn\": \"$AppArn\", \"AppVersion\": \"$AppVersion\",  \"GoogleAdminEmailArn\": \"$SecretGoogleAdminEmail\", \"GoogleCredentialsArn\": \"$SecretGoogleCredentials\", \"SCIMEndpointUrlArn\": \"$SecretSCIMEndpoint\", \"SCIMAccessTokenArn\": \"$SecretSCIMAccessToken\", \"RegionArn\": \"$SecretRegion\", \"IdentityStoreIdArn\": \"$SecretIdentityStoreID\", \"GroupMatch\": \"name:Del*\"}" \
+        --argjson StackPolicy "{\"Statement\":[{\"Effect\": \"Allow\", \"NotAction\": \"Update:Delete\", \"Principal\": \"*\", \"Resource\": \"*\"}]}" \
+        '$ARGS.named' > ./deploy/delegated.json
+      - cat ./deploy/delegated.json
+
+      # Update params with the values for this run for non-delegated account
+      - |
+        jq -n \
+        --argjson Parameters "{\"AppArn\": \"$AppArn\", \"AppVersion\": \"$AppVersion\", \"GoogleAdminEmailArn\": \"$SecretGoogleAdminEmail\", \"GoogleCredentialsArn\": \"$SecretGoogleCredentials\", \"SCIMEndpointUrlArn\": \"$SecretSCIMEndpoint\", \"SCIMAccessTokenArn\": \"$SecretSCIMAccessToken\", \"RegionArn\": \"$SecretRegion\", \"IdentityStoreIdArn\": \"$SecretIdentityStoreID\", \"GroupMatch\": \"name:Non*\"}" \
+        --argjson StackPolicy "{\"Statement\":[{\"Effect\": \"Allow\", \"NotAction\": \"Update:Delete\", \"Principal\": \"*\", \"Resource\": \"*\"}]}" \
+        '$ARGS.named' > ./deploy/nondelegated.json
+      - cat ./deploy/nondelegated.json
+
+
+artifacts:
+  files:
+    - ssosync
+    - deploy/**/*
+    - tests/**/*

cicd/account_execution/staging/buildspec.yml

@@ -0,0 +1,77 @@
+AWSTemplateFormatVersion: '2010-09-09'
+Transform: 'AWS::Serverless-2016-10-31'
+
+Description: 
+  This CloudFormation template will deploy an instance of the SSOSync-Staging
+  candidate releases (via privately shared app in the AWS Serverless Application
+  Repository (SAR) within the Staging Account.
+
+Parameters:
+  AppArn:
+    Description: The candidate release in the SAR
+    Default: 'arn:aws:serverlessrepo:<AWS::Region>:<AccountId>:applications/<ApplicationName>'
+    Type: String
+  AppVersion:
+    Description: The version of this build in SAR
+    Default: 'v1.0.0-rc.10'
+    Type: String
+  GoogleAdminEmailArn:
+    Type: String
+  GoogleCredentialsArn:
+    Type: String
+  SCIMEndpointUrlArn:
+    Type: String
+  SCIMAccessTokenArn:
+    Type: String
+  RegionArn:
+    Type: String
+  IdentityStoreIdArn:
+    Type: String
+  GroupMatch:
+    Description: The search string to match Groups in Google Workspace
+    Default: 'name:AWS*'
+    Type: String
+
+Resources:
+  SARApp:
+    Type: AWS::Serverless::Application
+    Properties:
+      Location:
+        ApplicationId: !Ref AppArn
+        SemanticVersion: !Ref AppVersion
+      Parameters:
+        FunctionName: SSOSyncFunction
+        GoogleAdminEmail: !Join
+          - ''
+          - - '{{resolve:secretsmanager:'
+            - !Ref GoogleAdminEmailArn
+            - '}}'
+        GoogleCredentials: !Join
+          - ''
+          - - '{{resolve:secretsmanager:'
+            - !Ref GoogleCredentialsArn
+            - '}}'
+        SCIMEndpointUrl: !Join
+          - ''
+          - - '{{resolve:secretsmanager:'
+            - !Ref SCIMEndpointUrlArn
+            - '}}'
+        SCIMEndpointAccessToken: !Join
+          - ''
+          - - '{{resolve:secretsmanager:'
+            - !Ref SCIMAccessTokenArn
+            - '}}'
+        Region: !Join
+          - ''
+          - - '{{resolve:secretsmanager:'
+            - !Ref RegionArn
+            - '}}'
+        IdentityStoreID: !Join
+          - ''
+          - - '{{resolve:secretsmanager:'
+            - !Ref IdentityStoreIdArn
+            - '}}'
+        SyncMethod: groups
+        GoogleGroupMatch: !Ref GroupMatch
+        LogLevel: warn
+        LogFormat: json

cicd/staging/build/params.json …cd/account_execution/staging/params.jsoncicd/staging/build/params.json renamed to cicd/account_execution/staging/params.json cicd/staging/build/params.json renamed to cicd/account_execution/staging/params.json

@@ -3,7 +3,6 @@ version: 0.2
 env:
   variables:
     ShareWith: "NOT-SHARED"
-    pipeline: "SSOSync-Test"
     interval: 10
     Success: '"Succeeded"'
     InProgress: '"InProgress"'

cicd/account_execution/staging/stack.yml

@@ -8,20 +8,28 @@ env:
 phases:
   install: 
     commands:
+      # Add goreleaser repo
+      - echo 'deb [trusted=yes] https://repo.goreleaser.com/apt/ /' | sudo tee /etc/apt/sources.list.d/goreleaser.list
+      
+      # Update the repos
+      - apt -qq --yes update
+      - apt -qq --yes upgrade
+      
       # Install go.lang
       - GoVersion=${GOLANG_20_VERSION}
 
-      # Install golint
+      # Install golint - now deprecated
       - go install golang.org/x/lint/golint@latest
 
-      # Install staticcheck
-      - go install honnef.co/go/tools/cmd/staticcheck@latest
+      # Install staticcheck - use static install from tarball
+      - wget -qO- https://github.com/dominikh/go-tools/releases/download/2023.1.6/staticcheck_linux_386.tar.gz | tar -xvz -C ./
 
       # Install Testify to use common assertions and mocks in tests
-      - go get -u github.com/stretchr/testify
+      - go get github.com/stretchr/testify
 
-      # Install goreleaser
-      - go install github.com/goreleaser/goreleaser@latest
+      # Install goreleaser - go install method broken due to dependancies using apt static binary approach
+      #      - go install github.com/goreleaser/goreleaser@latest
+      - apt -qq --yes install goreleaser
 
   pre_build: 
     commands:
@@ -33,7 +41,7 @@ phases:
       - go get ./...
 
       # Run staticcheck
-      - staticcheck ./...
+      - staticcheck/staticcheck ./...
 
       # Ensure code passes all lint tests
       #- golint -set_exit_status ./...
@@ -63,8 +71,8 @@ phases:
       # Tweak the .goreleaser.yml so it uses the vairables from .Env
       - patch .goreleaser.yml cicd/build/build/goreleaser.patch
 
-      # Make main but only for the lambda (linux amd64)
-      - goreleaser build --snapshot --rm-dist --single-target
+      # Make main 
+      - goreleaser build --snapshot --clean
 
 
       # Check we've packaged something useful

cicd/staging/testing/buildspec.yml …/account_execution/testing/buildspec.ymlcicd/staging/testing/buildspec.yml renamed to cicd/account_execution/testing/buildspec.yml cicd/staging/testing/buildspec.yml renamed to cicd/account_execution/testing/buildspec.yml

@@ -1,8 +1,21 @@
---- .goreleaser.yml	2022-06-15 08:38:24.000000000 +0100
-+++ .goreleaser-codebuild.yml	2022-06-21 12:33:43.000000000 +0100
-@@ -22,7 +22,7 @@
-     - goos: windows
-       goarch: 386
+--- .goreleaser.yml.default	2023-10-25 11:30:58
++++ .goreleaser.yml	2023-10-25 11:32:18
+@@ -9,20 +9,11 @@
+     - CGO_ENABLED=0
+   goos:
+     - linux
+-    - darwin
+-    - windows
+   goarch:
+-    - 386
+     - amd64
+-    - arm
+     - arm64
+-  ignore:
+-    - goos: darwin
+-      goarch: 386
+-    - goos: windows
+-      goarch: 386
    ldflags:
 -     - -s -w -X github.com/awslabs/ssosync/cmd.version={{.Version}} -X github.com/awslabs/ssosync/cmd.commit={{.Commit}} -X github.com/awslabs/ssosync/cmd.date={{.Date}} -X github.com/awslabs/ssosync/cmd.builtBy=goreleaser
 +     - -s -w -X github.com/awslabs/ssosync/cmd.version={{.Env.GitTag}} -X github.com/awslabs/ssosync/cmd.commit={{.Env.GitCommit}} -X github.com/awslabs/ssosync/cmd.date={{.Date}} -X github.com/awslabs/ssosync/cmd.builtBy=goreleaser -X github.com/awslabs/ssosync/cmd.goversion={{.Env.GoVersion}}