Skip to content

Commit 6927cf6

Browse files
authored
Merge pull request #21 from awslabs/1.2.1
Updates for 1.3.0
2 parents 1c4f16d + bd24ed0 commit 6927cf6

9 files changed

Lines changed: 255 additions & 9 deletions

CHANGELOG.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,3 +99,19 @@
9999

100100
### Added
101101
* CHANGELOG file
102+
103+
## [1.3.0] - 2021-07-08
104+
105+
### Added
106+
* Code for cleaning up AWS resources created by Assisted Log Enabler for AWS.
107+
* Amazon Route 53 Resolver Query Logging in single account mode is only currently supported.
108+
* Options for running cleanup mode within the main function.
109+
* IAM Permissions example for cleanup operations.
110+
* Information within the Step-by-Step instructions for multi-account to reflect details about AWS CloudFormation StackSets Delegated Administrator.
111+
112+
### Changed
113+
* README documentation.
114+
* Updated Cleanup section to reflect new cleanup capabilities.
115+
* Updated IAM Permissions examples within the README.
116+
* AWS CloudFormation template for deploying IAM Permissions to run cleanup code.
117+
* Header in files to reflect "Assisted Log Enabler for AWS", instead of "Assisted Log Enabler (ALE)".

README.md

Lines changed: 57 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,9 @@ The following is a simple diagram on how Assisted Log Enabler for AWS works in a
2323

2424
## Prerequisites
2525
### Permissions
26-
The following permissions are needed within AWS IAM for Assisted Log Enabler for AWS to run:
26+
The following permissions are needed within AWS IAM for Assisted Log Enabler for AWS to run. Please see each section for a breakdown per AWS Service and functionality:
2727
```
28+
# All permissions used within Assisted Log Enabler for AWS:
2829
"ec2:DescribeVpcs",
2930
"ec2:DescribeFlowLogs",
3031
"ec2:CreateFlowLogs",
@@ -42,7 +43,55 @@ The following permissions are needed within AWS IAM for Assisted Log Enabler for
4243
"route53resolver:ListResolverQueryLogConfigAssociations",
4344
"route53resolver:CreateResolverQueryLogConfig",
4445
"route53resolver:AssociateResolverQueryLogConfig",
46+
"iam:CreateServiceLinkRole", # This is used to create the AWSServiceRoleForRoute53 Resolver, which is used for creating the Amazon Route 53 Query Logging Configurations.
47+
"route53resolver:ListResolverQueryLogConfigs",
48+
"route53resolver:ListTagsForResource",
49+
"route53resolver:DisassociateResolverQueryLogConfig",
50+
"route53resolver:DeleteResolverQueryLogConfig"
51+
52+
# For adding AWS CloudTrail logs:
53+
"s3:GetBucketPolicy",
54+
"s3:PutBucketPolicy",
55+
"s3:PutLifecycleConfiguration"
56+
"s3:PutObject",
57+
"s3:CreateBucket",
58+
"cloudtrail:StartLogging",
59+
"cloudtrail:CreateTrail",
60+
"cloudtrail:DescribeTrails"
61+
62+
# For adding Amazon VPC Flow Logs:
63+
"s3:GetBucketPolicy",
64+
"s3:PutBucketPolicy",
65+
"s3:PutLifecycleConfiguration"
66+
"s3:PutObject",
67+
"s3:CreateBucket",
68+
"ec2:DescribeVpcs",
69+
"ec2:DescribeFlowLogs",
70+
"ec2:CreateFlowLogs"
71+
72+
# For adding Amazon EKS logs:
73+
"eks:UpdateClusterConfig",
74+
"eks:ListClusters",
75+
"logs:CreateLogDelivery"
76+
77+
# For adding Amazon Route 53 Resolver Query Logs:
78+
"s3:GetBucketPolicy",
79+
"s3:PutBucketPolicy",
80+
"s3:PutLifecycleConfiguration"
81+
"s3:PutObject",
82+
"s3:CreateBucket",
83+
"ec2:DescribeVpcs",
84+
"route53resolver:ListResolverQueryLogConfigAssociations",
85+
"route53resolver:CreateResolverQueryLogConfig",
86+
"route53resolver:AssociateResolverQueryLogConfig",
4587
"iam:CreateServiceLinkRole" # This is used to create the AWSServiceRoleForRoute53 Resolver, which is used for creating the Amazon Route 53 Query Logging Configurations.
88+
89+
# NEW! For cleanup of Amazon Route 53 Resolver Query Logs created by Assisted Log Enabler for AWS:
90+
"route53resolver:ListResolverQueryLogConfigs",
91+
"route53resolver:ListTagsForResource",
92+
"route53resolver:ListResolverQueryLogConfigAssociations",
93+
"route53resolver:DisassociateResolverQueryLogConfig",
94+
"route53resolver:DeleteResolverQueryLogConfig"
4695
```
4796
Additionally, if running from within a AWS Lambda function, the function will need the AWSLambdaBasicExecutionRole in order to run successfully. Please refer to the following link for more details: https://docs.aws.amazon.com/lambda/latest/dg/lambda-intro-execution-role.html
4897

@@ -154,6 +203,7 @@ python3 assisted_log_enabler.py --mode single_account --cloudtrail
154203
### Step-by-Step Instructions (for running in AWS CloudShell, multi account mode)
155204
1. Log into the AWS Console of the account you want to run the Assisted Log Enabler for AWS.
156205
* Ensure that the AWS Account you're in is the account you want to store the logs. Additionally, ensure that the AWS account you're in has access to the AWS Organizations information within your AWS environment.
206+
* You may have to register your AWS account as a delegated administrator within AWS CloudFormation, in order to run this code in an AWS account of your choosing. Please see the following link for more details: [Register a delegated administrator](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/stacksets-orgs-delegated-admin.html)
157207
2. Within the AWS Console, go to AWS CloudFormation.
158208
3. Within AWS CloudFormation, go to StackSets.
159209
4. Within the StackSets screen, select Create StackSet.
@@ -225,6 +275,12 @@ Once the logs have been enabled, you can safely remove any of the downloaded fil
225275

226276
For any AWS IAM Roles that are created, either manually or using AWS CloudFormation StackSets, those can be safely deleted upon enablement of logs through the Assisted Log Enabler for AWS.
227277

278+
NEW! A cleanup mode is available within the Assisted Log Enabler for AWS (currently only for single account, Amazon Route 53 Resover Query Logs). Collected logs within Amazon S3 will NOT be removed, however, logging resources can be removed by following the below commands:
279+
```
280+
# To remove Amazon Route 53 Resolver Query Logs (single account):
281+
python3 assisted_log_enabler.py --mode cleanup --single_r53querylogs
282+
```
283+
228284

229285
## Costs
230286
For answers to cost-related questions involved with this solution, refer to the following links:

assisted_log_enabler.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
22
#// SPDX-License-Identifier: Apache-2.0
3-
# Assisted Log Enabler (ALE) - Find resources that are not logging, and turn them on.
3+
# Assisted Log Enabler for AWS - Find resources that are not logging, and turn them on.
44
# Joshua "DozerCat" McKiddy - Team DragonCat - AWS
55

66
import logging
@@ -16,6 +16,7 @@
1616

1717
from subfunctions import ALE_multi_account
1818
from subfunctions import ALE_single_account
19+
from subfunctions import ALE_cleanup_single
1920

2021

2122
current_date = datetime.datetime.now(tz=timezone.utc)
@@ -81,6 +82,9 @@ def assisted_log_enabler():
8182
function_parser_group.add_argument('--r53querylogs', action='store_true', help=' Turns on Amazon Route 53 Resolver Query Logs.')
8283
function_parser_group.add_argument('--cloudtrail', action='store_true', help=' Turns on AWS CloudTrail.')
8384

85+
cleanup_parser_group = parser.add_argument_group('Cleanup Options', 'Use these flags to choose which resources you want to turn logging off for.')
86+
cleanup_parser_group.add_argument('--single_r53querylogs', action='store_true', help=' Turns on Amazon Route 53 Resolver Query Logs.')
87+
8488
args = parser.parse_args()
8589
banner()
8690

@@ -110,6 +114,9 @@ def assisted_log_enabler():
110114
ALE_multi_account.lambda_handler(event, context)
111115
else:
112116
logging.info("No valid option selected. Please run with -h to display valid options.")
117+
elif args.mode == 'cleanup':
118+
if args.single_r53querylogs:
119+
ALE_cleanup_single.run_r53_cleanup()
113120
else:
114121
print("No valid option selected. Please run with -h to display valid options.")
115122

permissions/ALE_child_account_role.yaml

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
22
#// SPDX-License-Identifier: Apache-2.0
3-
# Assisted Log Enabler (ALE) - Find resources that are not logging, and turn them on.
3+
# Assisted Log Enabler for AWS - Find resources that are not logging, and turn them on.
44
# Joshua "DozerCat" McKiddy - Team DragonCat - AWS
55
# This sample template is for creating an IAM Role within child accounts, for the purpose of running Assisted Log Enabler across a multi-account environment.
66

@@ -61,6 +61,17 @@ Resources:
6161
Condition:
6262
StringLike:
6363
'iam:AWSServiceName': 'route53resolver.amazonaws.com'
64+
- Effect: Allow
65+
Action:
66+
- route53resolver:ListResolverQueryLogConfigs
67+
- route53resolver:ListTagsForResource
68+
- route53resolver:ListResolverQueryLogConfigAssociations
69+
- route53resolver:DisassociateResolverQueryLogConfig
70+
- route53resolver:DeleteResolverQueryLogConfig
71+
Resource: '*'
72+
Condition:
73+
StringEquals:
74+
'aws:PrincipalOrgId': !Ref OrgId
6475

6576
AssistedLogEnablerRole:
6677
Type: AWS::IAM::Role
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
// SPDX-License-Identifier: Apache-2.0
3+
// Assisted Log Enabler for AWS - Find resources that are not logging, and turn them on.
4+
// Joshua "DozerCat" McKiddy - Team DragonCat - AWS
5+
6+
{
7+
"Version": "2012-10-17",
8+
"Statement": [
9+
{
10+
"Sid": "AssistedLogEnablerCleanup1",
11+
"Effect": "Allow",
12+
"Action": [
13+
"route53resolver:ListResolverQueryLogConfigs",
14+
"route53resolver:ListTagsForResource",
15+
"route53resolver:ListResolverQueryLogConfigAssociations",
16+
"route53resolver:DisassociateResolverQueryLogConfig",
17+
"route53resolver:DeleteResolverQueryLogConfig"
18+
],
19+
"Resource": "*",
20+
"Condition": {
21+
"StringEquals": {
22+
"aws:SourceAccount": "<enter_account_number_here>"
23+
}
24+
}
25+
}
26+
]
27+
}

permissions/ALE_permissions_example_single_account.json

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
22
// SPDX-License-Identifier: Apache-2.0
3-
// Assisted Log Enabler (ALE) - Find resources that are not logging, and turn them on.
3+
// Assisted Log Enabler for AWS - Find resources that are not logging, and turn them on.
44
// Joshua "DozerCat" McKiddy - Team DragonCat - AWS
55

66
{
@@ -27,7 +27,6 @@
2727
"route53resolver:ListResolverQueryLogConfigAssociations",
2828
"route53resolver:CreateResolverQueryLogConfig",
2929
"route53resolver:AssociateResolverQueryLogConfig"
30-
3130
],
3231
"Resource": "*",
3332
"Condition": {

subfunctions/ALE_cleanup_single.py

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
#// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
#// SPDX-License-Identifier: Apache-2.0
3+
# Assisted Log Enabler for AWS - Find resources that are not logging, and turn them on.
4+
# Joshua "DozerCat" McKiddy - Team DragonCat - AWS
5+
6+
7+
import logging
8+
import os
9+
import json
10+
import boto3
11+
import time
12+
import datetime
13+
from botocore.exceptions import ClientError
14+
from datetime import timezone
15+
16+
17+
current_date = datetime.datetime.now(tz=timezone.utc)
18+
current_date_string = str(current_date)
19+
timestamp_date = datetime.datetime.now(tz=timezone.utc).strftime("%Y-%m-%d-%H%M%S")
20+
timestamp_date_string = str(timestamp_date)
21+
22+
23+
logFormatter = '%(asctime)s - %(levelname)s - %(message)s'
24+
logging.basicConfig(format=logFormatter, level=logging.INFO)
25+
logger = logging.getLogger()
26+
logger.setLevel(logging.INFO)
27+
output_handle = logging.FileHandler('ALE_' + timestamp_date_string + '.log')
28+
output_handle.setLevel(logging.INFO)
29+
logger.addHandler(output_handle)
30+
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
31+
output_handle.setFormatter(formatter)
32+
33+
34+
region = os.environ['AWS_REGION']
35+
36+
37+
region_list = ['af-south-1', 'ap-east-1', 'ap-south-1', 'ap-northeast-1', 'ap-northeast-2', 'ap-northeast-3', 'ap-southeast-1', 'ap-southeast-2', 'ca-central-1', 'eu-central-1', 'eu-west-1', 'eu-west-2', 'eu-west-3', 'eu-north-1', 'eu-south-1', 'me-south-1', 'sa-east-1', 'us-east-1', 'us-east-2', 'us-west-1', 'us-west-2']
38+
39+
40+
# 1. Remove the Route 53 Resolver Query Logging Resources created by Assisted Log Enabler
41+
def r53_cleanup():
42+
"""Function to clean up Route 53 Query Logging Resources"""
43+
for aws_region in region_list:
44+
logging.info("---- LINE BREAK BETWEEN REGIONS ----")
45+
logging.info("Cleaning up Route 53 Query Logging Resources in region " + aws_region + ".")
46+
route53resolver = boto3.client('route53resolver', region_name=aws_region)
47+
try:
48+
QueryLogList: list = []
49+
QueryLogArnRemoveList: list = []
50+
QueryLogIdRemoveList: list = []
51+
logging.info("ListResolverQueryLogConfigs API Call")
52+
ale_r53_logs = route53resolver.list_resolver_query_log_configs() # Collecting Arn of all Query Logs
53+
for r53_arn in ale_r53_logs['ResolverQueryLogConfigs']:
54+
QueryLogList.append(r53_arn['Arn'])
55+
for r53_tag_info in QueryLogList:
56+
logging.info("Listing Tags for " + r53_tag_info)
57+
logging.info("ListTagsForResource API Call")
58+
r53_tags = route53resolver.list_tags_for_resource( # Looking at tags for each Arn collected
59+
ResourceArn=r53_tag_info
60+
)
61+
for value in r53_tags['Tags']:
62+
if (value['Key'] == 'Workflow' and value['Value'] == 'assisted-log-enabler'):
63+
logging.info("The following Route 53 Query Logger was created by Assisted Log Enabler for AWS, and will be removed within this function: " + r53_tag_info)
64+
QueryLogArnRemoveList.append(r53_tag_info)
65+
for Id in QueryLogArnRemoveList:
66+
logging.info("Gathering Resource ID for Route 53 Query Logging Resource to be removed.")
67+
logging.info("ListResolverQueryLogConfigs API Call")
68+
r53_resource_id = route53resolver.list_resolver_query_log_configs()['ResolverQueryLogConfigs'][QueryLogArnRemoveList.index(Id)]['Id'] # Collecting Resource ID for each Arn collected
69+
QueryLogIdRemoveList.append(r53_resource_id)
70+
logging.info(r53_resource_id + " added to removal list.")
71+
logging.info("The following Resource IDs were created by Assisted Log Enabler for AWS, and will be removed within this function.")
72+
print(QueryLogIdRemoveList)
73+
for r53_remove in QueryLogIdRemoveList:
74+
logging.info("Gathering Query Log Config Associations for " + r53_remove)
75+
logging.info("ListResolverQueryLogConfigAssociations API Call")
76+
associated_vpcs = route53resolver.list_resolver_query_log_config_associations(
77+
)
78+
if associated_vpcs['TotalCount'] > 0 and associated_vpcs['ResolverQueryLogConfigAssociations'][0]['ResolverQueryLogConfigId'] == r53_remove:
79+
logging.info("The following Route 53 Query Logger is associated with a VPC, and will be removed within this function: " + r53_remove)
80+
VPCRemovalList = []
81+
for vpc_info in associated_vpcs['ResolverQueryLogConfigAssociations']:
82+
VPCRemovalList.append(vpc_info['ResourceId'])
83+
logging.info("List of VPCs to be disassociated:")
84+
print(VPCRemovalList)
85+
for vpc in VPCRemovalList:
86+
logging.info("Removing " + vpc + " from Route 53 Query Logging configuration " + r53_remove)
87+
logging.info("DisassociateResolverQueryLogConfig API Call")
88+
removing_vpc = route53resolver.disassociate_resolver_query_log_config(
89+
ResolverQueryLogConfigId=r53_remove,
90+
ResourceId=vpc
91+
)
92+
logging.info(vpc + " removed from " + r53_remove)
93+
time.sleep(1)
94+
logging.info("60 second pause to ensure disassociation of Amazon VPCs...")
95+
time.sleep(60)
96+
logging.info("Removing Route 53 Query Logger: " + r53_remove)
97+
logging.info("DeleteResolverQueryLogConfig")
98+
r53_cleanup = route53resolver.delete_resolver_query_log_config(
99+
ResolverQueryLogConfigId=r53_remove
100+
)
101+
logging.info(r53_remove + " has been removed.")
102+
time.sleep(2)
103+
else:
104+
logging.info("Removing Route 53 Query Logger: " + r53_remove)
105+
logging.info("DeleteResolverQueryLogConfig")
106+
r53_cleanup = route53resolver.delete_resolver_query_log_config(
107+
ResolverQueryLogConfigId=r53_remove
108+
)
109+
logging.info(r53_remove + " has been removed.")
110+
time.sleep(2)
111+
except Exception as exception_handle:
112+
logging.error(exception_handle)
113+
114+
115+
def run_r53_cleanup():
116+
"""Function to run the r53_cleanup function"""
117+
r53_cleanup()
118+
logging.info("This is the end of the script. Please feel free to validate that logging resources have been cleaned up.")
119+
120+
121+
def lambda_handler(event, context):
122+
"""Function that runs all of the previously defined functions"""
123+
r53_cleanup()
124+
logging.info("This is the end of the script. Please feel free to validate that logging resources have been cleaned up.")
125+
126+
127+
if __name__ == '__main__':
128+
event = "event"
129+
context = "context"
130+
lambda_handler(event, context)

subfunctions/ALE_multi_account.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
22
#// SPDX-License-Identifier: Apache-2.0
3-
# Assisted Log Enabler (ALE) - Find resources that are not logging, and turn them on.
3+
# Assisted Log Enabler for AWS - Find resources that are not logging, and turn them on.
44
# Joshua "DozerCat" McKiddy - Team DragonCat - AWS
55

66

subfunctions/ALE_single_account.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
22
#// SPDX-License-Identifier: Apache-2.0
3-
# Assisted Log Enabler (ALE) - Find resources that are not logging, and turn them on.
3+
# Assisted Log Enabler for AWS - Find resources that are not logging, and turn them on.
44
# Joshua "DozerCat" McKiddy - Team DragonCat - AWS
55

66

@@ -256,8 +256,8 @@ def route_53_query_logs(region_list, account_number):
256256
logging.info("Route 53 Query Logging Created. Resource ID:" + r53_query_log_id)
257257
for vpc in r53_working_list:
258258
logging.info("Associating " + vpc + " with the created Route 53 Query Logging.")
259-
logging.info("AssocateResolverQueryLogConfig")
260-
activate_r5_logs = route53resolver.associate_resolver_query_log_config(
259+
logging.info("AssociateResolverQueryLogConfig")
260+
activate_r53_logs = route53resolver.associate_resolver_query_log_config(
261261
ResolverQueryLogConfigId=r53_query_log_id,
262262
ResourceId=vpc
263263
)

0 commit comments

Comments
 (0)