Skip to content

Commit 802ad65

Browse files
authored
Merge pull request #21 from kmcquade/fix/botocore-credentials-loading
Fix credentials loading for download command. Set required minimum Policy Sentry version
2 parents 47c20de + 10d7a71 commit 802ad65

File tree

9 files changed

+82
-169
lines changed

9 files changed

+82
-169
lines changed

.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ default-iam-report.csv
99
private/default-iam-results.json
1010
default-results-summary.csv
1111
private/*
12+
current.json
1213

1314
Pipfile.lock
1415

CHANGELOG.md

+5
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# CHANGELOG
22

3+
## 0.0.10 (2020-05-05)
4+
* Removed the recursive credentials method from the `download` command.
5+
* Fixed occasional installation error occurring from outdated Policy Sentry versions.
6+
* Fixed instructions for the `download` command.
7+
38
## 0.0.9 (2020-05-03)
49
* HTML report now always shows Trust Policies for Roles, even if they do not allow assumption from a Compute Service. This can help assessors with triaging and pentesters for targeting.
510

README.md

+10-13
Original file line numberDiff line numberDiff line change
@@ -101,16 +101,11 @@ pip3 install --user cloudsplaining
101101

102102
We can scan an entire AWS account and generate reports. To do this, we leverage the AWS IAM [get-account-authorization-details](https://docs.aws.amazon.com/cli/latest/reference/iam/get-account-authorization-details.html) API call, which downloads a large JSON file (around 100KB per account) that contains all of the IAM details for the account. This includes data on users, groups, roles, customer-managed policies, and AWS-managed policies.
103103

104-
* To do this, set your AWS access keys as environment variables:
104+
* You must have AWS credentials configured that can be used by the CLI.
105105

106-
```bash
107-
export AWS_ACCESS_KEY_ID=...
108-
export AWS_SECRET_ACCESS_KEY=...
109-
# If you are using MFA or STS; optional but highly recommended
110-
export AWS_SESSION_TOKEN=...
111-
```
106+
* You must have the privileges to run [iam:GetAccountAuthorizationDetails](https://docs.aws.amazon.com/IAM/latest/APIReference/API_GetAccountAuthorizationDetails.html). The `arn:aws:iam::aws:policy/SecurityAudit` policy includes this, as do many others that allow Read access to the IAM Service.
112107

113-
* Then run `cloudsplaining`'s `download` command:
108+
* To download the account authorization details, ensure you are authenticated to AWS, then run `cloudsplaining`'s `download` command:
114109

115110
```bash
116111
cloudsplaining download
@@ -119,10 +114,10 @@ cloudsplaining download
119114
* If you prefer to use your `~/.aws/credentials` file instead of environment variables, you can specify the profile name:
120115

121116
```bash
122-
cloudsplaining download --profile default
117+
cloudsplaining download --profile myprofile
123118
```
124119

125-
It will download a file titled `default.json` in your current directory.
120+
It will download a JSON file in your current directory that contains your account authorization detail information.
126121

127122
#### Create Exclusions file
128123

@@ -279,8 +274,6 @@ Actions: ecr:BatchDeleteImage, ecr:CompleteLayerUpload, ecr:CreateRepository, ec
279274
cloudsplaining download
280275
# Download from a specific profile
281276
cloudsplaining download --profile someprofile
282-
# Download authorization details for **all** of your AWS profiles
283-
cloudsplaining download --profile all
284277
285278
# Scan Authorization details
286279
cloudsplaining scan --input default.json
@@ -302,14 +295,18 @@ No, it will only scan policies that are attached to IAM principals.
302295

303296
Not by default. If you want to do this, specify the `--include-non-default-policy-versions` flag. Note that the `scan` tool does not currently operate on non-default versions.
304297

305-
**I followed the installation instructions but can't execute the program via command line. What do I do?**
298+
**I followed the installation instructions but can't execute the program via command line at all. What do I do?**
306299

307300
This is likely an issue with your PATH. Your PATH environment variable is not considering the binary packages installed by `pip3`. On a Mac, you can likely fix this by entering the command below, depending on the versions you have installed. YMMV.
308301

309302
```bash
310303
export PATH=$HOME/Library/Python/3.7/bin/:$PATH
311304
```
312305

306+
**I followed the installation instructions but I am receiving a `ModuleNotFoundError` that says `No module named policy_sentry.analysis.expand`. What should I do?**
307+
308+
Try upgrading to the latest version of Cloudsplaining. This error was fixed in version 0.0.10.
309+
313310
## References
314311

315312
* [Policy Sentry](https://github.com/salesforce/policy_sentry/) by [Kinnaird McQuade](https://twitter.com/kmcquade3) at Salesforce

cloudsplaining/bin/cloudsplaining

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
"""
88
Cloudsplaining is an AWS IAM Assessment tool that identifies violations of least privilege and generates a risk-prioritized HTML report with a triage worksheet.
99
"""
10-
__version__ = "0.0.9"
10+
__version__ = "0.0.10"
1111
import click
1212
from cloudsplaining import command
1313

cloudsplaining/command/download.py

+15-119
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,6 @@
88
import os
99
import json
1010
import logging
11-
import configparser
1211
from pathlib import Path
1312
import boto3
1413
import click
@@ -26,8 +25,7 @@
2625
@click.option(
2726
"--profile",
2827
type=str,
29-
default="default",
30-
required=True,
28+
required=False,
3129
help="Specify 'all' to authenticate to AWS and analyze *all* existing IAM policies. Specify a non-default "
3230
"profile here. Defaults to the 'default' profile.",
3331
)
@@ -37,12 +35,6 @@
3735
default=Path.cwd(),
3836
help="Path to store the output. Defaults to current directory.",
3937
)
40-
@click.option(
41-
"--credentials-file",
42-
type=click.Path(exists=False),
43-
help="Path to the AWS credentials file.",
44-
default=str(Path.home()) + "/.aws/credentials",
45-
)
4638
@click.option(
4739
"--include-non-default-policy-versions",
4840
is_flag=True,
@@ -51,82 +43,23 @@
5143
" Note that this will dramatically increase the size of the downloaded file.",
5244
)
5345
@click_log.simple_verbosity_option(logger)
54-
def download(profile, output, credentials_file, include_non_default_policy_versions):
46+
def download(profile, output, include_non_default_policy_versions):
5547
"""
5648
Runs aws iam get-authorization-details on all accounts specified in the aws credentials file, and stores them in
5749
account-alias.json
5850
"""
59-
# Just default profile
60-
if profile == "default":
61-
profiles = ["default"]
62-
print("profile: default")
63-
# Get all profiles
64-
elif profile == "all":
65-
profiles = get_list_of_aws_profiles(credentials_file)
66-
print("profile: all profiles")
67-
print(profiles)
68-
else:
69-
credentials_file_profiles = get_list_of_aws_profiles(credentials_file)
70-
# If it exists in ~/.aws/credentials,
71-
if profile in credentials_file_profiles:
72-
print(f"profile: {profile}")
73-
profiles = [profile]
74-
else:
75-
raise Exception(
76-
"The profile %s is not in the ~/.aws/credentials file", profile
77-
)
78-
for profile in profiles:
79-
print("Running get_account_authorization_details for profile: ", profile)
80-
get_account_authorization_details(
81-
profile, output, include_non_default_policy_versions
82-
)
83-
84-
85-
def get_account_authorization_details(
86-
profile, output, include_non_default_policy_versions=False
87-
):
88-
"""
89-
Run aws iam get-account-authorization-details and store locally.
90-
91-
:param profile: Name of the profile in the AWS Credentials file
92-
:param output: The path of a directory to store the results.
93-
:param include_non_default_policy_versions: When downloading AWS managed policy documents, also include the non-default policy versions. Note that this will dramatically increase the size of the downloaded file.
94-
:return:
95-
"""
96-
97-
def get_session_via_environment_variables():
98-
aws_access_key_id = os.getenv("AWS_ACCESS_KEY_ID")
99-
aws_secret_access_key = os.getenv("AWS_SECRET_ACCESS_KEY")
100-
aws_session_token = os.getenv("AWS_SESSION_TOKEN")
101-
if aws_access_key_id and aws_secret_access_key and aws_session_token:
102-
session = boto3.Session(
103-
aws_access_key_id=aws_access_key_id,
104-
aws_secret_access_key=aws_secret_access_key,
105-
aws_session_token=aws_session_token,
106-
)
107-
return session
108-
elif aws_access_key_id and aws_secret_access_key and not aws_session_token:
109-
print(
110-
"It looks like you are authenticating with static credentials rather than temporary credentials. "
111-
"Static credentials usage is a security concern. We suggest you use MFA to generate short lived "
112-
"credentials and set those as environment variables instead."
113-
)
114-
session = boto3.Session(
115-
aws_access_key_id=aws_access_key_id,
116-
aws_secret_access_key=aws_secret_access_key,
117-
)
118-
return session
119-
else:
120-
return False
51+
default_region = "us-east-1"
52+
session_data = {"region_name": default_region}
12153

122-
config = Config(connect_timeout=5, retries={"max_attempts": 10})
123-
if get_session_via_environment_variables():
124-
print("AWS Credentials found in Environment variables.")
125-
boto3_session = get_session_via_environment_variables()
54+
if profile:
55+
session_data["profile_name"] = profile
56+
output_filename = os.path.join(output, f"{profile}.json")
12657
else:
127-
boto3_session = boto3.Session(profile_name=profile)
58+
output_filename = "default.json"
12859

129-
iam_client = boto3_session.client("iam", config=config)
60+
session = boto3.Session(**session_data)
61+
config = Config(connect_timeout=5, retries={"max_attempts": 10})
62+
iam_client = session.client("iam", config=config)
13063

13164
results = {
13265
"UserDetailList": [],
@@ -182,46 +115,9 @@ def get_session_via_environment_variables():
182115
}
183116
results["Policies"].append(entry)
184117

185-
filename = os.path.join(output, f"{profile}.json")
186-
if os.path.exists(filename):
187-
os.remove(filename)
188-
with open(filename, "w") as file:
118+
if os.path.exists(output_filename):
119+
os.remove(output_filename)
120+
with open(output_filename, "w") as file:
189121
json.dump(results, file, indent=4, default=str)
190-
print(f"Saved results to {filename}")
122+
print(f"Saved results to {output_filename}")
191123
return 1
192-
193-
194-
def get_list_of_aws_profiles(credentials_file):
195-
"""Get a list of profiles from the AWS Credentials file"""
196-
config = configparser.RawConfigParser()
197-
config.read(credentials_file)
198-
sections = config.sections()
199-
legitimate_sections = []
200-
for section in sections:
201-
# https://github.com/broamski/aws-mfa#credentials-file-setup
202-
broamski_suffix = "-long-term"
203-
# pylint: disable=no-else-continue
204-
if section.endswith(broamski_suffix):
205-
# skip it if it's not a real profile we want to evaluate
206-
continue
207-
else:
208-
legitimate_sections.append(section)
209-
return legitimate_sections
210-
211-
212-
def login(profile_name, service="iam"):
213-
"""Log in to AWS and return a boto3 session."""
214-
default_region = os.environ.get("AWS_REGION", "us-east-1")
215-
session_data = {"region_name": default_region}
216-
if profile_name:
217-
session_data["profile_name"] = profile_name
218-
219-
session = boto3.Session(**session_data)
220-
221-
# Return the service requested by the function - either sts or iam
222-
if service:
223-
this_session = session.client(service)
224-
# By default return IAM
225-
else:
226-
this_session = session.client("iam")
227-
return this_session

docs/user-guide/download.md

+10-21
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,26 @@
1-
# Downloading Account Authorization Details
1+
#### Downloading Account Authorization Details
22

3-
The `download` command downloads a large JSON file containing all the AWS IAM information in your account. This is done via the [aws iam get-account-authorization-details](https://docs.aws.amazon.com/cli/latest/reference/iam/get-account-authorization-details.html) API call. It stores them in `account-alias.json`.
3+
We can scan an entire AWS account and generate reports. To do this, we leverage the AWS IAM [get-account-authorization-details](https://docs.aws.amazon.com/cli/latest/reference/iam/get-account-authorization-details.html) API call, which downloads a large JSON file (around 100KB per account) that contains all of the IAM details for the account. This includes data on users, groups, roles, customer-managed policies, and AWS-managed policies.
44

5-
The `scan` command requires that file.
5+
* You must have AWS credentials configured that can be used by the CLI.
66

7-
## Quick start
7+
* You must have the privileges to run [iam:GetAccountAuthorizationDetails](https://docs.aws.amazon.com/IAM/latest/APIReference/API_GetAccountAuthorizationDetails.html). The `arn:aws:iam::aws:policy/SecurityAudit` policy includes this, as do many others that allow Read access to the IAM Service.
88

9-
* Set your AWS access keys as environment variables:
9+
* To download the account authorization details, ensure you are authenticated to AWS, then run `cloudsplaining`'s `download` command:
1010

1111
```bash
12-
export AWS_ACCESS_KEY_ID=...
13-
export AWS_SECRET_ACCESS_KEY=...
14-
# If you are using MFA or STS; optional but highly recommended
15-
export AWS_SESSION_TOKEN=...
12+
cloudsplaining download
1613
```
1714

18-
* Download the account authorization details
15+
* If you prefer to use your `~/.aws/credentials` file instead of environment variables, you can specify the profile name:
1916

2017
```bash
21-
cloudsplaining download
18+
cloudsplaining download --profile myprofile
2219
```
2320

24-
## Additional Details
25-
26-
#### Order of Precedence
27-
28-
* **Environment variables**: The `download` command will first look for the existence of your AWS access keys in environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN`).
29-
- Note: If you do not have AWS_SESSION_TOKEN set and are using static access keys, I highly recommend the use of [aws-mfa](https://github.com/broamski/aws-mfa) for security reasons.
30-
31-
* **Shared Credentials file**:
32-
- If those environment variables are not set, it will then use the `default` profile in your `~/.aws/credentials` file, if a different profile name is not provided via the argument `--profile`.
33-
- If you specify `--profile all`, it will run the download command recursively for every profile in your `~/.aws/credentials` file.
21+
It will download a JSON file in your current directory that contains your account authorization detail information.
3422

23+
## Additional Details
3524

3625
### Required AWS IAM Policy
3726

docs/user-guide/scan-account.md

+34-8
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,40 @@
1-
# Scanning an Account
1+
#### Scanning the Authorization Details file
22

3-
Scan the Account Authorization details file with the following command
3+
Now that we've downloaded the account authorization file, we can scan *all* of the AWS IAM policies with `cloudsplaining`.
44

5+
Run the following command:
6+
7+
```bash
8+
cloudsplaining scan --exclusions-file exclusions.yml --input examples/files/example.json --output examples/files/
59
```
6-
cloudsplaining scan --input default.json --exclusions-file my-exclusions.yml
7-
```
810

9-
* It will generate three files:
10-
1. The single-file HTML report
11-
2. The triage CSV worksheet, and
12-
3. The raw JSON data file
11+
It will create an HTML report like [this](https://opensource.salesforce.com/cloudsplaining/):
12+
13+
> ![](docs/_images/cloudsplaining-report.gif)
1314
1415

16+
It will also create a raw JSON data file:
17+
18+
* `default-iam-results.json`: This contains the raw JSON output of the report. You can use this data file for operating on the scan results for various purposes. For example, you could write a Python script that parses this data and opens up automated JIRA issues or Salesforce Work Items. An example entry is shown below. The full example can be viewed at [examples/output/example-authz-details-results.json](examples/files/iam-results-example.json)
19+
20+
```json
21+
{
22+
"example-authz-details": [
23+
{
24+
"AccountID": "012345678901",
25+
"ManagedBy": "Customer",
26+
"PolicyName": "InsecureUserPolicy",
27+
"Arn": "arn:aws:iam::012345678901:user/userwithlotsofpermissions",
28+
"ActionsCount": 2,
29+
"ServicesCount": 1,
30+
"Actions": [
31+
"s3:PutObject",
32+
"s3:PutObjectAcl"
33+
],
34+
"Services": [
35+
"s3"
36+
]
37+
}
38+
]
39+
}
40+
```

docs/user-guide/troubleshooting.md

+5-6
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,13 @@
11
# Troubleshooting
22

3-
### Running the command (Path issues)
4-
5-
* *I followed the installation instructions but can't execute the program via command line. What do I do?*
3+
**I followed the installation instructions but can't execute the program via command line at all. What do I do?**
64

75
This is likely an issue with your PATH. Your PATH environment variable is not considering the binary packages installed by `pip3`. On a Mac, you can likely fix this by entering the command below, depending on the versions you have installed. YMMV.
86

97
```bash
10-
# Python 3.7
118
export PATH=$HOME/Library/Python/3.7/bin/:$PATH
12-
# Python 3.8
13-
export PATH=$HOME/Library/Python/3.8/bin/:$PATH
149
```
10+
11+
**I followed the installation instructions but I am receiving a `ModuleNotFoundError` that says `No module named policy_sentry.analysis.expand`. What should I do?**
12+
13+
Try upgrading to the latest version of Cloudsplaining. This error was fixed in version 0.0.10.

setup.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ def get_version():
3737
packages=setuptools.find_packages(exclude=['test*', 'tmp*']),
3838
tests_require=TESTS_REQUIRE,
3939
install_requires=[
40-
'policy_sentry',
40+
'policy_sentry>=0.8.0.3',
4141
'click',
4242
'click_log',
4343
'schema',

0 commit comments

Comments
 (0)