Skip to content

Commit e926afd

Browse files
lcouzenslcouzens
authored
Merge branch 'main' into esebesto/pef_subs_resources
2 parents 851d9c3 + c87c693 commit e926afd

30 files changed

+1095
-405
lines changed

‎.github/workflows/publish-to-pypi.yml

+2-1
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,8 @@ jobs:
2323
- name: Install dependencies
2424
run: |
2525
python3 -m pip install --upgrade pip
26-
python3 -m pip install pipenv-setup setuptools wheel twine chardet vistir==0.6.1
26+
python3 -m pip install pipenv-setup setuptools wheel twine chardet vistir==0.6.1 -c requirements/build-constraints.txt
27+
2728
- name: Build and publish
2829
env:
2930
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

‎.github/workflows/unittest.yml

+4-2
Original file line numberDiff line numberDiff line change
@@ -56,10 +56,12 @@ jobs:
5656
run: pipenv run coverage xml
5757

5858
- name: Upload coverage to Codecov
59-
uses: codecov/[email protected]
59+
uses: codecov/[email protected]
60+
env:
61+
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
6062
with:
61-
token: ${{ secrets.CODECOV_TOKEN }}
6263
file: ./coverage.xml
6364
flags: unittests
6465
name: Python-${{ matrix.python-version}}
6566
fail_ci_if_error: true
67+
plugin: pycoverage # Only run one plugin even though we don't want any to run.

‎.gitleaks.toml

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
[allowlist]
2+
regexes = [
3+
'''AssumedRole:AROAYSLL3JVQ6DYUNKWQJ:1637692740557658269''',
4+
]

‎Pipfile.lock

+214-113
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎README.md

+2
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,8 @@ The `make run-iqe` command by default will run the smoke tests. However, if you
140140
--insights-upload UPLOAD_URL optional, Use local directory path to populate a
141141
"local upload directory".
142142
--ros-ocp-info Optional, Generate ROS for Openshift data.
143+
--constant-values-ros-ocp Optional, Generate constant values for ROS for OpenShift data only
144+
when used with the ros-ocp-info parameter.
143145

144146
OCI Report Options:
145147
--oci-bucket-name BUCKET_NAME optional, OCI bucket name.

‎example_azure_static_data.yml

+7
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,13 @@ generators:
2323
resource_location: "US West"
2424
tags: {"environment": "prod", "project":"p3"}
2525
additional_info: {"ConsumptionMeter": "1111aaaa-22bb-33cc-44dd-555555eeeeee"}
26+
- ManagedDiskGenerator:
27+
start_date: last_month
28+
meter_id: 55555555-4444-3333-2222-111111111116
29+
resource_location: "US North Central"
30+
tags: {"environment": "prod", "project":"p3"}
31+
meter_name: "P4 LRS Disk"
32+
resource_name: "azure-cloud-prefix-pvc-volume_2"
2633

2734

2835
# SubscriptionGuid

‎example_ocp_static_data.yml

+11-2
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
generators:
33
- OCPGenerator:
4-
start_date: 2020-04-01
4+
start_date: 2024-05-01
55
nodes:
66
- node:
77
node_name: alpha
@@ -35,8 +35,10 @@ generators:
3535
- volume:
3636
volume_name: pvc-volume_1
3737
storage_class: gp2
38+
csi_driver:
39+
csi_volume_handle:
3840
volume_request_gig: 20
39-
labels: label_key3:label_value3|label_key4:label_value4|label_storageclass:bravo
41+
labels: label_key3:label_value3|label_key4:label_value4|label_storageclass:gp2
4042
volume_claims:
4143
- volume_claim:
4244
volume_claim_name: pod_name1_data
@@ -51,3 +53,10 @@ generators:
5153
# 2-4-2019: 4
5254
# 2-5-2019: 3
5355
# 2-6-2019: 4
56+
- volume:
57+
volume_name: pvc-volume_claimless
58+
storage_class: gp3-csi
59+
csi_driver: ebs.csi.aws.com
60+
csi_volume_handle: vol-55555555
61+
volume_request_gig: 20
62+
labels: label_key5:label_value5|label_key6:label_value6|label_storageclass:gp3-csi

‎examples/ocp_on_aws/aws_static_data.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -86,11 +86,11 @@ generators:
8686
start_date: 2019-04-01
8787
end_date: 2019-07-31
8888
product_sku: VEAJHRNBBBBA
89-
resource_id: 12345671
89+
resource_id: vol-12345671
9090
amount: 10
9191
rate: 0.01
9292
tags:
93-
resourceTags/user:storageclass: bravo
93+
resourceTags/user:storageclass: gp3-csi
9494
- EBSGenerator:
9595
start_date: 2019-04-01
9696
end_date: 2019-07-31

‎examples/ocp_on_aws/ocp_static_data.yml

+6-4
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
generators:
33
- OCPGenerator:
4-
start_date: 2019-05-01
4+
start_date: 2019-04-01
55
end_date: 2019-07-31
66
nodes:
77
- node:
@@ -24,14 +24,16 @@ generators:
2424
volumes:
2525
- volume:
2626
volume_name: pvc-volume_1
27-
storage_class: gp2
27+
csi_driver: ebs.csi.aws.com
28+
csi_volume_handle: vol-12345671
29+
storage_class: gp3-csi
2830
volume_request_gig: 20
29-
labels: label_environment:dev|label_app:catalog|label_version:prod|label_storageclass:bravo
31+
labels: label_environment:dev|label_app:catalog|label_version:prod|label_storageclass:gp3-csi
3032
volume_claims:
3133
- volume_claim:
3234
volume_claim_name: pod_name1_data
3335
pod_name: pod_name1
34-
labels: label_environment:dev|label_app:catalog|label_version:prod|label_storageclass:bravo
36+
labels: label_environment:dev|label_app:catalog|label_version:prod|label_storageclass:gp3-csi
3537
capacity_gig: 20
3638
catalog:
3739
pods:

‎nise/__init__.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
1-
__version__ = "4.4.14"
1+
__version__ = "4.6.5"
22

33
VERSION = __version__.split(".")

‎nise/__main__.py

+25-11
Original file line numberDiff line numberDiff line change
@@ -196,12 +196,12 @@ def add_azure_parser_args(parser):
196196
help="Azure container to place the data.",
197197
)
198198
parser.add_argument(
199-
"-v2",
200-
"--version-two",
201-
dest="version_two",
199+
"-rg",
200+
"--resource-group",
201+
dest="resource_group_export",
202202
action="store_true",
203203
required=False,
204-
help="Generate version two of the azure report.",
204+
help="Generate resource group based azure report.",
205205
)
206206

207207

@@ -299,6 +299,13 @@ def add_ocp_parser_args(parser):
299299
action="store_true",
300300
help="Flag used to add the `daily_reports` marker to manifests.",
301301
)
302+
parser.add_argument(
303+
"--constant-values-ros-ocp",
304+
dest="constant_values_ros_ocp",
305+
required=False,
306+
action="store_true",
307+
help="Flag to generate constant values for ROS for Openshift",
308+
)
302309

303310

304311
def add_oci_parser_args(parser):
@@ -693,18 +700,25 @@ def _load_static_report_data(options):
693700
end_dates = []
694701
static_report_data = load_yaml(static_file)
695702
for generator_dict in static_report_data.get("generators"):
696-
for _, attributes in generator_dict.items():
703+
for attributes in generator_dict.values():
697704
start_date = get_start_date(attributes, options)
698705
generated_start_date = calculate_start_date(start_date)
699706
start_dates.append(generated_start_date)
700-
if attributes.get("end_date"):
701-
generated_end_date = calculate_end_date(generated_start_date, attributes.get("end_date"))
702-
elif options.get("end_date") and options.get("end_date").date() != today().date():
703-
generated_end_date = calculate_end_date(generated_start_date, options.get("end_date"))
704-
else:
705-
generated_end_date = today()
707+
708+
end_date = attributes.get("end_date", options.get("end_date"))
709+
generated_end_date = today()
710+
if end_date and (
711+
end_date != today().date()
712+
or (
713+
isinstance(end_date, datetime.datetime)
714+
and (end_date.date() != today().date() or end_date.hour != 0)
715+
)
716+
):
717+
generated_end_date = calculate_end_date(generated_start_date, end_date)
718+
706719
if options.get("provider") == "azure":
707720
generated_end_date += datetime.timedelta(hours=24)
721+
708722
end_dates.append(generated_end_date)
709723

710724
attributes["start_date"] = str(generated_start_date)

‎nise/generators/aws/aws_constants.py

+1
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
"""AWS Report Constants"""
1818

1919
REGIONS = (
20+
# (location description, region code, sub region code, storage region)
2021
("US East (N. Virginia)", "us-east-1", "us-east-1a", "USE1-EBS"),
2122
("US East (N. Virginia)", "us-east-1", "us-east-1b", "USE1-EBS"),
2223
("US East (N. Virginia)", "us-east-1", "us-east-1c", "USE1-EBS"),

‎nise/generators/aws/aws_generator.py

+1-2
Original file line numberDiff line numberDiff line change
@@ -324,8 +324,7 @@ def _init_data_row(self, start, end, **kwargs): # noqa: C901
324324
def _get_location(self):
325325
"""Pick instance location."""
326326
options = None
327-
if self.attributes and self.attributes.get("region"):
328-
region = self.attributes.get("region")
327+
if region := self.attributes.get("region"):
329328
options = [option for option in REGIONS if region in option]
330329
if options:
331330
location = choice(options)

‎nise/generators/aws/data_transfer_generator.py

+28-27
Original file line numberDiff line numberDiff line change
@@ -25,45 +25,46 @@ class DataTransferGenerator(AWSGenerator):
2525
"""Generator for Data Transfer data."""
2626

2727
DATA_TRANSFER = (
28-
("{}-{}-AWS-In-Bytes", "PublicIP-In", "InterRegion Inbound"),
29-
("{}-{}-AWS-Out-Bytes", "PublicIP-Out", "InterRegion Outbound"),
28+
# (usage type, operation, transfer type)
29+
("{region1}-{region2}-AWS-{direction}-Bytes", "PublicIP-{direction}", "InterRegion {direction}bound"),
30+
("DataTransfer-{direction}-Bytes", "RunInstances", ""),
31+
("{region1}-DataTransfer-Regional-Bytes", "PublicIP-{direction}", ""),
32+
("{region1}-DataTransfer-Regional-Bytes", "InterZone-{direction}", ""),
3033
)
34+
DATA_TRANSFER_DIRECTIONS = ("in", "out")
3135

3236
def __init__(self, start_date, end_date, currency, payer_account, usage_accounts, attributes=None, tag_cols=None):
3337
"""Initialize the data transfer generator."""
3438
super().__init__(start_date, end_date, currency, payer_account, usage_accounts, attributes, tag_cols)
35-
self._amount = None
36-
self._rate = None
37-
self._saving = None
38-
self._product_sku = None
39-
self._resource_id = None
40-
self._product_code = "AmazonEC2"
41-
self._product_name = "Amazon Elastic Compute Cloud"
42-
if attributes:
43-
if attributes.get("product_code"):
44-
self._product_code = attributes.get("product_code")
45-
if attributes.get("product_name"):
46-
self._product_name = attributes.get("product_name")
47-
if attributes.get("resource_id"):
48-
self._resource_id = attributes.get("resource_id")
49-
if attributes.get("amount"):
50-
self._amount = float(attributes.get("amount"))
51-
if attributes.get("rate"):
52-
self._rate = float(attributes.get("rate"))
53-
if attributes.get("product_sku"):
54-
self._product_sku = attributes.get("product_sku")
55-
if attributes.get("tags"):
56-
self._tags = attributes.get("tags")
57-
if attributes.get("saving"):
58-
self._saving = float(attributes.get("saving"))
39+
40+
self._amount = float(self.attributes.get("amount", 0)) or None
41+
self._data_direction = self.attributes.get("data_direction")
42+
self._product_code = self.attributes.get("product_code", "AmazonEC2")
43+
self._product_name = self.attributes.get("product_name", "Amazon Elastic Compute Cloud")
44+
self._product_sku = self.attributes.get("product_sku")
45+
self._rate = float(self.attributes.get("rate", 0)) or None
46+
self._resource_id = f"i-{self.attributes.get('resource_id', self.fake.ean8())}"
47+
self._saving = float(self.attributes.get("saving", 0)) or None
48+
self._tags = self.attributes.get("tags", self._tags)
49+
50+
@property
51+
def data_direction(self):
52+
if self._data_direction is not None:
53+
return self._data_direction.capitalize()
54+
55+
# Purposefully not caching this value so a different value is returned on each call
56+
return choice(self.DATA_TRANSFER_DIRECTIONS).capitalize()
5957

6058
def _get_data_transfer(self, rate):
6159
"""Get data transfer info."""
6260
location1, aws_region, _, storage_region1 = self._get_location()
6361
location2, _, _, storage_region2 = self._get_location()
6462
trans_desc, operation, trans_type = choice(self.DATA_TRANSFER)
65-
trans_desc = trans_desc.format(storage_region1, storage_region2)
63+
trans_desc = trans_desc.format(region1=storage_region1, region2=storage_region2, direction=self.data_direction)
64+
operation = operation.format(direction=self.data_direction)
65+
trans_type = trans_type.format(direction=self.data_direction)
6666
description = f"${rate} per GB - {location1} data transfer to {location2}"
67+
6768
return trans_desc, operation, description, location1, location2, trans_type, aws_region
6869

6970
def _get_product_sku(self):

0 commit comments

Comments
 (0)