AWS Cloud Cost Pricing docs [SA-7086]

content/integrations/aws-pricing.md

@@ -0,0 +1,185 @@
+# AWS Pricing Model
+
+MTO supports AWS pricing model via the `integrationConfig.components.showbackOpts.cloudIntegrationSecretRef` field. Following 3 types of pricing are supported:
+
+- [`AWS Standard Pricing`](#aws-standard-pricing)
+- [`AWS Spot Instance Datafeed`](#aws-spot-instance-pricing)
+
+## AWS Standard Pricing
+
+OpenCost will automatically read the node information `node.spec.providerID` to determine the cloud service provider (CSP) in use. If it detects the CSP is AWS, it will attempt to pull the AWS on-demand pricing from the configured public API URL with no further configuration required.
+
+OpenCost will request pricing data from the `us-east-1` region for your `node_region` using the template:
+
+```url
+https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/${node_region}/index.json
+```
+
+## AWS Spot Instance Pricing
+
+To enable the AWS Spot Instance pricing, subscribe to AWS Spot Instance Data Feed with the following command
+
+```sh
+aws ec2 create-spot-datafeed-subscription \
+    --bucket <BUCKET_NAME> \
+    [--prefix <BUCKET_PREFIX>]
+```
+
+- `<BUCKET_NAME>` — Name of the bucket Amazon S3 bucket to store the data feed files
+- `<BUCKET_PREFIX>` — Optional prefix directory in which data feed files will be stored
+
+### Amazon S3 bucket requirements
+
+When you subscribe to the data feed, you must specify an Amazon S3 bucket to store the data feed files.
+
+Before you choose an Amazon S3 bucket for the data feed, consider the following:
+
+- You must have `FULL_CONTROL` permission to the bucket. If you're the bucket owner, you have this permission by default. Otherwise, the bucket owner must grant your AWS account this permission.
+
+- When you subscribe to a data feed, these permissions are used to update the bucket ACL to give the AWS data feed account `FULL_CONTROL` permission. The AWS data feed account writes data feed files to the bucket. If your account doesn't have the required permissions, the data feed files cannot be written to the bucket. For more information, see Logs sent to Amazon S3 in the Amazon CloudWatch Logs User Guide.
+
+- If you update the ACL and remove the permissions for the AWS data feed account, the data feed files cannot be written to the bucket. You must resubscribe to the data feed to receive the data feed files.
+
+- Each data feed file has its own ACL (separate from the ACL for the bucket). The bucket owner has `FULL_CONTROL` permission to the data files. The AWS data feed account has read and write permissions.
+
+- If you delete your data feed subscription, Amazon EC2 doesn't remove the read and write permissions for the AWS data feed account on either the bucket or the data files. You must remove these permissions yourself.
+
+- If you encrypt your Amazon S3 bucket using server-side encryption with an AWS KMS key stored in AWS Key Management Service (SSE-KMS), you must use a customer managed key. For more information, see Amazon S3 bucket server-side encryption in the Amazon CloudWatch Logs User Guide.
+
+### Configure an IAM Policy
+
+- Create a user or role for OpenCost with access to the spot feed bucket. Attach the policy below to the role/user and replace `<BUCKET_NAME>` with your spot bucket name
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Action": [
+                "s3:ListAllMyBuckets",
+                "s3:ListBucket",
+                "s3:HeadBucket",
+                "s3:HeadObject",
+                "s3:List*",
+                "s3:Get*"
+            ],
+            "Resource": [
+                "arn:aws:s3:::<BUCKET_NAME>"
+            ],
+            "Effect": "Allow",
+            "Sid": "SpotDataAccess"
+        }
+    ]
+}
+```
+
+### Option 1: Service Keys
+
+Generate Access Keys with following steps
+
+1. Navigate to the AWS IAM Console.
+1. Select Access Management > Users from the left navigation.
+1. Find the OpenCost User and select Security credentials > Create access key.
+1. Follow along to receive the Access Key ID and Secret Access Key (AWS will not provide you the Secret Access Key in the future, so make sure you save this value).
+
+Create a `service-key.json` and replace the values with the user access keys
+
+```json
+{
+  "aws_access_key_id": "<ACCESS_KEY_ID>",
+  "aws_secret_access_key": "<SECRET_ACCESS_KEY>"
+}
+```
+
+Create a Kubernetes secret with the following command
+
+```sh
+kubectl create secret generic <SECRET_NAME> --from-file=service-key.json --namespace multi-tenant-operator
+```
+
+!!! note
+    When managing the service account key as a Kubernetes secret, the secret must reference the service account key JSON file, and that file must be named `service-key.json`.
+
+The data feed will provide specific pricing information about any Spot instances in your account on an hourly basis. After setting this up, the bucket information can be provided through options in the IntegrationConfig via `integrationConfig.components.showbackOpts.custom` object.
+
+- `awsSpotDataBucket` - The name of the S3 bucket Spot Instance Data Feed is publishing to.
+- `awsSpotDataRegion` - The region configured for Spot Instance Data Feed
+- `awsSpotDataPrefix` - The prefix (if any) configured for Spot Instance Data Feed
+- `projectID` - The AWS Account ID
+- `<SECRET_NAME>` - Name of the service key secret created in previous step
+
+Example:
+
+```yaml
+components:
+  showback: true # should be enabled
+  showbackOpts:
+    custom:
+      provider: aws
+      description: "AWS Provider Configuration. Provides default values used if instance type or spot information is not found."
+      CPU: "0.031611"
+      spotCPU: "0.006655"
+      RAM: "0.004237"
+      GPU: "0.95"
+      spotRAM: "0.000892"
+      storage: "0.000138888889"
+      zoneNetworkEgress: "0.01"
+      regionNetworkEgress: "0.01"
+      internetNetworkEgress: "0.143"
+      spotLabel: "kops.k8s.io/instancegroup"
+      spotLabelValue: "spotinstance-nodes"
+      awsSpotDataRegion: "us-east-2"
+      awsSpotDataBucket: "my-spot-bucket"
+      awsSpotDataPrefix: "spot_pricing_prefix"
+      projectID: "012345678901"
+
+    cloudPricingSecretRef:
+      name: <SECRET_NAME>
+      namespace: multi-tenant-operator
+```
+
+### Option 2: IRSA (IAM Roles for Service Accounts)
+
+After creating the role and policy, attach the role ARN as an annotation to the opencost gateway account via following command
+
+```sh
+kubectl annotate serviceaccounts -n multi-tenant-operator tenant-operator-opencost-gateway eks.amazonaws.com/role-arn=<ROLE_ARN>
+```
+
+Apply the IntegrationConfig with current spot bucket data feed information
+
+Example:
+
+```yaml
+components:
+  showback: true # should be enabled
+  showbackOpts:
+    custom:
+      provider: aws
+      description: "AWS Provider Configuration. Provides default values used if instance type or spot information is not found."
+      CPU: "0.031611"
+      spotCPU: "0.006655"
+      RAM: "0.004237"
+      GPU: "0.95"
+      spotRAM: "0.000892"
+      storage: "0.000138888889"
+      zoneNetworkEgress: "0.01"
+      regionNetworkEgress: "0.01"
+      internetNetworkEgress: "0.143"
+      spotLabel: "kops.k8s.io/instancegroup"
+      spotLabelValue: "spotinstance-nodes"
+      awsSpotDataRegion: "us-east-2"
+      awsSpotDataBucket: "my-spot-bucket"
+      awsSpotDataPrefix: "spot_pricing_prefix"
+      projectID: "012345678901"
+```
+
+## References
+
+For more information, refer to the following resources
+
+- [`OpenCost documentation`](https://www.opencost.io/docs/configuration/aws)
+- [`Kubecost documentation`](https://docs.kubecost.com/install-and-configure/install/cloud-integration/aws-cloud-integrations)
+- [`Assign IAM roles to Kubernetes service accounts`](https://docs.aws.amazon.com/eks/latest/userguide/associate-service-account-role.html)
+- [`Track your Spot Instance costs using the Spot Instance data feed`](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-data-feeds.html)
+- [`Manage access keys from IAM users`](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html)

content/integrations/azure-pricing.md

@@ -1,12 +1,11 @@
-# Azure Pricing
+# Azure Pricing Model
 
-MTO supports Azure pricing model via the `integrationConfig.components.showbackOpts.azurePricingSecretRef` field. Following 3 types of pricing are supported:
+MTO supports Azure pricing model via the `integrationConfig.components.showbackOpts.cloudPricingSecretRef` field. Following 3 types of pricing are supported:
 
-- [`Azure Pricing Configuration`](https://www.opencost.io/docs/configuration/azure#azure-pricing-configuration)
-- [`Customer-specific pricing`](https://www.opencost.io/docs/configuration/azure#customer-specific-pricing)
-- [`Azure Cloud Costs`](https://www.opencost.io/docs/configuration/azure#azure-cloud-costs)
+- [`Azure Standard Pricing`](#azure-standard-pricing)
+- [`Customer-specific pricing`](#customer-specific-pricing)
 
-## Azure Pricing Configuration
+## Azure Standard Pricing
 
 For Azure pricing configuration, OpenCost needs access to the Microsoft Azure Billing Rate Card API to access accurate pricing data for your Kubernetes resources.
 
@@ -86,7 +85,7 @@ components:
     console: true # should be enabled
     showback: true # should be enabled
     showbackOpts:
-      azurePricingSecretRef:
+      cloudPricingSecretRef:
         name: azure-service-key
         namespace: multi-tenant-operator
 ```
@@ -140,7 +139,7 @@ components:
     console: true # should be enabled
     showback: true # should be enabled
     showbackOpts:
-      azurePricingSecretRef:
+      cloudPricingSecretRef:
         name: customer-specific-pricing
         namespace: multi-tenant-operator
 ```
@@ -195,70 +194,6 @@ RESPONSE="$(curl --silent --show-error -X PUT "${URL}" \
 echo "Response: ${RESPONSE}"
 ```
 
-## Azure Cloud Costs
-
-The following values can be found in the Azure Portal under *Cost Management > Exports*, or Storage accounts:
-
-- `<SUBSCRIPTION_ID>` is the Subscription ID belonging to the Storage account which stores your exported Azure cost report data.
-- `<STORAGE_ACCOUNT>` is the name of the Storage account where the exported Azure cost report data is being stored.
-- `<STORAGE_ACCESS_KEY>` can be found by selecting Access Keys from the navigation sidebar then selecting Show keys. Using either of the two keys will work.
-- `<STORAGE_CONTAINER>` is the name that you chose for the exported cost report when you set it up. This is the name of the container where the CSV cost reports are saved in your Storage account.
-- `<CONTAINER_PATH>` should be used if there is more than one billing report that is exported to the configured container. The path provided should have only one billing export because OpenCost will retrieve the most recent billing report for a given month found within the path. If this configuration is not used, it should be set to an empty string "".
-- `<CLOUD>` is the value which denotes the cloud where the storage account exists. Possible values are public and gov. The default is public if an empty string is provided.
-
-Set these values to the Azure array in the `cloud-integration.json` file:
-
-```json
-{
-  "azure": {
-    "storage": [
-      {
-        "subscriptionID": "<SUBSCRIPTON_ID>",
-        "account": "<STORAGE_ACCOUNT>",
-        "container": "<STORAGE_CONTAINER>",
-        "path": "<CONTAINER_PATH>",
-        "cloud": "<CLOUD>",
-        "authorizer": {
-          "accessKey": "<STORAGE_ACCESS_KEY>",
-          "account": "<STORAGE_ACCOUNT>",
-          "authorizerType": "AzureAccessKey"
-        }
-      },
-      {
-        "subscriptionID": "<SUBSCRIPTION_ID>",
-        "account": "<STORAGE_ACCOUNT>",
-        "container": "<EXPORT_CONTAINER>",
-        "path": "",
-        "cloud": "<CLOUD>",
-        "authorizer": {
-          "accessKey": "<ACCOUNT_ACCESS_KEY>",
-          "account": "<STORAGE_ACCOUNT>",
-          "authorizerType": "AzureAccessKey"
-        }
-      }
-    ]
-  }
-}
-```
-
-Load the `cloud-integration.json` into a Kubernetes secret in your opencost namespace.
-
-```bash
-kubectl create secret generic cloud-costs --from-file=./cloud-integration.json --namespace opencost
-```
-
-Update your IntegrationConfig to use the Cloud Costs pricing model:
-
-```yaml
-components:
-    console: true # should be enabled
-    showback: true # should be enabled
-    showbackOpts:
-      azurePricingSecretRef:
-        name: cloud-costs
-        namespace: multi-tenant-operator
-```
-
 ## Conclusion
 
 In this guide, we have seen how to configure OpenCost to use Azure pricing model. We have seen how to configure Azure Pricing Configuration, Customer-specific pricing, and Azure Cloud Costs. To enable all three pricing models, you can create a single secret with all the required values and update the IntegrationConfig to use the secret.
@@ -276,7 +211,7 @@ components:
     console: true # should be enabled
     showback: true # should be enabled
     showbackOpts:
-      azurePricingSecretRef:
+      cloudPricingSecretRef:
         name: azure-pricing
         namespace: multi-tenant-operator
 ```

content/kubernetes-resources/integration-config.md

@@ -23,9 +23,19 @@ spec:
         zoneNetworkEgress: "0.01"
         regionNetworkEgress: "0.01"
         internetNetworkEgress: "0.12"
-      azurePricingSecretRef:
-        name: azure-pricing
+        spotLabel: "kops.k8s.io/instancegroup"
+        spotLabelValue: "spotinstance-nodes"
+        awsSpotDataRegion: "us-east-2"
+        awsSpotDataBucket: "my-spot-bucket"
+        awsSpotDataPrefix: "spot_pricing_prefix"
+        projectID: "012345678901"
+
+      cloudPricingSecretRef:
+        name: secret-name
         namespace: multi-tenant-operator
+
+      opencostServiceRoleArn: arn:aws:iam::123456789012:role/S3Access
+
     ingress:
       ingressClassName: 'nginx'
       keycloak:
@@ -153,7 +163,7 @@ Following are the different components that can be used to configure multi-tenan
         zoneNetworkEgress: "0.01"
         regionNetworkEgress: "0.01"
         internetNetworkEgress: "0.12"
-      azurePricingSecretRef:
+      cloudPricingSecretRef:
         name: azure-pricing
         namespace: multi-tenant-operator
     ingress:
@@ -184,7 +194,8 @@ Following are the different components that can be used to configure multi-tenan
         - `tlsSecretName:` Name of the secret containing the TLS certificate and key for the Keycloak's ingress.
 - `components.showbackOpts:` Configures the showback feature with the following options:
     - `custom:` Custom pricing model for showback.
-    - `azurePricingSecretRef:` Secret reference for Azure pricing model.
+    - `cloudPricingSecretRef:` Secret reference for AWS / Azure Pricing model.
+    - `opencostServiceRoleArn`: Role ARN to be used by OpenCost gateway's service account
 
 Here's an example of how to generate the secrets required to configure MTO:
 
@@ -233,14 +244,22 @@ After modifying your default IntegrationConfig in `multi-tenant-operator` namesp
 
 ### Azure Pricing Model
 
-MTO supports Azure pricing model via the `showbackOpts.azurePricingSecretRef` field. Following 3 types of pricing are supported:
+MTO supports Azure pricing model via the `showbackOpts.cloudIntegrationSecretRef` field. Following 2 types of pricing are supported:
 
-- [`Azure Pricing Configuration`](https://www.opencost.io/docs/configuration/azure#azure-pricing-configuration)
-- [`Customer-specific pricing`](https://www.opencost.io/docs/configuration/azure#customer-specific-pricing)
-- [`Azure Cloud Costs`](https://www.opencost.io/docs/configuration/azure#azure-cloud-costs)
+- [`Azure Standard Pricing`](../integrations/azure-pricing.md#azure-standard-pricing)
+- [`Customer-specific pricing`](../integrations/azure-pricing.md#customer-specific-pricing)
 
 More details on Azure pricing can be found [here](../integrations/azure-pricing.md).
 
+### AWS Pricing Model
+
+MTO supports AWS pricing model via the `integrationConfig.components.showbackOpts.cloudIntegrationSecretRef` field. Following 2 types of pricing are supported:
+
+- [`AWS Standard Pricing`](../integrations/aws-pricing.md#aws-standard-pricing)
+- [`AWS Spot Instance Pricing`](../integrations/aws-pricing.md#aws-spot-instance-pricing)
+
+More details on AWS pricing can be found [here](../integrations/aws-pricing.md).
+
 ## Access Control
 
 ```yaml

theme_override/mkdocs.yml

@@ -92,6 +92,7 @@ nav:
       - integrations/mattermost.md
       - integrations/keycloak.md
       - integrations/azure-pricing.md
+      - integrations/aws-pricing.md
   - pricing.md
   - changelog.md
   - eula.md