Skip to content

Update module reference including instances fetch - #451

Open
trinacriax wants to merge 2 commits into
aws-solutions-library-samples:mainfrom
trinacriax:axrusso-kpiview
Open

Update module reference including instances fetch#451
trinacriax wants to merge 2 commits into
aws-solutions-library-samples:mainfrom
trinacriax:axrusso-kpiview

Conversation

@trinacriax

Copy link
Copy Markdown

Summary

Extends the module-reference Lambda to collect instance pricing data from the AWS Pricing API (ElastiCache, RDS, OpenSearch) and adds an Athena view that computes instance generation, processor type, and upgrade paths for the KPI dashboard.

Changes

  • Lambda enhancements:

    • Add pricing:GetProducts IAM permission
    • Switch architecture from x86_64 to arm64
    • Increase MemorySize to 2688 MB
    • Collect pricing data for ElastiCache, RDS, and OpenSearch instances via Pricing API paginator
  • Glue tables: Add three new tables (reference_pricing_elasticache_instances, reference_pricing_rds_instances, reference_pricing_opensearch_instances) for pricing data stored in S3

  • Athena view (kpi_instance_mapping): Cross-service view that resolves instance family, generation (Current/Previous), processor (Intel/AMD/Graviton), and upgrade paths using EC2 instance types as the authoritative processor source

  • Tests: Add integration tests for the three new pricing tables

Testing

  • Integration tests added in test/test_from_scratch.py validating data presence in each pricing table

@trinacriax trinacriax changed the title Axrusso kpiview Update module reference including instances fetch Jul 15, 2026
FROM elasticache_reserved_cache_nodes_offerings
CROSS JOIN UNNEST(recurringcharges) AS t(charge)

KpiInstanceMappingView:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest not to release it as a part of CFN, but as a separate guidance under https://docs.aws.amazon.com/guidance/latest/cloud-intelligence-dashboards/data-collection-utilize-data.html.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we can have it deployed as a part of KPI dashboard

rec['region'] = region
write(rec)

# Pull pricing data for instance types (pricing API is global, us-east-1 only)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@trinacriax, https://github.com/aws-solutions-library-samples/cloud-intelligence-dashboards-data-collection/blob/main/data-collection/deploy/module-pricing.yaml module collects pricing data; could you please check if what is collected there is sufficient enough? I think it should be

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants