[Bugfix] EMF exporter not handling prometheus config in container envs (EKS/ECS) correctly

[movence]

Description of the issue

The config translation logic for AWS EMF exporter doesn't handle prometheus block correctly by picking up a wrong component config template when there are additional config blocks such as kubernetes or ecs. This issue makes prometheus data being exported to undefiend log stream under containerinsights log group (/aws/containerinsights/{clusterName}/performance)

EKS Addon advanced config:

{
  "agent": {
    "config": {
      "logs": {
        "metrics_collected": {
          "application_signals": {},
          "kubernetes": {
            "enhanced_container_insights": true
          },
          "prometheus": {
            "cluster_name": "test-cluster",
            "prometheus_config_path": "/etc/prometheusconfig/prometheus.yaml"
          }
        }
      },
      "traces": {
        "traces_collected": {
          "application_signals": {}
        }
      }
    },
    "prometheus": {
      "config": {
        "global": {
          "scrape_interval": "1m",
          "scrape_timeout": "10s"
        },
        "scrape_configs": [
          {
            "bearer_token_file": "/var/run/secrets/kubernetes.io/serviceaccount/token",
            "job_name": "kubernetes-apiservers",
            "kubernetes_sd_configs": [
              {
                "role": "endpoints"
              }
            ],
            "relabel_configs": [
              {
                "action": "keep",
                "regex": "kubernetes;https",
                "source_labels": [
                  "__meta_kubernetes_service_name",
                  "__meta_kubernetes_endpoint_port_name"
                ]
              },
              {
                "action": "replace",
                "source_labels": [
                  "__meta_kubernetes_namespace"
                ],
                "target_label": "Namespace"
              },
              {
                "action": "replace",
                "source_labels": [
                  "__meta_kubernetes_service_name"
                ],
                "target_label": "Service"
              }
            ],
            "scheme": "https",
            "tls_config": {
              "ca_file": "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt",
              "insecure_skip_verify": true
            }
          }
        ]
      }
    }
  }
}

Resulting prometheus config block in amazon-cloudwatch-agent.yaml:

exporters:
...(trimmed)
    awsemf/prometheus:
        add_entity: true
        certificate_file_path: ""
        detailed_metrics: false
        dimension_rollup_option: NoDimensionRollup
        disable_metric_extraction: true
        eks_fargate_container_insights_enabled: false
        endpoint: https://fake_endpoint
        enhanced_container_insights: true
        external_id: ""
        imds_retries: 2
        local_mode: true
        log_group_name: /aws/containerinsights/{ClusterName}/performance
        log_retention: 0
        log_stream_name: '{NodeName}'
        max_retries: 2
        metric_declarations:
            - dimensions:
                - - ClusterName
                - - ClusterName
                  - ContainerName
                  - FullPodName
                  - Namespace
                  - PodName
                - - ClusterName
                  - ContainerName
                  - Namespace
                  - PodName
              metric_name_selectors:
                - container_cpu_utilization
                - container_cpu_utilization_over_container_limit
                - container_cpu_limit
                - container_cpu_request
                - container_memory_utilization
                - container_memory_utilization_over_container_limit
                - container_memory_failures_total
                - container_memory_limit
                - container_memory_request
                - container_filesystem_usage
                - container_filesystem_available
                - container_filesystem_utilization
...(trimmed)

Description of changes

• Add pipeline name for prometheus and ECS environment validations to condition checks for more robust config processing
• Move kubernetes processing to the bottom of if/else chain to avoid any accidental translation issues

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

unit tests and a test EKS cluster to verify translated yaml file and prometheus EMF logs being emitted to the correct log stream/group.

Requirements

Before commiting your code, please do the following steps.

1. Run make fmt and make fmt-sh
2. Run make lint

---

Integration Tests

To run integration tests against this PR, add the ready for testing label.

[lisguo]

nit: Can move this function into the translator struct to make it easier to mock