Implement Data Collection Rule for Container Insights

README.md

@@ -243,6 +243,8 @@ No modules.
 | [azurerm_kubernetes_cluster_node_pool.node_pool_create_before_destroy](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/kubernetes_cluster_node_pool) | resource |
 | [azurerm_log_analytics_solution.main](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/log_analytics_solution) | resource |
 | [azurerm_log_analytics_workspace.main](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/log_analytics_workspace) | resource |
+| [azurerm_monitor_data_collection_rule.dcr](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/monitor_data_collection_rule) | resource |
+| [azurerm_monitor_data_collection_rule_association.dcra](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/monitor_data_collection_rule_association) | resource |
 | [azurerm_role_assignment.acr](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/role_assignment) | resource |
 | [azurerm_role_assignment.application_gateway_byo_vnet_network_contributor](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/role_assignment) | resource |
 | [azurerm_role_assignment.application_gateway_existing_vnet_network_contributor](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/role_assignment) | resource |
@@ -319,6 +321,7 @@ No modules.
 | <a name="input_cost_analysis_enabled"></a> [cost\_analysis\_enabled](#input\_cost\_analysis\_enabled) | (Optional) Enable Cost Analysis. | `bool` | `false` | no |
 | <a name="input_create_role_assignment_network_contributor"></a> [create\_role\_assignment\_network\_contributor](#input\_create\_role\_assignment\_network\_contributor) | (Deprecated) Create a role assignment for the AKS Service Principal to be a Network Contributor on the subnets used for the AKS Cluster | `bool` | `false` | no |
 | <a name="input_create_role_assignments_for_application_gateway"></a> [create\_role\_assignments\_for\_application\_gateway](#input\_create\_role\_assignments\_for\_application\_gateway) | (Optional) Whether to create the corresponding role assignments for application gateway or not. Defaults to `true`. | `bool` | `true` | no |
+| <a name="input_data_collection_settings"></a> [data\_collection\_settings](#input\_data\_collection\_settings) | `data_collection_interval` -  Determines how often the agent collects data. Valid values are 1m - 30m in 1m intervals. Default is 1m.<br/>`namespace_filtering_mode_for_data_collection` - Can be 'Include', 'Exclude', or 'Off'. Determines how namespaces are filtered for data collection.<br/>`namespaces_for_data_collection` - List of Kubernetes namespaces for data collection based on the filtering mode.<br/>`container_log_v2_enabled` - Flag to enable the ContainerLogV2 schema for collecting logs.<br/>See more details: https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-data-collection-configure?tabs=cli#configure-dcr-with-azure-portal-1 | <pre>object({<br/>    data_collection_interval                     = string<br/>    namespace_filtering_mode_for_data_collection = string<br/>    namespaces_for_data_collection               = list(string)<br/>    container_log_v2_enabled                     = bool<br/>  })</pre> | <pre>{<br/>  "container_log_v2_enabled": true,<br/>  "data_collection_interval": "1m",<br/>  "namespace_filtering_mode_for_data_collection": "Off",<br/>  "namespaces_for_data_collection": [<br/>    "kube-system",<br/>    "gatekeeper-system",<br/>    "azure-arc"<br/>  ]<br/>}</pre> | no |
 | <a name="input_default_node_pool_fips_enabled"></a> [default\_node\_pool\_fips\_enabled](#input\_default\_node\_pool\_fips\_enabled) | (Optional) Should the nodes in this Node Pool have Federal Information Processing Standard enabled? Changing this forces a new resource to be created. | `bool` | `null` | no |
 | <a name="input_disk_encryption_set_id"></a> [disk\_encryption\_set\_id](#input\_disk\_encryption\_set\_id) | (Optional) The ID of the Disk Encryption Set which should be used for the Nodes and Volumes. More information [can be found in the documentation](https://docs.microsoft.com/azure/aks/azure-disk-customer-managed-keys). Changing this forces a new resource to be created. | `string` | `null` | no |
 | <a name="input_ebpf_data_plane"></a> [ebpf\_data\_plane](#input\_ebpf\_data\_plane) | (Optional) Specifies the eBPF data plane used for building the Kubernetes network. Possible value is `cilium`. Changing this forces a new resource to be created. | `string` | `null` | no |
@@ -367,6 +370,9 @@ No modules.
 | <a name="input_maintenance_window_auto_upgrade"></a> [maintenance\_window\_auto\_upgrade](#input\_maintenance\_window\_auto\_upgrade) | - `day_of_month` - (Optional) The day of the month for the maintenance run. Required in combination with RelativeMonthly frequency. Value between 0 and 31 (inclusive).<br/>- `day_of_week` - (Optional) The day of the week for the maintenance run. Options are `Monday`, `Tuesday`, `Wednesday`, `Thurday`, `Friday`, `Saturday` and `Sunday`. Required in combination with weekly frequency.<br/>- `duration` - (Required) The duration of the window for maintenance to run in hours.<br/>- `frequency` - (Required) Frequency of maintenance. Possible options are `Weekly`, `AbsoluteMonthly` and `RelativeMonthly`.<br/>- `interval` - (Required) The interval for maintenance runs. Depending on the frequency this interval is week or month based.<br/>- `start_date` - (Optional) The date on which the maintenance window begins to take effect.<br/>- `start_time` - (Optional) The time for maintenance to begin, based on the timezone determined by `utc_offset`. Format is `HH:mm`.<br/>- `utc_offset` - (Optional) Used to determine the timezone for cluster maintenance.<br/>- `week_index` - (Optional) The week in the month used for the maintenance run. Options are `First`, `Second`, `Third`, `Fourth`, and `Last`.<br/><br/>---<br/>`not_allowed` block supports the following:<br/>- `end` - (Required) The end of a time span, formatted as an RFC3339 string.<br/>- `start` - (Required) The start of a time span, formatted as an RFC3339 string. | <pre>object({<br/>    day_of_month = optional(number)<br/>    day_of_week  = optional(string)<br/>    duration     = number<br/>    frequency    = string<br/>    interval     = number<br/>    start_date   = optional(string)<br/>    start_time   = optional(string)<br/>    utc_offset   = optional(string)<br/>    week_index   = optional(string)<br/>    not_allowed = optional(set(object({<br/>      end   = string<br/>      start = string<br/>    })))<br/>  })</pre> | `null` | no |
 | <a name="input_maintenance_window_node_os"></a> [maintenance\_window\_node\_os](#input\_maintenance\_window\_node\_os) | - `day_of_month` -<br/>- `day_of_week` - (Optional) The day of the week for the maintenance run. Options are `Monday`, `Tuesday`, `Wednesday`, `Thurday`, `Friday`, `Saturday` and `Sunday`. Required in combination with weekly frequency.<br/>- `duration` - (Required) The duration of the window for maintenance to run in hours.<br/>- `frequency` - (Required) Frequency of maintenance. Possible options are `Daily`, `Weekly`, `AbsoluteMonthly` and `RelativeMonthly`.<br/>- `interval` - (Required) The interval for maintenance runs. Depending on the frequency this interval is week or month based.<br/>- `start_date` - (Optional) The date on which the maintenance window begins to take effect.<br/>- `start_time` - (Optional) The time for maintenance to begin, based on the timezone determined by `utc_offset`. Format is `HH:mm`.<br/>- `utc_offset` - (Optional) Used to determine the timezone for cluster maintenance.<br/>- `week_index` - (Optional) The week in the month used for the maintenance run. Options are `First`, `Second`, `Third`, `Fourth`, and `Last`.<br/><br/>---<br/>`not_allowed` block supports the following:<br/>- `end` - (Required) The end of a time span, formatted as an RFC3339 string.<br/>- `start` - (Required) The start of a time span, formatted as an RFC3339 string. | <pre>object({<br/>    day_of_month = optional(number)<br/>    day_of_week  = optional(string)<br/>    duration     = number<br/>    frequency    = string<br/>    interval     = number<br/>    start_date   = optional(string)<br/>    start_time   = optional(string)<br/>    utc_offset   = optional(string)<br/>    week_index   = optional(string)<br/>    not_allowed = optional(set(object({<br/>      end   = string<br/>      start = string<br/>    })))<br/>  })</pre> | `null` | no |
 | <a name="input_microsoft_defender_enabled"></a> [microsoft\_defender\_enabled](#input\_microsoft\_defender\_enabled) | (Optional) Is Microsoft Defender on the cluster enabled? Requires `var.log_analytics_workspace_enabled` to be `true` to set this variable to `true`. | `bool` | `false` | no |
+| <a name="input_monitor_data_collection_rule_data_sources_syslog_facilities"></a> [monitor\_data\_collection\_rule\_data\_sources\_syslog\_facilities](#input\_monitor\_data\_collection\_rule\_data\_sources\_syslog\_facilities) | Syslog supported facilities as documented here: https://learn.microsoft.com/en-us/azure/azure-monitor/agents/data-sources-syslog | `list(string)` | <pre>[<br/>  "auth",<br/>  "authpriv",<br/>  "cron",<br/>  "daemon",<br/>  "mark",<br/>  "kern",<br/>  "local0",<br/>  "local1",<br/>  "local2",<br/>  "local3",<br/>  "local4",<br/>  "local5",<br/>  "local6",<br/>  "local7",<br/>  "lpr",<br/>  "mail",<br/>  "news",<br/>  "syslog",<br/>  "user",<br/>  "uucp"<br/>]</pre> | no |
+| <a name="input_monitor_data_collection_rule_data_sources_syslog_levels"></a> [monitor\_data\_collection\_rule\_data\_sources\_syslog\_levels](#input\_monitor\_data\_collection\_rule\_data\_sources\_syslog\_levels) | List of syslog levels | `list(string)` | <pre>[<br/>  "Debug",<br/>  "Info",<br/>  "Notice",<br/>  "Warning",<br/>  "Error",<br/>  "Critical",<br/>  "Alert",<br/>  "Emergency"<br/>]</pre> | no |
+| <a name="input_monitor_data_collection_rule_extensions_streams"></a> [monitor\_data\_collection\_rule\_extensions\_streams](#input\_monitor\_data\_collection\_rule\_extensions\_streams) | An array of container insights table streams. See documentation in DCR for a list of the valid streams and their corresponding table: https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-data-collection-configure?tabs=portal#stream-values-in-dcr | `list(any)` | <pre>[<br/>  "Microsoft-ContainerLog",<br/>  "Microsoft-ContainerLogV2",<br/>  "Microsoft-KubeEvents",<br/>  "Microsoft-KubePodInventory",<br/>  "Microsoft-KubeNodeInventory",<br/>  "Microsoft-KubePVInventory",<br/>  "Microsoft-KubeServices",<br/>  "Microsoft-KubeMonAgentEvents",<br/>  "Microsoft-InsightsMetrics",<br/>  "Microsoft-ContainerInventory",<br/>  "Microsoft-ContainerNodeInventory",<br/>  "Microsoft-Perf"<br/>]</pre> | no |
 | <a name="input_monitor_metrics"></a> [monitor\_metrics](#input\_monitor\_metrics) | (Optional) Specifies a Prometheus add-on profile for the Kubernetes Cluster<br/>object({<br/>  annotations\_allowed = "(Optional) Specifies a comma-separated list of Kubernetes annotation keys that will be used in the resource's labels metric."<br/>  labels\_allowed      = "(Optional) Specifies a Comma-separated list of additional Kubernetes label keys that will be used in the resource's labels metric."<br/>}) | <pre>object({<br/>    annotations_allowed = optional(string)<br/>    labels_allowed      = optional(string)<br/>  })</pre> | `null` | no |
 | <a name="input_msi_auth_for_monitoring_enabled"></a> [msi\_auth\_for\_monitoring\_enabled](#input\_msi\_auth\_for\_monitoring\_enabled) | (Optional) Is managed identity authentication for monitoring enabled? | `bool` | `null` | no |
 | <a name="input_net_profile_dns_service_ip"></a> [net\_profile\_dns\_service\_ip](#input\_net\_profile\_dns\_service\_ip) | (Optional) IP address within the Kubernetes service address range that will be used by cluster service discovery (kube-dns). Changing this forces a new resource to be created. | `string` | `null` | no |

log_analytics.tf

@@ -81,3 +81,65 @@ resource "azurerm_log_analytics_solution" "main" {
     publisher = "Microsoft"
   }
 }
+
+locals {
+  dcr_location = try(coalesce(try(local.log_analytics_workspace.location, null), try(data.azurerm_log_analytics_workspace.main[0].location, null)), null)
+}
+
+resource "azurerm_monitor_data_collection_rule" "dcr" {
+  count               = (local.create_analytics_workspace && var.oms_agent_enabled) ? 1 : 0
+  name                = "MSCI-${local.dcr_location}-${azurerm_kubernetes_cluster.main.name}"
+  resource_group_name = var.resource_group_name
+  location            = local.dcr_location
+  tags                = var.tags
+
+  destinations {
+    log_analytics {
+      workspace_resource_id = local.log_analytics_workspace.id
+      name                  = local.log_analytics_workspace.name
+    }
+  }
+
+  data_flow {
+    streams      = var.monitor_data_collection_rule_extensions_streams
+    destinations = [local.log_analytics_workspace.name]
+  }
+
+  data_flow {
+    streams      = ["Microsoft-Syslog"]
+    destinations = [local.log_analytics_workspace.name]
+  }
+
+  data_sources {
+    syslog {
+      streams        = ["Microsoft-Syslog"]
+      facility_names = var.monitor_data_collection_rule_data_sources_syslog_facilities
+      log_levels     = var.monitor_data_collection_rule_data_sources_syslog_levels
+      name           = "sysLogsDataSource"
+    }
+
+    extension {
+      streams        = var.monitor_data_collection_rule_extensions_streams
+      extension_name = "ContainerInsights"
+      extension_json = jsonencode({
+        "dataCollectionSettings" : {
+          interval               = var.data_collection_settings.data_collection_interval
+          namespaceFilteringMode = var.data_collection_settings.namespace_filtering_mode_for_data_collection
+          namespaces             = var.data_collection_settings.namespaces_for_data_collection
+          enableContainerLogV2   = var.data_collection_settings.container_log_v2_enabled
+        }
+      })
+      name = "ContainerInsightsExtension"
+    }
+  }
+
+  description = "DCR for Azure Monitor Container Insights"
+}
+
+resource "azurerm_monitor_data_collection_rule_association" "dcra" {
+  count                   = (local.create_analytics_workspace && var.oms_agent_enabled) ? 1 : 0
+  name                    = "ContainerInsightsExtension"
+  target_resource_id      = azurerm_kubernetes_cluster.main.id
+  data_collection_rule_id = azurerm_monitor_data_collection_rule.dcr[0].id
+  description             = "Association of container insights data collection rule. Deleting this association will break the data collection for this AKS Cluster."
+}

variables.tf

@@ -1479,3 +1479,43 @@ variable "workload_identity_enabled" {
   default     = false
   description = "Enable or Disable Workload Identity. Defaults to false."
 }
+
+variable "monitor_data_collection_rule_extensions_streams" {
+  type        = list(any)
+  default     = ["Microsoft-ContainerLog", "Microsoft-ContainerLogV2", "Microsoft-KubeEvents", "Microsoft-KubePodInventory", "Microsoft-KubeNodeInventory", "Microsoft-KubePVInventory", "Microsoft-KubeServices", "Microsoft-KubeMonAgentEvents", "Microsoft-InsightsMetrics", "Microsoft-ContainerInventory", "Microsoft-ContainerNodeInventory", "Microsoft-Perf"]
+  description = "An array of container insights table streams. See documentation in DCR for a list of the valid streams and their corresponding table: https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-data-collection-configure?tabs=portal#stream-values-in-dcr"
+}
+
+variable "monitor_data_collection_rule_data_sources_syslog_levels" {
+  type        = list(string)
+  default     = ["Debug", "Info", "Notice", "Warning", "Error", "Critical", "Alert", "Emergency"]
+  description = "List of syslog levels"
+}
+
+variable "monitor_data_collection_rule_data_sources_syslog_facilities" {
+  type        = list(string)
+  default     = ["auth", "authpriv", "cron", "daemon", "mark", "kern", "local0", "local1", "local2", "local3", "local4", "local5", "local6", "local7", "lpr", "mail", "news", "syslog", "user", "uucp"]
+  description = "Syslog supported facilities as documented here: https://learn.microsoft.com/en-us/azure/azure-monitor/agents/data-sources-syslog"
+}
+
+variable "data_collection_settings" {
+  type = object({
+    data_collection_interval                     = string
+    namespace_filtering_mode_for_data_collection = string
+    namespaces_for_data_collection               = list(string)
+    container_log_v2_enabled                     = bool
+  })
+  default = {
+    data_collection_interval                     = "1m"
+    namespace_filtering_mode_for_data_collection = "Off"
+    namespaces_for_data_collection               = ["kube-system", "gatekeeper-system", "azure-arc"]
+    container_log_v2_enabled                     = true
+  }
+  description = <<-EOT
+    `data_collection_interval` -  Determines how often the agent collects data. Valid values are 1m - 30m in 1m intervals. Default is 1m.
+    `namespace_filtering_mode_for_data_collection` - Can be 'Include', 'Exclude', or 'Off'. Determines how namespaces are filtered for data collection.
+    `namespaces_for_data_collection` - List of Kubernetes namespaces for data collection based on the filtering mode.
+    `container_log_v2_enabled` - Flag to enable the ContainerLogV2 schema for collecting logs.
+    See more details: https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-data-collection-configure?tabs=cli#configure-dcr-with-azure-portal-1
+  EOT
+}