Mimir-ruler doesn't evaluate prometheusrule resources

[genseb13011]

Hi,

I'd like to use PrometheusRule resource (prometheus CRD) to create alerts.

Currently, mimir-ruler doesn't seem to evaluate my PrometheusRule because I can't see any alert rules in Grafana.

Am I missing something?

Thanks.

Technical Informations

• Prometheus CRDs has been deployed using helm chart kube-prometheus-stack version 62.3.0
• Mimir-distributed helm version: 5.4.0

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: mimir-install
  namespace: argocd
spec:
  source:
    repoURL: https://grafana.github.io/helm-charts
    targetRevision: 5.4.0
    chart: mimir-distributed
    helm:
      values: |
        mimir:
          structuredConfig:
            limits:
              compactor_blocks_retention_period: 7d
            ingester:
                ring:
                  replication_factor: 1
            
            common:
              storage:
                backend: s3
                s3:
                  access_key_id: xxxxxxxxxxxxxxxxxxxxxx
                  endpoint: xxxxxxxxxxxxxxxxxxxx
                  secret_access_key: xxxxxxxxxxxxxxxxxxxxxxx
            blocks_storage:
              s3:
                bucket_name: mimir-blocks-storage
            ruler_storage:
              s3:
                bucket_name: mimir-ruler
            alertmanager_storage:
              backend: s3
              s3:
                bucket_name: mimir-ruler
        alertmanager:
          enabled: true
        #gateway required for alerts displayed in Grafana
        gateway:
          enabledNonEnterprise: true
        distributor:
          replicas: 1
        ingester:
          replicas: 1
          zoneAwareReplication:
            enabled: false
        ruler:
          replicas: 1
        ruler_querier:
          replicas: 1
        ruler_query_frontend:
          replicas: 1
        ruler_query_scheduler:
          replicas: 1
        querier:
          replicas: 1
        query_frontend:
          replicas: 1
        query_scheduler:
          replicas: 1
        store_gateway:
          replicas: 1
          zoneAwareReplication:
            enabled: false
        compactor:
          replicas: 1
        minio:
          enabled: false
        nginx:
          enabled: false
        metaMonitoring:
          prometheusRule:
            enabled: true
            mimirAlerts: true
            mimirRules: true
  destination:
    namespace: observability
    server: https://kubernetes.default.svc
  project: default
  syncPolicy:
    automated:
      prune: true
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true

• Mimir-ruler config API seems to be well enabled

curl -X GET http://mimir-install-ruler.observability.svc.cluster.local:8080/api/v1/status/buildinfo
{"status":"success","data":{"application":"Grafana Mimir","version":"2.13.0","revision":"4775ec1","branch":"HEAD","goVersion":"go1.22.5","features":{"ruler_config_api":"true","alertmanager_config_api":"true","query_sharding":"true","federated_rules":"false"}}}/

• PrometheusRule YAML

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  labels:
    app: oauth2-proxy
    namespace: oauth2-proxy
  name: oauth-proxy-prometheus-rules
spec:
  groups:
  - name: oauth-proxy-test
    rules:
    - alert: oauth-proxy-test
      annotations:
        description: Creation alert test
        summary: random alert for testing purpose
      expr: up{pod="oauth2-proxy-install-8594db9789-kvb8v"} == 0
      labels:
        namespace: oauth2-proxy
        severity: critical

Thanks.

[narqo]

Mimir doesn't know anything about PrometheusRule resources. One need to have an intermediate component, that will observe these resources in Kubernetes and materialize them into Mimir. For example, Grafana Alloy has mimir.rules.kubernetes component, that can do that.

Maybe there is a prometheus-operator-native way to configure it to do that. Please, check their documentation.