Skip to content

The nginx_ingress_controller_requests metric not exposing the complete requestURI as label #12763

Closed
@Laveshsp

Description

@Laveshsp

What happened:

I have installed nginx ingress controller and enabled controller metrics. Also added prometheus pod annotations to scrape metrics. When I use nginx_ingress_controller_requests metrics, it doesn't give me the entire requestURI in labels but just gives the host and path configured in kubernetes ingress manifest.

Query Ran :

rate(nginx_ingress_controller_requests{status=~"5.."}[1m]) > 0

Below are the labels that were output

{app_kubernetes_io_component="controller", app_kubernetes_io_instance="ingress-nginx", app_kubernetes_io_managed_by="Helm", app_kubernetes_io_name="ingress-nginx", app_kubernetes_io_part_of="ingress-nginx", app_kubernetes_io_version="1.9.3", cluster="SHARED-CUSTOMER-DEV-CL", controller_class="k8s.io/ingress-nginx", controller_namespace="ingress", controller_pod="ingress-nginx-controller-7cf96d6db6-5rrm4", helm_sh_chart="ingress-nginx-4.8.2", host="unique.py.lav.test", ingress="error-ingress", instance="10.135.69.73:10254", job="kubernetes-pods", kubernetes_namespace="ingress", kubernetes_pod_name="ingress-nginx-controller-7cf96d6db6-5rrm4", method="GET", namespace="monitor", path="/", pod_template_hash="7cf96d6db6", prometheus="prometheus/kube-prometheus-stack-prometheus", region="us-west-2", service="error-service", status="503"}

What you expected to happen:

I wanted the nginx_ingress_controller_requests to provide me the requestURI as label output which gives me the complete URI of the request

NGINX Ingress controller version : 1.9.3

Kubernetes version : 1.30

Environment:

  • Cloud provider or hardware configuration: AWS EKS 1.30

  • How was the ingress-nginx-controller installed:

    • If helm was used then please show output of ingress-nginx ingress 36 2024-11-04 11:53:23.847501744 +0000 UTC deployed ingress-nginx-4.8.2 1.9.3
    • If helm was used then please show output of helm -n <ingresscontrollernamespace> get values <helmreleasename>
USER-SUPPLIED VALUES:
clusterName: SHARED-CUSTOMER-DEV-CL
controller:
  allowSnippetAnnotations: true
  autoscaling:
    enabled: true
    maxReplicas: 2
  config:
    enable-modsecurity: true
    enable-real-ip: true
    enable-underscores-in-headers: true
    http-snippet: 'more_set_headers "X-ModSecurity: enabled";'
    log-format-escape-json: true
    log-format-upstream: '{"timestamp": "$time_iso8601", "requestID": "$req_id", "proxyUpstreamName":
      "$proxy_upstream_name", "proxyAlternativeUpstreamName": "$proxy_alternative_upstream_name","upstreamStatus":
      $upstream_status, "upstreamAddr": "$upstream_addr", "httpRequest": {"requestMethod":
      "$request_method", "requestUrl": "$host$request_uri", "status": $status,"requestSize":
      $request_length, "responseSize": $upstream_response_length, "userAgent": "$http_user_agent",
      "remoteIp": "$remote_addr", "referer": "$http_referer", "latency": "$upstream_response_time
      s", "protocol": "$server_protocol"}}'
    modsecurity-snippet: |
      # this enables the default OWASP Core Rule Set
      Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf

      # Enable prevention mode. Options: DetectionOnly On Off (default is DetectionOnly)
      SecRuleEngine On

      # Enable scanning of the request body
      SecRequestBodyAccess On

      # Reject if larger (we could also let it pass with ProcessPartial)
      SecRequestBodyLimitAction Reject

      # Send ModSecurity audit logs to the stdout (only for rejected requests)
      SecAuditLog /dev/stdout

      # Format the logs in JSON
      SecAuditLogFormat JSON

      # could be On/Off/RelevantOnly
      SecAuditEngine RelevantOnly
    use-forwarded-headers: true
    use-proxy-protocol: true
  extraArgs:
    default-ssl-certificate: ingress/default-tls
  metrics:
    enabled: true
  podAnnotations:
    '"prometheus.io/port"': 10254
    '"prometheus.io/scrape"': true
  replicaCount: 1
tolerations:
- effect: NoSchedule
  key: mainnode
  operator: Exists

Can you please let me know if there is an option to get the complete requestURI in mertics scraped by prometheus from nginx ingress controller

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-kindIndicates a PR lacks a `kind/foo` label and requires one.needs-priorityneeds-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.

    Type

    No type

    Projects

    • Status

      Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions