How to enable Grafana HTTPS in V5 on Openshift.

[bjwyun]

Hi all, I'm trying to enable https grafana url in the openshift cluster.
And I create a secret to save tls.key, tls.crt.

---

In

Grafana Operator
4.10.1 provided by Red Hat

I can create the Grafana CR as follows to enable Https:

apiVersion: integreatly.org/v1alpha1
kind: Grafana
metadata:
  creationTimestamp: '2024-07-16T09:28:24Z'
  generation: 2
  managedFields:
    - apiVersion: integreatly.org/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:spec':
          .: {}
          'f:config':
            .: {}
            'f:server':
              .: {}
              'f:cert_file': {}
              'f:cert_key': {}
              'f:protocol': {}
          'f:secrets': {}
      manager: Mozilla
      operation: Update
      time: '2024-07-16T09:30:30Z'
    - apiVersion: integreatly.org/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        'f:status':
          .: {}
          'f:message': {}
          'f:phase': {}
          'f:previousServiceName': {}
      manager: manager
      operation: Update
      subresource: status
      time: '2024-07-16T09:30:47Z'
  name: example
  namespace: my-grafana
  resourceVersion: '20356640'
  uid: f7e0b464-3dcb-4edc-b78b-f65d0e057196
spec:
  config:
    server:
      cert_file: /etc/grafana-secrets/grafana-tls-secret/tls.crt
      cert_key: /etc/grafana-secrets/grafana-tls-secret/tls.key
      protocol: https
  secrets:
    - grafana-tls-secret
status:
  message: success
  phase: reconciling
  previousServiceName: grafana-service

---

but in V5 if I do the similar things, the grafana-tls-secret will not be mounted into container. Could you help to give an example about enabling https with V5 operator ? thanks.

[theSuess]

You can use the spec.deployment field to override the created deployment. In there you are able to customize the whole deployment object which includes the ability to mount arbitrary secrets

[bjwyun]

Hi @theSuess , I tried to use

---

spec:
  config:
    auth:
      disable_login_form: 'false'
    log:
      mode: console
    security:
      admin_password: start
      admin_user: root
    server:
      cert_file: /etc/grafana-secrets/grafana-tls-secret/tls.crt
      cert_key: /etc/grafana-secrets/grafana-tls-secret/tls.key
      protocol: https
  deployment:
    metadata: {}
    spec:
      template:
        metadata: {}
        spec:
          containers:
            - name: grafana
              volumeMounts:
                - mountPath: /etc/grafana/
                  name: grafana-a-ini
                - mountPath: /var/lib/grafana
                  name: grafana-data
                - mountPath: /var/log/grafana
                  name: grafana-logs
                - mountPath: /etc/grafana-secrets/grafana-tls-secret
                  name: grafana-tls-secret
          volumes:
            - name: grafana-tls-secret
              secret:
                defaultMode: 420
                secretName: grafana-tls-secret

but got error in pod init

logger=provisioning.datasources t=2024-07-17T03:00:11.513313874Z level=error msg="can't read datasource provisioning files from directory" path=/etc/grafana/provisioning/datasources error="open /etc/grafana/provisioning/datasources: no such file or directory"
logger=provisioning.plugins t=2024-07-17T03:00:11.513342415Z level=error msg="Failed to read plugin provisioning files from directory" path=/etc/grafana/provisioning/plugins error="open /etc/grafana/provisioning/plugins: no such file or directory"
logger=provisioning.notifiers t=2024-07-17T03:00:11.513401325Z level=error msg="Can't read alert notification provisioning files from directory" path=/etc/grafana/provisioning/notifiers error="open /etc/grafana/provisioning/notifiers: no such file or directory"
logger=provisioning.alerting t=2024-07-17T03:00:11.513459347Z level=error msg="can't read alerting provisioning files from directory" path=/etc/grafana/provisioning/alerting error="open /etc/grafana/provisioning/alerting: no such file or directory"
logger=provisioning.alerting t=2024-07-17T03:00:11.513467207Z level=info msg="starting to provision alerting"
logger=provisioning.alerting t=2024-07-17T03:00:11.513483487Z level=info msg="finished to provision alerting"
logger=grafanaStorageLogger t=2024-07-17T03:00:11.513665Z level=info msg="Storage starting"
logger=ngalert.state.manager t=2024-07-17T03:00:11.514203677Z level=info msg="Warming state cache for startup"
logger=ngalert.multiorg.alertmanager t=2024-07-17T03:00:11.514357398Z level=info msg="Starting MultiOrg Alertmanager"
logger=provisioning.dashboard t=2024-07-17T03:00:11.51451666Z level=error msg="can't read dashboard provisioning files from directory" path=/etc/grafana/provisioning/dashboards error="open /etc/grafana/provisioning/dashboards: no such file or directory"
logger=http.server t=2024-07-17T03:00:11.517814955Z level=info msg="HTTP Server TLS settings" MinTLSVersion=TLS1.2

Do you have any idea of it ? thanks.

[bjwyun]

Update with

kind: Deployment
apiVersion: apps/v1
metadata:
  annotations:
    deployment.kubernetes.io/revision: '3'
  resourceVersion: '21164177'
  name: grafana-a-deployment
  uid: a2eb6b86-4ae3-4520-889a-5c6527b4861c
  creationTimestamp: '2024-07-17T02:46:30Z'
  generation: 3
  managedFields:
    - manager: v5
      operation: Update
      apiVersion: apps/v1
      time: '2024-07-17T03:05:59Z'
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:ownerReferences':
            .: {}
            'k:{"uid":"c05c4ea8-6954-43ee-85d1-f8362c6b2dec"}': {}
        'f:spec':
          'f:progressDeadlineSeconds': {}
          'f:replicas': {}
          'f:revisionHistoryLimit': {}
          'f:selector': {}
          'f:strategy':
            'f:rollingUpdate':
              .: {}
              'f:maxSurge': {}
              'f:maxUnavailable': {}
            'f:type': {}
          'f:template':
            'f:metadata':
              'f:labels':
                .: {}
                'f:app': {}
              'f:name': {}
            'f:spec':
              'f:volumes':
                .: {}
                'k:{"name":"grafana-a-ini"}':
                  .: {}
                  'f:configMap':
                    .: {}
                    'f:defaultMode': {}
                    'f:name': {}
                  'f:name': {}
                'k:{"name":"grafana-data"}':
                  .: {}
                  'f:emptyDir': {}
                  'f:name': {}
                'k:{"name":"grafana-logs"}':
                  .: {}
                  'f:emptyDir': {}
                  'f:name': {}
                'k:{"name":"grafana-tls-secret"}':
                  .: {}
                  'f:name': {}
                  'f:secret':
                    .: {}
                    'f:defaultMode': {}
                    'f:secretName': {}
              'f:containers':
                'k:{"name":"grafana"}':
                  'f:image': {}
                  'f:volumeMounts':
                    .: {}
                    'k:{"mountPath":"/etc/grafana-secrets/grafana-tls-secret"}':
                      .: {}
                      'f:mountPath': {}
                      'f:name': {}
                    'k:{"mountPath":"/etc/grafana/"}':
                      .: {}
                      'f:mountPath': {}
                      'f:name': {}
                    'k:{"mountPath":"/var/lib/grafana"}':
                      .: {}
                      'f:mountPath': {}
                      'f:name': {}
                    'k:{"mountPath":"/var/log/grafana"}':
                      .: {}
                      'f:mountPath': {}
                      'f:name': {}
                  'f:terminationMessagePolicy': {}
                  .: {}
                  'f:resources':
                    .: {}
                    'f:limits':
                      .: {}
                      'f:memory': {}
                    'f:requests':
                      .: {}
                      'f:cpu': {}
                      'f:memory': {}
                  'f:args': {}
                  'f:env':
                    .: {}
                    'k:{"name":"CONFIG_HASH"}':
                      .: {}
                      'f:name': {}
                      'f:value': {}
                    'k:{"name":"GF_INSTALL_PLUGINS"}':
                      .: {}
                      'f:name': {}
                    'k:{"name":"GF_SECURITY_ADMIN_PASSWORD"}':
                      .: {}
                      'f:name': {}
                      'f:valueFrom':
                        .: {}
                        'f:secretKeyRef': {}
                    'k:{"name":"GF_SECURITY_ADMIN_USER"}':
                      .: {}
                      'f:name': {}
                      'f:valueFrom':
                        .: {}
                        'f:secretKeyRef': {}
                    'k:{"name":"PLUGINS_HASH"}':
                      .: {}
                      'f:name': {}
                      'f:valueFrom':
                        .: {}
                        'f:configMapKeyRef': {}
                    'k:{"name":"TMPDIR"}':
                      .: {}
                      'f:name': {}
                      'f:value': {}
                  'f:readinessProbe':
                    .: {}
                    'f:failureThreshold': {}
                    'f:httpGet':
                      .: {}
                      'f:path': {}
                      'f:port': {}
                      'f:scheme': {}
                    'f:initialDelaySeconds': {}
                    'f:periodSeconds': {}
                    'f:successThreshold': {}
                    'f:timeoutSeconds': {}
                  'f:securityContext':
                    .: {}
                    'f:allowPrivilegeEscalation': {}
                    'f:capabilities':
                      .: {}
                      'f:drop': {}
                    'f:privileged': {}
                    'f:readOnlyRootFilesystem': {}
                    'f:runAsNonRoot': {}
                  'f:terminationMessagePath': {}
                  'f:imagePullPolicy': {}
                  'f:ports':
                    .: {}
                    'k:{"containerPort":3000,"protocol":"TCP"}':
                      .: {}
                      'f:containerPort': {}
                      'f:name': {}
                      'f:protocol': {}
                  'f:name': {}
              'f:dnsPolicy': {}
              'f:serviceAccount': {}
              'f:restartPolicy': {}
              'f:schedulerName': {}
              'f:terminationGracePeriodSeconds': {}
              'f:serviceAccountName': {}
              'f:securityContext':
                .: {}
                'f:seccompProfile':
                  .: {}
                  'f:type': {}
    - manager: kube-controller-manager
      operation: Update
      apiVersion: apps/v1
      time: '2024-07-17T03:06:10Z'
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:annotations':
            .: {}
            'f:deployment.kubernetes.io/revision': {}
        'f:status':
          'f:availableReplicas': {}
          'f:conditions':
            .: {}
            'k:{"type":"Available"}':
              .: {}
              'f:lastTransitionTime': {}
              'f:lastUpdateTime': {}
              'f:message': {}
              'f:reason': {}
              'f:status': {}
              'f:type': {}
            'k:{"type":"Progressing"}':
              .: {}
              'f:lastTransitionTime': {}
              'f:lastUpdateTime': {}
              'f:message': {}
              'f:reason': {}
              'f:status': {}
              'f:type': {}
          'f:observedGeneration': {}
          'f:readyReplicas': {}
          'f:replicas': {}
          'f:updatedReplicas': {}
      subresource: status
  namespace: my-grafana
  ownerReferences:
    - apiVersion: grafana.integreatly.org/v1beta1
      kind: Grafana
      name: grafana-a
      uid: c05c4ea8-6954-43ee-85d1-f8362c6b2dec
spec:
  replicas: 1
  selector:
    matchLabels:
      app: grafana-a
  template:
    metadata:
      name: grafana-a-deployment
      creationTimestamp: null
      labels:
        app: grafana-a
    spec:
      restartPolicy: Always
      serviceAccountName: grafana-a-sa
      schedulerName: default-scheduler
      terminationGracePeriodSeconds: 30
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      containers:
        - resources:
            limits:
              memory: 1Gi
            requests:
              cpu: 100m
              memory: 256Mi
          readinessProbe:
            httpGet:
              path: /api/health
              port: 3000
              scheme: HTTP
            initialDelaySeconds: 5
            timeoutSeconds: 3
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 1
          terminationMessagePath: /dev/termination-log
          name: grafana
          env:
            - name: PLUGINS_HASH
              valueFrom:
                configMapKeyRef:
                  name: grafana-a-plugins
                  key: PLUGINS_HASH
                  optional: true
            - name: CONFIG_HASH
              value: de45f88b5d91e5a68192330ec849d230e6951be0377a3514630a524b1e5ac0e6
            - name: GF_INSTALL_PLUGINS
            - name: TMPDIR
              value: /var/lib/grafana
            - name: GF_SECURITY_ADMIN_USER
              valueFrom:
                secretKeyRef:
                  name: grafana-a-admin-credentials
                  key: GF_SECURITY_ADMIN_USER
            - name: GF_SECURITY_ADMIN_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: grafana-a-admin-credentials
                  key: GF_SECURITY_ADMIN_PASSWORD
          securityContext:
            capabilities:
              drop:
                - ALL
            privileged: false
            runAsNonRoot: true
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
          ports:
            - name: grafana-http
              containerPort: 3000
              protocol: TCP
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: grafana-a-ini
              mountPath: /etc/grafana/
            - name: grafana-data
              mountPath: /var/lib/grafana
            - name: grafana-logs
              mountPath: /var/log/grafana
            - name: grafana-tls-secret
              mountPath: /etc/grafana-secrets/grafana-tls-secret
          terminationMessagePolicy: File
          image: 'docker.io/grafana/grafana:10.4.3'
          args:
            - '-config=/etc/grafana/grafana.ini'
      serviceAccount: grafana-a-sa
      volumes:
        - name: grafana-tls-secret
          secret:
            secretName: grafana-tls-secret
            defaultMode: 420
        - name: grafana-a-ini
          configMap:
            name: grafana-a-ini
            defaultMode: 420
        - name: grafana-logs
          emptyDir: {}
        - name: grafana-data
          emptyDir: {}
      dnsPolicy: ClusterFirst
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600
status:
  observedGeneration: 3
  replicas: 1
  updatedReplicas: 1
  readyReplicas: 1
  availableReplicas: 1
  conditions:
    - type: Available
      status: 'True'
      lastUpdateTime: '2024-07-17T03:06:10Z'
      lastTransitionTime: '2024-07-17T03:06:10Z'
      reason: MinimumReplicasAvailable
      message: Deployment has minimum availability.
    - type: Progressing
      status: 'True'
      lastUpdateTime: '2024-07-17T03:06:10Z'
      lastTransitionTime: '2024-07-17T03:00:06Z'
      reason: NewReplicaSetAvailable
      message: ReplicaSet "grafana-a-deployment-86cf5c74f6" has successfully progressed.

but still failed to setup the pod

here's the pod log file

grafana-a-deployment-8696f996f6-j2289-grafana.log

[bjwyun]

@theSuess finally worked out - -！. it's related to container readiness...default value is

              readinessProbe:
                httpGet:
                  path: /api/health
                  port: 3000
                  scheme: HTTP

by changing to HTTPS. it worked.

---

apiVersion: grafana.integreatly.org/v1beta1
kind: Grafana
metadata:
  generation: 7
  labels:
    dashboards: grafana-a
    folders: grafana-a
spec:
  config:
    auth:
      disable_login_form: 'false'
    log:
      mode: console
    security:
      admin_password: start
      admin_user: root
    server:
      cert_file: /etc/grafana-secrets/grafana-tls-secret/tls.crt
      cert_key: /etc/grafana-secrets/grafana-tls-secret/tls.key
      protocol: https
  deployment:
    metadata: {}
    spec:
      template:
        metadata: {}
        spec:
          containers:
            - name: grafana
              readinessProbe:
                httpGet:
                  path: /api/health
                  port: 3000
                  scheme: HTTPS
              volumeMounts:
                - mountPath: /etc/grafana/
                  name: grafana-a-ini
                - mountPath: /var/lib/grafana
                  name: grafana-data
                - mountPath: /var/log/grafana
                  name: grafana-logs
                - mountPath: /etc/grafana-secrets/grafana-tls-secret
                  name: grafana-tls-secret
                - mountPath: /etc/grafana/provisioning/datasources
                  name: grafana-datasources
                - mountPath: /etc/grafana/provisioning/plugins
                  name: grafana-plugins
                - mountPath: /etc/grafana/provisioning/notifiers
                  name: grafana-notifiers
                - mountPath: /etc/grafana/provisioning/alerting
                  name: grafana-alerting
                - mountPath: /etc/grafana/provisioning/dashboards
                  name: grafana-dashboards
          volumes:
            - name: grafana-tls-secret
              secret:
                defaultMode: 420
                secretName: grafana-tls-secret
            - emptyDir: {}
              name: grafana-datasources
            - emptyDir: {}
              name: grafana-plugins
            - emptyDir: {}
              name: grafana-notifiers
            - emptyDir: {}
              name: grafana-alerting
            - emptyDir: {}
              name: grafana-dashboards
  version: 10.4.3