diff --git a/observability/otel/otel-config-multicluster.adoc b/observability/otel/otel-config-multicluster.adoc index 9d07343a8362..6bc8434d4b2d 100644 --- a/observability/otel/otel-config-multicluster.adoc +++ b/observability/otel/otel-config-multicluster.adoc @@ -15,9 +15,11 @@ For a multicluster configuration, you can create one OpenTelemetry Collector ins .Procedure -. Mount the following certificates in the OpenTelemetry Collector instance, skipping already mounted certificates. +.Generate Certificates -.. An Issuer to generate the certificates by using the {cert-manager-operator}. +. Skipping if certificates are already mounted on the clusters, certificates will be generated using the {cert-manager-operator}. The following steps assume you already have it installed on central cluster. + +..An Issuer to generate the certificates by using the {cert-manager-operator}. + [source,yaml] ---- @@ -55,7 +57,7 @@ spec: group: cert-manager.io ---- -.. A CA issuer. +.. Create a CA issuer. + [source,yaml] ---- @@ -68,7 +70,7 @@ spec: secretName: ca-secret ---- -.. The client and server certificates. +.. The server certificates user in Central cluster. + [source,yaml] ---- @@ -77,83 +79,159 @@ kind: Certificate metadata: name: server spec: - secretName: server-tls + secretName: server-tls #<1> isCA: false usages: - server auth - client auth dnsNames: - - "otel.observability.svc.cluster.local" # <1> + - "central.observability-cluster.com" # <2> issuerRef: name: ca-issuer ---- +---- +<1> Secret will contain certificate +<2> List of exact DNS names to be mapped to a solver in the server OpenTelemetry Collector instance. + +.. The client certificates, this certificate will be used in the Edge cluster ++ +[source,yaml] +---- apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: client spec: - secretName: client-tls + secretName: client-tls #<1> isCA: false usages: - server auth - client auth dnsNames: - - "otel.observability.svc.cluster.local" # <2> + - "edge.observability-cluster.com" # <2> issuerRef: name: ca-issuer ---- -<1> List of exact DNS names to be mapped to a solver in the server OpenTelemetry Collector instance. +<1> Secret will contain certificate <2> List of exact DNS names to be mapped to a solver in the client OpenTelemetry Collector instance. -. Create a service account for the OpenTelemetry Collector instance. + +.Central Cluster + +. Create the YAML file to define the `OpenTelemetryCollector` custom resource (CR) in the central cluster. + -.Example ServiceAccount +.Example `OpenTelemetryCollector` custom resource for the central cluster [source,yaml] ---- -apiVersion: v1 -kind: ServiceAccount +apiVersion: opentelemetry.io/v1alpha1 +kind: OpenTelemetryCollector metadata: - name: otel-collector-deployment + name: otlp-receiver + namespace: chainsaw-multi-cluster-receive +spec: + config: | + receivers: + otlp: + protocols: + http: + tls: # <1> + cert_file: /certs/server.crt + key_file: /certs/server.key + client_ca_file: /certs/ca.crt + grpc: + tls: + cert_file: /certs/server.crt + key_file: /certs/server.key + client_ca_file: /certs/ca.crt + exporters: + otlp: + endpoint: "tempo--distributor:4317" #<2> + tls: + insecure: true + service: + pipelines: + traces: + receivers: [otlp] + exporters: [otlp] + ingress: + route: + termination: passthrough #<3> + type: route + mode: deployment + volumeMounts: + - mountPath: /certs + name: otel-certs + volumes: + - name: otel-certs #<4> + secret: + name: server-tls ---- +<1> The Collector receiver requires the certificates listed in the Generate Certificates step. +<2> The Collector exporter is configured to export OTLP and points to the Tempo distributor endpoint, which in this example is `"tempo-simplest-distributor:4317"` and already created. +<3> The ingress type should be configure to route and termination will be passthrough, in order to not terminate TLS and forward certificates to the collector. +<4> The volume with the secrets generated by cert manager following the Generate Certificates section should be mounted -. Create a cluster role for the service account. +.For Edge cluster +. Create ServiceAccount , ClusterRole and ClusterRoleBinding for the OpenTelemetry collector, this is neeede to use resourceatributesdetector processor. + -.Example ClusterRole [source,yaml] ---- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: otel-edge-multi-cluster +--- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: - name: otel-collector + name: otel-edge-multi-cluster rules: - # <1> - # <2> -- apiGroups: ["", "config.openshift.io"] - resources: ["pods", "namespaces", "infrastructures", "infrastructures/status"] - verbs: ["get", "watch", "list"] ----- -<1> The `k8sattributesprocessor` requires permissions for pods and namespace resources. -<2> The `resourcedetectionprocessor` requires permissions for infrastructures and status. - -. Bind the cluster role to the service account. -+ -.Example ClusterRoleBinding -[source,yaml] ----- +- apiGroups: + - config.openshift.io + resources: + - infrastructures + - infrastructures/status + verbs: + - get + - watch + - list +- apiGroups: + - apps + resources: + - replicasets + verbs: + - get + - watch + - list +- apiGroups: + - "" + resources: + - pods + verbs: + - get + - watch + - list +- apiGroups: + - "" + resources: + - namespaces + verbs: + - get + - watch + - list +--- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: - name: otel-collector -subjects: -- kind: ServiceAccount - name: otel-collector-deployment - namespace: otel-collector- + name: otel-edge-multi-cluster roleRef: - kind: ClusterRole - name: otel-collector apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: otel-edge-multi-cluster +subjects: +- kind: ServiceAccount + name: otel-edge-multi-cluster ---- - ++ . Create the YAML file to define the `OpenTelemetryCollector` custom resource (CR) in the edge clusters. + .Example `OpenTelemetryCollector` custom resource for the edge clusters @@ -166,7 +244,7 @@ metadata: namespace: otel-collector- spec: mode: daemonset - serviceAccount: otel-collector-deployment + serviceAccount: otel-edge-multi-cluster config: | receivers: jaeger: @@ -192,7 +270,7 @@ spec: detectors: [openshift] exporters: otlphttp: - endpoint: https://observability-cluster.com:443 # <1> + endpoint: https://central.observability-cluster.com:443 # <1> tls: insecure: false cert_file: /certs/server.crt @@ -207,57 +285,10 @@ spec: volumes: - name: otel-certs secret: - name: otel-certs - volumeMounts: + name: client-tls + volumeMounts: #<2> - name: otel-certs mountPath: /certs ---- <1> The Collector exporter is configured to export OTLP HTTP and points to the OpenTelemetry Collector from the central cluster. - -. Create the YAML file to define the `OpenTelemetryCollector` custom resource (CR) in the central cluster. -+ -.Example `OpenTelemetryCollector` custom resource for the central cluster -[source,yaml] ----- -apiVersion: opentelemetry.io/v1alpha1 -kind: OpenTelemetryCollector -metadata: - name: otlp-receiver - namespace: observability -spec: - mode: "deployment" - ingress: - type: route - route: - termination: "passthrough" - config: | - receivers: - otlp: - protocols: - http: - tls: # <1> - cert_file: /certs/server.crt - key_file: /certs/server.key - client_ca_file: /certs/ca.crt - exporters: - logging: {} - otlp: - endpoint: "tempo--distributor:4317" # <2> - tls: - insecure: true - service: - pipelines: - traces: - receivers: [otlp] - processors: [] - exporters: [otlp] - volumes: - - name: otel-certs - secret: - name: otel-certs - volumeMounts: - - name: otel-certs - mountPath: /certs ----- -<1> The Collector receiver requires the certificates listed in the first step. -<2> The Collector exporter is configured to export OTLP and points to the Tempo distributor endpoint, which in this example is `"tempo-simplest-distributor:4317"` and already created. +<2> The volume with the secrets generated by cert manager for the client certificated following the Generate Certificates section should be mounted,