|
1 | 1 | <a href="https://opensource.newrelic.com/oss-category/#community-plus"><picture><source media="(prefers-color-scheme: dark)" srcset="https://github.com/newrelic/opensource-website/raw/main/src/images/categories/dark/Community_Plus.png"><source media="(prefers-color-scheme: light)" srcset="https://github.com/newrelic/opensource-website/raw/main/src/images/categories/Community_Plus.png"><img alt="New Relic Open Source community plus project banner." src="https://github.com/newrelic/opensource-website/raw/main/src/images/categories/Community_Plus.png"></picture></a> |
2 | 2 |
|
3 | | -# Kubernetes Metadata injection for New Relic APM agents [](https://codecov.io/gh/newrelic/k8s-metadata-injection) [](https://travis-ci.com/newrelic/k8s-metadata-injection) [](https://goreportcard.com/report/github.com/newrelic/k8s-metadata-injection) |
| 3 | +# Kubernetes Metadata injection for New Relic APM agents [](https://codecov.io/gh/newrelic/k8s-metadata-injection) [](https://goreportcard.com/report/github.com/newrelic/k8s-metadata-injection) |
4 | 4 |
|
5 | 5 | # Table of contents |
6 | 6 |
|
|
18 | 18 | - [Custom](#custom) |
19 | 19 | - [Contributing](#contributing) |
20 | 20 | - [License](#license) |
21 | | -- [Release a new version](#release-a-new-version) |
22 | 21 |
|
23 | 22 | ## Documentation |
24 | 23 |
|
@@ -56,32 +55,76 @@ For further information of the configuration needed for the chart just read the |
56 | 55 |
|
57 | 56 | ### Prerequisites |
58 | 57 |
|
59 | | -For the development process [Minikube](https://kubernetes.io/docs/getting-started-guides/minikube) and [Skaffold](https://github.com/GoogleCloudPlatform/skaffold) tools are used. |
| 58 | +For the development process [Minikube](https://kubernetes.io/docs/getting-started-guides/minikube) and [Helm](https://helm.sh/) tools are used. |
60 | 59 |
|
61 | | -- [Install Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/). |
62 | | -- [Install Skaffold](https://github.com/GoogleCloudPlatform/skaffold#installation). |
| 60 | +- [Install Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/) |
| 61 | +- [Install Helm](https://helm.sh/docs/intro/install/) |
| 62 | +- [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) |
63 | 63 |
|
64 | | -Currently the project compiles with **Go 1.11.4**. |
| 64 | +Currently the project compiles with **Go 1.25.6**. |
65 | 65 |
|
66 | 66 | ### Dependency management |
67 | 67 |
|
68 | 68 | [Go modules](https://github.com/golang/go/wiki/Modules) are used for managing dependencies. This project does not need to be in your GOROOT, if you wish so. |
69 | 69 |
|
70 | | -Currently for K8s libraries it uses version 1.13.1. Only couple of libraries are direct dependencies, the rest are indirect. You need to point all of them to the same K8s version to make sure that everything works as expected. For the moment this process is manual. |
| 70 | +Currently for K8s libraries it uses version 0.35.0. Only couple of libraries are direct dependencies, the rest are indirect. You need to point all of them to the same K8s version to make sure that everything works as expected. For the moment this process is manual. |
71 | 71 |
|
72 | | -### Configuration |
| 72 | +### Local Development Setup |
73 | 73 |
|
74 | | -- Copy the deployment file `deploy/newrelic-metadata-injection.yaml` to `deploy/local.yaml`. |
75 | | -- Edit the file and set the following value as container image: `internal/k8s-metadata-injector`. |
76 | | -- Make sure that `imagePullPolicy: Always` is not present in the file (otherwise, the image won't be pulled). |
| 74 | +To run the webhook locally with Minikube: |
77 | 75 |
|
78 | | -### Run |
| 76 | +1. **Start Minikube**: |
| 77 | + ```bash |
| 78 | + minikube start |
| 79 | + ``` |
79 | 80 |
|
80 | | -Run `skaffold run`. This will build a docker image, build the webhook server inside it, and finally deploy the webhook server to your Minikube and use the Kubernetes API server to sign its TLS certificate ([see section about certificates](#3-install-the-certificates)). |
| 81 | +2. **Build the Docker image in Minikube's Docker environment**: |
| 82 | + ```bash |
| 83 | + # Configure shell to use Minikube's Docker daemon |
| 84 | + eval $(minikube docker-env) |
81 | 85 |
|
82 | | -To follow the logs, you can run `skaffold run --tail`. To delete the resources created by Skaffold you can run `skaffold delete`. |
| 86 | + # Build the image |
| 87 | + make compile build-container DOCKER_IMAGE_TAG=local-dev |
| 88 | + ``` |
83 | 89 |
|
84 | | -If you would like to enable automatic redeploy on changes to the repository, you can run `skaffold dev`. It automatically tails the logs and delete the resources when interrupted (i.e. with a `Ctrl + C`). |
| 90 | +3. **Install the webhook using Helm**: |
| 91 | + ```bash |
| 92 | + # Add New Relic Helm repository |
| 93 | + helm repo add newrelic https://helm-charts.newrelic.com |
| 94 | + |
| 95 | + # Build chart dependencies |
| 96 | + helm dependency build ./charts/nri-metadata-injection |
| 97 | + |
| 98 | + # Install the chart with local image |
| 99 | + helm upgrade --install nri-metadata-injection ./charts/nri-metadata-injection \ |
| 100 | + --set cluster=local-cluster \ |
| 101 | + --set image.tag=local-dev \ |
| 102 | + --set image.pullPolicy=Never |
| 103 | + ``` |
| 104 | + |
| 105 | +4. **View logs**: |
| 106 | + ```bash |
| 107 | + kubectl logs -l app.kubernetes.io/name=nri-metadata-injection -f |
| 108 | + ``` |
| 109 | + |
| 110 | +5. **Test the webhook** by deploying a sample pod: |
| 111 | + ```bash |
| 112 | + kubectl create deployment test-deployment --image=nginx:latest |
| 113 | + kubectl get pods |
| 114 | + kubectl exec <pod-name> -- env | grep NEW_RELIC_METADATA_KUBERNETES |
| 115 | + ``` |
| 116 | + |
| 117 | +6. **Cleanup**: |
| 118 | + ```bash |
| 119 | + helm uninstall nri-metadata-injection |
| 120 | + kubectl delete deployment test-deployment |
| 121 | + ``` |
| 122 | + |
| 123 | +After making code changes, rebuild the Docker image (step 2) and upgrade the Helm release: |
| 124 | +```bash |
| 125 | +helm upgrade nri-metadata-injection ./charts/nri-metadata-injection \ |
| 126 | + --reuse-values |
| 127 | +``` |
85 | 128 |
|
86 | 129 | ### Tests |
87 | 130 |
|
@@ -129,47 +172,50 @@ Please refer to [docs/performance.md](docs/performance.md). |
129 | 172 |
|
130 | 173 | ## Certificates management |
131 | 174 |
|
132 | | -Admission webhooks are called by the Kubernetes API server and it needs to authenticate the webhooks using TLS. In this project we offer 2 different options of certificate management. |
| 175 | +Admission webhooks are called by the Kubernetes API server and it needs to authenticate the webhooks using TLS. The Helm chart offers 3 different options for certificate management: |
| 176 | + |
| 177 | +1. **Automatic (Default)**: Helm pre-install/pre-upgrade jobs automatically create and manage certificates |
| 178 | +2. **cert-manager**: Use cert-manager for automated certificate lifecycle management |
| 179 | +3. **Custom**: Provide your own TLS certificates |
133 | 180 |
|
134 | | -Either certificate management choice made, the important thing is to have the secret created with the correct name and namespace, and also to have the correct CA bundle in the MutatingWebhookConfiguration resource. As long as this is done the webhook server will be able to pick it up. |
| 181 | +Regardless of which option you choose, the important thing is to have the secret created with the correct name and namespace, and also to have the correct CA bundle in the MutatingWebhookConfiguration resource. As long as this is done the webhook server will be able to pick it up. |
135 | 182 |
|
136 | | -### Automatic |
| 183 | +### Automatic (Default) |
137 | 184 |
|
138 | 185 | Please refer to the [setup instructions in the official documentation](https://docs.newrelic.com/docs/integrations/kubernetes-integration/metadata-injection/kubernetes-apm-metadata-injection#install). |
139 | 186 |
|
140 | | -For the automatic certificate management, the [k8s-webhook-cert-manager](https://github.com/newrelic/k8s-webhook-cert-manager) is used. Feel free to check the repository to know more about it. |
| 187 | +The Helm chart uses Kubernetes jobs (see [charts/nri-metadata-injection/templates/admission-webhooks/job-patch/](./charts/nri-metadata-injection/templates/admission-webhooks/job-patch/)) that run during Helm hooks to automatically create self-signed certificates and patch the MutatingWebhookConfiguration. |
141 | 188 |
|
142 | | -The manifest file at [deploy/job.yaml](./deploy/job.yaml) contains a service account that has the following **cluster** permissions (**RBAC based**) to be capable of automatically manage the certificates: |
| 189 | +The jobs run with a service account that has the following **cluster** permissions (**RBAC based**): |
143 | 190 |
|
144 | 191 | - `MutatingWebhookConfiguration` - **get**, **create** and **patch**: to be able to create the webhook and patch its CA bundle. |
145 | 192 | - `CertificateSigningRequests` - **create**, **get** and **delete**: to be able to sign the certificate required for the webhook server without leaving duplicates. |
146 | 193 | - `CertificateSigningRequests/Approval` - **update**: to be able to approve CertificateSigningRequests. |
147 | 194 | - `Secrets` - **create**, **get** and **patch**: to be able to manage the TLS secret used to store the key/cert pair used in the webhook server. |
148 | | -- `ConfigMaps` - **get**: to be able go get the k8s api server's CA bundle, used in the MutatingWebhookConfiguration. |
| 195 | +- `ConfigMaps` - **get**: to be able to get the k8s api server's CA bundle, used in the MutatingWebhookConfiguration. |
| 196 | + |
| 197 | +### cert-manager |
| 198 | + |
| 199 | +To use cert-manager for certificate management, set `certManager.enabled=true` in your Helm values. This delegates certificate lifecycle management to cert-manager. |
149 | 200 |
|
150 | 201 | If you wish to learn more about TLS certificates management inside Kubernetes, check out [the official documentation for Managing TLS Certificates in a Cluster](https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/#create-a-certificate-signing-request-object-to-send-to-the-kubernetes-api). |
151 | 202 |
|
152 | 203 | ### Custom |
153 | 204 |
|
154 | | -Otherwise, if you want to use the custom certificate management option you have to create the TLS secret with the signed certificate/key pair and patch the webhook's CA bundle: |
| 205 | +If you want to use the custom certificate management option, set `customTLSCertificate=true` in your Helm values, then create the TLS secret with the signed certificate/key pair and patch the webhook's CA bundle: |
155 | 206 |
|
156 | 207 | ```bash |
157 | 208 | $ kubectl create secret tls newrelic-metadata-injection-secret \ |
158 | 209 | --key=server-key.pem \ |
159 | 210 | --cert=signed-server-cert.pem \ |
160 | 211 | --dry-run -o yaml | |
161 | | - kubectl -n default apply -f - |
| 212 | + kubectl -n <namespace> apply -f - |
162 | 213 |
|
163 | | -$ caBundle=$(cat caBundle.pem | base64 | td -d '\n') |
164 | | -$ kubectl patch mutatingwebhookconfiguration newrelic-metadata-injection-cfg --type='json' -p "[{'op': 'replace', 'path': '/webhooks/0/clientConfig/caBundle', 'value':'${caBundle}'}]" |
| 214 | +$ caBundle=$(cat caBundle.pem | base64 | tr -d '\n') |
| 215 | +$ kubectl patch mutatingwebhookconfiguration <webhook-name> --type='json' -p "[{'op': 'replace', 'path': '/webhooks/0/clientConfig/caBundle', 'value':'${caBundle}'}]" |
165 | 216 | ``` |
166 | 217 |
|
167 | | -## Release a new version |
168 | | - |
169 | | -- Update the version in `deploy/newrelic-metadata-injection.yaml`. |
170 | | -- Update the version in `WEBHOOK_DOCKER_IMAGE_TAG` in the `Makefile`. |
171 | | -- Create a Github release. |
172 | | -- Launch the `k8s-metadata-injection-release` job in Jenkins. |
| 218 | +Replace `<namespace>` with your installation namespace and `<webhook-name>` with the name of your webhook configuration. |
173 | 219 |
|
174 | 220 | ## Support |
175 | 221 |
|
|
0 commit comments