Skip to content

Commit 414841a

Browse files
mohamed-essammohamed-essam
committed
WIP
1 parent 00cf45b commit 414841a

File tree

14 files changed

+311
-87
lines changed

14 files changed

+311
-87
lines changed

‎README.md‎

Lines changed: 20 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -1,101 +1,49 @@
11
# NetBird Kubernetes Operator
22
For easily provisioning access to Kubernetes resources using NetBird.
33

4+
https://github.com/user-attachments/assets/5472a499-e63d-4301-a513-ad84cfe5ca7b
5+
46
## Description
57

68
This operator enables easily provisioning NetBird access on kubernetes clusters, allowing users to access internal resources directly.
79

810
## Getting Started
911

1012
### Prerequisites
11-
- helm version 3+
13+
- (Recommended) helm version 3+
1214
- kubectl version v1.11.3+.
1315
- Access to a Kubernetes v1.11.3+ cluster.
14-
- (Optional for Helm chart installation) Cert Manager.
15-
16-
### To Deploy on the cluster
16+
- (Recommended) Cert Manager.
1717

18-
**Using the install.yaml**
1918

20-
```sh
21-
kubectl create namespace netbird
22-
kubectl apply -n netbird -f https://github.com/netbirdio/kubernetes-operator/releases/latest/manifests/install.yaml
23-
```
19+
### Deployment
20+
> [!NOTE]
21+
> Helm Installation method is recommended due to automation of multiple settings within the deployment.
2422
25-
**Using the Helm Chart**
23+
#### Using Helm
2624

25+
1. Add helm repository.
2726
```sh
2827
helm repo add netbirdio https://netbirdio.github.io/kubernetes-operator
29-
helm install -n netbird kubernetes-operator netbirdio/kubernetes-operator
3028
```
29+
2. (Recommended) Install [cert-manager](https://cert-manager.io/docs/installation/#default-static-install).
30+
1. (Recommended) Create a values.yaml file, check `helm show values netbirdio/kubernetes-operator` for more info.
31+
1. Install using `helm install --create-namespace -f values.yaml -n netbird netbird-operator netbirdio/kubernetes-operator`.
3132

32-
For more options, check the default values by running
33-
```sh
34-
helm show values netbirdio/kubernetes-operator
35-
```
33+
#### Using install.yaml
3634

37-
### To Uninstall
38-
**Using install.yaml**
35+
> [!IMPORTANT]
36+
> install.yaml only includes a very basic template for deploying a stripped down version of kubernetes-operator.
37+
> This excludes any and all configuration for ingress capabilities, and requires cert-manager to be installed.
3938
4039
```sh
41-
kubectl delete -n netbird -f https://github.com/netbirdio/kubernetes-operator/releases/latest/manifests/install.yaml
42-
kubectl delete namespace netbird
43-
```
44-
45-
**Using helm**
46-
47-
```sh
48-
helm uninstall -n netbird kubernetes-operator
40+
kubectl create namespace netbird
41+
kubectl apply -n netbird -f https://raw.githubusercontent.com/netbirdio/kubernetes-operator/refs/heads/main/manifests/install.yaml
4942
```
5043

51-
### Provision pods with NetBird access
52-
53-
1. Create a Setup Key in your [NetBird console](https://docs.netbird.io/how-to/register-machines-using-setup-keys#using-setup-keys).
54-
1. Create a Secret object in the namespace where you need to provision NetBird access (secret name and field can be anything).
55-
```yaml
56-
apiVersion: v1
57-
stringData:
58-
setupkey: EEEEEEEE-EEEE-EEEE-EEEE-EEEEEEEEEEEE
59-
kind: Secret
60-
metadata:
61-
name: test
62-
```
63-
1. Create an NBSetupKey object referring to your secret.
64-
```yaml
65-
apiVersion: netbird.io/v1
66-
kind: NBSetupKey
67-
metadata:
68-
name: test
69-
spec:
70-
# Optional, overrides management URL for this setupkey only
71-
# defaults to https://api.netbird.io
72-
managementURL: https://netbird.example.com
73-
secretKeyRef:
74-
name: test # Required
75-
key: setupkey # Required
76-
```
77-
1. Annotate the pods you need to inject NetBird into with `netbird.io/setup-key`.
78-
```yaml
79-
apiVersion: apps/v1
80-
kind: Deployment
81-
metadata:
82-
name: deployment
83-
spec:
84-
selector:
85-
matchLabels:
86-
app: myapp
87-
template:
88-
metadata:
89-
labels:
90-
app: myapp
91-
annotations:
92-
netbird.io/setup-key: test # Must match the name of an NBSetupKey object in the same namespace
93-
spec:
94-
containers:
95-
- image: yourimage
96-
name: container
44+
### Usage
9745

98-
```
46+
Checks [usage.md](docs/usage.md).
9947

10048
## Contributing
10149

‎docs/usage.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# Usage
2+
3+
## Provision pods with NetBird access
4+
5+
1. Create a Setup Key in your [NetBird console](https://docs.netbird.io/how-to/register-machines-using-setup-keys#using-setup-keys).
6+
1. Create a Secret object in the namespace where you need to provision NetBird access (secret name and field can be anything).
7+
```yaml
8+
apiVersion: v1
9+
stringData:
10+
setupkey: EEEEEEEE-EEEE-EEEE-EEEE-EEEEEEEEEEEE
11+
kind: Secret
12+
metadata:
13+
name: test
14+
```
15+
1. Create an NBSetupKey object referring to your secret.
16+
```yaml
17+
apiVersion: netbird.io/v1
18+
kind: NBSetupKey
19+
metadata:
20+
name: test
21+
spec:
22+
# Optional, overrides management URL for this setupkey only
23+
# defaults to https://api.netbird.io
24+
managementURL: https://netbird.example.com
25+
secretKeyRef:
26+
name: test # Required
27+
key: setupkey # Required
28+
```
29+
1. Annotate the pods you need to inject NetBird into with `netbird.io/setup-key`.
30+
```yaml
31+
kind: Deployment
32+
...
33+
spec:
34+
...
35+
template:
36+
metadata:
37+
annotations:
38+
netbird.io/setup-key: test # Must match the name of an NBSetupKey object in the same namespace
39+
...
40+
spec:
41+
containers:
42+
...
43+
```
44+
45+
## Provisioning Networks (Ingress Functionality)
46+
47+
### Granting controller access to NetBird Management
48+
49+
> [!IMPORTANT]
50+
> NetBird kubernetes operator generates configurations using NetBird API, editing or deleting these configurations in the NetBird console may cause temporary network disconnection until the operator reconciles the configuration.
51+
52+
1. Create a Service User on your NetBird dashboard (Must be Admin). [Doc](https://docs.netbird.io/how-to/access-netbird-public-api#creating-a-service-user).
53+
1. Create access token for the Service User (Must be Admin). [Doc](https://docs.netbird.io/how-to/access-netbird-public-api#creating-a-service-user).
54+
1. Add access token to your helm values file under `netbirdAPI.key`.
55+
1. Alternatively, provision secret in the same namespace as the operator and set the key `NB_API_KEY` to the access token generated.
56+
1. Set `netbirdAPI.keyFromSecret` to the name of the secret created.
57+
1. Set `ingress.enabled` to `true`.
58+
1. Optionally, to provision network immediately, set `ingress.router.enabled` to `true`.
59+
1. Optionally, to provision 1 network per kubernetes namespace, set `ingress.namespacedNetworks` to `true`.
60+
1. Run `helm install` or `helm upgrade`.
61+
62+
### Exposing a Service
63+
64+
> [!IMPORTANT]
65+
> Ingress DNS Resolution requires DNS Wildcard Routing to be enabled, and at least one DNS Nameserver configured for clients.
66+
67+
|Annotation|Description|Default|Valid Values|
68+
|---|---|---|---|
69+
|`netbird.io/expose`|Expose service using NetBird Network Resource||(`null`, `true`)|
70+
|`netbird.io/groups`|Comma-separated list of group names to assign to Network Resource|`{ClusterName}-{Namespace}-{Service}`|Any comma-separated list of strings.|
71+
|`netbird.io/resource-name`|Network Resource name|`{Namespace}-{Service}`|Any valid network resource name, make sure they're unique!|
72+
|`netbird.io/policy`|Name of NBPolicy to propagate service ports as destination.||Name of any NBPolicy resource|
73+
|`netbird.io/policy-ports`|Narrow down exposed ports in policy, leave empty for all ports.||Comma-separated integer list, integers must be between 0-65535|
74+
|`netbird.io/policy-protocol`|Narrow down protocol for use in policy, leave empty for all protocols.||(`tcp`,`udp`)|
75+
76+
### Notes
77+
* `netbird.io/expose` will interpret any string as true value, the only false value is `null`.
78+
* The operator does **not** handle duplicate resource names within the same network, it is up to you to ensure resource names are unique within the same network.
79+
* While the NetBird console will allow group names to contain commas, this is not allowed in `netbird.io/groups` annotation as commas are used as separators.
80+
* If a group already exists on NetBird console with the same name, NetBird Operator will use that group ID instead of creating a new group.
81+
* NetBird Operator will attempt to clean up any resources created, including groups created for resources.
82+
* In case a group is used by resources that cannot be cleaned up by the operator, the operator will eventually ignore the group in NetBird.
83+
* It's recommended to use unique groups per NetBird Operator installation to remove any possible conflicts.
84+
* NetBird Operator does not validate service annotations on update, as this may cause unnecessary overhead on any Service update.
85+
86+
### Managing Policies
87+
88+
Simply add policies under `ingress.policies`, for example:
89+
1. Add the following configuration in your `values.yaml` file.
90+
```yaml
91+
ingress:
92+
policies:
93+
default:
94+
name: Kubernetes Default Policy # Required, name of policy in NetBird console
95+
description: Default # Optional
96+
sourceGroups: # Required, name of groups to assign as source in Policy.
97+
- All
98+
ports: # Optional, resources annotated 'netbird.io/policy=default' will append to this.
99+
- 443
100+
protocols: # Optional, restricts protocols allowed to resources, defaults to ['tcp', 'udp'].
101+
- tcp
102+
- udp
103+
bidirectional: true # Optional, defaults to true
104+
```
105+
2. Reference policy in Services using `netbird.io/policy=default`, this will add relevant ports and destination groups to Policy.
106+
3. (Optional) limit specific ports in exposed service by adding `netbird.io/policy-ports=443`.
107+
4. (Optional) limit specific protocol in exposed service by adding `netbird.io/policy-protocol=tcp`.
108+
109+
#### Notes
110+
* Each NBPolicy will only create policies in NetBird console when information provided is enough to create one. If there are no services acting as destination, or specified services do not conform to the protocol(s) defined, policy will not be created.
111+
* Each NBPolicy will create one Policy in NetBird console per protocol specified as long as protocol has destinations, this ensures better secured policies by separating ports for TCP and UDP.
112+
* Policies currently do not support ICMP protocol, as ICMP is not supported in kubernetes services, and there are [no current plans to support it](https://discuss.kubernetes.io/t/icmp-support-for-kubernetes-service/21738).

‎examples/ingress/exposed-nginx.yaml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
apiVersion: apps/v1
2+
kind: Deployment
3+
metadata:
4+
labels:
5+
app: test
6+
name: test
7+
namespace: default
8+
spec:
9+
replicas: 1
10+
selector:
11+
matchLabels:
12+
app: test
13+
strategy:
14+
rollingUpdate:
15+
maxSurge: 25%
16+
maxUnavailable: 25%
17+
type: RollingUpdate
18+
template:
19+
metadata:
20+
labels:
21+
app: test
22+
spec:
23+
containers:
24+
- image: nginx
25+
imagePullPolicy: Always
26+
name: nginx
27+
---
28+
apiVersion: v1
29+
kind: Service
30+
metadata:
31+
annotations:
32+
netbird.io/expose: "true"
33+
netbird.io/policy: default
34+
netbird.io/resource-name: nginx
35+
labels:
36+
app: test
37+
name: test
38+
namespace: default
39+
spec:
40+
ports:
41+
- name: http
42+
port: 80
43+
protocol: TCP
44+
targetPort: 80
45+
selector:
46+
app: test
47+
type: ClusterIP

‎examples/ingress/values.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
ingress:
2+
enabled: true
3+
router:
4+
enabled: true
5+
policies:
6+
default:
7+
name: Kubernetes Default Policy
8+
sourceGroups:
9+
- All
10+
11+
netbirdAPI:
12+
key: "nbp_m0LM9ZZvDUzFO0pY50iChDOTxJgKFM3DIqmZ" # Replace with valid NetBird Service Account token

‎examples/setup-keys/example.yaml‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
apiVersion: v1
2+
kind: Secret
3+
metadata:
4+
name: test
5+
namespace: default
6+
stringData:
7+
SETUP_KEY: 50445ABC-8901-4050-8047-0A390658A79B # Replace with valid setup key
8+
---
9+
apiVersion: netbird.io/v1
10+
kind: NBSetupKey
11+
metadata:
12+
name: test
13+
namespace: default
14+
spec:
15+
secretKeyRef:
16+
name: test
17+
key: SETUP_KEY
18+
---
19+
apiVersion: apps/v1
20+
kind: Deployment
21+
metadata:
22+
labels:
23+
app: test
24+
name: test
25+
namespace: default
26+
spec:
27+
replicas: 1
28+
selector:
29+
matchLabels:
30+
app: test
31+
strategy:
32+
rollingUpdate:
33+
maxSurge: 25%
34+
maxUnavailable: 25%
35+
type: RollingUpdate
36+
template:
37+
metadata:
38+
labels:
39+
app: test
40+
annotations:
41+
netbird.io/setup-key: test
42+
spec:
43+
containers:
44+
- image: ubuntu
45+
imagePullPolicy: Always
46+
name: ubuntu
47+
command:
48+
- sleep
49+
- inf

‎helm/kubernetes-operator/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@ apiVersion: v2
22
name: kubernetes-operator
33
description: A Helm chart for Kubernetes
44
type: application
5-
version: 0.1.1
5+
version: 0.1.2
66
appVersion: "v0.1.0-rc.3"

‎helm/kubernetes-operator/templates/_helpers.tpl‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ caCert: {{ index $secret.data "ca.crt" }}
9292
clientCert: {{ index $secret.data "tls.crt" }}
9393
clientKey: {{ index $secret.data "tls.key" }}
9494
{{- else -}}
95-
{{- $altNames := list (printf "%s.%s" $serviceName .Release.Namespace) (printf "%s.%s.svc" $serviceName .Release.Namespace) (printf "%s.%s.svc.%s" $serviceName .Release.Namespace .Values.webhook.cluster.dnsDomain) -}}
95+
{{- $altNames := list (printf "%s.%s" $serviceName .Release.Namespace) (printf "%s.%s.svc" $serviceName .Release.Namespace) (printf "%s.%s.%s" $serviceName .Release.Namespace .Values.cluster.dns) -}}
9696
{{- $ca := genCA "kubernetes-operator-ca" 3650 -}}
9797
{{- $cert := genSignedCert (include "kubernetes-operator.fullname" .) nil $altNames 3650 $ca -}}
9898
caCert: {{ $ca.Cert | b64enc }}

‎helm/kubernetes-operator/templates/webhook.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -235,7 +235,7 @@ metadata:
235235
spec:
236236
dnsNames:
237237
- {{ template "kubernetes-operator.webhookService" . }}.{{ .Release.Namespace }}.svc
238-
- {{ template "kubernetes-operator.webhookService" . }}.{{ .Release.Namespace }}.svc.{{ .Values.webhook.cluster.dnsDomain }}
238+
- {{ template "kubernetes-operator.webhookService" . }}.{{ .Release.Namespace }}.{{ .Values.cluster.dns }}
239239
issuerRef:
240240
kind: Issuer
241241
name: {{ template "kubernetes-operator.fullname" . }}-selfsigned-issuer

0 commit comments

Comments
 (0)