netbirdio · mohamed-essam · Mar 5, 2025 · Mar 2, 2025 · Mar 5, 2025 · Mar 5, 2025
diff --git a/README.md b/README.md
@@ -1,107 +1,56 @@
 # NetBird Kubernetes Operator
 For easily provisioning access to Kubernetes resources using NetBird.
 
+https://github.com/user-attachments/assets/5472a499-e63d-4301-a513-ad84cfe5ca7b
+
 ## Description
 
-This operator enables easily provisioning NetBird access on kubernetes clusters, allowing users to access internal resources directly.
+This operator easily provides NetBird access on Kubernetes clusters, allowing users to access internal resources directly.
 
 ## Getting Started
 
 ### Prerequisites
-- helm version 3+
+- (Recommended) helm version 3+
 - kubectl version v1.11.3+.
 - Access to a Kubernetes v1.11.3+ cluster.
-- (Optional for Helm chart installation) Cert Manager.
-
-### To Deploy on the cluster
+- (Recommended) Cert Manager.
 
-**Using the install.yaml**
 
-```sh
-kubectl create namespace netbird
-kubectl apply -n netbird -f https://github.com/netbirdio/kubernetes-operator/releases/latest/manifests/install.yaml
-```
+### Deployment
+> [!NOTE]
+> Helm Installation method is recommended due to the automation of multiple settings within the deployment.
 
-**Using the Helm Chart**
+#### Using Helm
 
+1. Add helm repository.
 ```sh
 helm repo add netbirdio https://netbirdio.github.io/kubernetes-operator
-helm install -n netbird kubernetes-operator netbirdio/kubernetes-operator
 ```
+2. (Recommended) Install [cert-manager](https://cert-manager.io/docs/installation/#default-static-install).
+3. (Recommended) Create a values.yaml file, check `helm show values netbirdio/kubernetes-operator` for more info.
+4. Install using `helm install --create-namespace -f values.yaml -n netbird netbird-operator netbirdio/kubernetes-operator`.
 
-For more options, check the default values by running
-```sh
-helm show values netbirdio/kubernetes-operator
-```
+> Learn more about the values.yaml options [here](helm/kubernetes-operator/values.yaml) and  [Granting controller access to NetBird Management](docs/usage.md#granting-controller-access-to-netbird-management).
+#### Using install.yaml
 
-### To Uninstall
-**Using install.yaml**
+> [!IMPORTANT]
+> install.yaml only includes a very basic template for deploying a stripped-down version of Kubernetes-operator.
+> This excludes any and all configurations for ingress capabilities and requires the cert-manager to be installed.
 
 ```sh
-kubectl delete -n netbird -f https://github.com/netbirdio/kubernetes-operator/releases/latest/manifests/install.yaml
-kubectl delete namespace netbird
-```
-
-**Using helm**
-
-```sh
-helm uninstall -n netbird kubernetes-operator
+kubectl create namespace netbird
+kubectl apply -n netbird -f https://raw.githubusercontent.com/netbirdio/kubernetes-operator/refs/heads/main/manifests/install.yaml
 ```
 
-### Provision pods with NetBird access
-
-1. Create a Setup Key in your [NetBird console](https://docs.netbird.io/how-to/register-machines-using-setup-keys#using-setup-keys).
-1. Create a Secret object in the namespace where you need to provision NetBird access (secret name and field can be anything).
-```yaml
-apiVersion: v1
-stringData:
-  setupkey: EEEEEEEE-EEEE-EEEE-EEEE-EEEEEEEEEEEE
-kind: Secret
-metadata:
-  name: test
-```
-1. Create an NBSetupKey object referring to your secret.
-```yaml
-apiVersion: netbird.io/v1
-kind: NBSetupKey
-metadata:
-  name: test
-spec:
-  # Optional, overrides management URL for this setupkey only
-  # defaults to https://api.netbird.io
-  managementURL: https://netbird.example.com 
-  secretKeyRef:
-    name: test # Required
-    key: setupkey # Required
-```
-1. Annotate the pods you need to inject NetBird into with `netbird.io/setup-key`.
-```yaml
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: deployment
-spec:
-  selector:
-    matchLabels:
-      app: myapp
-  template:
-    metadata:
-      labels:
-        app: myapp
-      annotations:
-        netbird.io/setup-key: test # Must match the name of an NBSetupKey object in the same namespace
-    spec:
-      containers:
-      - image: yourimage
-        name: container
+### Usage
 
-```
+Check the usage of [usage.md](docs/usage.md) and examples.
 
 ## Contributing
 
 ### Prerequisites
 
-To be able to develop on this project, you need to have the following tools installed:
+To be able to develop this project, you need to have the following tools installed:
 
 - [Git](https://git-scm.com/).
 - [Make](https://www.gnu.org/software/make/).

diff --git a/docs/usage.md b/docs/usage.md
@@ -0,0 +1,162 @@
+# Usage
+
+## Provision pods with NetBird access
+
+1. Create a Setup Key in your [NetBird console](https://docs.netbird.io/how-to/register-machines-using-setup-keys#using-setup-keys).
+1. Create a Secret object in the namespace where you need to provision NetBird access (secret name and field can be anything).
+```yaml
+apiVersion: v1
+stringData:
+  setupkey: EEEEEEEE-EEEE-EEEE-EEEE-EEEEEEEEEEEE
+kind: Secret
+metadata:
+  name: test
+```
+1. Create an NBSetupKey object referring to your secret.
+```yaml
+apiVersion: netbird.io/v1
+kind: NBSetupKey
+metadata:
+  name: test
+spec:
+  # Optional, overrides management URL for this setupkey only
+  # defaults to https://api.netbird.io
+  managementURL: https://netbird.example.com 
+  secretKeyRef:
+    name: test # Required
+    key: setupkey # Required
+```
+1. Annotate the pods you need to inject NetBird into with `netbird.io/setup-key`.
+```yaml
+kind: Deployment
+...
+spec:
+...
+  template:
+    metadata:
+      annotations:
+        netbird.io/setup-key: test # Must match the name of an NBSetupKey object in the same namespace
+...
+    spec:
+      containers:
+...
+```
+
+## Provisioning Networks (Ingress Functionality)
+
+### Granting controller access to NetBird Management
+
+> [!IMPORTANT]
+> The NetBird Kubernetes operator generates configurations using NetBird API; editing or deleting these configurations in the NetBird console may cause temporary network disconnection until the operator reconciles the configuration.
+
+1. Create a Service User on your NetBird dashboard (Must be Admin). [Doc](https://docs.netbird.io/how-to/access-netbird-public-api#creating-a-service-user).
+1. Create an access token for the Service User (Must be Admin). [Doc](https://docs.netbird.io/how-to/access-netbird-public-api#creating-a-service-user).
+1. Add access token to your helm values file under `netbirdAPI.key`.
+    1. Alternatively, provision secret in the same namespace as the operator and set the key `NB_API_KEY` to the access token generated.
+    1. Set `netbirdAPI.keyFromSecret` to the name of the secret created.
+1. Set `ingress.enabled` to `true`.
+    1. Optionally, to provision the network immediately, set `ingress.router.enabled` to `true`.
+    1. Optionally, to provision 1 network per > The NetBird Kubernetes operator generates configurations using NetBird API; editing or deleting these configurations in the NetBird console may cause temporary network disconnection until the operator reconciles the configuration. namespace, set `ingress.namespacedNetworks` to `true`.
+1. Run `helm install` or `helm upgrade`.
+
+Minimum values.yaml example:
+```yaml
+netbirdAPI:
+  key: "nbp_XXxxxxxxXXXXxxxxXXXXXxxx"
+
+ingress:
+  enabled: true
+cluster:
+  name: kubernetes
+```
+
+### Exposing a Service
+
+> [!IMPORTANT]  
+> Ingress DNS Resolution requires DNS Wildcard Routing to be enabled and at least one DNS Nameserver configured for clients.
+
+|Annotation|Description|Default|Valid Values|
+|---|---|---|---|
+|`netbird.io/expose`| Expose service using NetBird Network Resource ||(`null`, `true`)|
+|`netbird.io/groups`| Comma-separated list of group names to assign to Network Resource |`{ClusterName}-{Namespace}-{Service}`|Any comma-separated list of strings.|
+|`netbird.io/resource-name`| Network Resource name |`{Namespace}-{Service}`|Any valid network resource name, make sure they're unique!|
+|`netbird.io/policy`| Name of NBPolicy to propagate service ports as destination. ||Name of any NBPolicy resource|
+|`netbird.io/policy-ports`| Narrow down exposed ports in a policy. Leave empty for all ports. ||Comma-separated integer list, integers must be between 0-65535|
+|`netbird.io/policy-protocol`| Narrow down protocol for use in a policy. Leave empty for all protocols. ||(`tcp`,`udp`)|
+
+Example service:
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: nginx-deployment
+spec:
+  replicas: 0
+  selector:
+    matchLabels:
+      app: nginx
+  template:
+    metadata:
+      labels:
+        app: nginx
+    spec:
+      containers:
+        - name: nginx
+          image: nginx:latest
+          ports:
+            - containerPort: 80
+
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: nginx-service
+  annotations:
+    netbird.io/expose: "true"
+    netbird.io/groups: "groupA,groupB"
+spec:
+  selector:
+    app: nginx
+  ports:
+    - protocol: TCP
+      port: 8080
+      targetPort: 80
+  type: ClusterIP
+```
+### Notes
+* `netbird.io/expose` will interpret any string as a `true` value; the only `false` value is `null`.
+* The operator does **not** handle duplicate resource names within the same network, it is up to you to ensure resource names are unique within the same network.
+* While the NetBird console will allow group names to contain commas, this is not allowed in `netbird.io/groups` annotation as commas are used as separators.
+* If a group already exists on NetBird console with the same name, NetBird Operator will use that group ID instead of creating a new group.
+* NetBird Operator will attempt to clean up any resources created, including groups created for resources.
+    * If a group is used by resources that the operator cannot clean up, the operator will eventually ignore the group in NetBird.
+    * It's recommended that unique groups be used per NetBird Operator installation to remove any possible conflicts.
+* The Operator does not validate service annotations on updates, as this may cause unnecessary overhead on any Service update.
+
+### Managing Policies
+
+Simply add policies under `ingress.policies`, for example:
+1. Add the following configuration in your `values.yaml` file.
+```yaml
+ingress:
+  policies:
+    default:
+      name: Kubernetes Default Policy # Required, name of policy in NetBird console
+      description: Default # Optional
+      sourceGroups: # Required, name of groups to assign as source in Policy.
+      - All
+      ports: # Optional, resources annotated 'netbird.io/policy=default' will append to this.
+      - 443
+      protocols: # Optional, restricts protocols allowed to resources, defaults to ['tcp', 'udp'].
+      - tcp
+      - udp
+      bidirectional: true # Optional, defaults to true
+```
+2. Reference policy in Services using `netbird.io/policy=default`, this will add relevant ports and destination groups to policy.
+3. (Optional) Limit specific ports in exposed service by adding `netbird.io/policy-ports=443`.
+4. (Optional) Limit specific protocol in exposed service by adding `netbird.io/policy-protocol=tcp`.
+
+#### Notes
+* Each NBPolicy will only create policies in the NetBird console when the information provided is enough to create one. If no services act as a destination or specified services do not conform to the protocol(s) defined, the policy will not be created.
+* Each NBPolicy will create one policy in the NetBird console per protocol specified as long as the protocol has destinations; this ensures better-secured policies by separating ports for TCP and UDP.
+* Policies currently do not support ICMP protocol, as ICMP is not supported in Kubernetes services, and there are [no current plans to support it](https://discuss.kubernetes.io/t/icmp-support-for-kubernetes-service/21738).
diff --git a/examples/ingress/exposed-nginx.yaml b/examples/ingress/exposed-nginx.yaml
@@ -0,0 +1,47 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+    app: test
+  name: test
+  namespace: default
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: test
+  strategy:
+    rollingUpdate:
+      maxSurge: 25%
+      maxUnavailable: 25%
+    type: RollingUpdate
+  template:
+    metadata:
+      labels:
+        app: test
+    spec:
+      containers:
+      - image: nginx
+        imagePullPolicy: Always
+        name: nginx
+---
+apiVersion: v1
+kind: Service
+metadata:
+  annotations:
+    netbird.io/expose: "true"
+    netbird.io/policy: default
+    netbird.io/resource-name: nginx
+  labels:
+    app: test
+  name: test
+  namespace: default
+spec:
+  ports:
+  - name: http
+    port: 80
+    protocol: TCP
+    targetPort: 80
+  selector:
+    app: test
+  type: ClusterIP
diff --git a/examples/ingress/values.yaml b/examples/ingress/values.yaml
@@ -0,0 +1,12 @@
+ingress:
+  enabled: true
+  router:
+    enabled: true
+  policies:
+    default:
+      name: Kubernetes Default Policy
+      sourceGroups:
+      - All
+
+netbirdAPI:
+  key: "nbp_m0LM9ZZvDUzFO0pY50iChDOTxJgKFM3DIqmZ" # Replace with valid NetBird Service Account token