-
-
Notifications
You must be signed in to change notification settings - Fork 1k
Description
I am setting up a self-hosted NetBird instance using the new Embedded IdP feature (introduced in v0.62.0). I have encountered a "chicken-and-egg" deadlock during the Management service startup because of my external reverse proxy architecture.
When EmbeddedIdP is enabled, the Management service attempts to validate its own OIDC configuration by making a network request to its public domain (https://netbird.example.com/...). This request travels out to my external proxy and loops back to the container. However, since the Management service is blocked waiting for this check to finish, it is not yet serving traffic. The proxy receives a "Connection Refused" or timeout, returns a 502 Bad Gateway to the Management service, and the service retries indefinitely or panics.
Architecture:
VM 1 (Proxy Host): Runs Traefik (External). Terminates SSL (443) and forwards traffic via H2C (HTTP/2 cleartext) to VM 2.
VM 2 (NetBird Host): Runs the NetBird Docker stack with a local Caddy acting as an internal router on Port 80.
Steps to Reproduce:
Configure management.json with "EmbeddedIdP": { "Enabled": true, ... }.
Set HttpConfig endpoints to the public domain (https://netbird.example.com/...).
Start the container (docker compose up).
Observe the boot loop in logs.
Observed Logs (The 502 Loop):
Plaintext
2026-01-10T12:19:56Z INFO [context: SYSTEM] management/cmd/management.go:157: loading OIDC configuration from the provided IDP configuration endpoint https://netbird.example.com/oauth2/.well-known/openid-configuration
Error: failed reading provided config file: /etc/netbird/management.json: failed fetching OIDC configuration from endpoint https://netbird.example.com/oauth2/.well-known/openid-configuration Get "https://netbird.example.com/oauth2/.well-known/openid-configuration": remote error: tls: internal error
...
(Repeats indefinitely or crashes with 502 Bad Gateway)
Observed Logs (The Panic - if HttpConfig is removed to bypass check): If I remove the HttpConfig block to try and bypass the check, the service crashes immediately:
Plaintext
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x78 pc=0x195347e]
goroutine 1 [running]:
github.com/netbirdio/netbird/management/cmd.loadMgmtConfig(...)
Configuration (Traefik - VM1):
YAML
netbird-backend:
loadBalancer:
serversTransport: netbird-grpc-transport
servers:
- url: "h2c://192.168.X.X:80" # Points to VM2 internal IP
Configuration (Management.json - VM2):
JSON
"HttpConfig": {
"AuthIssuer": "https://netbird.example.com/oauth2",
"AuthAudience": "netbird-dashboard",
"OIDCConfigEndpoint": "https://netbird.example.com/oauth2/.well-known/openid-configuration"
},
"EmbeddedIdP": {
"Enabled": true,
"Issuer": "https://netbird.example.com/oauth2",
...
}
Workaround Used: I was forced to disable EmbeddedIdP and run a Standalone Dex container. This works perfectly because Dex starts independently, breaking the dependency loop.
The Request: While the Standalone Dex workaround fixes the crash, it causes the NetBird Dashboard to hide the new "Identity Providers" UI tab, which I specifically wanted to use for configuring Entra ID easily.
Fix: Can we add a flag (e.g., --skip-oidc-startup-check) to the Management service so it trusts the config without trying to curl itself?
Feature: Can we force-enable the "Identity Providers" UI menu even when using a custom/standalone IdP?Describe the problem
A clear and concise description of what the problem is.
To Reproduce
Steps to reproduce the behavior:
- Go to '...'
- Click on '....'
- Scroll down to '....'
- See error
Expected behavior
A clear and concise description of what you expected to happen.
Are you using NetBird Cloud?
Please specify whether you use NetBird Cloud or self-host NetBird's control plane.
NetBird version
netbird version
Is any other VPN software installed?
If yes, which one?
Debug output
To help us resolve the problem, please attach the following anonymized status output
netbird status -dA
Create and upload a debug bundle, and share the returned file key:
netbird debug for 1m -AS -U
Uploaded files are automatically deleted after 30 days.
Alternatively, create the file only and attach it here manually:
netbird debug for 1m -AS
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
Have you tried these troubleshooting steps?
- Reviewed client troubleshooting (if applicable)
- Checked for newer NetBird versions
- Searched for similar issues on GitHub (including closed ones)
- Restarted the NetBird client
- Disabled other VPN software
- Checked firewall settings