Skip to content

v0.62.0 Embedded IdP Deadlock behind External Reverse Proxy (Two-VM Setup) & Missing IdP UI in Standalone Mode #5084

@Der-Papst

Description

@Der-Papst

I am setting up a self-hosted NetBird instance using the new Embedded IdP feature (introduced in v0.62.0). I have encountered a "chicken-and-egg" deadlock during the Management service startup because of my external reverse proxy architecture.

When EmbeddedIdP is enabled, the Management service attempts to validate its own OIDC configuration by making a network request to its public domain (https://netbird.example.com/...). This request travels out to my external proxy and loops back to the container. However, since the Management service is blocked waiting for this check to finish, it is not yet serving traffic. The proxy receives a "Connection Refused" or timeout, returns a 502 Bad Gateway to the Management service, and the service retries indefinitely or panics.

Architecture:

VM 1 (Proxy Host): Runs Traefik (External). Terminates SSL (443) and forwards traffic via H2C (HTTP/2 cleartext) to VM 2.

VM 2 (NetBird Host): Runs the NetBird Docker stack with a local Caddy acting as an internal router on Port 80.

Steps to Reproduce:

Configure management.json with "EmbeddedIdP": { "Enabled": true, ... }.

Set HttpConfig endpoints to the public domain (https://netbird.example.com/...).

Start the container (docker compose up).

Observe the boot loop in logs.

Observed Logs (The 502 Loop):

Plaintext

2026-01-10T12:19:56Z INFO [context: SYSTEM] management/cmd/management.go:157: loading OIDC configuration from the provided IDP configuration endpoint https://netbird.example.com/oauth2/.well-known/openid-configuration
Error: failed reading provided config file: /etc/netbird/management.json: failed fetching OIDC configuration from endpoint https://netbird.example.com/oauth2/.well-known/openid-configuration Get "https://netbird.example.com/oauth2/.well-known/openid-configuration": remote error: tls: internal error
...
(Repeats indefinitely or crashes with 502 Bad Gateway)
Observed Logs (The Panic - if HttpConfig is removed to bypass check): If I remove the HttpConfig block to try and bypass the check, the service crashes immediately:

Plaintext

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x78 pc=0x195347e]
goroutine 1 [running]:
github.com/netbirdio/netbird/management/cmd.loadMgmtConfig(...)
Configuration (Traefik - VM1):

YAML

netbird-backend:
  loadBalancer:
    serversTransport: netbird-grpc-transport
    servers:
      - url: "h2c://192.168.X.X:80" # Points to VM2 internal IP

Configuration (Management.json - VM2):

JSON

"HttpConfig": {
    "AuthIssuer": "https://netbird.example.com/oauth2",
    "AuthAudience": "netbird-dashboard",
    "OIDCConfigEndpoint": "https://netbird.example.com/oauth2/.well-known/openid-configuration"
},
"EmbeddedIdP": {
    "Enabled": true,
    "Issuer": "https://netbird.example.com/oauth2",
    ...
}

Workaround Used: I was forced to disable EmbeddedIdP and run a Standalone Dex container. This works perfectly because Dex starts independently, breaking the dependency loop.

The Request: While the Standalone Dex workaround fixes the crash, it causes the NetBird Dashboard to hide the new "Identity Providers" UI tab, which I specifically wanted to use for configuring Entra ID easily.

Fix: Can we add a flag (e.g., --skip-oidc-startup-check) to the Management service so it trusts the config without trying to curl itself?

Feature: Can we force-enable the "Identity Providers" UI menu even when using a custom/standalone IdP?Describe the problem

A clear and concise description of what the problem is.

To Reproduce

Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior

A clear and concise description of what you expected to happen.

Are you using NetBird Cloud?

Please specify whether you use NetBird Cloud or self-host NetBird's control plane.

NetBird version

netbird version

Is any other VPN software installed?

If yes, which one?

Debug output

To help us resolve the problem, please attach the following anonymized status output

netbird status -dA

Create and upload a debug bundle, and share the returned file key:

netbird debug for 1m -AS -U

Uploaded files are automatically deleted after 30 days.

Alternatively, create the file only and attach it here manually:

netbird debug for 1m -AS

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

Have you tried these troubleshooting steps?

  • Reviewed client troubleshooting (if applicable)
  • Checked for newer NetBird versions
  • Searched for similar issues on GitHub (including closed ones)
  • Restarted the NetBird client
  • Disabled other VPN software
  • Checked firewall settings

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions