Resources extreamly slow but connection estabilished and stable (Gerbil <-> Newt)

[szszl0]

Describe the Bug

I recently moved from free Pangolin Cloud to selfhosted Pangolin on my VPS. I have exactly same public resources setup as i had before. Newt site is also installed exactly the same way as it was when i was using Pangolin Cloud - in an LXC container as systemd service inside my Proxmox home server. My home is behind cgnat without ipv6 but it has not been an issue with Pangolin Cloud. My VPS is both ipv4 and v6 capable.
Network speed and latency are flawless on both VPS and home server. Resources "work" in a way that they are loading eventually but it takes from 10 to about 60 seconds... Exposed services arent a problem - they work fast when i connect to them using local network or tailscale vpn.
So long story short - exactly same setup worked great on Pangolin Cloud but sucks on my VPS. Any ideas why? VPS's CPU and memory are fine during resource loading.
and
Pangolin dashboard itself loads super fast, in less than a second. Only the resources are slow. For example Home Assistant dashboard is loading 10-15s, Immich loads in 10s but uploading one photo takes few minutes. Gotify and Termix load much longer than 60s and sometimes dont load at all (it loads the service ui background and sometimes some ui elements but doesnt load data at all)

VPSs bandwidth and speed are fine, it is located near to my home.

I suspect the problem is between Gerbil and Newt but i dont know why as both services are installed and configured exactly as described in docs (with default settings)

Environment

• OS Type & Version: Debian 13
• Pangolin Version: latest (currently 1.17.1)
• Gerbil Version: 1.3.1
• Traefik Version: 3.6
• Newt Version: 1.11

To Reproduce

just install Pangolin exactly the way described in quick start guide. Everything default. Site connected, domain configured. Newt running on my homelab as a systemd service. Public resources configured, healthchecks healthy. Click resource link.

Expected Behavior

I expected my resources to open and work. Instead they load very slowly or eventually end up "half loaded" (only some ui elements)

[szszl0]

here are some test i ran with ChatGPT

Environment

• Self-hosted setup using:
  • VPS with Docker (Pangolin, Gerbil, Traefik)
  • Home server behind NAT
  • WireGuard tunnel (Gerbil ↔ Newt LXC)
• Domain-based routing via reverse proxy
• Backend services hosted on LAN (e.g. 192.168.x.x:8080)

---

🧪 Docker Environment

CONTAINER ID IMAGE NAME
... traefik:v3.6 traefik
... fosrl/gerbil:1.3.1 gerbil
... fosrl/pangolin:ee-1.17.1 pangolin
... henrygd/beszel-agent beszel-agent

---

🧪 Network / Interface State

Docker + Gerbil networking

Gateway: 172.18.0.1
IPAddress: 172.18.0.3
IPv6: fdbc:xxxx::3
Docker MTU: 1500
wg0 MTU (inside container): 1280

---

🧪 MTU / Packet Test

ping -M do -s 1472 1.1.1.1 → 100% packet loss
ping -M do -s 1400 1.1.1.1 → 100% packet loss

---

🧪 Tunnel Performance

curl http://127.0.0.1/

connect: ~0.03s
total: ~0.04s

---

🧪 Backend (Home Server)

curl http://192.168.x.x:8080

HTTP/1.1 200 OK
Server: nginx/1.22.1
Content-Type: text/html
Content-Length: 1817

---

🧪 Notes Service (Working Case)

External access

curl -L https://notes/
.

HTTP/2 200
content-type: text/html; charset=utf-8
x-powered-by: Next.js

Headers

cache-control: s-maxage=31536000
x-nextjs-cache: HIT
x-nextjs-prerender: 1

Local proxy test

curl http://127.0.0.1/
-H "Host: notes."

HTTP/1.1 302 Found
Location: https://notes/
./

Assets

/favicon.ico → HTTP/2 200
/manifest.json → HTTP/2 200
/assets/ → HTTP/2 308 → /assets

---

🧪 SSH Service (Problem Case)

Root request

curl -s -D - https://ssh/
.

HTTP/2 302
Location:
https:///auth/resource/?redirect=https://ssh/
./

Assets

/favicon.ico → HTTP/2 302 → /auth/resource/...
/manifest.json → HTTP/2 302 → /auth/resource/...
/assets/ → HTTP/2 302 → /auth/resource/...

Local proxy test

curl -H "Host: ssh." http://127.0.0.1/

HTTP/1.1 302 Found
Location: https://ssh/
./

---

🧪 TLS / HTTP Details

TLSv1.3 handshake: OK
HTTP/2: enabled
ALPN: h2
Certificates: valid (Let's Encrypt)

---

🧪 Timing

connect: ~0.02s
start: ~0.11s
total: ~0.11s

---

📌 Summary of Observed Behavior

• Notes service:

  • Returns HTTP 200
  • Serves Next.js frontend correctly
  • Assets load successfully

• SSH service:

  • Returns HTTP 302 redirects
  • Redirects to Pangolin auth resource endpoint
  • All paths (/, assets, static files) are intercepted

• Tunnel and backend:

  • WireGuard tunnel stable
  • Home backend services respond correctly on LAN
  • No packet loss observed after MTU adjustment

---

[AstralDestiny]

More concerned if you're getting 100% packet loss seems whatever that vps is doing when it needs to do some actual work,

Not sure what curling 127.0.0.1 is going to do as a test honestly.. traefik doesn't route based on ip alone and if you wanted to curl against a resource you'd need to find the wireguard ip given to it via like docker exec pangolin curl http://localhost:3001/api/v1/traefik-config | jq

Then like, #1464 (comment) and see if you just have something live within the stack alongside gerbil and traefik if it can reach the resource properly.

Backend test doesn't really need to be censored,

Ssh isn't a http service though? unless that's hosting a thing like guac or kasm or so.. 302 doesn't say much that's just a redirect, would've been nice with curl -L to follow that 302 and see if it chokes then and your local proxy test are you doing vps > newt > local reverse proxy > service? or newt to local rp?

[szszl0]

yeah, in terms of VPS resources - it is exactly as stated as recommended in Pangolin docs. The CPU usage rarely goes above 25%, memory is somewhere in between 45-60% usually (90% if you count cached)

i am not sure how to follow the thing form linked comment. First of all i dont run Newt in docker so i assume just run "iperf3 -s" in Newt's LXC? Then the test resource i have no idea what should be set up, i did:

(192.168.0.150 is Newt's LXC local ip)

then i found this "test" resource ip: "http://100.89.128.4:62140"
then ran on my VPS: sudo docker run -it --network container:gerbil --rm debian bash -c "apt-get update && apt-get install -y iperf3 && iperf3 -c 100.89.128.4 -p 62140 -t 700"

output on VPS:

`Connecting to host 100.89.128.4, port 62140
[  5] local 100.89.128.1 port 60026 connected to 100.89.128.4 port 62140
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  5.00 MBytes  41.9 Mbits/sec    2   1.20 KBytes       
[  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   1.20 KBytes       
[  5]   2.00-3.00   sec  2.12 MBytes  17.8 Mbits/sec    1   1.68 MBytes       
[  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    0   1.68 MBytes       
[  5]   4.00-5.00   sec  3.50 MBytes  29.3 Mbits/sec    1   1.76 MBytes       
[  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   1.76 MBytes       
[  5]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec  164   1.23 MBytes       
[  5]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0   1.23 MBytes       
[  5]   8.00-9.00   sec  3.00 MBytes  25.2 Mbits/sec    1    783 KBytes       
[  5]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0    783 KBytes       
[  5]  10.00-11.00  sec  0.00 Bytes  0.00 bits/sec    0    783 KBytes       
[  5]  11.00-12.00  sec  1.38 MBytes  11.5 Mbits/sec    1   1.24 MBytes       
[  5]  12.00-13.00  sec  0.00 Bytes  0.00 bits/sec    0   1.24 MBytes       
[  5]  13.00-14.00  sec  1.25 MBytes  10.5 Mbits/sec    1   1.31 MBytes       
[  5]  14.00-15.00  sec  0.00 Bytes  0.00 bits/sec    0   1.31 MBytes       
[  5]  15.00-16.00  sec  1.62 MBytes  13.6 Mbits/sec    1   1.37 MBytes       
[  5]  16.00-17.00  sec  0.00 Bytes  0.00 bits/sec    0   1.37 MBytes       
[  5]  17.00-18.00  sec  0.00 Bytes  0.00 bits/sec    0   1.37 MBytes       
     [...]
[  5]  97.00-98.00  sec  0.00 Bytes  0.00 bits/sec    0    621 KBytes       
[  5]  98.00-99.00  sec  1.38 MBytes  11.5 Mbits/sec    1    458 KBytes       
[  5]  99.00-100.00 sec  0.00 Bytes  0.00 bits/sec    0    458 KBytes       
[  5] 100.00-101.00 sec  0.00 Bytes  0.00 bits/sec    0    458 KBytes       
[  5] 101.00-102.00 sec  0.00 Bytes  0.00 bits/sec    1    496 KBytes       
[  5] 102.00-103.00 sec  0.00 Bytes  0.00 bits/sec    0    496 KBytes       
[  5] 103.00-104.00 sec  1.62 MBytes  13.6 Mbits/sec    1    526 KBytes       
[  5] 104.00-105.00 sec  0.00 Bytes  0.00 bits/sec    0    526 KBytes       
[  5] 105.00-106.00 sec  0.00 Bytes  0.00 bits/sec    0    526 KBytes       
[  5] 106.00-107.00 sec  0.00 Bytes  0.00 bits/sec    1    571 KBytes       
[  5] 107.00-108.00 sec  0.00 Bytes  0.00 bits/sec    0    571 KBytes       
[  5] 108.00-109.00 sec  1.50 MBytes  12.6 Mbits/sec    1    618 KBytes       
[  5] 109.00-110.00 sec  0.00 Bytes  0.00 bits/sec    0    618 KBytes       
[  5] 110.00-111.00 sec  0.00 Bytes  0.00 bits/sec    0    618 KBytes       
[  5] 111.00-112.00 sec  0.00 Bytes  0.00 bits/sec    1    648 KBytes       
[  5] 112.00-113.00 sec  0.00 Bytes  0.00 bits/sec    0    648 KBytes       
[  5] 113.00-114.00 sec  1.75 MBytes  14.7 Mbits/sec    1    705 KBytes       
[  5] 114.00-115.00 sec  0.00 Bytes  0.00 bits/sec    0    705 KBytes       
[  5] 115.00-116.00 sec  0.00 Bytes  0.00 bits/sec    0    705 KBytes       
^C[  5] 116.00-116.81 sec  0.00 Bytes  0.00 bits/sec    1    758 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-116.81 sec  81.5 MBytes  5.85 Mbits/sec  1228            sender
[  5]   0.00-116.81 sec  0.00 Bytes  0.00 bits/sec                  receiver
iperf3: interrupt - the client has terminated`

and on Newt LXC:

`Accepted connection from 192.168.0.150, port 45986
[  5] local 192.168.0.150 port 5201 connected to 192.168.0.150 port 45990
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  3.12 MBytes  26.2 Mbits/sec                  
[  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec                  
[  5]   2.00-3.00   sec  1.88 MBytes  15.7 Mbits/sec                  
[  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec                  
[  5]   4.00-5.00   sec  2.88 MBytes  24.1 Mbits/sec                  
[  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec                  
[  5]   6.00-7.00   sec  1.88 MBytes  15.7 Mbits/sec                  
[  5]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec                  
[  5]   8.00-9.00   sec   512 KBytes  4.19 Mbits/sec                  
[  5]   9.00-10.00  sec   512 KBytes  4.19 Mbits/sec                  
[  5]  10.00-11.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  11.00-12.00  sec  1.62 MBytes  13.6 Mbits/sec                  
[  5]  12.00-13.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  13.00-14.00  sec  1.50 MBytes  12.6 Mbits/sec                  
[  5]  14.00-15.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  15.00-16.00  sec  1.75 MBytes  14.7 Mbits/sec                  
[  5]  16.00-17.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  17.00-18.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  18.00-19.00  sec  1.25 MBytes  10.5 Mbits/sec                  
[  5]  19.00-20.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  20.00-21.00  sec   896 KBytes  7.34 Mbits/sec                  
[  5]  21.00-22.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  22.00-23.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  23.00-24.00  sec  1.88 MBytes  15.7 Mbits/sec                  
[  5]  24.00-25.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  25.00-26.00  sec  1.25 MBytes  10.5 Mbits/sec                  
[  5]  26.00-27.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  27.00-28.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  28.00-29.00  sec   896 KBytes  7.34 Mbits/sec                  
[  5]  29.00-30.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  30.00-31.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  31.00-32.00  sec  1.12 MBytes  9.44 Mbits/sec                  
[  5]  32.00-33.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  33.00-34.00  sec  0.00 Bytes  0.00 bits/sec                  
[  5]  34.00-35.00  sec  1.12 MBytes  9.43 Mbits/sec                  
[...]       
[  5] 101.00-102.00 sec   768 KBytes  6.30 Mbits/sec                  
[  5] 102.00-103.00 sec  0.00 Bytes  0.00 bits/sec                  
[  5] 103.00-104.00 sec   640 KBytes  5.24 Mbits/sec                  
[  5] 104.00-105.00 sec  0.00 Bytes  0.00 bits/sec                  
[  5] 105.00-106.00 sec  0.00 Bytes  0.00 bits/sec                  
[  5] 106.00-107.00 sec   896 KBytes  7.34 Mbits/sec                  
[  5] 107.00-108.00 sec  0.00 Bytes  0.00 bits/sec                  
[  5] 108.00-109.00 sec  1.00 MBytes  8.39 Mbits/sec                  
[  5] 109.00-110.00 sec  0.00 Bytes  0.00 bits/sec                  
[  5] 110.00-111.00 sec  0.00 Bytes  0.00 bits/sec                  
[  5] 111.00-112.00 sec   768 KBytes  6.29 Mbits/sec                  
[  5] 112.00-113.00 sec  0.00 Bytes  0.00 bits/sec                  
[  5] 113.00-114.00 sec  1.25 MBytes  10.5 Mbits/sec                  
[  5] 114.00-115.00 sec  0.00 Bytes  0.00 bits/sec                  
[  5] 115.00-116.00 sec  0.00 Bytes  0.00 bits/sec                  
[  5] 115.00-116.00 sec  0.00 Bytes  0.00 bits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-116.00 sec  79.9 MBytes  5.78 Mbits/sec                  receiver
iperf3: the client has terminated`

Shortened the outputs a little bit with ([...])

i didnt actually proceed to the last step:

"Now go about your use with what ever you are using if you see huge dips on iperf3 then it's something choking, A dip down by a few Mbps isn't an issue but huge dips you know them when you see them."

because there is nothing to use, my resources do not load.

Sorry if i did something wrong, i have no idea what i am doing

[AstralDestiny]

Well you're not even getting consistent speed over the tunnel at all so either something on the vps or maybe the vps provider itself is overprosvioned or so not really sure be curious if it's just between the two locations or if I run a site here do you get the same slow or non existant anywhere..

[szszl0]

I will open a ticket at my VPS provider and keep you updated

[AstralDestiny]

Mind throwing me a site config and see if it's just between me and your vps or whatnot? In slack or discord or does github have dm's? I'm not fully sure I've been responding to you from discord.

[szszl0]

Yeah sure. But i am not quite sure what do you mean by site config. Should i create new site and send you the credemtials to it?

[AstralDestiny]

Yep