-
-
Notifications
You must be signed in to change notification settings - Fork 89
Description
Describe the bug
This is going to be the most specific bug ever, but our ISP won't let us hit HTTPS services at all. A lot of stuff in TGS starts breaking when it cant reach HTTPS services and I feel like there's not enough error handling for specifically this.
This problem only started happening when we couldn't reach https services.
To Reproduce
Steps to reproduce the behavior:
- On your firewall, drop all requests outgoing to 443
- Run Monkestation
- Let the game server reboot normally
- dd.exe gets killed (thats normal) but never restarts and TGS shows it alive.
Expected behavior
Server to restart normally
Logs
Please include full server logs to help diagnose your problem
2025-05-25T03:35:15.3120009-07:00
Instance:12|Job:11843|Request:|User:|Monitor:138|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Components.Session.SessionPersistor: Saved reattach information: "Session: 7533, PID: 15856, Access Identifier m5pZRxiEONhgGpcU2ffbbrArodsuIboE2BiPH//f, RebootState: Normal, Port: 1541" ()
2025-05-25T03:35:15.3120404-07:00
Instance:12|Job:11843|Request:|User:|Monitor:138|Bridge:|Chat:|IR:): [INF] Tgstation.Server.Host.Components.Watchdog.WindowsWatchdog: Watchdog starting session ID 7533 ()
2025-05-25T03:35:16.4727078-07:00
Instance:12|Job:11843|Request:|User:|Monitor:138|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Components.Session.SessionPersistor: Updated reattach information: "Session: 7533, PID: 15856, Access Identifier m5pZRxiEONhgGpcU2ffbbrArodsuIboE2BiPH//f, RebootState: Normal, Port: 1541" ()
2025-05-25T03:35:16.4727403-07:00
Instance:12|Job:11843|Request:|User:|Monitor:138|Bridge:|Chat:|IR:): [INF] Tgstation.Server.Host.Components.Watchdog.WindowsWatchdog: Controller(s) initialized successfully ()
2025-05-25T03:35:16.4727667-07:00
Instance:12|Job:11843|Request:|User:|Monitor:138|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Components.Watchdog.WindowsWatchdog: Relaunch successful, resuming monitor... ()
2025-05-25T03:38:34.7152297-07:00
Instance:15|Job:11886|Request:|User:|Monitor:|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Jobs.JobProgressReporter: Job progress reporter created. Stage: "(null)" ()
2025-05-25T03:38:34.7152329-07:00
Instance:15|Job:11886|Request:|User:|Monitor:|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Jobs.JobProgressReporter: Job progress reporter created. Stage: "(null)" ()
2025-05-25T03:38:35.3089625-07:00
Instance:17|Job:11887|Request:|User:|Monitor:|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Jobs.JobProgressReporter: Job progress reporter created. Stage: "(null)" ()
2025-05-25T03:38:35.3090345-07:00
Instance:17|Job:11887|Request:|User:|Monitor:|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Jobs.JobProgressReporter: Job progress reporter created. Stage: "(null)" ()
2025-05-25T03:38:35.8717826-07:00
Instance:1|Job:|Request:|User:|Monitor:|Bridge:9|Chat:|IR:): [INF] Tgstation.Server.Host.Components.Session.SessionController: Bridge requested process termination! ()
2025-05-25T03:39:21.2115829-07:00
Instance:1|Job:11841|Request:|User:|Monitor:141|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Components.Session.SessionController: Not sending topic request HealthCheck to server that is rebooting/starting. ()
2025-05-25T03:39:21.2115847-07:00
Instance:1|Job:11841|Request:|User:|Monitor:141|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Components.Watchdog.WindowsWatchdog: DEFCON 4: Game server missed first health check! ()
2025-05-25T03:40:21.2112772-07:00
Instance:1|Job:11841|Request:|User:|Monitor:142|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Components.Session.SessionController: Not sending topic request HealthCheck to server that is rebooting/starting. ()
2025-05-25T03:40:21.2112852-07:00
Instance:1|Job:11841|Request:|User:|Monitor:142|Bridge:|Chat:|IR:): [INF] Tgstation.Server.Host.Components.Watchdog.WindowsWatchdog: DEFCON 3: Game server has missed 2 health checks! ()
2025-05-25T03:41:21.2120048-07:00
Instance:1|Job:11841|Request:|User:|Monitor:143|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Components.Session.SessionController: Not sending topic request HealthCheck to server that is rebooting/starting. ()
2025-05-25T03:41:21.2120332-07:00
Instance:1|Job:11841|Request:|User:|Monitor:143|Bridge:|Chat:|IR:): [WRN] Tgstation.Server.Host.Components.Watchdog.WindowsWatchdog: DEFCON 2: Game server has missed 3 health checks! If it does not respond to the next one, the watchdog will "be restarted"! ()
2025-05-25T03:42:21.2127418-07:00
Instance:1|Job:11841|Request:|User:|Monitor:144|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Components.Session.SessionController: Not sending topic request HealthCheck to server that is rebooting/starting. ()
2025-05-25T03:42:21.2129013-07:00
Instance:1|Job:11841|Request:|User:|Monitor:144|Bridge:|Chat:|IR:): [WRN] Tgstation.Server.Host.Components.Watchdog.WindowsWatchdog: DEFCON 1: Four health checks have been missed! "Restarting"... ()
2025-05-25T03:42:21.2129457-07:00
Instance:1|Job:11841|Request:|User:|Monitor:144|Bridge:|Chat:|IR:): [DBG] Tgstation.Server.Host.Components.Watchdog.WindowsWatchdog: DumpOnHealthCheckRestart enabled. ()
2025-05-25T03:42:21.2146993-07:00
Instance:1|Job:11841|Request:|User:|Monitor:144|Bridge:|Chat:|IR:): [INF] Tgstation.Server.Host.Components.Watchdog.WindowsWatchdog: Dumping session to "C:\SS13\Monkestation-MRP1\Diagnostics\ProcessDumps\DreamDaemon-20250525104221.dmp"... ()
2025-05-25T03:42:21.2154010-07:00
Instance:1|Job:11841|Request:|User:|Monitor:144|Bridge:|Chat:|IR:): [WRN] Tgstation.Server.Host.Components.Watchdog.WindowsWatchdog: Creating dump failed! ()
Tgstation.Server.Host.Jobs.JobException: Cannot perform this operation as the game server is not currently running!
at Tgstation.Server.Host.System.WindowsProcessFeatures.CreateDump(Process process, String outputFile, Boolean minidump, CancellationToken cancellationToken) in D:\a\tgstation-server\tgstation-server\src\Tgstation.Server.Host\System\WindowsProcessFeatures.cs:line 141
at Tgstation.Server.Host.Components.Watchdog.WatchdogBase.CreateDumpNoLock(CancellationToken cancellationToken) in D:\a\tgstation-server\tgstation-server\src\Tgstation.Server.Host\Components\Watchdog\WatchdogBase.cs:line 1308
at Tgstation.Server.Host.Components.Watchdog.WatchdogBase.HandleHealthCheck(CancellationToken cancellationToken) in D:\a\tgstation-server\tgstation-server\src\Tgstation.Server.Host\Components\Watchdog\WatchdogBase.cs:line 1204
Server State: (please complete the following information):
- OS: Windows Server 2022
- Version: v6.17.0
- Database Type/Version: MariaDB 11.7
- BYOND Version Used: 516.1661
- git Repository Used: https://github.com/monkestation/monkestation2.0
- Origin Commit hash Used: d8abe48d3e159e3f199d0f390eacd3946307842e
- Active Test Merges: #6780, #6648, #6710, #3867, #4807, #6569, #6615, #6704, #6714, #6759, #6771, #6781, #6783, #6800, #6763, #6802
- Client Version: ??
Additional context
Our ISP is blocking outgoing requests with a target port of 443 since we've recently been hit with a DDOS.
Another smaller issue, we can't choose a preinstalled BYOND version via the engine page because BYOND is also, under a DDOS attack so it just gets stuck on Loading BYOND Version Information...