elastic-agent installed as Fleet server remains stuck in uninstalling with retry errors for over 4-5 minutes.

[amolnater-qasource]

Kibana Build details:

VERSION: 8.16.0 SNAPSHOT
BUILD: 78993
COMMIT: 6eb8471c3124046eca03cccf20e0cc4f9706bcd5

Artifact: https://snapshots.elastic.co/8.16.0-106cdbc2/downloads/beats/elastic-agent/elastic-agent-8.16.0-SNAPSHOT-windows-x86_64.zip

Host: Windows Server 2022- Test Signing ON

Preconditions:

1. 8.16.0 SNAPSHOT Kibana self-managed environment should be available.
2. Agent should be installed using Quickstart or Production mode of security.

Steps to reproduce:

1. Observe Fleet Server is installed.
2. Run .\Elastic\Agent\elastic-agent.exe uninstall.
3. Observe notify: Fleet network error[retry] error is stuck for more than 4 minutes.
4. Observe folder is already blank before the completion.

Expected Result:
elastic-agent installed as Fleet server should not remain stuck in uninstalling with retry errors for over 4-5 minutes.

Screenshots:

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[amolnater-qasource]

@muskangulati-qasource Please review.

[muskangulati-qasource]

Secondary review is Done for this ticket!

[cmacknz]

@michel-laterman this is probably from the uninstall notification, at minimum there should be better error messages here if Fleet Server was legitimately unavailable.

[amolnater-qasource]

Hi Team,
We have revalidated this issue on latest 8.16.0 BC1 kibana cloud environment and had below observations:

Observations:

• Multiple error logs are observed consistently on running uninstall command for installed fleet-server.

Screen Recording:

Amol.Self.WindowsBC.-.ec2-35-174-4-132.compute-1.amazonaws.com.-.Remote.Desktop.Connection.2024-10-24.11-22-29.mp4

Amol.Self.WindowsBC.-.ec2-35-174-4-132.compute-1.amazonaws.com.-.Remote.Desktop.Connection.2024-10-24.12-00-38.mp4

Build details:
VERSION: 8.16.0 BC1
BUILD: 79314
COMMIT: 5575428dd3aef69366cddb4ccf07a2a26d30ce48
Artifact Link: https://staging.elastic.co/8.16.0-e8d5928a/downloads/beats/elastic-agent/elastic-agent-8.16.0-windows-x86_64.zip

Hence, we are reopening this issue.

Thanks!!

[michel-laterman]

Hi @amolnater-qasource,

For the failures you posted:

Did you destroy the ES deployment before uninstalling the agent instance? If so the 1st instances failure messages are expected; it should have taken a shorter amount of time then 4m as originally reported. do you recall how long it took?

Is the panic that occured in the 2nd scenario common? Do you have steps on how to recreate? I would like to recreate this with a binary where the symbols are still present because the failure when trying to marshall json seems strange to me.

[amolnater-qasource]

Hi @michel-laterman

Thank you for looking into this issue.

Did you destroy the ES deployment before uninstalling the agent instance? If so the 1st instances failure messages are expected; it should have taken a shorter amount of time then 4m as originally reported. do you recall how long it took?

Please find below steps for this:

• Install Fleet Server on kibana cloud environment.
• Keep it running for sometime.
• Simply run uninstall command: C:\"Program Files"\Elastic\Agent\elastic-agent.exe uninstall
• Observe multiple log lines in the powershell.
• Note: ES deployment isn't destroyed.

The issue is reproducible for 8.17.0-SNAPSHOT agent.

Screen Recording:

Amol.Windows.2025.-.ec2-18-212-146-218.compute-1.amazonaws.com.-.Remote.Desktop.Connection.2024-11-19.15-32-00.mp4

Is the panic that occured in the 2nd scenario common? Do you have steps on how to recreate? I would like to recreate this with a binary where the symbols are still present because the failure when trying to marshall json seems strange to me.

This issue is inconsistently observed when VM is shut down for a long time and then agent is uninstalled.
This issue is reproduced multiple times, however no consistent steps are available.

Build details:
VERSION: 8.17.0 SNAPSHOT
BUILD: 80188
COMMIT: fdb16ae8cbdf4236db3696aa00d0bb98c943d864
Artifact Link: https://snapshots.elastic.co/8.17.0-0c51e25e/downloads/beats/elastic-agent/elastic-agent-8.17.0-SNAPSHOT-windows-x86_64.zip

Please let us know if anything else is required from our end.
Thanks!