Skip to content

[procmgr] Supervise process-agent on Windows via dd-procmgr - #54735

Open
jose-manuel-almaza wants to merge 5 commits into
jose/procmgr-secret-backend-gatesfrom
jose/procmgr-windows-process-agent
Open

[procmgr] Supervise process-agent on Windows via dd-procmgr#54735
jose-manuel-almaza wants to merge 5 commits into
jose/procmgr-secret-backend-gatesfrom
jose/procmgr-windows-process-agent

Conversation

@jose-manuel-almaza

Copy link
Copy Markdown
Contributor

What does this PR do?

Moves Windows process-agent supervision to dd-procmgr using the same dual-mode pattern as PAR:

  • Fleet installer writes processes.d/datadog-agent-process.yaml (Privileged spawn profile)
  • Legacy datadog-process-agent SCM service is suppressed when procmgr owns process-agent
  • Agent startup waits on dd-procmgr-service before starting gated legacy children; independent services (sysprobe, security-agent, installer) start without blocking on procmgr

Also runs dd-procmgr-service as LocalSystem (MSI service custom action) so the supervisor can spawn Privileged children while agent-profile processes still spawn as ddagentuser.

Stack context: PR 4/4 split from #53249. Builds on #54734#54732#54731. Spawn profiles, config gates, and secret backend resolution land in those PRs.

Motivation

We want subservices on dd-procmgr instead of SCM. Process-agent needs LocalSystem on Windows; other agent children should stay on the agent user. This PR wires the product integration once the procmgr foundation (PRs 1–3) is in place.

Describe how you validated your changes

  • Go unit tests for dependent Windows service startup (dependent_services_windows_test.go)
  • Go unit tests for fleet installer templates and YAML path substitution (processmanager/...)
  • New Windows E2E: process-agent supervised by procmgr, runs as LocalSystem, legacy SCM stopped
  • E2E: dd-procmgr-service runs as LocalSystem; agent-profile children run as agent user
  • E2E: privileged spawn catalog enforcement when processes.d YAML is tampered with
  • PAR/procmgr Windows E2E regression

Additional Notes

  • Linux process-agent stays on systemd for now. Legacy SCM service registration is not removed yet.
  • Process-agent processes.d config uses the shared install_root helper (same as PAR/ADP). MSI RemoveFolderEx handles uninstall/rollback cleanup; no per-file MSI rollback custom actions.
  • Includes release note windows-process-procmgr-dual-mode-b7d2e4a1c8f03962.yaml.
  • Closes/supersedes #53249 once the full stack merges.

@jose-manuel-almaza jose-manuel-almaza added the qa/done QA done before merge and regressions are covered by tests label Aug 11, 2026
@jose-manuel-almaza
jose-manuel-almaza requested review from a team as code owners August 11, 2026 16:07
@dd-octo-sts dd-octo-sts Bot added the internal Identify a non-fork PR label Aug 11, 2026
@github-actions github-actions Bot added the long review PR is complex, plan time to review it label Aug 11, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7a9ca39dfe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +217 to +218
startServices(independent, false)
startServices(gated, <-procmgrWait)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cancel the gated starter during agent shutdown

When the Agent is stopped while dd-procmgr-service is still in the newly added startup wait, stopDependentServices can stop procmgr and the already-launched goroutine then receives false here and starts the gated legacy process/PAR/DDOT services after their shutdown pass has completed. This can race or prolong Windows service shutdown and leave dependent workloads starting as the core service exits; tie this work to the Agent lifecycle or wait for the starter before stopping dependents.

AGENTS.md reference: AGENTS.md:L190-L192

Useful? React with 👍 / 👎.

Comment on lines +484 to +486
func run[Env any](t *testing.T, s e2e.Suite[Env], systemProbeConfig string, agentConfig string, securityAgentConfig string) {
opts := []e2e.SuiteOption{e2e.WithProvisioner(awsHostWindows.ProvisionerNoFakeIntake(
awsHostWindows.WithRunOptions(runOpts...),
awsHostWindows.WithRunOptions(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the larger VM for Driver Verifier tests

In the checked new-e2e-windows-service-test matrix, all six Driver Verifier entries now reach this helper without the former t3.xlarge run option because the helper no longer accepts extra options. They therefore fall back to the framework's t3.large Windows default, undoing the explicit 4-vCPU mitigation for Driver Verifier's kernel overhead and making SCM deadline failures and CI flakes likely; retain the per-test instance override while keeping ordinary suites on the default.

AGENTS.md reference: test/new-e2e/tests/windows/service-test/AGENTS.md:L18-L21

Useful? React with 👍 / 👎.

@datadog-prod-us1-4

This comment has been minimized.

@clarkb7
clarkb7 requested review from clarkb7 and removed request for lrbhatia August 11, 2026 16:40
@jose-manuel-almaza
jose-manuel-almaza requested review from a team as code owners August 12, 2026 11:26

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

The Windows supervision changes keep legacy fallback behavior tied to a successful dd-procmgr startup, add the process-agent definition and preserve the existing configuration gate inputs. No reproducible diff-only behavioral regression was found; Windows-specific execution remains unvalidated because the sandbox lacks the required toolchain and cannot download it.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Datadog Autotest · Commit 7a9ca39 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from 7a9ca39 to 98a223d Compare August 12, 2026 11:52
@jose-manuel-almaza
jose-manuel-almaza removed request for a team August 12, 2026 11:53
@jose-manuel-almaza
jose-manuel-almaza requested a review from a team as a code owner August 12, 2026 12:13
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from 33449de to 43513c9 Compare August 14, 2026 10:28
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from 43513c9 to 13decb9 Compare August 14, 2026 10:46
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from 13decb9 to 3b5bc84 Compare August 14, 2026 11:01
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from 3b5bc84 to 7adf791 Compare August 14, 2026 11:20
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from 7adf791 to 67034f9 Compare August 14, 2026 13:51
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch 2 times, most recently from 2855d38 to b2a9ee3 Compare August 14, 2026 14:18
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from b2a9ee3 to a1a89c1 Compare August 14, 2026 14:19
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from a1a89c1 to c241a83 Compare August 14, 2026 14:34
jose-manuel-almaza added a commit that referenced this pull request Aug 17, 2026
Agent-profile spawn needs dd-procmgr-service as LocalSystem to read the
agent password from LSA. Re-add the test in #54735 with the Wix change.
jose-manuel-almaza added a commit that referenced this pull request Aug 17, 2026
Agent-profile spawn needs dd-procmgr-service as LocalSystem to read the
agent password from LSA. Re-add the test in #54735 with the Wix change.
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from a03bf2d to 0961a62 Compare August 17, 2026 08:53
jose-manuel-almaza added a commit that referenced this pull request Aug 17, 2026
…les PR

Restore procmgr_win_test.go to main and drop the extracted helpers file.
The only E2E delta on this PR is removing the PAR test for #54735.
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from 0961a62 to 433e146 Compare August 17, 2026 09:47
…it PR 4/4)

Wire dual-mode agent startup, fleet processes.d install for process-agent,
dd-procmgr-service LocalSystem credentials, and Windows E2E coverage. Completes
the process-agent migration off legacy SCM when procmgr owns supervision.
Add process-agent to COAT adoption telemetry alongside the Windows dual-mode
processes.d integration so management_mode and procmgr_process_running gauges
reflect process-agent supervision once fleet installer wiring is active.
Assert dd-procmgr-service runs as LocalSystem and agent-profile children
run as the agent user with a loaded user profile. Requires the MSI installer
to register dd-procmgr-service as LocalSystem (ServiceCustomAction in this PR).
Match process-agent and system-probe: the Wix service installer now uses
LocalSystem instead of ddagentuser. ServiceCustomAction already sets
LocalSystem at install time; this closes the initial-registration gap.
Re-add the test dropped from #54731. Agent-profile spawn for PAR requires
dd-procmgr-service as LocalSystem (Wix change in this PR) to read the
agent password from LSA.
@jose-manuel-almaza
jose-manuel-almaza force-pushed the jose/procmgr-windows-process-agent branch from 433e146 to b982f1b Compare August 17, 2026 13:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal Identify a non-fork PR long review PR is complex, plan time to review it qa/done QA done before merge and regressions are covered by tests team/agent-build team/agent-runtimes team/windows-products

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants