Skip to content

Updated osquery perf for #45550 #48935

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
getvictor wants to merge 3 commits into
base: main
Choose a base branch
from
+71 −3
Open

Updated osquery perf for #45550 #48935

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
328489c
Updated osquery perf for #45550
getvictor Jul 8, 2026
8f76ee9
Review fixes
getvictor Jul 8, 2026
66d8ddd
Review fixes
getvictor Jul 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions cmd/osquery-perf/agent.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -547,8 +547,15 @@ type agent struct {
// cache of this host's per-host certificates (the certs unique to this host, excluding the shared certs every host
// reports). Note that this requires a mutex even though only used in a.processQuery, that's because both the runLoop
// and the live query goroutines may call DistributedWrite (which calls processQuery).
certificatesMutex sync.RWMutex
hostCertSpecs []simulatedCert
certificatesMutex sync.RWMutex
hostCertSpecs []simulatedCert
// scepCertSpecs holds the certs issued to this host during Windows MDM SCEP exchanges, keyed by the SCEP CSP
// unique ID so a re-issued cert replaces its predecessor.
scepCertSpecs map[string]simulatedCert

@ksykulev ksykulev Jul 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a stylistic thing, but the pattern I have seen with other maps is to use make to initialize them in newAgent in

agent := &agent{
...
scepCertSpecs: make(map[string]simulatedCert)
}

// withholdSCEPCert marks ~5% of hosts that never report their issued SCEP certs, exercising the server's SCEP
// verification backstop (test-only failure).
withholdSCEPCert bool

commonSoftwareNameSuffix string

entraIDDeviceID string
Expand Down Expand Up @@ -727,6 +734,8 @@ func newAgent(
cachedLastOpenedAt: make(map[string]*time.Time),
commonSoftwareNameSuffix: commonSoftwareNameSuffix,
mdmProfileFailureProb: mdmProfileFailureProb,
// Every 20th host (5%) withholds its issued SCEP certs to exercise the server's verification backstop (test-only failure).
withholdSCEPCert: agentIndex%20 == 0,
Comment on lines +737 to +738

@coderabbitai coderabbitai Bot Jul 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Inspect the relevant file and nearby context.
git ls-files cmd/osquery-perf/agent.go
wc -l cmd/osquery-perf/agent.go
sed -n '680,780p' cmd/osquery-perf/agent.go

# Find definitions and call sites for the relevant identifiers.
rg -n "withholdSCEPCert|agentIndex|hostIndexOffset|newAgent\(" cmd/osquery-perf/agent.go cmd/osquery-perf -S

Repository: fleetdm/fleet

Length of output: 8286

🏁 Script executed:

#!/bin/bash
set -euo pipefail

git ls-files cmd/osquery-perf/agent.go
wc -l cmd/osquery-perf/agent.go
sed -n '680,780p' cmd/osquery-perf/agent.go

rg -n "withholdSCEPCert|agentIndex|hostIndexOffset|newAgent\(" cmd/osquery-perf/agent.go cmd/osquery-perf -S

Repository: fleetdm/fleet

Length of output: 8286

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Show the newAgent signature and the call sites that pass hostIndexOffset / agentIndex.
rg -n "func newAgent|newAgent\(" cmd/osquery-perf/agent.go cmd/osquery-perf -S -A 4 -B 4

# Show any calculations that define agentIndex or hostIndexOffset in the same file.
rg -n "hostIndexOffset|agentIndex" cmd/osquery-perf/agent.go -S -A 3 -B 3

Repository: fleetdm/fleet

Length of output: 7544

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Read only the relevant slice with line numbers.
nl -ba cmd/osquery-perf/agent.go | sed -n '700,760p'

# Show the declaration/call site context for hostIndexOffset and agentIndex.
rg -n "hostIndexOffset|agentIndex|withholdSCEPCert" cmd/osquery-perf/agent.go -n -A 6 -B 6

Repository: fleetdm/fleet

Length of output: 191

🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
p = Path("cmd/osquery-perf/agent.go")
lines = p.read_text().splitlines()
for start, end in [(2168,2208),(3884,3924),(728,742)]:
    print(f"\n--- {start}-{end} ---")
    for i in range(start, end+1):
        if i <= len(lines):
            print(f"{i:4d}: {lines[i-1]}")
PY

Repository: fleetdm/fleet

Length of output: 4188

🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
p = Path("cmd/osquery-perf/agent.go")
lines = p.read_text().splitlines()

for start, end in [(3648, 3672), (3880, 3935), (2188, 2210)]:
    print(f"\n--- {start}-{end} ---")
    for i in range(start, end + 1):
        if i <= len(lines):
            print(f"{i:4d}: {lines[i-1]}")
PY

Repository: fleetdm/fleet

Length of output: 5219

Base the withholding cohort on the global host index. agentIndex is 1-based, so agentIndex%20 == 0 only withholds on each shard’s local 20th agent. Use (hostIndexOffset + agentIndex - 1) % 20 == 0 so the 5% cohort is consistent across distributed runs.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cmd/osquery-perf/agent.go` around lines 737 - 738, The withholding cohort is
currently derived from agentIndex, which only selects every shard’s local 20th
agent instead of a consistent global 5% sample. Update the withholdSCEPCert
assignment in agent.go to base the decision on the global host index using
hostIndexOffset plus agentIndex, so the test-only failure cohort is stable
across distributed runs. Use the existing hostIndexOffset and agentIndex symbols
in the same host-generation logic to locate and adjust the condition.


entraIDDeviceID: uuid.NewString(),
entraIDUserPrincipalName: fmt.Sprintf("fake-%s@example.com", randomString(5)),
Expand Down Expand Up @@ -1624,6 +1633,16 @@ func (a *agent) doWindowsMDMCheckIn(onDemand bool) (newPollInterval time.Duratio
continue
}
a.stats.IncrementMDMSCEPSuccess()
if res.Cert == nil {
continue
}
// Report the issued cert via the certificates detail query so the server observes the
// fleet-<profileUUID> renewal-ID marker and marks the SCEP profile verified. The withheld ~5% never
// report theirs, so the server's verification backstop fails those profiles.
if a.withholdSCEPCert {
continue
}
a.storeSCEPCertSpec(res.UniqueID, res.Cert)
}
}()
} else {
Expand Down
51 changes: 50 additions & 1 deletion cmd/osquery-perf/certificates.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@ package main

import (
"crypto/sha1" //nolint:gosec
"crypto/x509"
"encoding/hex"
"fmt"
"maps"
// osquery-perf shares one global math/rand RNG seeded from the --seed flag so load-test runs are reproducible
"math/rand" //nolint:depguard
"slices"
"strings"
"time"

Expand Down Expand Up @@ -151,12 +153,59 @@ func (a *agent) generateCertSpecs() []simulatedCert {
a.churnPerHostCertSpecs()
}

specs := make([]simulatedCert, 0, len(sharedCerts)+len(a.hostCertSpecs))
specs := make([]simulatedCert, 0, len(sharedCerts)+len(a.hostCertSpecs)+len(a.scepCertSpecs))
specs = append(specs, sharedCerts...)
specs = append(specs, a.hostCertSpecs...)
// Include the certs issued via Windows MDM SCEP exchanges (never churned; replaced only on re-issuance). Sorted
// by CSP unique ID so the report order is stable across refreshes.
for _, id := range slices.Sorted(maps.Keys(a.scepCertSpecs)) {
specs = append(specs, a.scepCertSpecs[id])
}
return specs
}

// storeSCEPCertSpec records a certificate issued during a Windows MDM SCEP exchange
func (a *agent) storeSCEPCertSpec(uniqueID string, cert *x509.Certificate) {
a.certificatesMutex.Lock()
defer a.certificatesMutex.Unlock()
if a.scepCertSpecs == nil {
a.scepCertSpecs = make(map[string]simulatedCert)
}
a.scepCertSpecs[uniqueID] = scepIssuedCertSpec(cert)
}

// scepIssuedCertSpec converts a certificate issued during a Windows MDM SCEP exchange into a simulatedCert. The
// cert's subject carries the fleet-<profileUUID> renewal-ID marker (expanded from the profile's
// $FLEET_VAR_SCEP_RENEWAL_ID), which the server matches on ingestion to flip the SCEP profile to verified. Reported
// machine-scoped: osquery-perf drives SCEP CSPs on the device channel, so the cert lands in the LocalMachine store.
func scepIssuedCertSpec(cert *x509.Certificate) simulatedCert {
return simulatedCert{
commonName: cert.Subject.CommonName,
subjectCommonName: cert.Subject.CommonName,
subjectOrg: firstOrEmpty(cert.Subject.Organization),
// Join multiple OUs with "+OU=", mirroring how osquery reports multi-OU certs
subjectOrgUnit: strings.Join(cert.Subject.OrganizationalUnit, "+OU="),
subjectCountry: firstOrEmpty(cert.Subject.Country),
issuerCommonName: cert.Issuer.CommonName,
issuerOrg: firstOrEmpty(cert.Issuer.Organization),
issuerCountry: firstOrEmpty(cert.Issuer.Country),
keyAlgorithm: "rsaEncryption",
keyStrength: "2048",
keyUsage: "Key Encipherment, Digital Signature",
signingAlgorithm: cert.SignatureAlgorithm.String(),
serial: cert.SerialNumber.String(),
notValidAfterUnix: fmt.Sprint(cert.NotAfter.Unix()),
notValidBeforeUnix: fmt.Sprint(cert.NotBefore.Unix()),
}
}

func firstOrEmpty(vals []string) string {
if len(vals) == 0 {
return ""
}
return vals[0]
}

// newPerHostCertSpecs generates 0-10 certificates unique to this host
func (a *agent) newPerHostCertSpecs() []simulatedCert {
count := rand.Intn(11) // 0..10
Expand Down
Loading