chore(antithesis): Frechet Distance and intake rework - #2228
Conversation
dd-octo-sts
Bot
added
the
area/test
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
This comment has been minimized.
This comment has been minimized.
Binary Size Analysis (Agent Data Plane)Baseline: 8326377 · Comparison: 9253a9a · diff ✅ Binary size difference within thresholdChanges by Module
Detailed Symbol Changes |
Regression Detector (Agent Data Plane)Run ID: Optimization Goals: ✅ No significant changes detectedFine details of change detection per experiment (5)Experiments configured
Bounds Checks: ✅ Passed (5)
ExplanationA change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression ( |
ba9004d to
82feb2c
Compare
82feb2c to
8728f6c
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 82feb2c0b2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
8728f6c to
b25826a
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b25826a1cd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
b25826a to
352e269
Compare
98dc187 to
aaebc86
Compare
352e269 to
49cf71c
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 49cf71c7b1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
aaebc86 to
986783b
Compare
49cf71c to
84df7c4
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 84df7c4dd7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
986783b to
e607b72
Compare
84df7c4 to
42d9ed1
Compare
3fa6ad4 to
f0318e1
Compare
b7a894e to
7b80b84
Compare
f0318e1 to
32dd7ae
Compare
7b80b84 to
f96c13f
Compare
32dd7ae to
3be9402
Compare
57141a5 to
91e00cc
Compare
a69729f to
4d5058b
Compare
91e00cc to
335b960
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4d5058bdac
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
4d5058b to
0997c6e
Compare
335b960 to
a07c2b7
Compare
0997c6e to
ebe1b03
Compare
a07c2b7 to
9d4c44d
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ebe1b03f0d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
71edde4 to
87b8a69
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 87b8a698c8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
87b8a69 to
5c990e2
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5c990e26d8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bb145b60c6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8c9fb2439f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aa0978f3d2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4f62b802cd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return; | ||
| } | ||
| *weight += w; | ||
| *mean += (point.value - *mean) * (w / *weight); |
There was a problem hiding this comment.
When a bucket contains multiple positive-interval rate points with large opposite signs, this update still overflows in point.value - *mean; for example, both [f64::MAX, f64::MIN] and [f64::MAX, -f64::MAX / 2.0] fold to -inf even though their representable means are 0 and f64::MAX / 4.0. The workload explicitly samples both finite extrema, and distance treats matching infinities as agreement, so a real lane divergence can pass the differential assertion. Fresh evidence relative to the prior weighting comment is that the replacement running-mean formula removed multiplication overflow but retained subtraction overflow; rescale the operands or use another overflow-safe weighted-mean update.
AGENTS.md reference: test/antithesis/AGENTS.md:L46-L49
Useful? React with 👍 / 👎.
webern
left a comment
webern
left a comment
There was a problem hiding this comment.
Rubber stamp since it's all harness code 🕹️
This commit introduces a new protocol to the intake node to allow for context bounding in runs. The purposes are twofold. First, while it's interesting to explore timelines in which there are too many contexts for either lane to handle these are less relevant to me _now_, so I've capped total contexts at 1M. Second, in order for timeseries equality to _work_ I need to be certain that I have at least _some_ timeseries with more than one point emitted into them. Previously drivers were emitting essentially random contexts and it was very unlikely for more than one point to be emitted per context. Oops. Drivers now request N contexts from the intake service for each sub-kind of dogstatsd they emit, meaning all contexts are sourced from a single spot in a topology and not from pure randomness.
The essential property we want to determine is whether ADP-on/ADP-off emits payloads to intake API that the intake API rejects. This is especially of interest for inputs that we know are ultimately rejected by intake API -- non-utf8 bytes in the wrong spot -- but are _not_ rejected by the SUT. The old mechanism had a feral/clean 'vibe' which served for a while but was confusing to debug. Is feral malformed? Is feral well-formed but wild? Anyway I got tired of it. There's now a predicate which defines whether a payload is well-formed or not -- that is, accepted by ADP-off Datadog Agent -- independent of whether intake API ultimately accepts the payloads that ingress inspires. Later I will build a generator that only emits malformed ingress but that is only hinted at in this work.
Pyld54 asserts a v3 tagset back-reference resolves to an earlier tagset. Agent emits that back-reference only when a metric carries both a shared prefix tagset and its own tags. Previously we only ever emitted load that carried no shared prefix. This commit adds provider_kind into the sampled datadog.yaml. Agent will attach this as a global tag which acts as a shared prefix. 1/3 of configs will not have provider_kind present.
This commit introduces the Frechet distance calculation into the differential scenario, allowing us to determine if ADP-on and ADP-off emit equivalent series for all input sequences. The distance calculation has been chosen to be mathematically defensible and also to have the right visual check for our latent understanding of what 'equivalent' series look like in practice.
2 participants
Summary
This commit introduces the Frechet distance calculation into the
differential scenario, allowing us to determine if ADP-on and ADP-off
emit equivalent series for all input sequences. The distance calculation
has been chosen to be mathematically defensible and also to have the
right visual check for our latent understanding of what 'equivalent' series
look like in practice.
Change Type
How did you test this PR?
References