Skip to content

Conversation

@anxolin
Copy link

@anxolin anxolin commented Dec 5, 2025

Disclaimer: I implemented this feature mostly using claude, so please carefully review. The logic fills good now to me. Also, feel free to take ownership of this PR and apply commits on top.

Problem

The current metrics-based bad token detection incorrectly bans entire tokens when settlement simulations fail, even though failures are often caused by solver-specific issues, not the tokens themselves. This leads to legitimate tokens like
WETH or USDC being banned (for specific colocated solvers).

Example scenario:

  • A solution with WETH→USDC + DAI→USDT fails due to a: solver routing bug, flashloan reverting, hook failing, other reasons
  • All 4 tokens (WETH, USDC, DAI, USDT) get failure marks
  • Eventually, good tokens get banned

Settlement simulations can fail for many reasons unrelated to token quality:

  • Solver bugs
  • Slippage issues
  • Insufficient solver balance
  • RPC/infrastructure errors
  • Invalid solution encoding

Solution

Implement order-level banning with metrics-based detection tracking failures per order UID instead of per token.

Changes

  • New bad_orders module mirroring bad_tokens structure
  • Added order_uids() to solutions for tracking
  • Modified encoding flow to track both tokens and orders
  • New configuration options for order-level banning
  • Added bad_orders_detected Prometheus metric

Configuration

[solver.my-solver]
enable-metrics-bad-order-detection = true
metrics-bad-order-detection-failure-ratio = 0.9
metrics-bad-order-detection-required-measurements = 5
metrics-bad-order-detection-order-freeze-time = "1h"

Note: Simulation-based bad token detection (which directly tests token behavior) remains unchanged and continues to work correctly.

Caveats

I think this PR improves things, preventing to blame tokens for an issue with a specific order. We can have now a similar issue with one problematic order making some other order to be banned because they are part of the same solution.

In practice, this affects only to one solver (is done in the driver), and I don't think will happen often. However, future PRs could enhance the granularity and try to simulate things in isolation if possible to find the culpit of reverts. Never the less, this should be way less intrusive and give fewer false positives than the old check.

Test

cargo test --package driver bad_orders

@github-actions
Copy link

github-actions bot commented Dec 5, 2025

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@anxolin anxolin changed the title Add bad order detection Fix: Ban problematic orders instead of tokens on simulation failures Dec 5, 2025
@anxolin anxolin marked this pull request as ready for review December 5, 2025 12:38
@anxolin anxolin requested a review from a team as a code owner December 5, 2025 12:38
@anxolin
Copy link
Author

anxolin commented Dec 5, 2025

I have read the CLA Document and I hereby sign the CLA

github-actions bot added a commit that referenced this pull request Dec 5, 2025
pub struct Detector {
failure_ratio: f64,
required_measurements: u32,
counter: Arc<DashMap<order::Uid, OrderStatistics>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The order needs to be dropped from this cache once it is executed. We shouldn't accumulate all the failed orders here indefinitely.

Comment on lines +103 to +111
.and_modify(|counter| {
counter.attempts += 1;
counter.fails += u32::from(failure);
})
.or_insert_with(|| OrderStatistics {
attempts: 1,
fails: u32::from(failure),
flagged_unsupported_at: None,
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And we shouldn't accumulate successful orders here for sure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants