Skip to content

Add expensive op extension#3181

Open
MitchTurner wants to merge 17 commits intomasterfrom
feature/minimize-impact-of-expensive-block-calls
Open

Add expensive op extension#3181
MitchTurner wants to merge 17 commits intomasterfrom
feature/minimize-impact-of-expensive-block-calls

Conversation

@MitchTurner
Copy link
Copy Markdown
Member

@MitchTurner MitchTurner commented Jan 13, 2026

Linked Issues/PRs

Description

This is a protection against DoS by spamming expensive requests that can block the threadpool. We will be migrating to a better, paginated request pattern, but that will be a breaking change.

The plan for now is add this change as a stop-gap, depricate this endpoint and move users to the new pattern.

Checklist

  • New behavior is reflected in tests

Before requesting review

  • I have reviewed the code myself
  • I have created follow-up issues caused by this PR and linked them here

@MitchTurner MitchTurner self-assigned this Jan 13, 2026
@MitchTurner MitchTurner marked this pull request as ready for review January 14, 2026 17:49
@MitchTurner MitchTurner requested review from a team, Dentosal, rymnc and xgreenx as code owners January 14, 2026 17:49
@cursor
Copy link
Copy Markdown

cursor bot commented Jan 14, 2026

PR Summary

Medium Risk
Adds a new GraphQL execution guard that can reject or time out block/blocks requests, which may change client-visible behavior under load and introduces new tunable limits that could be misconfigured.

Overview
Adds an "expensive op" GraphQL extension that identifies queries whose root fields include block/blocks and applies a semaphore-based concurrency cap plus a per-request timeout, returning GraphQL errors when the limit is exceeded or execution times out.

Wires the guard into the GraphQL schema builder and exposes new CLI/config knobs (--concurrent-full-block-requests, --full-block-request-timeout) with defaults, updating the service config accordingly.

Expands integration tests to issue full-block GraphQL queries (via cynic) and assert the new timeout and rate-limit failure modes.

Written by Cursor Bugbot for commit bd06491. This will update automatically on new commits. Configure here.

Voxelot
Voxelot previously approved these changes Jan 16, 2026
Dentosal
Dentosal previously approved these changes Jan 26, 2026
Comment on lines +112 to +113
#[clap(long = "full-block-request-timeout", default_value = "3s", env)]
pub full_block_request_timeout: humantime::Duration,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be nice if this was optional for local envs, but just setting a high limit should be fine

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked into this, it makes the code a bit more complicated. Still worth considering, but not prioritizing now.

self.semaphore.available_permits(),
);

match is_expensive {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: why not if?

let out = tokio::time::timeout(self.timeout, fut).await;
tracing::warn!(
"finished executing in {:?}ns, success: {:?}",
starting_time.elapsed().as_nanos(),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could probably just use the Debug impl of Duration here, it gives nice units by default

@MitchTurner MitchTurner dismissed stale reviews from Dentosal and Voxelot via f0c89d8 March 13, 2026 22:19
@fuel-cla-bot
Copy link
Copy Markdown

fuel-cla-bot bot commented Mar 13, 2026

Thanks for the contribution! Unfortunately we can't verify the commit author(s): Codex <c***@.local>. One possible solution is to add that email to your GitHub account. Alternatively you can change your commits to another email and force push the change. After getting your commits associated with your GitHub account, sign the Fuel Labs Contributor License Agreement and this Pull Request will be revalidated.

"finished executing in {:?}ns, success: {:?}",
starting_time.elapsed().as_nanos(),
out.is_ok(),
);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warn-level logging on every expensive request execution

Medium Severity

tracing::warn! is used unconditionally for every expensive operation execution, including successful ones. In production with many legitimate block/blocks queries, this generates a warning-level log entry per request. Warning level is for conditions that may need attention — routine successful completions belong at debug! or info! level. This will create significant log noise and may mask real warnings.

Fix in Cursor Fix in Web

)
})
.sum()
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guard counts fields across all operations, not executed one

Medium Severity

expensive_root_field_count iterates over all operations in the document (doc.operations.iter()) and sums their expensive fields. In GraphQL, a document can contain multiple named operations but only one is executed per request. This over-counts permits needed, potentially rejecting legitimate requests that execute a cheap operation but whose document happens to also define expensive ones.

Fix in Cursor Fix in Web

@MitchTurner MitchTurner force-pushed the feature/minimize-impact-of-expensive-block-calls branch from f0c89d8 to d97ac7e Compare March 13, 2026 22:24
@MitchTurner MitchTurner force-pushed the feature/minimize-impact-of-expensive-block-calls branch from d97ac7e to c57d35a Compare March 13, 2026 22:29
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

"finished executing in {:?}ns, success: {:?}",
starting_time.elapsed(),
out.is_ok(),
);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Log message appends spurious "ns" after Duration debug format

Low Severity

The format string "finished executing in {:?}ns, success: {:?}" appends a literal ns after the Debug representation of a Duration, which already includes its own unit. This produces garbled output like "finished executing in 7msns" or "finished executing in 1.234sns". The ns suffix needs to be removed, or starting_time.elapsed().as_nanos() used instead.

Fix in Cursor Fix in Web

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants