feat: consider average tenure block fullness for tx fee estimations #2

rafaelcr · 2025-05-07T14:31:47Z

Modifies the Stacks core fee estimator proxy to follow this algorithm:

If the current and recent tenure costs are not full, always return the minimum transaction fee
If tenures start receiving transactions but do not yet reach a 90% fullness in any dimension, keep returning the minimum fee
Once a dimension exceeds 90% based on a weighted average of past tenures, use the Stacks core estimation (with a configured multiplier)
If at any time the current tenure has at least 5 blocks and less than 50% utilization, return to minimum fees

All of the previous numbers are configurable via ENV.

haneytron · 2025-05-07T15:33:15Z

src/api/routes/core-node-rpc-proxy.ts

+// https://github.com/stacks-network/stacks-core/blob/eb865279406d0700474748dc77df100cba6fa98e/stackslib/src/core/mod.rs#L212-L218
+const BLOCK_LIMIT_WRITE_LENGTH = 15_000_000;
+const BLOCK_LIMIT_WRITE_COUNT = 15_000;
+const BLOCK_LIMIT_READ_LENGTH = 100_000_000;
+const BLOCK_LIMIT_READ_COUNT = 15_000;
+const BLOCK_LIMIT_RUNTIME = 5_000_000_000;


Consider not hardcoding these. They could be pulled from either ENV config (less ideal since still duplicates from canonical node data) or perhaps from node directly to keep things in sync. If pulling from node, probably need an interval check for most recent values to update.

haneytron · 2025-05-07T15:34:13Z

src/api/routes/core-node-rpc-proxy.ts

@@ -23,6 +23,12 @@ function getReqUrl(req: { url: string; hostname: string }): URL {

 // https://github.com/stacks-network/stacks-core/blob/20d5137438c7d169ea97dd2b6a4d51b8374a4751/stackslib/src/chainstate/stacks/db/blocks.rs#L338
 const MINIMUM_TX_FEE_RATE_PER_BYTE = 1;


Where does this value come from? I see the link above, but if there's a way to derive it at runtime vs hard-coded that may be more sustainable for future changes (less to rework).

haneytron · 2025-05-07T15:36:08Z

src/api/routes/core-node-rpc-proxy.ts

+  /// only happens if there is no fee market i.e. if the last N block tenures have not been full. We
+  /// use a threshold of 90% to determine if a block size dimension is full.
+  async function shouldUseTransactionMinimumFee(): Promise<boolean> {
+    const tenureWindow = parseInt(process.env['STACKS_CORE_FEE_TENURE_FULLNESS_WINDOW'] ?? '5');


There's always a balance in infinite configuration, but consider not hardcoding the fallback to 5, or if doing so, define it more prominently perhaps at the top of the file or something where constants generally appear. Could get lost in line 141 of functions. This is totally splitting hairs.

haneytron · 2025-05-07T15:40:31Z

src/api/routes/core-node-rpc-proxy.ts

+    const averageCosts = await fastify.db.getTenureAverageExecutionCosts(tenureWindow);
+    if (!averageCosts) return false;
+    return (
+      averageCosts.read_count < BLOCK_LIMIT_READ_COUNT * 0.9 &&


Consider taking 0.9 literal and make it a variable (const) at least, don't repeat it so that we don't change it multiple places later and accidentally miss a spot. Could derive from ENV with 0.9 default as with the 5 above.

haneytron · 2025-05-07T15:41:29Z

src/api/routes/core-node-rpc-proxy.ts

+      averageCosts.runtime < BLOCK_LIMIT_RUNTIME * 0.9
+    );
+  }
+
  const maxBodySize = 10_000_000; // 10 MB max POST body size


Where is this derived from? Web server or proxy constraints? Could be made configurable and also should be verified as it could cause failure if it exceeds the limits set by whatever web server is running.

haneytron · 2025-05-07T15:42:47Z

src/api/routes/core-node-rpc-proxy.ts

-          reply.statusCode === 200 &&
-          feeEstimationModifier !== null
-        ) {
+        } else if (getReqUrl(req).pathname === '/v2/fees/transaction' && reply.statusCode === 200) {


If the API route ever changes, this breaks. Should be configurable or pulled from the Node? routing engine directly if possible.

Case sensitivity may also matter? /V2 vs /v2 etc.

haneytron · 2025-05-07T15:48:30Z

src/api/routes/core-node-rpc-proxy.ts

+            // multiplier.
+            responseJson.estimations.forEach(estimation => {
+              // max(min fee, estimate returned by node * configurable modifier)
+              estimation.fee = Math.max(minFee, Math.round(estimation.fee * modifier));


Check: is whole number rounding intended here? I assume so because you probably can't pay fees in fractions.

haneytron · 2025-05-07T15:51:08Z

src/datastore/pg-store.ts

+          WHERE type_id = ${DbTxTypeId.TenureChange}
+            AND canonical = TRUE
+            AND microblock_canonical = TRUE


Assuming TenureChange is an enum or const this is not prone to SQL injection. Ensure an appropriate index exists against txs table to cover the WHERE query here or else scans will murder this.

haneytron · 2025-05-07T15:52:01Z

src/datastore/pg-store.ts

+          WHERE type_id = ${DbTxTypeId.TenureChange}
+            AND canonical = TRUE
+            AND microblock_canonical = TRUE
+          ORDER BY block_height DESC


Micro-optimization: always try to order in mem instead of making the DB do it, offloads the single point of failure from DB CPU cycles. Probably doesn't matter, this is a nit.

haneytron · 2025-05-07T15:53:21Z

src/datastore/pg-store.ts

+        )
+        SELECT block_height FROM tenure_change_blocks ORDER BY block_height ASC
+      `;
+      if (tenureChanges.count != numTenures + 1) return undefined;


Consider adding comment explaining this logic, as unintuitive to me.

haneytron · 2025-05-07T15:56:08Z

src/datastore/pg-store.ts

+      tenureCond = sql`${tenureCond} ELSE ${tenureChanges.length}`;
+
+      // Sum and return each tenure's execution costs.
+      const result = await sql<


Consider not using DB as math engine. Pull data and do the SUM and AVG in Node to help alleviate DB contention. The tradeoff is if you do this multiple places, you are prone to missing a spot on changes, however that is true of this logic as-is now also.

haneytron · 2025-05-07T15:56:32Z

src/datastore/pg-store.ts

+      const currentTenureBlock = tenureChanges[numTenures].block_height;
+
+      // Group blocks by tenure.
+      let tenureCond = sql``;


Consider a few lines of comment on algorithm for generating the conditional clause for the SQL.

feat: consider average tenure block fullness for tx fee estimations

58ad59c

haneytron reviewed May 7, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: consider average tenure block fullness for tx fee estimations #2

feat: consider average tenure block fullness for tx fee estimations #2

Uh oh!

rafaelcr commented May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

haneytron May 7, 2025

Uh oh!

Uh oh!

		@@ -23,6 +23,12 @@ function getReqUrl(req: { url: string; hostname: string }): URL {

		// https://github.com/stacks-network/stacks-core/blob/20d5137438c7d169ea97dd2b6a4d51b8374a4751/stackslib/src/chainstate/stacks/db/blocks.rs#L338
		const MINIMUM_TX_FEE_RATE_PER_BYTE = 1;

feat: consider average tenure block fullness for tx fee estimations #2

Are you sure you want to change the base?

feat: consider average tenure block fullness for tx fee estimations #2

Uh oh!

Conversation

rafaelcr commented May 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!