feat: configurable max gas fee with message deferral

docs/content/configuration/index.md

@@ -36,6 +36,12 @@ Most settings require a restart. The following can be changed at runtime via the
 |-----|------|---------|-------------|
 | [`pdp.aggregation.manager.poll_interval`](pdp/aggregation/manager.md#poll_interval) | duration | `30s` | How often the aggregation manager polls for new work |
 | [`pdp.aggregation.manager.batch_size`](pdp/aggregation/manager.md#batch_size) | duration | `10` | Maximum number of items to process in a single batch |
+| [`pdp.gas.max_fee.prove`](pdp/gas.md#max_feeprove) | uint (wei) | `0` | Max gas fee for proof submission |
+| [`pdp.gas.max_fee.proving_period`](pdp/gas.md#max_feeproving_period) | uint (wei) | `0` | Max gas fee for advancing proving period |
+| [`pdp.gas.max_fee.proving_init`](pdp/gas.md#max_feeproving_init) | uint (wei) | `0` | Max gas fee for initiating proving |
+| [`pdp.gas.max_fee.add_roots`](pdp/gas.md#max_feeadd_roots) | uint (wei) | `0` | Max gas fee for adding roots |
+| [`pdp.gas.max_fee.default`](pdp/gas.md#max_feedefault) | uint (wei) | `0` | Fallback max gas fee for other messages |
+| [`pdp.gas.retry_wait`](pdp/gas.md#retry_wait) | duration | `5m` | Wait between gas fee re-checks |
 
 ## Minimal Example
 

docs/content/configuration/pdp/gas.md

@@ -0,0 +1,127 @@
+# Gas Fee Limits
+
+Per-message-type gas fee limits with automatic deferral during high congestion.
+
+| Key | Default | Env | Dynamic |
+|-----|---------|-----|---------|
+| `pdp.gas.max_fee.prove` | `0` (no limit) | `PIRI_PDP_GAS_MAX_FEE_PROVE` | Yes |
+| `pdp.gas.max_fee.proving_period` | `0` (no limit) | `PIRI_PDP_GAS_MAX_FEE_PROVING_PERIOD` | Yes |
+| `pdp.gas.max_fee.proving_init` | `0` (no limit) | `PIRI_PDP_GAS_MAX_FEE_PROVING_INIT` | Yes |
+| `pdp.gas.max_fee.add_roots` | `0` (no limit) | `PIRI_PDP_GAS_MAX_FEE_ADD_ROOTS` | Yes |
+| `pdp.gas.max_fee.default` | `0` (no limit) | `PIRI_PDP_GAS_MAX_FEE_DEFAULT` | Yes |
+| `pdp.gas.retry_wait` | `5m` | `PIRI_PDP_GAS_RETRY_WAIT` | Yes |
+
+## Overview
+
+Piri sends several types of on-chain messages during normal operation. During network congestion, gas fees can spike dramatically — in one observed incident, base fees rose 489x above normal, costing a node operator ~3 FIL to onboard 0.5 TB of data.
+
+Gas fee limits let you set a maximum cost (in wei) per message type. When the estimated gas cost exceeds the configured limit, Piri **defers the message** rather than sending it — the message is automatically retried after `retry_wait` elapses. During deferral:
+
+- Data ingestion, storage, and retrieval continue normally
+- Only on-chain message submission is paused
+- Deferred messages do not consume the task's retry budget
+- Messages are sent automatically once fees drop below the limit
+
+**Dynamic Configuration:** All gas fee settings can be changed at runtime using the admin API. Changes take effect immediately on the next send attempt.
+
+## Message Types
+
+| Config Key | Message Type | Time Sensitivity |
+|-----------|-------------|-----------------|
+| `max_fee.prove` | Proof submission (`provePossession`) | High — must land within challenge window |
+| `max_fee.proving_period` | Advancing proving period (`nextProvingPeriod`) | High — epoch-constrained |
+| `max_fee.proving_init` | Initiating first proving period | High — per proof set |
+| `max_fee.add_roots` | Adding roots to proof set | Low — data already stored |
+| `max_fee.default` | Fallback for all other messages | Varies |
+
+Messages without a dedicated config key (e.g., provider registration, proof set creation, root deletion) use the `default` limit.
+
+## Fields
+
+### `max_fee.prove`
+
+Maximum gas fee (in wei) for proof submission messages. Set this high enough to ensure proofs land within their challenge window — missing a challenge window means the proof set is not proven for that period.
+
+### `max_fee.proving_period`
+
+Maximum gas fee (in wei) for advancing the proving period. Similar time sensitivity to `prove` — delays here postpone the next challenge cycle.
+
+### `max_fee.proving_init`
+
+Maximum gas fee (in wei) for initiating the first proving period on a proof set.
+
+### `max_fee.add_roots`
+
+Maximum gas fee (in wei) for adding roots to a proof set. This is the safest to cap aggressively — data is already stored and served, the root just isn't registered on-chain yet.
+
+### `max_fee.default`
+
+Fallback maximum gas fee (in wei) for any message type without a dedicated limit. Applies to one-time operations like provider registration, proof set creation, and root deletion.
+
+### `retry_wait`
+
+How long to wait before re-checking gas fees after a deferral. Default is 5 minutes. During sustained fee spikes, this prevents tight polling of the RPC endpoint.
+
+## Recommendations
+
+**Conservative (cost-sensitive):**
+
+Set aggressive limits on non-time-sensitive operations and generous limits on proving:
+
+```toml
+[pdp.gas.max_fee]
+prove = 100000000000000000          # 0.1 FIL
+proving_period = 100000000000000000 # 0.1 FIL
+proving_init = 50000000000000000    # 0.05 FIL
+add_roots = 10000000000000000       # 0.01 FIL
+default = 10000000000000000         # 0.01 FIL
+```
+
+**Permissive (availability-focused):**
+
+Only cap the lowest-priority messages:
+
+```toml
+[pdp.gas.max_fee]
+add_roots = 50000000000000000  # 0.05 FIL
+default = 50000000000000000    # 0.05 FIL
+```
+
+**No limits (default):**
+
+All values default to `0`, which means no gas fee checking — Piri pays whatever the network demands. This is the pre-existing behavior.
+
+## Runtime Adjustment
+
+During a gas spike, you can tighten limits without restarting:
+
+```bash
+# Check current settings
+piri client admin config list
+
+# Lower the limit for root additions
+piri client admin config set pdp.gas.max_fee.add_roots 10000000000000000
+
+# Increase the retry interval during sustained spikes
+piri client admin config set pdp.gas.retry_wait 15m
+```
+
+To persist changes across restarts, add `--persist`:
+
+```bash
+piri client admin config set --persist pdp.gas.max_fee.add_roots 10000000000000000
+```
+
+## TOML
+
+```toml
+[pdp.gas]
+retry_wait = "5m"
+
+[pdp.gas.max_fee]
+prove = 100000000000000000
+proving_period = 100000000000000000
+proving_init = 50000000000000000
+add_roots = 10000000000000000
+default = 10000000000000000
+```

docs/content/configuration/pdp/index.md

@@ -25,6 +25,10 @@ owner_address = "0x1234..."
 lotus_endpoint = "wss://lotus.example.com/rpc/v1"
 ```
 
+### [gas](gas.md)
+
+Gas fee limit configuration. Set per-message-type maximums to defer on-chain messages during congestion.
+
 ### [aggregation](aggregation/index.md)
 
 Aggregation system configuration.

docs/mkdocs.yml

@@ -72,6 +72,7 @@ nav:
       - server: configuration/server.md
       - pdp:
           - configuration/pdp/index.md
+          - gas: configuration/pdp/gas.md
           - aggregation:
               - configuration/pdp/aggregation/index.md
               - commp: configuration/pdp/aggregation/commp.md

pkg/config/app/pdp.go

@@ -35,6 +35,31 @@ type PDPServiceConfig struct {
 	PayerAddress common.Address
 	// Aggregation contains aggregation manager configuration
 	Aggregation AggregationConfig
+	// Gas contains gas fee limit configuration
+	Gas GasConfig
+}
+
+// GasConfig configures per-message-type gas fee limits.
+// Values are in wei. A value of 0 means no limit (default).
+type GasConfig struct {
+	MaxFee    GasMaxFeeConfig
+	RetryWait time.Duration
+}
+
+// GasMaxFeeConfig holds per-message-type maximum gas fees in wei.
+type GasMaxFeeConfig struct {
+	Prove         uint
+	ProvingPeriod uint
+	ProvingInit   uint
+	AddRoots      uint
+	Default       uint
+}
+
+// DefaultGasConfig returns a GasConfig with no limits set (all zero).
+func DefaultGasConfig() GasConfig {
+	return GasConfig{
+		RetryWait: 5 * time.Minute,
+	}
 }
 
 // SigningServiceConfig configures the signing service for PDP operations

pkg/config/defaults.go

@@ -36,6 +36,16 @@ const (
 	ManagerJobQueueRetryDelay Key = "pdp.aggregation.manager.job_queue.retry_delay"
 )
 
+// PDP Gas Fee Limits (dynamic - can change at runtime)
+const (
+	GasMaxFeeProve         Key = "pdp.gas.max_fee.prove"
+	GasMaxFeeProvingPeriod Key = "pdp.gas.max_fee.proving_period"
+	GasMaxFeeProvingInit   Key = "pdp.gas.max_fee.proving_init"
+	GasMaxFeeAddRoots      Key = "pdp.gas.max_fee.add_roots"
+	GasMaxFeeDefault       Key = "pdp.gas.max_fee.default"
+	GasRetryWait           Key = "pdp.gas.retry_wait"
+)
+
 var defaultValues = map[Key]any{
 	CommPJobQueueWorkers:    runtime.NumCPU(),
 	CommPJobQueueRetries:    50,

pkg/config/pdp.go

@@ -35,6 +35,7 @@ type PDPServiceConfig struct {
 	ChainID        string               `mapstructure:"chain_id" validate:"required" flag:"chain-id" toml:"chain_id,omitempty"`
 	PayerAddress   string               `mapstructure:"payer_address" validate:"required" flag:"payer-address" toml:"payer_address,omitempty"`
 	Aggregation    AggregationConfig    `mapstructure:"aggregation" toml:"aggregation,omitempty"`
+	Gas            GasConfig            `mapstructure:"gas" toml:"gas,omitempty"`
 }
 
 func (c PDPServiceConfig) Validate() error {
@@ -109,6 +110,7 @@ func (c PDPServiceConfig) ToAppConfig() (app.PDPServiceConfig, error) {
 		ChainID:      chainID,
 		PayerAddress: common.HexToAddress(c.PayerAddress),
 		Aggregation:  aggregationCfg,
+		Gas:          c.Gas.ToAppConfig(),
 	}, nil
 }
 
@@ -264,6 +266,38 @@ func (c AggregationConfig) ToAppConfig() (app.AggregationConfig, error) {
 	}, nil
 }
 
+// GasConfig configures per-message-type gas fee limits.
+type GasConfig struct {
+	MaxFee    GasMaxFeeConfig `mapstructure:"max_fee" toml:"max_fee,omitempty"`
+	RetryWait time.Duration   `mapstructure:"retry_wait" toml:"retry_wait,omitempty"`
+}
+
+// GasMaxFeeConfig holds per-message-type maximum gas fees in wei.
+type GasMaxFeeConfig struct {
+	Prove         uint `mapstructure:"prove" toml:"prove,omitempty"`
+	ProvingPeriod uint `mapstructure:"proving_period" toml:"proving_period,omitempty"`
+	ProvingInit   uint `mapstructure:"proving_init" toml:"proving_init,omitempty"`
+	AddRoots      uint `mapstructure:"add_roots" toml:"add_roots,omitempty"`
+	Default       uint `mapstructure:"default" toml:"default,omitempty"`
+}
+
+func (c GasConfig) ToAppConfig() app.GasConfig {
+	retryWait := c.RetryWait
+	if retryWait == 0 {
+		retryWait = 5 * time.Minute
+	}
+	return app.GasConfig{
+		MaxFee: app.GasMaxFeeConfig{
+			Prove:         c.MaxFee.Prove,
+			ProvingPeriod: c.MaxFee.ProvingPeriod,
+			ProvingInit:   c.MaxFee.ProvingInit,
+			AddRoots:      c.MaxFee.AddRoots,
+			Default:       c.MaxFee.Default,
+		},
+		RetryWait: retryWait,
+	}
+}
+
 // DefaultAggregationConfig returns an AggregationConfig with sensible defaults.
 // These values match the viper defaults in defaults.go.
 func DefaultAggregationConfig() AggregationConfig {

pkg/fx/app/common.go

@@ -28,6 +28,7 @@ func CommonModules(cfg app.AppConfig) fx.Option {
 		fx.Supply(cfg.Replicator),
 		fx.Supply(cfg.PDPService.SigningService),
 		fx.Supply(cfg.PDPService.Aggregation.Manager),
+		fx.Supply(cfg.PDPService.Gas),
 
 		identity.Module, // Provides principal.Signer
 		proofs.Module,   // Provides service for requesting service proofs

pkg/fx/scheduler/messages.go

@@ -7,6 +7,8 @@ import (
 	"go.uber.org/fx"
 	"gorm.io/gorm"
 
+	"github.com/storacha/piri/pkg/config/app"
+	"github.com/storacha/piri/pkg/config/dynamic"
 	"github.com/storacha/piri/pkg/pdp/chainsched"
 	"github.com/storacha/piri/pkg/pdp/ethereum"
 	"github.com/storacha/piri/pkg/pdp/scheduler"
@@ -46,9 +48,11 @@ var MessageModule = fx.Module("scheduler-messages",
 
 type SenderETHParams struct {
 	fx.In
-	DB     *gorm.DB `name:"engine_db"`
-	Client service.EthClient
-	Wallet wallet.Wallet
+	DB        *gorm.DB `name:"engine_db"`
+	Client    service.EthClient
+	Wallet    wallet.Wallet
+	Registry  *dynamic.Registry
+	GasConfig app.GasConfig
 }
 
 // SenderETHPair holds both the sender and task to ensure they're created together
@@ -58,7 +62,7 @@ type SenderETHPair struct {
 }
 
 func ProvideSenderETHPair(params SenderETHParams) (*SenderETHPair, error) {
-	sender, sendTask, err := tasks.NewSenderETH(params.Client, params.Wallet, params.DB)
+	sender, sendTask, err := tasks.NewSenderETH(params.Client, params.Wallet, params.DB, tasks.WithGasConfig(params.Registry), tasks.WithGasDefaults(params.GasConfig))
 	return &SenderETHPair{
 		Sender:   sender,
 		SendTask: sendTask,