feat(db): add per-user CachedSettingsStore decorator (#2425)

* feat(db): add per-user CachedSettingsStore decorator

SettingsStore methods hit the database on every call. The v2 engine
path (effect_adapter) and the dispatcher's per-turn tool permission
loading both called get_all_settings() without caching, adding
unnecessary DB round-trips on every agentic loop iteration.

Add a write-through CachedSettingsStore decorator that caches
get_all_settings() results per user_id. Write operations (set_setting,
delete_setting, set_all_settings) delegate to the inner store then
invalidate that user's cache entry. The write lock is held across DB
loads to prevent stale-data races from concurrent invalidations.

Wire the cache into TenantScope via a new settings_store field on
AgentDeps, so all settings reads in the agent loop go through the
cache. Remove the per-turn cached_tool_permissions Mutex hack from
ChatDelegate that was working around the missing cache layer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review feedback

- Store Arc<HashMap> in cache instead of bare HashMap to avoid cloning
  the full settings map on every cache hit. get_setting/has_settings now
  only clone the single requested value or check emptiness through the Arc.
- Route get_setting_with_admin_fallback() through self.settings() instead
  of self.inner so both the per-user and admin lookups go through the cache.
- Update settings section comment to accurately describe which methods
  delegate through settings().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address all PR review feedback

- Use `crate::db::` imports instead of `super::` (convention fix)
- Add `wrap()` factory fn to CachedSettingsStore, simplify app.rs construction
- Store `Arc<HashMap>` in cache to avoid full map clones on hits
- Expose `invalidate_user()` and `flush()` public methods
- Wire `flush()` into SIGHUP handler via concrete `settings_cache` on AppComponents
- Wire `settings_store` into GatewayState and route all settings handlers
  through it so web UI writes invalidate the cache (critical fix)
- Route `get_setting_with_admin_fallback()` through `self.settings()`
- Add error-path test (FailingStore mock, cache not poisoned on error)
- Add concurrent-access test (8 concurrent readers, inner store hit once)
- Add TenantScope caller-level test (read/write through cache wiring)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: reuse resolve_settings_store() in settings_tools_set_handler

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address all review feedback on CachedSettingsStore

- Add TTL (300s) and max-entries cap (1000) to bound cache staleness
  and memory growth. Entries expire after TTL; cache clears when cap
  exceeded.
- Route admin tool_policy GET/PUT through resolve_settings_store() so
  writes invalidate the __admin__ cache entry.
- Route settings_export_handler and settings_tools_list_handler through
  resolve_settings_store() (were bypassing cache on reads).
- Wire invalidate_user() into users_delete_handler and
  users_suspend_handler so deleted/suspended users' settings are evicted.
- Replace GatewayState.settings_store (trait object) with
  settings_cache (concrete CachedSettingsStore) — single field for both
  trait dispatch and cache management, no desync risk.
- Add settings_override to ExtensionManager with with_settings_store()
  builder. All settings reads/writes in ExtensionManager now route
  through the cached store when available.
- Make ExtensionManager::settings_store() pub(crate); update
  AuthManager to call it instead of database(), closing the auth
  descriptor cache bypass.
- Remove unused wrap() method; merge redundant invalidate/invalidate_user.
- Add tracing::debug on SIGHUP cache flush.
- Expand module docs with design assumptions, known bypass paths, TTL
  and eviction semantics.
- Add tests: expired_entry_triggers_reload, fresh_entry_does_not_reload,
  max_entries_cap_triggers_eviction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: collapse nested if into filter to satisfy clippy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: henrypark133

src/agent/agent_loop.rs

@@ -196,6 +196,9 @@ pub struct AgentDeps {
     /// Resolved durable owner scope for the instance.
     pub owner_id: String,
     pub store: Option<Arc<dyn Database>>,
+    /// Cached settings store. When set, `TenantScope` routes settings reads
+    /// through this cache instead of hitting the raw `Database` directly.
+    pub settings_store: Option<Arc<dyn crate::db::SettingsStore + Send + Sync>>,
     pub llm: Arc<dyn LlmProvider>,
     /// Cheap/fast LLM for lightweight tasks (heartbeat, routing, evaluation).
     /// Falls back to the main `llm` if None.
@@ -482,10 +485,13 @@ impl Agent {
         let user_id = identity.owner_id.as_str();
         let rate = self.deps.tenant_rates.get_or_create(user_id).await;
 
-        let store =
-            self.deps.store.as_ref().map(|db| {
-                crate::tenant::TenantScope::with_identity(identity.clone(), Arc::clone(db))
-            });
+        let store = self.deps.store.as_ref().map(|db| {
+            let scope = crate::tenant::TenantScope::with_identity(identity.clone(), Arc::clone(db));
+            match &self.deps.settings_store {
+                Some(ss) => scope.with_settings_store(Arc::clone(ss)),
+                None => scope,
+            }
+        });
 
         // Reuse the owner workspace if user matches, otherwise create per-user.
         // Per-user workspaces are seeded on first creation so they get identity

src/agent/dispatcher.rs

@@ -261,7 +261,6 @@ impl Agent {
             force_text_at,
             user_tz,
             turn_usage: std::sync::Mutex::new(TurnUsageSummary::default()),
-            cached_tool_permissions: std::sync::Mutex::new(None),
             cached_admin_tool_policy: tokio::sync::OnceCell::new(),
         };
 
@@ -377,8 +376,6 @@ struct ChatDelegate<'a> {
     force_text_at: usize,
     user_tz: chrono_tz::Tz,
     turn_usage: std::sync::Mutex<TurnUsageSummary>,
-    cached_tool_permissions:
-        std::sync::Mutex<Option<std::collections::HashMap<String, PermissionState>>>,
     cached_admin_tool_policy: crate::tools::permissions::AdminToolPolicyCache,
 }
 
@@ -478,48 +475,25 @@ impl<'a> LoopDelegate for ChatDelegate<'a> {
         // AlwaysAllow tools are pre-approved in session so the approval
         // flow is skipped — unless the tool declares ApprovalRequirement::Always,
         // which is an unbypassable hard floor.
-        let tool_permissions = {
-            // Check the cache first (brief lock, no await while held).
-            let cached = {
-                let cache = self
-                    .cached_tool_permissions
-                    .lock()
-                    .unwrap_or_else(|e| e.into_inner());
-                cache.clone()
-            };
-            if let Some(perms) = cached {
-                perms
-            } else {
-                // Cache miss — load from DB (async).
-                let perms = if let Some(store) = self.tenant.store() {
-                    match store.get_all_settings().await {
-                        Ok(db_map) => {
-                            crate::settings::Settings::from_db_map(&db_map).tool_permissions
-                        }
-                        Err(e) => {
-                            tracing::warn!(
-                                "Failed to load tool permissions, keeping existing session state: {}",
-                                e
-                            );
-                            // Fail closed: preserve the previously filtered available_tools
-                            // rather than publishing the unfiltered tool list, which could
-                            // re-expose tools explicitly marked Disabled.
-                            return None;
-                        }
-                    }
-                } else {
-                    std::collections::HashMap::new()
-                };
-                // Store in cache for subsequent iterations.
-                {
-                    let mut cache = self
-                        .cached_tool_permissions
-                        .lock()
-                        .unwrap_or_else(|e| e.into_inner());
-                    *cache = Some(perms.clone());
+        //
+        // The SettingsStore is wrapped in CachedSettingsStore so repeated
+        // calls within the same session are cheap (in-memory lookup).
+        let tool_permissions = if let Some(store) = self.tenant.store() {
+            match store.get_all_settings().await {
+                Ok(db_map) => crate::settings::Settings::from_db_map(&db_map).tool_permissions,
+                Err(e) => {
+                    tracing::warn!(
+                        "Failed to load tool permissions, keeping existing session state: {}",
+                        e
+                    );
+                    // Fail closed: preserve the previously filtered available_tools
+                    // rather than publishing the unfiltered tool list, which could
+                    // re-expose tools explicitly marked Disabled.
+                    return None;
                 }
-                perms
             }
+        } else {
+            std::collections::HashMap::new()
         };
 
         // Filter tool definitions and collect AlwaysAllow names for session
@@ -2011,6 +1985,7 @@ mod tests {
         let deps = AgentDeps {
             owner_id: "default".to_string(),
             store: None,
+            settings_store: None,
             llm: Arc::new(StaticLlmProvider),
             cheap_llm: None,
             safety: Arc::new(SafetyLayer::new(&SafetyConfig {
@@ -2320,6 +2295,7 @@ mod tests {
         let deps = AgentDeps {
             owner_id: "default".to_string(),
             store: None,
+            settings_store: None,
             llm: Arc::new(AuthThenApprovalProvider),
             cheap_llm: None,
             safety: Arc::new(SafetyLayer::new(&SafetyConfig {
@@ -3300,6 +3276,7 @@ mod tests {
         let deps = AgentDeps {
             owner_id: "default".to_string(),
             store: None,
+            settings_store: None,
             llm,
             cheap_llm: None,
             safety: Arc::new(SafetyLayer::new(&SafetyConfig {
@@ -3448,6 +3425,7 @@ mod tests {
         let deps = AgentDeps {
             owner_id: "default".to_string(),
             store: Some(db),
+            settings_store: None,
             llm: llm as Arc<dyn LlmProvider>,
             cheap_llm: None,
             safety: Arc::new(SafetyLayer::new(&SafetyConfig {
@@ -3578,6 +3556,7 @@ mod tests {
             let deps = AgentDeps {
                 owner_id: "default".to_string(),
                 store: None,
+                settings_store: None,
                 llm,
                 cheap_llm: None,
                 safety: Arc::new(SafetyLayer::new(&SafetyConfig {

src/agent/thread_ops.rs

@@ -2569,6 +2569,7 @@ mod tests {
         let deps = crate::agent::AgentDeps {
             owner_id: "default".to_string(),
             store: None,
+            settings_store: None,
             llm: Arc::new(StaticLlmProvider),
             cheap_llm: None,
             safety: Arc::new(ironclaw_safety::SafetyLayer::new(
@@ -3137,6 +3138,7 @@ mod tests {
         let deps = AgentDeps {
             owner_id: "default".to_string(),
             store: None,
+            settings_store: None,
             llm: Arc::new(StubLlm::default()),
             cheap_llm: None,
             safety: Arc::new(SafetyLayer::new(&SafetyConfig {

src/app.rs

@@ -49,6 +49,9 @@ pub struct AppComponents {
     /// runtime settings writes flow through the workspace and pick up schema
     /// validation.
     pub settings_store: Option<Arc<dyn crate::db::SettingsStore + Send + Sync>>,
+    /// Concrete cache handle for `flush()` / `invalidate_user()`.
+    /// Same instance backing `settings_store` when a cache is active.
+    pub settings_cache: Option<Arc<crate::db::cached_settings::CachedSettingsStore>>,
     pub extension_manager: Option<Arc<ExtensionManager>>,
     pub mcp_session_manager: Arc<McpSessionManager>,
     pub mcp_process_manager: Arc<McpProcessManager>,
@@ -823,7 +826,7 @@ impl AppBuilder {
             Arc::new(InMemorySecretsStore::new(crypto))
         };
         let extension_manager = {
-            let manager = Arc::new(ExtensionManager::new(
+            let mut em = ExtensionManager::new(
                 Arc::clone(&mcp_session_manager),
                 Arc::clone(&mcp_process_manager),
                 ext_secrets,
@@ -836,7 +839,11 @@ impl AppBuilder {
                 self.config.owner_id.clone(),
                 self.db.clone(),
                 catalog_entries.clone(),
-            ));
+            );
+            if let Some(ref ss) = settings_store_override {
+                em = em.with_settings_store(Arc::clone(ss));
+            }
+            let manager = Arc::new(em);
             tools.register_extension_tools(Arc::clone(&manager));
 
             // Register permission management tool and upgrade tool_list with
@@ -949,22 +956,30 @@ impl AppBuilder {
         // `upgrade_tool_list`) can be wired with the adapter from the start.
         // The same adapter instance is then exposed on `AppComponents.settings_store`
         // and reused by main.rs (e.g. for the SIGHUP reload handler).
-        let settings_store: Option<Arc<dyn crate::db::SettingsStore + Send + Sync>> =
-            match (&workspace, &self.db) {
-                (Some(ws), Some(db)) => {
-                    let adapter = Arc::new(crate::workspace::WorkspaceSettingsAdapter::new(
-                        Arc::clone(ws),
-                        Arc::clone(db),
-                    ));
-                    if let Err(e) = adapter.ensure_system_config().await {
-                        tracing::debug!(
-                            "WorkspaceSettingsAdapter eager seed failed (lazy seed will retry): {e}"
-                        );
-                    }
-                    Some(adapter as Arc<dyn crate::db::SettingsStore + Send + Sync>)
+        let (settings_store, settings_cache): (
+            Option<Arc<dyn crate::db::SettingsStore + Send + Sync>>,
+            Option<Arc<crate::db::cached_settings::CachedSettingsStore>>,
+        ) = match (&workspace, &self.db) {
+            (Some(ws), Some(db)) => {
+                let adapter = Arc::new(crate::workspace::WorkspaceSettingsAdapter::new(
+                    Arc::clone(ws),
+                    Arc::clone(db),
+                ));
+                if let Err(e) = adapter.ensure_system_config().await {
+                    tracing::debug!(
+                        "WorkspaceSettingsAdapter eager seed failed (lazy seed will retry): {e}"
+                    );
                 }
-                _ => None,
-            };
+                let cached = Arc::new(crate::db::cached_settings::CachedSettingsStore::new(
+                    adapter as Arc<dyn crate::db::SettingsStore + Send + Sync>,
+                ));
+                (
+                    Some(Arc::clone(&cached) as Arc<dyn crate::db::SettingsStore + Send + Sync>),
+                    Some(cached),
+                )
+            }
+            _ => (None, None),
+        };
 
         let (
             mcp_session_manager,
@@ -1099,6 +1114,7 @@ impl AppBuilder {
             embeddings,
             workspace,
             settings_store,
+            settings_cache,
             extension_manager,
             mcp_session_manager,
             mcp_process_manager,

src/bridge/auth_manager.rs

@@ -123,11 +123,9 @@ impl AuthManager {
                     .map(|db| db.as_ref() as &dyn crate::db::SettingsStore)
             })
             .or_else(|| {
-                self.extension_manager.as_ref().and_then(|manager| {
-                    manager
-                        .database()
-                        .map(|db| db.as_ref() as &dyn crate::db::SettingsStore)
-                })
+                self.extension_manager
+                    .as_ref()
+                    .and_then(|manager| manager.settings_store())
             })
     }
 

src/bridge/router.rs

@@ -4308,6 +4308,7 @@ mod tests {
         let deps = crate::agent::AgentDeps {
             owner_id: "default".to_string(),
             store: None,
+            settings_store: None,
             llm: Arc::new(StaticLlmProvider),
             cheap_llm: None,
             safety: Arc::new(ironclaw_safety::SafetyLayer::new(
@@ -4930,6 +4931,7 @@ mod tests {
         let deps = AgentDeps {
             owner_id: "default".to_string(),
             store: None,
+            settings_store: None,
             llm: Arc::new(StubLlm::default()),
             cheap_llm: None,
             safety: Arc::new(SafetyLayer::new(&SafetyConfig {