Skip to content

[POC] [AAP-64471] Add dynamic DB/cache session manager#927

Draft
john-westcott-iv wants to merge 1 commit intoansible:develfrom
john-westcott-iv:AAP-64471-POC
Draft

[POC] [AAP-64471] Add dynamic DB/cache session manager#927
john-westcott-iv wants to merge 1 commit intoansible:develfrom
john-westcott-iv:AAP-64471-POC

Conversation

@john-westcott-iv
Copy link
Copy Markdown
Member

Description

  • What is being changed? Added a feature flag FEATURE_GATEWAY_SIDECAR_CACHE_ENABLED that allows the session store to dynamically switch between cached+DB and DB-only storage modes at runtime.
  • Why is this change needed? In clustered deployments with sidecar caches, session invalidation (logout) doesn't propagate instantly across nodes. When a user logs out on one node, their session gets cleared from that node's cache and the DB, but other nodes may still have the session cached, causing "zombie sessions" where users appear to still be logged in.
  • How does this change address the issue? When the feature flag is enabled, sessions bypass the cache entirely and go straight to the DB, ensuring logout is immediately consistent across all cluster nodes.

Type of Change

  • New feature (non-breaking change which adds functionality)

Self-Review Checklist

  • I have performed a self-review of my code
  • I have added relevant comments to complex code sections
  • I have considered the security impact of these changes
  • I have considered performance implications
  • I have thought about error handling and edge cases
  • I have tested the changes in my local environment

Testing Instructions

Prerequisites

  • DAB test environment

Steps to Test

  1. Run the tests: tox -e py312 -- test_app/tests/lib/sessions/stores/test_cached_dynamic_timeout.py -v
  2. All 10 session tests should pass

Expected Results

  • When FEATURE_GATEWAY_SIDECAR_CACHE_ENABLED=False (default): Sessions are stored in both cache and DB
  • When FEATURE_GATEWAY_SIDECAR_CACHE_ENABLED=True: Sessions are stored in DB only

Additional Context

This is a POC to validate the approach for supporting gateway deployments with distributed sidecar caches where session consistency is critical.

Key implementation detail

The core logic is just 4 lines:

def _dispatch(self, method_name, *args, **kwargs):
    use_cache = not flag_state('FEATURE_GATEWAY_SIDECAR_CACHE_ENABLED')
    parent = CachedDBSessionStore if use_cache else DBSessionStore
    return getattr(parent, method_name)(self, *args, **kwargs)

Note: This PR was developed with assistance from Claude AI assistant.

Add FEATURE_GATEWAY_SIDECAR_CACHE_ENABLED feature flag to dynamically
switch between cached+DB and DB-only session storage at runtime.

When sidecar cache is disabled (default): sessions use cache + DB
When sidecar cache is enabled: sessions use DB only to ensure
logout/session invalidation is immediately consistent across all
cluster nodes (no stale cached sessions in sidecar deployments).

Co-authored-by: Claude <[email protected]>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 4, 2026

DVCS PR Check Results:

PR appears valid (JIRA key(s) found)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer these to have a descriptive name, but I seem to be losing that battle.

# Sidecar cache enabled = DB only (no centralized cache)
# Sidecar cache disabled = use centralized cache + DB
use_cache = not flag_state('FEATURE_GATEWAY_SIDECAR_CACHE_ENABLED')
parent = CachedDBSessionStore if use_cache else DBSessionStore
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that the cases were:

  • flag true --> uses the local redis as a cache
  • flag false --> uses the old clustered redis as the cache

What I see here is

  • flag false --> uses the database as the cache

This behavior is actually what I wanted! It makes complete sense to add the sidecar redis, delete the redis cluster, and keep the DB cache (worse performance) as a fallback in case something goes wrong with the sidecar redis, which I find to be entirely reasonable.

But that only makes sense if we are getting rid of the clustered redis. And this is the best possible plan. We should do that, although the awkward part is that it isn't really a "feature flag" as opposed to a general setting or behavior-toggle. We would be abandoning the old behavior. But again, this is the best path forward. But not the plan you wrote up.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For clarity, DBSessionStore is not a DB cache per say, its just sessions stored in the DB. CachedDBSessionStore is DB stored sessions but sessions are put into the cache to avoid the DB call. In the case of sidecar redis having sessions only in the DB helps prevent the user logout issue and sessions being left around in the cache on other machines (no need to call dispatcherd on a user logout to sync the sessions). In my research, this is what AWX is doing already.

The feature flag would eventually "go away" once side car was GA and the SessionStore class in this file would just inherit from DBSessionStore only.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For clarity, DBSessionStore is not a DB cache per say, its just sessions stored in the DB.

Going cross-eyed here.

So if you're using DBSessionStore... is it accurate to say that you're not caching sessions?

Yes, sessions are in a table somewhere. That's what they are, and the table comes from some Django contrib app like contrib.auth. So I think it makes sense that doing a kv cache in the database makes absolutely no sense as opposed to just reading the values.

But it sounds like you're saying that the ultimate end-state with Redis sidecar would be that sessions aren't cached. That's not good, right? We need to cache sessions, right?

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Feb 4, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants