Skip to content

Commit 83ec117

Browse files
fix(config): add cache_config field to SQLAlchemy configs (#731)
Closes #730. ## Summary - `SQLAlchemyAsyncConfig` and `SQLAlchemySyncConfig` now accept `cache_config: Optional[CacheConfig]` and `cache_manager: Optional[CacheManager]` kwargs, closing the documentation/implementation gap from #636. - When `cache_config` is set, `__post_init__` instantiates a `CacheManager` (unless one was passed explicitly) and publishes it to sessions via `session_config.info["cache_manager"]`. Repositories pick it up automatically through the existing `session.info.get("cache_manager")` lookup in `repository/_async.py` and `repository/_sync.py`. - `session_config` and its `info` dict are shallow-copied during `__post_init__`, so two configs constructed with the same `session_config` instance no longer alias each other's `cache_manager` or `file_object_raise_on_error` flag. - `__hash__` now includes `id(cache_manager)` so configs that differ only by cache region remain distinguishable in the class-level engine/session registries. - `CacheConfig` / `CacheManager` stay behind `TYPE_CHECKING` at module load and are imported at runtime only when `cache_config` is set, so dogpile.cache probing is still avoided for users who don't opt in. After this change the "Quick Start" and "Repository Integration" examples in `docs/usage/caching.rst` work as documented: ```python db_config = SQLAlchemyAsyncConfig( connection_string="sqlite+aiosqlite:///app.db", cache_config=CacheConfig( backend="dogpile.cache.memory", expiration_time=300, ), ) async with db_config.get_session() as session: repo = UserRepository(session=session, auto_expunge=True) user = await repo.get(user_id) # DB + cache populate user = await repo.get(user_id) # cache hit (fresh deserialized instance) ``` > [!NOTE] > `auto_expunge=True` on the repository is required for cache hits to round-trip cleanly across consecutive calls — without it, a cached entity attached to a closed session triggers `MissingGreenlet` on attribute refresh in async usage. This matches the existing "Recommended with caching" guidance in the caching guide; the Quick Start example in the docs should likely be updated to show it inline (follow-up). ## Test plan - [x] Repro from the issue body runs without raising on both async and sync configs. - [x] New unit tests in `tests/unit/test_config/test_{async,sync}_config.py`: - `cache_config` builds a manager and propagates it to `session_config.info`. - An explicit `cache_manager` overrides `cache_config` and is preserved by identity. - Without `cache_config`, no `cache_manager` key is published. - Two configs sharing a `session_config` do not clobber each other's `cache_manager` / `file_object_raise_on_error` and keep user-supplied `info` keys. - `__hash__` distinguishes configs that differ only by `cache_manager`. - [x] New integration tests in `tests/integration/test_cache_repository.py` covering both sync and async paths: - `config.get_session()` + repository, populate cache, delete the underlying DB row, second `repo.get()` in a fresh session succeeds — proving the served value comes from the cache region rather than the database. - Documented Quick Start flow: two consecutive `repo.get()` calls in one session, second call is served from cache (fresh deserialized instance, identity differs). - [x] Existing cache/config/routing suites: 253 passed, 1 skipped (pre-existing `dogpile.cache`-absent gate). <!-- docs-preview --> <hr> 📚 Documentation preview: <a href="https://litestar-org.github.io/advanced-alchemy-docs-preview/731">https://litestar-org.github.io/advanced-alchemy-docs-preview/731</a> --------- Co-authored-by: Cody Fincher <cody@litestar.dev>
1 parent ed8c46b commit 83ec117

5 files changed

Lines changed: 451 additions & 8 deletions

File tree

advanced_alchemy/config/common.py

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import copy
12
from dataclasses import dataclass, field
23
from pathlib import Path
34
from typing import TYPE_CHECKING, Any, Callable, ClassVar, Generic, Optional, Union, cast
@@ -16,6 +17,7 @@
1617
from sqlalchemy.orm.session import JoinTransactionMode
1718
from sqlalchemy.sql import TableClause
1819

20+
from advanced_alchemy.cache import CacheConfig, CacheManager
1921
from advanced_alchemy.utils.dataclass import EmptyType
2022

2123
__all__ = (
@@ -187,6 +189,25 @@ class GenericSQLAlchemyConfig(Generic[EngineT, SessionT, SessionMakerT]):
187189
- ``False``: Log warnings on file operation failures, don't raise exceptions
188190
- ``True`` (default): Raise exceptions on file operation failures
189191
"""
192+
cache_config: "Optional[CacheConfig]" = None
193+
"""Optional :class:`CacheConfig <advanced_alchemy.cache.CacheConfig>` for dogpile.cache integration.
194+
195+
When set, a :class:`CacheManager <advanced_alchemy.cache.CacheManager>` is instantiated during
196+
:meth:`__post_init__` and stored in ``session_config.info["cache_manager"]``. Repositories
197+
created against sessions produced by this config will pick the manager up automatically.
198+
199+
Requires the optional ``dogpile.cache`` dependency (``pip install advanced-alchemy[dogpile]``);
200+
without it the manager falls back to a no-op :class:`NullRegion`.
201+
202+
.. seealso::
203+
:doc:`/usage/caching`
204+
"""
205+
cache_manager: "Optional[CacheManager]" = None
206+
"""Optional pre-built :class:`CacheManager <advanced_alchemy.cache.CacheManager>` instance.
207+
208+
Takes precedence over :attr:`cache_config`. Useful when sharing a single cache manager across
209+
multiple configs.
210+
"""
190211
_SESSION_SCOPE_KEY_REGISTRY: "ClassVar[set[str]]" = field(init=False, default=cast("set[str]", set()))
191212
"""Internal counter for ensuring unique identification of session scope keys in the class."""
192213
_ENGINE_APP_STATE_KEY_REGISTRY: "ClassVar[set[str]]" = field(init=False, default=cast("set[str]", set()))
@@ -203,12 +224,24 @@ def __post_init__(self) -> None:
203224
else:
204225
metadata_registry.set(self.bind_key, self.metadata)
205226

206-
# Store file_object_raise_on_error in session_config.info
207-
# Ensure session_config.info is a dict (convert from Empty if needed)
208-
if self.session_config.info is Empty:
209-
self.session_config.info = {}
210-
if isinstance(self.session_config.info, dict):
211-
self.session_config.info["file_object_raise_on_error"] = self.file_object_raise_on_error
227+
# Detach session_config and normalize info to a private dict so config writes
228+
# don't bleed between configs that share the same session_config object.
229+
self.session_config = copy.copy(self.session_config)
230+
configured_info = self.session_config.info
231+
session_info: dict[str, Any] = (
232+
{} if configured_info is Empty or configured_info is None else dict(configured_info)
233+
)
234+
session_info["file_object_raise_on_error"] = self.file_object_raise_on_error
235+
self.session_config.info = session_info
236+
237+
# Build a CacheManager from cache_config if one wasn't supplied explicitly,
238+
# then propagate it to sessions via session_config.info["cache_manager"].
239+
if self.cache_manager is None and self.cache_config is not None:
240+
from advanced_alchemy.cache import CacheManager
241+
242+
self.cache_manager = CacheManager(self.cache_config)
243+
if self.cache_manager is not None:
244+
session_info["cache_manager"] = self.cache_manager
212245

213246
def __hash__(self) -> int: # pragma: no cover
214247
return hash(
@@ -217,6 +250,7 @@ def __hash__(self) -> int: # pragma: no cover
217250
self.connection_string,
218251
self.engine_config.__class__.__qualname__,
219252
self.bind_key,
253+
id(self.cache_manager) if self.cache_manager is not None else None,
220254
)
221255
)
222256

docs/changelog.rst

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,17 @@
4141
and cache managers. The older ``list()`` and ``list_and_count()`` names
4242
remain available as deprecation wrappers until 2.0.
4343

44+
.. change:: configure repository caching from SQLAlchemy configs
45+
:type: feature
46+
:pr: 731
47+
:issue: 730
48+
49+
Adds ``cache_config`` and ``cache_manager`` support to SQLAlchemy config
50+
objects. Configured cache managers are stored in ``session.info`` so
51+
repositories created from config-managed sessions pick up caching
52+
consistently, including when ``session_config.info`` is explicitly
53+
``None``.
54+
4455
.. change:: migrate object storage tests to rustfs
4556
:type: misc
4657
:pr: 732

tests/integration/test_cache_repository.py

Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -553,3 +553,202 @@ class CachedAuthorRepository(SQLAlchemySyncRepository[Any]):
553553

554554
finally:
555555
CachedAuthor.metadata.drop_all(sqlite_engine)
556+
557+
558+
# Config-driven cache_config integration (issue #730)
559+
560+
561+
@pytest.mark.asyncio
562+
@pytest.mark.aiosqlite
563+
@pytest.mark.skipif(not DOGPILE_CACHE_INSTALLED, reason="dogpile.cache not installed")
564+
async def test_async_config_cache_config_repo_auto_pickup(
565+
aiosqlite_engine: AsyncEngine,
566+
request: pytest.FixtureRequest,
567+
) -> None:
568+
"""SQLAlchemyAsyncConfig(cache_config=...) wires session.info so the second get() is a cache hit."""
569+
from sqlalchemy import text
570+
571+
from advanced_alchemy.config import SQLAlchemyAsyncConfig
572+
573+
worker_id = get_worker_id(request)
574+
CachedAuthor = get_cached_author_model("aiosqlite_cfg", worker_id)
575+
table_name = CachedAuthor.__tablename__
576+
577+
async with aiosqlite_engine.begin() as conn:
578+
await conn.run_sync(CachedAuthor.metadata.create_all)
579+
580+
try:
581+
config = SQLAlchemyAsyncConfig(
582+
engine_instance=aiosqlite_engine,
583+
cache_config=CacheConfig(backend="dogpile.cache.memory", expiration_time=300, key_prefix="cfg:"),
584+
)
585+
586+
class CachedAuthorRepository(SQLAlchemyAsyncRepository[Any]):
587+
model_type = CachedAuthor
588+
589+
async with config.get_session() as session:
590+
repo = CachedAuthorRepository(session=session, auto_expunge=True)
591+
assert repo._cache_manager is config.cache_manager
592+
593+
author = CachedAuthor(name="Cfg Async")
594+
await repo.add(author)
595+
await session.commit()
596+
author_id = author.id
597+
598+
first = await repo.get(author_id)
599+
assert first.name == "Cfg Async"
600+
assert config.cache_manager is not None
601+
cached_region = config.cache_manager.get_entity_sync(table_name, author_id, CachedAuthor)
602+
assert cached_region is not None and cached_region.name == "Cfg Async"
603+
604+
# Delete the row out from under SQLAlchemy. Cache survives, so the next get() can only
605+
# succeed if it's served from the cache region, not the DB.
606+
async with aiosqlite_engine.begin() as conn:
607+
await conn.execute(text(f"DELETE FROM {table_name}"))
608+
609+
async with config.get_session() as session:
610+
repo = CachedAuthorRepository(session=session, auto_expunge=True)
611+
from_cache = await repo.get(author_id)
612+
assert from_cache.name == "Cfg Async"
613+
finally:
614+
async with aiosqlite_engine.begin() as conn:
615+
await conn.run_sync(CachedAuthor.metadata.drop_all)
616+
617+
618+
@pytest.mark.asyncio
619+
@pytest.mark.aiosqlite
620+
@pytest.mark.skipif(not DOGPILE_CACHE_INSTALLED, reason="dogpile.cache not installed")
621+
async def test_async_config_cache_config_same_session_double_get(
622+
aiosqlite_engine: AsyncEngine,
623+
request: pytest.FixtureRequest,
624+
) -> None:
625+
"""Documented example: two consecutive get() calls in one session — second is a cache hit.
626+
627+
Cache hit serves a freshly deserialized instance, so identity differs from the first call.
628+
"""
629+
from advanced_alchemy.config import SQLAlchemyAsyncConfig
630+
631+
worker_id = get_worker_id(request)
632+
CachedAuthor = get_cached_author_model("aiosqlite_same_session", worker_id)
633+
634+
async with aiosqlite_engine.begin() as conn:
635+
await conn.run_sync(CachedAuthor.metadata.create_all)
636+
637+
try:
638+
config = SQLAlchemyAsyncConfig(
639+
engine_instance=aiosqlite_engine,
640+
cache_config=CacheConfig(backend="dogpile.cache.memory", expiration_time=300, key_prefix="same:"),
641+
)
642+
643+
class CachedAuthorRepository(SQLAlchemyAsyncRepository[Any]):
644+
model_type = CachedAuthor
645+
646+
async with config.get_session() as session:
647+
repo = CachedAuthorRepository(session=session, auto_expunge=True)
648+
author = CachedAuthor(name="Same Session Async")
649+
await repo.add(author)
650+
await session.commit()
651+
author_id = author.id
652+
653+
async with config.get_session() as session:
654+
repo = CachedAuthorRepository(session=session, auto_expunge=True)
655+
first = await repo.get(author_id)
656+
second = await repo.get(author_id)
657+
assert first.name == second.name == "Same Session Async"
658+
# Cache hit returns a freshly deserialized instance, not the same Python object.
659+
assert first is not second
660+
finally:
661+
async with aiosqlite_engine.begin() as conn:
662+
await conn.run_sync(CachedAuthor.metadata.drop_all)
663+
664+
665+
@pytest.mark.sqlite
666+
@pytest.mark.skipif(not DOGPILE_CACHE_INSTALLED, reason="dogpile.cache not installed")
667+
def test_sync_config_cache_config_repo_auto_pickup(
668+
sqlite_engine: Engine,
669+
request: pytest.FixtureRequest,
670+
) -> None:
671+
"""SQLAlchemySyncConfig(cache_config=...) wires session.info so the second get() is a cache hit."""
672+
from sqlalchemy import text
673+
674+
from advanced_alchemy.config import SQLAlchemySyncConfig
675+
676+
worker_id = get_worker_id(request)
677+
CachedAuthor = get_cached_author_model("sqlite_cfg", worker_id)
678+
table_name = CachedAuthor.__tablename__
679+
680+
CachedAuthor.metadata.create_all(sqlite_engine)
681+
682+
try:
683+
config = SQLAlchemySyncConfig(
684+
engine_instance=sqlite_engine,
685+
cache_config=CacheConfig(backend="dogpile.cache.memory", expiration_time=300, key_prefix="cfg:"),
686+
)
687+
688+
class CachedAuthorRepository(SQLAlchemySyncRepository[Any]):
689+
model_type = CachedAuthor
690+
691+
with config.get_session() as session:
692+
repo = CachedAuthorRepository(session=session, auto_expunge=True)
693+
assert repo._cache_manager is config.cache_manager
694+
695+
author = CachedAuthor(name="Cfg Sync")
696+
repo.add(author)
697+
session.commit()
698+
author_id = author.id
699+
700+
first = repo.get(author_id)
701+
assert first.name == "Cfg Sync"
702+
assert config.cache_manager is not None
703+
cached_region = config.cache_manager.get_entity_sync(table_name, author_id, CachedAuthor)
704+
assert cached_region is not None and cached_region.name == "Cfg Sync"
705+
706+
with sqlite_engine.begin() as conn:
707+
conn.execute(text(f"DELETE FROM {table_name}"))
708+
709+
with config.get_session() as session:
710+
repo = CachedAuthorRepository(session=session, auto_expunge=True)
711+
from_cache = repo.get(author_id)
712+
assert from_cache.name == "Cfg Sync"
713+
finally:
714+
CachedAuthor.metadata.drop_all(sqlite_engine)
715+
716+
717+
@pytest.mark.sqlite
718+
@pytest.mark.skipif(not DOGPILE_CACHE_INSTALLED, reason="dogpile.cache not installed")
719+
def test_sync_config_cache_config_same_session_double_get(
720+
sqlite_engine: Engine,
721+
request: pytest.FixtureRequest,
722+
) -> None:
723+
"""Documented example: two consecutive get() calls in one session — second is a cache hit."""
724+
from advanced_alchemy.config import SQLAlchemySyncConfig
725+
726+
worker_id = get_worker_id(request)
727+
CachedAuthor = get_cached_author_model("sqlite_same_session", worker_id)
728+
729+
CachedAuthor.metadata.create_all(sqlite_engine)
730+
731+
try:
732+
config = SQLAlchemySyncConfig(
733+
engine_instance=sqlite_engine,
734+
cache_config=CacheConfig(backend="dogpile.cache.memory", expiration_time=300, key_prefix="same:"),
735+
)
736+
737+
class CachedAuthorRepository(SQLAlchemySyncRepository[Any]):
738+
model_type = CachedAuthor
739+
740+
with config.get_session() as session:
741+
repo = CachedAuthorRepository(session=session, auto_expunge=True)
742+
author = CachedAuthor(name="Same Session Sync")
743+
repo.add(author)
744+
session.commit()
745+
author_id = author.id
746+
747+
with config.get_session() as session:
748+
repo = CachedAuthorRepository(session=session, auto_expunge=True)
749+
first = repo.get(author_id)
750+
second = repo.get(author_id)
751+
assert first.name == second.name == "Same Session Sync"
752+
assert first is not second
753+
finally:
754+
CachedAuthor.metadata.drop_all(sqlite_engine)

0 commit comments

Comments
 (0)