feat(litellm): async retain, reflect support, and API cleanup #167
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add sync parameter to retain() for blocking vs background operation - Default to async retain (sync=False) for better performance - Add get_pending_retain_errors() to check async failures - Fix "Unclosed client session" warnings by properly closing clients - Fix "Timeout context manager" asyncio errors by creating fresh clients - Each API call now creates and closes its own client (aiohttp limitation) - Add _get_client() and _close_client() helpers for consistent handling - Update recall(), reflect(), _retain_sync() and _inject_memories() 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Make hindsight_query required when inject_memories=True to enforce intentional memory queries (no automatic last-user-message fallback) - Add reflect_context parameter for shaping LLM reasoning in reflect - Add reflect_response_schema for structured JSON output from reflect - Add _reflect_sync() and _reflect_async() methods in callbacks - Update wrappers.py to support response_schema in reflect/areflect This improves the developer experience by making memory injection explicit and adds full reflect API support through the integration. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
…ontext - Rename `recall_budget` parameter to `budget` for consistency with API - Add `hindsight_reflect_context` kwarg for per-call reflect context override - Fix reflect() to not pass None values for optional parameters Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Document configure() vs set_defaults() separation - Add hindsight_query requirement when inject_memories=True - Document async retain (sync=False default) and get_pending_retain_errors() - Add hindsight_reflect_context per-call override documentation - Document budget parameter (renamed from recall_budget) - Add reflect_context and reflect_response_schema options - Update all code examples to use new API structure - Add new functions to API Reference table Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Update tests to use separate configure() and set_defaults() calls - Fix test assertions to check config vs defaults appropriately - Add tests for legacy parameter backwards compatibility - Add new TestSetDefaults test class - Fix _format_memories test call signature (settings, config order) Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Add mission parameter to hindsight_client.create_bank() - Add set_bank_mission() function to hindsight_litellm - Deprecate set_bank_background() with DeprecationWarning - Update _create_or_update_bank() to support mission parameter - Update README and docstrings to document the new API The 'background' field has been deprecated in the Hindsight API in favor of 'mission' which is used for mental model generation. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Remove set_bank_background() in favor of set_bank_mission() - Remove background parameter from _create_or_update_bank() - Remove background parameter from hindsight_client.create_bank() - Remove legacy parameters from configure() (bank_id, document_id, budget, etc.) - These have been replaced by the set_defaults() API - Remove legacy test cases for deprecated parameters Co-Authored-By: Claude Opus 4.5 <[email protected]>
DK09876
force-pushed
the
feat/litellm-async-retain-client-cleanup
branch
2 times, most recently
from
b436f53 to
6b2856a
Compare
…ain-client-cleanup
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Improvements to the hindsight-litellm integration:
sync=False) for better performance, withget_pending_retain_errors()for error trackinghindsight_querywheninject_memories=Truefor intentional memory queriesreflect_contextandreflect_response_schemaparameters for reflect APIhindsight_reflect_contextkwarg for per-call context overriderecall_budgettobudgetfor API consistencyconfigure()(static settings) fromset_defaults()(per-call defaults)