feat(memory): add observational memory extractors#18653
Conversation
Introduces a public Extractor API for Observational Memory with inline XML extraction and structured follow-up extraction modes. Persists extracted values into thread OM metadata, carries them forward into future observer/reflector prompts, and exposes extraction results in OM streaming markers. Also threads agent context through processor execution so extractor hooks can access the running agent, updates storage types with extraction fields on BufferedObservationChunk, and documents the new API with focused unit and integration coverage. Co-Authored-By: Mastra Code (crof/glm-5.2) <noreply@mastra.ai>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
🦋 Changeset detectedLatest commit: 7fe1030 The changes in this PR will be included in the next version bump. This PR includes changesets to release 25 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
PR triageLinked issue check skipped for core contributor @TylerBarnes. PR complexity score
Applied label: Changed test gateChanged tests failed against the base branch as expected. Label: |
|
Dependency limit exceeded — report not shown. This pull request scan exceeded the 10,000-dependency limit applied to this scan, so the results are incomplete and may be inaccurate. To avoid reporting false positives, Socket has not posted a report. Upgrade your plan to raise the dependency limit and get complete reports, or view the partial scan in the dashboard. Socket is always free for open source. If this is a non-commercial open source project, contact us to request a free Team account. |
| }) | ||
| ``` | ||
|
|
||
| Extractor behavior is derived from whether you provide a schema. Schema-backed extractors run as a follow-up structured output call after the Observer or Reflector finishes. The follow-up call adds model latency and usage, but it uses the same temporary in-memory Observer or Reflector thread so providers that support prompt caching can reuse the shared prompt prefix. Schema-less extractors are inline string extractors emitted directly in the Observer or Reflector response. |
There was a problem hiding this comment.
this is overly explanatory. We can just say that
- adding a schema makes it do a follow up structured output request (don't need to talk about tradeoffs)
- it should be assumed that we're preserving prompt caching, don't need to over explain internals
- yes schema less extractors are inline in the first request
| }) | ||
| ``` | ||
|
|
||
| By default, OM includes the previous extraction in later extraction prompts. Set `includePreviousExtraction: false` for values that should only come from the current OM run. |
There was a problem hiding this comment.
wording is a bit confusing. we should say something like: when its false the observer does not see the last extracted value (or something)
| ```typescript title="src/mastra/agents/agent.ts" | ||
| new Extractor({ | ||
| name: 'Latest blocker', | ||
| instructions: 'Extract only the latest blocker mentioned in this conversation window.', |
There was a problem hiding this comment.
weird instructions. should be more like: extract any blockers the agent is running into
| Use `WorkingMemoryExtractor` when the Observer or Reflector should update working memory instead of relying on the main agent to call the working memory tool: | ||
|
|
||
| ```typescript title="src/mastra/agents/agent.ts" | ||
| import { Memory, WorkingMemoryExtractor } from '@mastra/memory' | ||
|
|
||
| const memory = new Memory({ | ||
| options: { | ||
| workingMemory: { | ||
| enabled: true, | ||
| injectTools: false, | ||
| }, | ||
| observationalMemory: { | ||
| observation: { | ||
| extract: [new WorkingMemoryExtractor()], | ||
| }, | ||
| }, | ||
| }, | ||
| }) | ||
| ``` | ||
|
|
||
| `WorkingMemoryExtractor` uses the normal extractor pipeline. It reads the active working memory template or schema, uses structured extraction when working memory has a JSON schema, writes updates through the memory instance, and doesn't duplicate the value under OM extracted metadata. |
There was a problem hiding this comment.
this doesn't belong here, we should add a note about working memory here and start with the sugar setting, then add a note about what that configures internally. But this should be in a working memory section.
There was a problem hiding this comment.
We should probably also update the working memory page linking over here
| }, | ||
| { | ||
| name: 'extract', | ||
| type: 'Extractor<any>[]', |
| }, | ||
| { | ||
| name: 'extract', | ||
| type: 'Extractor<any>[]', |
There was a problem hiding this comment.
also this section is duplicated?
| Use `WorkingMemoryExtractor` when OM should update working memory. Set `workingMemory.injectTools` to `false` when the main agent shouldn't receive the working memory update tool. | ||
|
|
||
| ```typescript title="src/mastra/agents/agent.ts" | ||
| import { Memory, WorkingMemoryExtractor } from '@mastra/memory' | ||
|
|
||
| const memory = new Memory({ | ||
| options: { | ||
| workingMemory: { | ||
| enabled: true, | ||
| injectTools: false, | ||
| }, | ||
| observationalMemory: { |
There was a problem hiding this comment.
this is out of date, and not sure it even belongs on this page at all?
| } | ||
|
|
||
| export async function createTemporaryOmMemoryContext(prefix: string): Promise<TemporaryOmMemoryContext> { | ||
| const { Memory } = await import('../../index'); |
There was a problem hiding this comment.
why a dynamic import?
Co-Authored-By: Mastra Code (crof/glm-5.2) <noreply@mastra.ai>
331daf4 to
7fe1030
Compare
Summary
Adds the observational memory extractor pipeline.
Extractors let OM capture structured values alongside observations, persist selected values into thread metadata, and carry extraction failures through the OM lifecycle. Extractor instructions and schemas can be static or resolved from runtime context, so extraction can adapt to the active agent and thread.
Test plan
pnpm --filter ./packages/memory test:unit -- --run src/processors/observational-memory/__tests__/extractor.test.ts src/processors/observational-memory/__tests__/observational-memory-api.test.ts --reporter=dot --bail 1pnpm --filter ./packages/memory checkStack