Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 92 additions & 7 deletions src/TokenGuard.Core/PackageReadme.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,113 @@
# TokenGuard.Core

Core TokenGuard package for .NET 10 agent loops. Tracks conversation growth, prepares provider-ready snapshots, and compacts older history when token pressure builds.
TokenGuard.Core keeps your agent loop conversation inside `ConversationContext`. That object is the source of truth for the session. Before each model call, TokenGuard reads that history, builds a provider-ready snapshot, and compacts only that snapshot when needed.

```csharp
// conversationContext is source of truth for this loop.
// System prompt lives there with every other message.
conversationContext.SetSystemPrompt("You are a careful coding assistant.");

// Add user turn to same stored conversation history.
conversationContext.AddUserMessage("Fix this, make no mistake.");

// Build next provider request from that history.
// TokenGuard may compact this snapshot to fit budget.
// Stored history inside conversationContext does not change.
var prepared = await conversationContext.PrepareAsync(cancellationToken);

// Send only prepared snapshot to provider.
var input = prepared.Messages.ForOpenAI();
var response = await chatClient.CompleteChatAsync(input, cancellationToken: cancellationToken);
```

You keep appending system, user, assistant, and tool messages to `ConversationContext`. Everything happens inside that object. `PrepareAsync()` returns a `PrepareResult` describing what should go to the model right now.

## What it does

- tracks token growth across the full turn sequence
- masks stale tool results using a sliding-window strategy when the conversation crosses a configurable soft threshold
- summarizes old history with your LLM when masking alone is not enough
- falls back to emergency truncation as a last resort
- pins durable context that survives all compaction stages
- stays provider-agnostic in core, with adapter helpers for OpenAI and Anthropic
- integrates in minutes via `AddConversationContext(...)` and a standard DI factory

## Install

```bash
dotnet add package TokenGuard.Core
```

## Use
## Quick start

### 1. Register at startup

```csharp
services.AddConversationContext(builder => builder
.WithMaxTokens(25_000)
.WithCompactionThreshold(0.80));
```

Default built-in pipeline starts compaction at **80%**, always runs sliding-window masking first, and keeps LLM summarization off until you register it explicitly.

Emergency truncation is **on by default at 1.0**. It fires only at the absolute token limit and acts as a last-resort safety net after the normal compaction pipeline has already run.

Override with `WithEmergencyThreshold(0.95)` to trigger earlier, or call `WithoutEmergencyThreshold()` to disable it entirely.

### 2. Create a context per conversation

using var conversation = serviceProvider
```csharp
using var conversationContext = serviceProvider
.GetRequiredService<IConversationContextFactory>()
.Create();
```

Configuration is singleton-scoped. Each `Create()` call returns an independent stateful context, safe to use across concurrent requests.

### 3. Run the loop

```csharp
using TokenGuard.Core.Enums;
using TokenGuard.Extensions.OpenAI;

var factory = serviceProvider.GetRequiredService<IConversationContextFactory>();

using var conversationContext = factory.Create();

conversation.SetSystemPrompt("You are a careful coding assistant.");
conversation.AddUserMessage("Summarize repo status.");
conversationContext.SetSystemPrompt("You are a precise coding assistant.");
conversationContext.AddPinnedMessage(MessageRole.User, "Repository root is /workspace/project.");
conversationContext.AddUserMessage("Summarize the failing tests.");

var prepared = await conversation.PrepareAsync(cancellationToken);
while (true)
{
var prepared = await conversationContext.PrepareAsync(cancellationToken);

if (prepared.Outcome == PrepareOutcome.CannotCompact)
throw new InvalidOperationException(prepared.BudgetFailureReason);

var response = await chatClient.CompleteChatAsync(
prepared.Messages.ForOpenAI(),
chatOptions,
cancellationToken);

conversationContext.RecordModelResponse(
response.ResponseSegments(),
response.InputTokens());

if (response.ToolCalls.Count == 0)
break;

foreach (var toolCall in response.ToolCalls)
{
var result = toolExecutor.Execute(toolCall);
conversationContext.RecordToolResult(toolCall.Id, toolCall.FunctionName, result);
}
}
```

Add `TokenGuard.Extensions.OpenAI` or `TokenGuard.Extensions.Anthropic` when you need provider adapters or LLM-backed summarization.
`PrepareAsync()` returns a `PrepareResult`, not just a message list. `PrepareResult.Messages` is the prepared snapshot to send to the provider. `ConversationContext.History` remains unchanged.

## More detail

- [Root README](https://github.com/svetstoykov/TokenGuard/blob/main/README.md)
- [How TokenGuard Thinks About Context](https://github.com/svetstoykov/TokenGuard/blob/main/docs/deep-dive/context-management.md)
4 changes: 2 additions & 2 deletions src/TokenGuard.Core/TokenGuard.Core.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<Version>1.0.0</Version>
<Version>1.0.1</Version>
<PackageId>TokenGuard.Core</PackageId>
<Title>TokenGuard.Core</Title>
<Authors>Svetlozar Stoykov</Authors>
Expand All @@ -18,7 +18,7 @@
<RepositoryUrl>https://github.com/svetstoykov/TokenGuard</RepositoryUrl>
<RepositoryType>git</RepositoryType>
<PublishRepositoryUrl>true</PublishRepositoryUrl>
<PackageReleaseNotes>Initial public release of TokenGuard core. Includes token budget tracking, sliding-window masking, optional LLM summarization support, pinned messages, and emergency truncation safeguards.</PackageReleaseNotes>
<PackageReleaseNotes>Refreshes the package README with clearer getting-started guidance.</PackageReleaseNotes>
</PropertyGroup>

<ItemGroup>
Expand Down
41 changes: 39 additions & 2 deletions src/TokenGuard.Extensions.Anthropic/PackageReadme.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,24 @@
# TokenGuard.Extensions.Anthropic

Anthropic adapter package for TokenGuard on .NET 10. Adds Anthropic-backed summarization and converts prepared TokenGuard messages to Anthropic request payloads.
Anthropic adapter package for TokenGuard on .NET 10. It converts prepared TokenGuard messages to Anthropic request parts and lets TokenGuard use Anthropic to summarize older history when context gets tight.

```csharp
services.AddConversationContext(builder => builder
.WithMaxTokens(25_000)
.UseLlmSummarization(anthropicClient, "claude-3-7-sonnet-latest"));

var prepared = await conversation.PrepareAsync(cancellationToken);
var (messages, systemPrompt) = prepared.Messages.ForAnthropic();
```

Use this package when you want the core conversation model to stay provider-agnostic and only adapt to Anthropic at the boundary.

## What it does

- adds Anthropic-backed summarization through `UseLlmSummarization(...)`
- converts prepared TokenGuard messages with `ForAnthropic()`
- returns system content separately, which matches the Anthropic API shape
- keeps the Anthropic-specific behavior out of `TokenGuard.Core`

## Install

Expand All @@ -9,13 +27,32 @@ dotnet add package TokenGuard.Core
dotnet add package TokenGuard.Extensions.Anthropic
```

## Use
## Quick start

### 1. Add the provider integration

```csharp
using TokenGuard.Extensions.Anthropic;

services.AddConversationContext(builder => builder
.WithMaxTokens(25_000)
.UseLlmSummarization(anthropicClient, "claude-3-7-sonnet-latest"));
```

### 2. Prepare the request

```csharp
var prepared = await conversation.PrepareAsync(cancellationToken);
var (messages, systemPrompt) = prepared.Messages.ForAnthropic();
```

### 3. Send the Anthropic request

Use `messages` as the Anthropic message list and `systemPrompt` as the separate system value in your request builder.

`ForAnthropic()` keeps the shape aligned with Anthropic's API, where system content is separate from the main message array.

## More detail

- [Root README](https://github.com/svetstoykov/TokenGuard/blob/main/README.md)
- [How TokenGuard Thinks About Context](https://github.com/svetstoykov/TokenGuard/blob/main/docs/deep-dive/context-management.md)
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<Version>1.0.0</Version>
<Version>1.0.1</Version>
<PackageId>TokenGuard.Extensions.Anthropic</PackageId>
<Title>TokenGuard.Extensions.Anthropic</Title>
<Authors>Svetlozar Stoykov</Authors>
Expand All @@ -18,7 +18,7 @@
<RepositoryUrl>https://github.com/svetstoykov/TokenGuard</RepositoryUrl>
<RepositoryType>git</RepositoryType>
<PublishRepositoryUrl>true</PublishRepositoryUrl>
<PackageReleaseNotes>Initial public release of TokenGuard's Anthropic integration. Adds Anthropic message conversion and Anthropic-backed history summarization for long-running agent loops.</PackageReleaseNotes>
<PackageReleaseNotes>Refreshes the package README with clearer getting-started guidance.</PackageReleaseNotes>
</PropertyGroup>

<ItemGroup>
Expand Down
55 changes: 51 additions & 4 deletions src/TokenGuard.Extensions.OpenAI/PackageReadme.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,27 @@
# TokenGuard.Extensions.OpenAI

OpenAI adapter package for TokenGuard on .NET 10. Adds OpenAI-backed summarization and converts prepared TokenGuard messages to OpenAI chat messages.
OpenAI adapter package for TokenGuard on .NET 10. It converts prepared TokenGuard messages to OpenAI chat messages and lets TokenGuard use OpenAI to summarize older history when context gets tight.

```csharp
services.AddConversationContext(builder => builder
.WithMaxTokens(25_000)
.UseLlmSummarization(chatClient));

var prepared = await conversation.PrepareAsync(cancellationToken);
var messages = prepared.Messages.ForOpenAI();

var response = await chatClient.CompleteChatAsync(messages, cancellationToken: cancellationToken);
conversation.RecordModelResponse(response.ResponseSegments(), response.InputTokens());
```

Use this package when you want TokenGuard to stay provider-agnostic in core and speak OpenAI at the edge.

## What it does

- adds OpenAI-backed summarization through `UseLlmSummarization(...)`
- converts prepared TokenGuard messages with `ForOpenAI()`
- validates tool-call and tool-result pairing before the request is sent
- keeps the OpenAI-specific behavior out of `TokenGuard.Core`

## Install

Expand All @@ -9,16 +30,42 @@ dotnet add package TokenGuard.Core
dotnet add package TokenGuard.Extensions.OpenAI
```

## Use
## Quick start

### 1. Add the provider integration

```csharp
using TokenGuard.Extensions.OpenAI;

services.AddConversationContext(builder => builder
.WithMaxTokens(25_000)
.WithSlidingWindowOptions(new SlidingWindowOptions(windowSize: 12))
.UseLlmSummarization(chatClient));
```

### 2. Prepare the request

```csharp
var prepared = await conversation.PrepareAsync(cancellationToken);
var messages = prepared.Messages.ForOpenAI();
```

var response = await chatClient.CompleteChatAsync(messages, cancellationToken: cancellationToken);
conversation.RecordModelResponse(response.ResponseSegments(), response.InputTokens());
### 3. Send the OpenAI request

```csharp
var response = await chatClient.CompleteChatAsync(
messages,
chatOptions,
cancellationToken);

conversation.RecordModelResponse(
response.ResponseSegments(),
response.InputTokens());
```

`ForOpenAI()` validates tool-call and tool-result structure. If the prepared history would create an orphaned tool result or a mismatched assistant/tool sequence, it throws before the request goes out.

## More detail

- [Root README](https://github.com/svetstoykov/TokenGuard/blob/main/README.md)
- [How TokenGuard Thinks About Context](https://github.com/svetstoykov/TokenGuard/blob/main/docs/deep-dive/context-management.md)
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<Version>1.0.0</Version>
<Version>1.0.1</Version>
<PackageId>TokenGuard.Extensions.OpenAI</PackageId>
<Title>TokenGuard.Extensions.OpenAI</Title>
<Authors>Svetlozar Stoykov</Authors>
Expand All @@ -26,7 +26,7 @@
<RepositoryUrl>https://github.com/svetstoykov/TokenGuard</RepositoryUrl>
<RepositoryType>git</RepositoryType>
<PublishRepositoryUrl>true</PublishRepositoryUrl>
<PackageReleaseNotes>Initial public release of TokenGuard's OpenAI integration. Adds OpenAI message conversion and OpenAI-backed history summarization for long-running agent loops.</PackageReleaseNotes>
<PackageReleaseNotes>Refreshes the package README with clearer getting-started guidance.</PackageReleaseNotes>
</PropertyGroup>

</Project>
Loading