Skip to content

Commit 4f6fc71

Browse files
committed
✨ feat: token tracking, console UI overhaul, compaction improvements, and docs redesign
- meeseeks_core/token_budget.py: per-agent token tracking with cache and reasoning subtotals - meeseeks_core/tool_use_loop.py: Langfuse per-invocation traces, steering message persistence, compaction anchored to real usage_metadata - meeseeks_core/compact.py: configurable compaction model with priority-ordered fallback list; caveman-mode flag; auto provider prompt caching hook - meeseeks_core/llm.py: MCP tool schema sanitizer covering $defs, items-as-list, nullable arrays, and missing array items - meeseeks_tools/core: resolve_safe_path honors symlinks inside project roots - meeseeks_api/backend.py: GET /sessions/:id/usage endpoint with build_usage_numbers helper - meeseeks_cli: usage footer with root/sub split and compaction count; /tokens context-window default corrected for sonnet-4-6 - meeseeks_console: shadcn/ui, TanStack Query, wouter, react-hook-form/zod, and vite-plugin-PWA adopted; all data hooks migrated to TanStack Query; token usage panel, context-window bar, FileReadCard, HighlightedCode, InputComposerBody, AgentIdChip added; ConfigMenu, InputBar, NavBar, NotificationPanel, and NewProjectForm rewritten against new primitives - docs/: 22-page journey-based site with Meeseeks branding, sidebar nav icons, syntax highlighting, Mermaid dark-mode support, and console CSS-variable palette - tests/: test_compact, test_llm, test_resolve_safe_path_symlinks, test_session_runtime, test_tools_integration added; test_token_budget and test_hooks expanded
1 parent dec15fe commit 4f6fc71

176 files changed

Lines changed: 25737 additions & 13656 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

AGENTS.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1-
This is a shim file for Claude agents.
1+
<!-- meeseeks:noload -->
2+
This is a shim file for external agents.
23

34
Read and follow [@CLAUDE.md](./CLAUDE.md) in this directory as the source of truth for project instructions.

README.md

Lines changed: 28 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11

2-
<h1 align="center">Meeseeks: The Personal Assistant 👋</h1>
2+
<p align="center">
3+
<img src="docs/logos/logo-transparent.svg" alt="Meeseeks logo" width="96" />
4+
</p>
5+
6+
<h1 align="center">Meeseeks: The Personal Assistant</h1>
37

48
<p align="center">
59
<a href="https://deepwiki.com/bearlike/Assistant"><img alt="Ask DeepWiki" src="https://deepwiki.com/badge.svg"></a>
@@ -35,6 +39,27 @@ The web console provides a task orchestration frontend backed by the REST API. I
3539
</tr>
3640
</table>
3741

42+
### Capabilities at a glance
43+
44+
<table align="center">
45+
<tr>
46+
<th>Plan mode approval</th>
47+
<th>Live diff on every edit</th>
48+
</tr>
49+
<tr>
50+
<td align="center"><img src="docs/meeseeks-console-03-plan-approval.jpg" alt="Plan approval in the Meeseeks console" height="300px"></td>
51+
<td align="center"><img src="docs/meeseeks-console-04-file-edit.jpg" alt="File-edit diff card in the Meeseeks console" height="300px"></td>
52+
</tr>
53+
<tr>
54+
<th>Plugin marketplace</th>
55+
<th>Virtual projects</th>
56+
</tr>
57+
<tr>
58+
<td align="center"><img src="docs/meeseeks-console-05-plugins.jpg" alt="Plugins page with installed plugins and marketplace listings" height="300px"></td>
59+
<td align="center"><img src="docs/meeseeks-console-06-projects.jpg" alt="Projects page showing virtual workspaces shared across sessions" height="300px"></td>
60+
</tr>
61+
</table>
62+
3863
## Features
3964

4065
### Core workflow
@@ -170,25 +195,9 @@ See [docs/index.md](docs/index.md) for the full architecture diagram.
170195

171196
## Documentation
172197

173-
**Overview**
174-
- [docs/index.md](docs/index.md) — product overview and architecture
175-
176-
**Setup and configuration**
177-
- [docs/getting-started.md](docs/getting-started.md) — setup guide (env, MCP, configs, run paths)
178-
179-
**Repository map**
180-
- [docs/components.md](docs/components.md) — monorepo map
181-
182-
**Clients**
183-
- [docs/clients-cli.md](docs/clients-cli.md) — terminal CLI
184-
- [docs/clients-web-api.md](docs/clients-web-api.md) — web console and REST API
185-
- [docs/clients-home-assistant.md](docs/clients-home-assistant.md) — Home Assistant voice integration
186-
- [docs/clients-nextcloud-talk.md](docs/clients-nextcloud-talk.md) — Nextcloud Talk chat integration
187-
- [docs/clients-email.md](docs/clients-email.md) — email channel (IMAP/SMTP)
198+
Full docs live at **[kanth.tech/Assistant](https://kanth.tech/Assistant/)** — including setup, every client surface, the capability reference, deployment guides, and the internals/SDK track. The source lives under [`docs/`](docs/) and is published with MkDocs.
188199

189-
**Reference**
190-
- [docs/reference.md](docs/reference.md) — API reference (mkdocstrings)
191-
- [docs/session-runtime.md](docs/session-runtime.md) — shared session runtime used by CLI + API
200+
If you are just getting started, jump to [Get Started](https://kanth.tech/Assistant/getting-started/).
192201

193202
## Development principles
194203

apps/meeseeks_api/src/meeseeks_api/backend.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1059,19 +1059,54 @@ def get(self, session_id: str) -> tuple[dict, int]:
10591059
"detail": e.get("payload", {}).get("detail"),
10601060
"status": e.get("payload", {}).get("status"),
10611061
"steps_completed": e.get("payload", {}).get("steps_completed", 0),
1062+
"input_tokens": e.get("payload", {}).get("input_tokens", 0),
1063+
"output_tokens": e.get("payload", {}).get("output_tokens", 0),
10621064
"ts": e.get("ts"),
10631065
}
10641066
for e in events
10651067
if e.get("type") == "sub_agent"
10661068
]
10671069
running = runtime.is_running(session_id)
1070+
stop_agents = [a for a in agents if a.get("action") == "stop"]
1071+
total_input_tokens = sum(a.get("input_tokens", 0) for a in stop_agents)
1072+
total_output_tokens = sum(a.get("output_tokens", 0) for a in stop_agents)
10681073
return {
10691074
"agents": agents,
10701075
"running": running,
10711076
"total_steps": total_steps,
1077+
"total_input_tokens": total_input_tokens,
1078+
"total_output_tokens": total_output_tokens,
10721079
}, 200
10731080

10741081

1082+
@ns.route("/sessions/<string:session_id>/usage")
1083+
class SessionUsage(Resource):
1084+
"""Return token usage broken down by root agent vs sub-agents."""
1085+
1086+
@api.doc(security="apikey")
1087+
def get(self, session_id: str) -> tuple[dict, int]:
1088+
"""Return root/sub-agent token usage + compaction stats."""
1089+
auth_error = _require_api_key()
1090+
if auth_error:
1091+
return auth_error
1092+
from meeseeks_core.token_budget import build_usage_numbers
1093+
1094+
events = runtime.load_events(session_id)
1095+
root_model: str | None = None
1096+
for event in reversed(events):
1097+
if event.get("type") != "context":
1098+
continue
1099+
payload = event.get("payload")
1100+
if isinstance(payload, dict):
1101+
candidate = payload.get("model")
1102+
if isinstance(candidate, str) and candidate:
1103+
root_model = candidate
1104+
break
1105+
if not root_model:
1106+
root_model = str(get_config_value("llm", "default_model", default="") or "")
1107+
return build_usage_numbers(events, root_model), 200
1108+
1109+
10751110
@ns.route("/sessions/<string:session_id>/archive")
10761111
class SessionArchive(Resource):
10771112
"""Archive or unarchive a session."""

apps/meeseeks_cli/src/meeseeks_cli/cli_agent_display.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@ def on_stop(self, handle: AgentHandle) -> None:
9999
state.last_tool = handle.last_tool_id
100100
state.error = str(handle.error)[:100] if handle.error else None
101101
state.stopped_at = handle.stopped_at
102+
state.token_count = handle.input_tokens + handle.output_tokens
102103

103104
# ------------------------------------------------------------------
104105
# Tool execution hooks (pre_tool_use / post_tool_use)

apps/meeseeks_cli/src/meeseeks_cli/cli_commands.py

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@
99
from urllib.request import Request, urlopen
1010

1111
from meeseeks_core.config import get_config_value, get_mcp_config_path
12-
from meeseeks_core.token_budget import get_token_budget
12+
from meeseeks_core.token_budget import get_token_budget, read_last_input_tokens
1313
from meeseeks_core.tool_registry import ToolRegistry, ToolSpec, load_registry
1414
from rich import box
1515
from rich.console import Console, Group
@@ -698,12 +698,31 @@ def _cmd_automatic(context: CommandContext, args: list[str]) -> bool:
698698
return True
699699

700700

701+
def _resolve_cli_model(context: CommandContext) -> str:
702+
"""Return the effective model name, falling back to config default.
703+
704+
The CLI leaves ``state.model_name`` empty when ``--model`` was not
705+
supplied and relies on ``build_chat_model`` to pick the default at
706+
LLM-call time. Usage/budget helpers that resolve the context window
707+
via LiteLLM need an explicit name — feed them the same default.
708+
"""
709+
name = context.state.model_name or ""
710+
if name:
711+
return name
712+
return str(get_config_value("llm", "default_model", default="") or "")
713+
714+
701715
@REGISTRY.command("/tokens", "Show token usage and remaining context")
702716
def _cmd_tokens(context: CommandContext, args: list[str]) -> bool:
703717
del args
704718
events = context.store.load_transcript(context.state.session_id)
705719
summary = context.store.load_summary(context.state.session_id)
706-
budget = get_token_budget(events, summary, context.state.model_name)
720+
budget = get_token_budget(
721+
events,
722+
summary,
723+
_resolve_cli_model(context),
724+
last_input_tokens=read_last_input_tokens(events),
725+
)
707726
table = Table(title="Token Budget", show_lines=True)
708727
table.add_column("Metric", style="cyan")
709728
table.add_column("Value")

apps/meeseeks_cli/src/meeseeks_cli/cli_dialogs.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,11 @@
1616
from textual.containers import Vertical
1717
from textual.widgets import Input, Label, OptionList, SelectionList
1818

19+
_DOTTED_BOX = box.Box(
20+
".:.:\n: ::\n.:.:\n: ::\n.:.:\n.:.:\n: ::\n.:.:",
21+
ascii=True,
22+
)
23+
1924

2025
def _textual_enabled() -> bool:
2126
if get_config_value("cli", "disable_textual", default=False):

apps/meeseeks_cli/src/meeseeks_cli/cli_master.py

Lines changed: 97 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,13 +9,15 @@
99
from collections.abc import Callable, Iterable
1010
from dataclasses import dataclass
1111
from pathlib import Path
12+
from typing import Any
1213

1314
from prompt_toolkit import PromptSession
1415
from prompt_toolkit.history import FileHistory
1516
from rich import box
1617
from rich.columns import Columns
1718
from rich.console import Console, Group, RenderableType
1819
from rich.live import Live
20+
from rich.markdown import Markdown
1921
from rich.panel import Panel
2022
from rich.rule import Rule
2123
from rich.status import Status
@@ -711,12 +713,70 @@ def _approval_with_keys(step: ActionStep) -> bool:
711713
border_style="bold green",
712714
)
713715
)
716+
_print_usage_footer(console, store, state.session_id, state.model_name)
714717

715718
# Surface recovery hint when the run ended in a recoverable failure so
716719
# users can type ``/retry`` or ``/continue`` at the next prompt.
717720
_maybe_print_recovery_hint(console, store, state.session_id)
718721

719722

723+
def _print_usage_footer(
724+
console: Console,
725+
store: SessionStoreBase,
726+
session_id: str,
727+
model_name: str | None,
728+
) -> None:
729+
"""Print a single-line token usage summary below the response panel.
730+
731+
Shows the same faceted view the console footer does: root agent
732+
headroom vs. model max (the actual context-window pressure), plus a
733+
sub-agent rollup and compaction count. Numbers come from the one
734+
``build_usage_numbers`` helper — no duplicate aggregation here.
735+
"""
736+
try:
737+
from meeseeks_core.token_budget import build_usage_numbers
738+
739+
effective_model = model_name or str(
740+
get_config_value("llm", "default_model", default="") or ""
741+
)
742+
events = store.load_transcript(session_id)
743+
u = build_usage_numbers(events, effective_model)
744+
except Exception:
745+
return # silent — footer is decorative
746+
max_in = u["root_max_input_tokens"]
747+
if max_in <= 0 and u["total_input_tokens_billed"] == 0:
748+
return # nothing meaningful yet
749+
pct = int(round(u["root_utilization"] * 100))
750+
line = Text()
751+
line.append(f"{u['root_model']}", style="dim cyan")
752+
line.append(" ", style="dim")
753+
line.append(
754+
f"root {_fmt_tokens(u['root_last_input_tokens'])}/{_fmt_tokens(max_in)} ({pct}%)",
755+
style="dim",
756+
)
757+
if u["sub_peak_input_tokens"] or u["sub_output_tokens"]:
758+
# Sub-agents run in isolated contexts; show combined peak pressure
759+
# (sum of per-agent peaks) + summed output.
760+
sub_peak = _fmt_tokens(u["sub_peak_input_tokens"])
761+
sub_out = _fmt_tokens(u["sub_output_tokens"])
762+
line.append(" · sub peak ", style="dim")
763+
line.append(f"{sub_peak} / {sub_out} out", style="dim")
764+
line.append(" · ", style="dim")
765+
line.append(f"{_fmt_tokens(u['tokens_until_compact'])} until compact", style="dim")
766+
if u["compaction_count"] > 0:
767+
line.append(f" · ⊙ {u['compaction_count']} compaction(s)", style="dim")
768+
console.print(line)
769+
770+
771+
def _fmt_tokens(n: int) -> str:
772+
"""Format a token count: 5200 → '5.2k', 1500000 → '1.5m'."""
773+
if n >= 1_000_000:
774+
return f"{n / 1_000_000:.1f}m"
775+
if n >= 1_000:
776+
return f"{n / 1_000:.1f}k"
777+
return str(n)
778+
779+
720780
def _maybe_print_recovery_hint(
721781
console: Console, store: SessionStoreBase, session_id: str
722782
) -> None:
@@ -1035,8 +1095,43 @@ def _build_cli_hook_manager(
10351095
) -> HookManager:
10361096
# When agent display is active, the live tree + integrated spinner
10371097
# replace the per-tool console.status() spinner.
1038-
def _on_compact(session_id: str) -> None:
1039-
console.print("[dim blue]Context compacted[/dim blue]")
1098+
def _on_compact(session_id: str, **kwargs: Any) -> None:
1099+
summary = kwargs.get("summary", "")
1100+
tokens_before = kwargs.get("tokens_before", 0)
1101+
tokens_saved = kwargs.get("tokens_saved", 0)
1102+
events_summarized = kwargs.get("events_summarized", 0)
1103+
tokens_after = tokens_before - tokens_saved
1104+
1105+
parts: list[RenderableType] = []
1106+
if tokens_before and tokens_saved:
1107+
pct = round((tokens_saved / tokens_before) * 100)
1108+
parts.append(
1109+
Text(f"{tokens_before:,}{tokens_after:,} tokens ({pct}% reduction)", style="dim")
1110+
)
1111+
elif tokens_before:
1112+
parts.append(Text(f"{tokens_before:,} tokens in context", style="dim"))
1113+
if events_summarized:
1114+
parts.append(Text(f"{events_summarized} events summarized", style="dim"))
1115+
if summary:
1116+
parts.append(Text(""))
1117+
parts.append(Markdown(summary))
1118+
elif parts:
1119+
parts.append(Text(""))
1120+
parts.append(
1121+
Text("Summary unavailable — structured compaction failed.", style="dim italic")
1122+
)
1123+
1124+
if parts:
1125+
console.print(
1126+
Panel(
1127+
Group(*parts),
1128+
title="[dim blue]Context Compacted[/dim blue]",
1129+
border_style="dim blue",
1130+
padding=(0, 1),
1131+
)
1132+
)
1133+
else:
1134+
console.print("[dim blue]Context compacted[/dim blue]")
10401135

10411136
if agent_display is not None:
10421137
return HookManager(

apps/meeseeks_console/.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,3 +24,7 @@ test-results
2424
*.njsproj
2525
*.sln
2626
*.sw?
27+
28+
# Re-include shadcn/ui scaffolding (root .gitignore excludes lib/ for Python packaging)
29+
!src/lib/
30+
!src/lib/**

0 commit comments

Comments
 (0)