Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,39 @@ Two details worth knowing:
- **Ollama streams OpenAI-shaped SSE, not NDJSON**, because smith talks to its `/v1/chat/completions` endpoint. OpenRouter, OpenAI and Ollama therefore share one reader; Anthropic has its own for its named-event format.
- **A stream that dies after text has already appeared is not retried.** Replaying the request would print the same text a second time. Failures before the first token — connection, HTTP status — still go through the normal retry handler.

### @-Mentions

Naming a file in the prompt costs a full provider roundtrip: the model reads the name, calls `read_file`, waits. `@path` skips that — the file is in the first request:

```
> explain the loop in @src/smith/agent.cr
📎 src/smith/agent.cr (174 lines)
```

The `@path` stays in the sentence so it still reads; the content is appended below it, in the same shape as a skill attachment.

| Form | Effect |
|---|---|
| `@src/agent.cr` | file content is appended |
| `@src/tools/` | **listing only** — a directory is never walked |
| `@"my notes.md"` | quotes carry a path with spaces |
| `foo@bar.com` | nothing — `@` only counts at the start of a word |

Paths resolve against the working directory, `~` expands, and trailing sentence punctuation is not part of the path (`@src/agent.cr.` finds the file). A path that does not exist stays in the text untouched with a warning — you may have meant a literal `@`.

```toml
[mentions]
max_lines = 2000 # per file, then it is truncated and marked
max_total_bytes = 262144 # across all mentions of one prompt
allow_outside = false
```

Three things worth knowing:

- **A mention that leaves the project is refused** unless `allow_outside` is set. A prompt does not always come from you — a skill body could otherwise pull in `@~/.ssh/id_rsa`.
- **Binary files are not embedded.** Detection is a null byte in the first kilobyte, the same test `grep` uses.
- **Skills expand first, mentions second**, so a skill body that references `@files` resolves too. Exactly one level: what a mention pulls in is never scanned again, so a file cannot drag itself back in through a skill.

### Thinking

Anthropic models can reason before they answer. smith keeps those blocks in the transcript — the API rejects the next request otherwise, because a thinking block carries a signature that has to come back untouched — and renders them as they stream, so a long research phase is no longer silent.
Expand Down Expand Up @@ -887,6 +920,7 @@ src/
├── output.cr # Human & JSON Lines renderers for the event stream
├── project_ctx.cr # SMITH.md & AGENTS.md discovery
├── skills.cr # Skill catalog discovery & $skill / /skill expansion
├── mentions.cr # @path expansion: file embedding, budgets & path guard
├── agents.cr # Custom agent definitions in .smith/agents/<name>.md
├── frontmatter.cr # Shared --- header parser for skills and agents
├── todos.cr # Todo list state, validation & change callback
Expand Down
28 changes: 28 additions & 0 deletions spec/smith/config_spec.cr
Original file line number Diff line number Diff line change
Expand Up @@ -503,6 +503,34 @@ describe "web settings" do
end
end

describe "mention settings" do
it "defaults to project-local mentions with Claude Code's line limit" do
with_sandbox do |temp_dir, _home|
mentions = Smith::Config.load(make_project(temp_dir)).mentions

mentions.max_lines.should eq(2000)
mentions.max_total_bytes.should eq(262144)
mentions.allow_outside?.should be_false
end
end

it "reads the section" do
with_sandbox do |temp_dir, _home|
project = make_project(temp_dir, <<-TOML)
[mentions]
max_lines = 50
max_total_bytes = 1024
allow_outside = true
TOML

mentions = Smith::Config.load(project).mentions
mentions.max_lines.should eq(50)
mentions.max_total_bytes.should eq(1024)
mentions.allow_outside?.should be_true
end
end
end

describe "thinking settings" do
it "is off by default, and leaves the legacy budget unset" do
with_sandbox do |temp_dir, _home|
Expand Down
201 changes: 201 additions & 0 deletions spec/smith/mentions_spec.cr
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
require "../spec_helper"
require "file_utils"
require "../../src/smith/mentions"

private def with_project(&)
dir = File.join(Dir.tempdir, "smith_mentions_#{Random::Secure.hex(4)}")
FileUtils.mkdir_p(dir)
# The tempdir is a symlink on macOS, so the project dir has to be resolved
# the same way the guard resolves candidate paths — otherwise every relative
# mention would look like an escape.
begin
yield File.realpath(dir)
ensure
FileUtils.rm_rf(dir)
end
end

private def expand(text : String, project_dir : String, **overrides)
settings = Smith::Mentions::Settings.new(**overrides)
Smith::Mentions.expand(text, project_dir, settings)
end

describe Smith::Mentions do
it "embeds a mentioned file and leaves the sentence readable" do
with_project do |dir|
File.write(File.join(dir, "notes.md"), "line one\nline two")

result = expand("look at @notes.md and explain", dir)

result.text.should contain("look at @notes.md and explain")
result.text.should contain("--- File: notes.md (2 lines) ---")
result.text.should contain("line one")
result.files.map(&.path).should eq(["notes.md"])
end
end

it "leaves email addresses alone" do
with_project do |dir|
result = expand("mail foo@bar.com about it", dir)

result.text.should eq("mail foo@bar.com about it")
result.files.should be_empty
end
end

it "reads a quoted path with spaces" do
with_project do |dir|
File.write(File.join(dir, "my notes.md"), "content")

result = expand(%(read @"my notes.md" please), dir)

result.files.map(&.path).should eq(["my notes.md"])
result.text.should contain("content")
end
end

it "lists a directory instead of inlining what is in it" do
with_project do |dir|
FileUtils.mkdir_p(File.join(dir, "src"))
File.write(File.join(dir, "src", "a.cr"), "puts 1")
File.write(File.join(dir, "src", "b.cr"), "puts 2")

result = expand("what is in @src/ ?", dir)

result.text.should contain("--- Directory: src/ (2 entries) ---")
result.text.should contain("a.cr")
result.text.should contain("b.cr")
result.text.should_not contain("puts 1")
end
end

it "leaves a missing path untouched and reports it rather than failing" do
with_project do |dir|
result = expand("look at @nope.cr", dir)

result.text.should eq("look at @nope.cr")
result.files.should be_empty
result.skipped.map(&.path).should eq(["nope.cr"])
result.skipped.first.reason.should contain("does not exist")
end
end

it "refuses a path that escapes the project" do
with_project do |dir|
result = expand("read @../../etc/passwd", dir)

result.files.should be_empty
result.skipped.first.reason.should contain("outside the project")
result.text.should_not contain("root:")
end
end

it "allows the escape when it is explicitly configured" do
with_project do |dir|
outside = File.join(Dir.tempdir, "smith_outside_#{Random::Secure.hex(4)}.txt")
File.write(outside, "external")

begin
result = expand("read @#{outside}", dir, allow_outside: true)

result.files.size.should eq(1)
result.text.should contain("external")
ensure
File.delete(outside) if File.exists?(outside)
end
end
end

it "truncates a file over the line limit and says so" do
with_project do |dir|
File.write(File.join(dir, "big.txt"), (1..10).map { |i| "line #{i}" }.join("\n"))

result = expand("see @big.txt", dir, max_lines: 3)

result.text.should contain("line 3")
result.text.should_not contain("line 4")
result.text.should contain("truncated")
result.files.first.truncated?.should be_true
end
end

it "stops embedding once the total budget is spent" do
with_project do |dir|
File.write(File.join(dir, "one.txt"), "a" * 200)
File.write(File.join(dir, "two.txt"), "b" * 200)

result = expand("@one.txt @two.txt", dir, max_total_bytes: 250)

result.files.map(&.path).should eq(["one.txt"])
result.skipped.map(&.path).should eq(["two.txt"])
result.skipped.first.reason.should contain("budget")
result.text.should_not contain("b" * 200)
end
end

it "does not embed a binary file" do
with_project do |dir|
File.write(File.join(dir, "blob.bin"), "PNG\u0000\u0001binary")

result = expand("what is @blob.bin", dir)

result.files.should be_empty
result.skipped.first.reason.should contain("binary")
end
end

it "embeds several mentions, each once" do
with_project do |dir|
File.write(File.join(dir, "a.txt"), "alpha")
File.write(File.join(dir, "b.txt"), "beta")

result = expand("compare @a.txt with @b.txt and @a.txt again", dir)

result.files.map(&.path).should eq(["a.txt", "b.txt"])
result.text.scan(/--- File: a\.txt/).size.should eq(1)
end
end

it "resolves a mention that a skill body brought in, but goes no deeper" do
# Skills expand first, so this is the combination the CLI actually builds.
with_project do |dir|
File.write(File.join(dir, "inner.txt"), "inner content")
File.write(File.join(dir, "outer.txt"), "see @inner.txt")

skill_expanded = "run it\n\n--- Skill Context: demo ---\ncheck @outer.txt"
result = expand(skill_expanded, dir)

result.files.map(&.path).should eq(["outer.txt"])
result.text.should contain("see @inner.txt")
# One level only: what outer.txt mentions is not pulled in as well.
result.text.should_not contain("inner content")
end
end

it "ignores a bare @ with nothing after it" do
with_project do |dir|
result = expand("what does @ mean", dir)

result.text.should eq("what does @ mean")
result.skipped.should be_empty
end
end

it "resolves ~ against the home directory" do
with_project do |dir|
File.write(File.join(dir, "home-notes.md"), "from home")

previous = ENV["HOME"]?
ENV["HOME"] = dir
begin
# Without expansion this would look for a directory literally named "~".
result = expand("read @~/home-notes.md", dir)

result.files.map(&.path).should eq(["~/home-notes.md"])
result.text.should contain("from home")
ensure
previous ? (ENV["HOME"] = previous) : ENV.delete("HOME")
end
end
end
end
30 changes: 30 additions & 0 deletions spec/smith/output_spec.cr
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,36 @@ describe "thinking output" do
stdout_io.to_s.should_not contain("encrypted-payload")
end

it "shows what a mention pulled in, and what it did not" do
stdout_io = IO::Memory.new
Smith::Output::HumanRenderer.new(stdout_io).handle(
Smith::Events::FilesMentioned.new(
[Smith::Mentions::Embedded.new("src/agent.cr", 174)],
[Smith::Mentions::Skip.new("secrets.env", "outside the project")]
)
)

output = stdout_io.to_s
output.should contain("src/agent.cr (174 lines)")
output.should contain("secrets.env — outside the project")
end

it "reports mentions structurally (json)" do
stdout_io = IO::Memory.new
Smith::Output::JsonRenderer.new(stdout_io, IO::Memory.new).handle(
Smith::Events::FilesMentioned.new(
[Smith::Mentions::Embedded.new("big.txt", 2000, truncated: true)],
[] of Smith::Mentions::Skip
)
)

line = parsed_lines(stdout_io).first
line["type"].as_s.should eq("files_mentioned")
line["files"][0]["path"].as_s.should eq("big.txt")
line["files"][0]["truncated"].as_bool.should be_true
line["skipped"].as_a.should be_empty
end

it "uses its own event types, leaving assistant_text consumers alone (json)" do
stdout_io = IO::Memory.new
renderer = Smith::Output::JsonRenderer.new(stdout_io, IO::Memory.new)
Expand Down
12 changes: 12 additions & 0 deletions src/smith/cli.cr
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ require "./agent"
require "./session"
require "./project_ctx"
require "./skills"
require "./mentions"
require "./agents"
require "./config"
require "./output"
Expand Down Expand Up @@ -568,7 +569,18 @@ module Smith
return false
end

# Skills first, mentions second: a skill body may itself reference
# @files, and this way those resolve too. Only one level deep — what a
# mention pulls in is never scanned again, so a file cannot drag itself
# back in through a skill.
expanded = @skills_catalog.expand_prompt(text)

mentions = Mentions.expand(expanded, Dir.current, @config.mentions)
if mentions.any?
renderer.handle(Events::FilesMentioned.new(mentions.files, mentions.skipped))
end
expanded = mentions.text

if context = outcome.additional_context
expanded = "#{expanded}\n\n#{context}"
end
Expand Down
11 changes: 11 additions & 0 deletions src/smith/config.cr
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ require "./paths"
require "./mode"
require "./hooks"
require "./subagents"
require "./mentions"

module Smith
# Resolved configuration, merged from (lowest to highest priority):
Expand Down Expand Up @@ -308,6 +309,16 @@ module Smith
lookup(*keys).try(&.as_a?).try(&.compact_map(&.as_s?)) || Array(String).new
end

# Budgets for @-mentions. allow_outside is off by default: a prompt coming
# from a skill file could otherwise pull in ~/.ssh/id_rsa.
def mentions : Mentions::Settings
Mentions::Settings.new(
max_lines: lookup("mentions", "max_lines").try(&.as_i?) || Mentions::DEFAULT_MAX_LINES,
max_total_bytes: lookup("mentions", "max_total_bytes").try(&.as_i?) || Mentions::DEFAULT_MAX_TOTAL_BYTES,
allow_outside: lookup("mentions", "allow_outside").try(&.as_bool?) || false
)
end

# Consumed by issue #3 (history compaction).
def context : ContextSettings
ContextSettings.new(
Expand Down
12 changes: 12 additions & 0 deletions src/smith/events.cr
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ require "./llm/types"
require "./todos"
require "./mode"
require "./hooks"
require "./mentions"

module Smith::Events
abstract class Event
Expand Down Expand Up @@ -165,5 +166,16 @@ module Smith::Events
end
end

# What an @-mention actually pulled into the prompt. Emitted even when
# everything was skipped: a mention that quietly did nothing is worse than
# one that says why.
class FilesMentioned < Event
getter files : Array(Mentions::Embedded)
getter skipped : Array(Mentions::Skip)

def initialize(@files : Array(Mentions::Embedded), @skipped : Array(Mentions::Skip))
end
end

alias Listener = Proc(Event, Nil)
end
Loading