From dd39c41988bf7d32af65be0fd67e4d0d77cd5217 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Fri, 21 Aug 2026 18:34:54 +0200 Subject: [PATCH 01/27] Let an agent investigate the heap dump in a window somebody is watching MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Shark Explorer's windows now answer MCP, so an agent reads the dump that is already open — the same tree, the same verdicts, the same notes — rather than a copy of its own, and `show` puts what it is looking at on the person's screen. The reason it is a server rather than a set of shark-cli commands is that a server can say no, and refusing is the whole mechanism: it works with any client and nothing here ever calls a model. - Every call takes a `reason`, enforced in AgentTool.call rather than only asked for in the schema, and it lands in the run's log beside the reads it caused. An investigation becomes something a person can follow afterwards instead of a conclusion they have to trust. - A verdict needs a reason another reader can check, and one that contradicts a verdict already recorded is refused with the list of what it disagrees with. - `conclude` is refused until the heap dump itself agrees that one reference is at fault, and the refusal says which of the three ways that fails it is. An agent that has narrowed a chain to three unexplained steps cannot report a root cause, however confident it is. The method — the LeakCanary method, as prose for a model — is handed over twice, in the handshake and again with open_heap_dumps, because some clients drop the handshake's instructions and a method nobody read is a method nobody followed. Two parts to the transport: a run publishes a loopback port and a token under ~/.shark-explorer/agents, and `--mcp-stdio` is a mode of the same app binary that pipes stdio to it, since an MCP client can be configured with a command and not with a port that changes every run. harness/start-harness.sh is how this actually gets tested: it opens a window and prints the command that throws an agent with no knowledge of this repository at it, prompted with nothing but "find the root cause". The documentation quotes that run. Co-Authored-By: Claude Opus 5 --- build.gradle.kts | 1 + docs/shark-explorer-changelog.md | 7 + docs/shark-explorer.md | 108 +++ gradle/libs.versions.toml | 5 + settings.gradle | 1 + shark/shark-explorer/AGENTS.md | 1 + .../shark-explorer-agent/AGENTS.md | 126 ++++ .../shark-explorer-agent/CLAUDE.md | 1 + .../shark-explorer-agent/build.gradle.kts | 20 + .../harness/start-harness.sh | 130 ++++ .../shark/explorer/agent/AgentHeapDump.kt | 84 +++ .../java/shark/explorer/agent/AgentJson.kt | 288 ++++++++ .../java/shark/explorer/agent/AgentMethod.kt | 104 +++ .../java/shark/explorer/agent/AgentServer.kt | 216 ++++++ .../shark/explorer/agent/AgentStdioBridge.kt | 160 +++++ .../java/shark/explorer/agent/AgentTool.kt | 223 +++++++ .../java/shark/explorer/agent/AgentTools.kt | 626 ++++++++++++++++++ .../java/shark/explorer/agent/McpSession.kt | 248 +++++++ .../shark/explorer/agent/AgentHeapDumps.kt | 122 ++++ .../shark/explorer/agent/AgentServerTest.kt | 158 +++++ .../explorer/agent/AgentStdioBridgeTest.kt | 164 +++++ .../shark/explorer/agent/AgentToolsTest.kt | 515 ++++++++++++++ .../shark/explorer/agent/FakeAgentHeapDump.kt | 73 ++ .../shark/explorer/agent/McpSessionTest.kt | 211 ++++++ .../java/shark/explorer/agent/RecordedLog.kt | 50 ++ .../shark-explorer-app/build.gradle.kts | 2 + .../java/shark/explorer/app/ExplorerAgents.kt | 170 +++++ .../java/shark/explorer/app/ExplorerWindow.kt | 10 + .../src/main/java/shark/explorer/app/Main.kt | 47 +- .../src/main/java/shark/explorer/NodeIds.kt | 9 +- 30 files changed, 3872 insertions(+), 8 deletions(-) create mode 100644 shark/shark-explorer/shark-explorer-agent/AGENTS.md create mode 100644 shark/shark-explorer/shark-explorer-agent/CLAUDE.md create mode 100644 shark/shark-explorer/shark-explorer-agent/build.gradle.kts create mode 100755 shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTool.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentHeapDumps.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/RecordedLog.kt create mode 100644 shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt diff --git a/build.gradle.kts b/build.gradle.kts index bebbfff38b..4d61c99b9f 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -197,6 +197,7 @@ val modulesWithoutPublicApi = listOf( "leakcanary-app-db", "leakcanary-app-service", "shark-cli", + "shark-explorer-agent", "shark-explorer-app", "shark-explorer-core", "shark-explorer-jdwp", diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index 2842fb3ffb..5713eeb1c5 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -38,6 +38,13 @@ uses, without the one for a newly recognized library leak: object as stuck makes it a leak and takes whatever it holds off the list. Kept between runs in `~/.shark-explorer/leak-statuses`, one file per heap dump. See [The verdict](shark-explorer.md#the-verdict). +* ✨ **Hand a heap dump to an agent**: the window is an MCP server too, so an agent investigates the heap dump + you have open — the same tree, the same verdicts, the same notes — and `show` puts what it is looking at on + your screen. What it can be held to is the point: every call has to say why it was made and lands in the + run's log beside the reads it caused, a verdict needs a reason another reader can check exactly as yours + does, and reporting a root cause is refused until the chain names one faulty reference. Point any MCP client + at the installed app with `--mcp-stdio`. + See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). * ✨ **The chain marks the faulty reference**: the one step going from an `Expected` object straight to a `Stuck` one reads `Holder.activity · faulty reference`, which is the leak itself rather than one of the objects it left behind, and the same reference the **Leaks** screen names that leak after. A chain whose diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 718085af1c..a2092a3f6f 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -213,6 +213,114 @@ expected takes it off the list. The one thing this costs is that a leak's finger LeakCanary reports only while nothing has been set by hand, since the fingerprint is the stretch of chain your verdict has just moved. +## Hand it to an agent + +The window is also an **[MCP](https://modelcontextprotocol.io) server**, so an agent — Claude Code, Cursor, +whatever you use — investigates *the heap dump you have open* rather than one of its own. It reads the same +tree, sets verdicts you watch appear, puts what it is looking at on your screen, and writes what it concluded +into the notes where you and the next reader will find it. + +Point your client at the app itself: + +```json +{ + "mcpServers": { + "shark-explorer": { + "command": "/Applications/Shark Explorer.app/Contents/MacOS/Shark Explorer", + "args": ["--mcp-stdio"] + } + } +} +``` + +That is the app's own launcher, and `--mcp-stdio` makes this copy of it a pipe to the window already open +rather than a second window. Nothing else to install and no port to configure: it talks to the run that +started most recently, says which one that was, and takes `--agent-run=` when several explorers are +open. Open a heap dump before you start — with no window there is nothing to investigate, and it says so +rather than waiting. + +Then ask for what you actually want. This is the whole prompt the session below was given: + +> A heap dump is open in Shark Explorer, which you can reach through its MCP tools. Something in it is +> leaking. Find the root cause. + +**The method comes with the tools**, so it doesn't have to come from you. The handshake hands over what a +leak is — one bad reference, the three zones of a chain, the rules that spread a verdict up and down it — and +the order that finds it, which is [the LeakCanary +method](https://engineering.block.xyz/blog/the-leakcanary-method) as the tools enforce it. + +| Tool | What it is | +| --- | --- | +| `open_heap_dumps` | Every window and what is open in it, with the method to follow. | +| `list_leaks` | The **Leaks** screen: what this heap dump says shouldn't be there. | +| `chain_from_gc_root` | One chain, every step with its labels and its verdict. | +| `describe_object` | What an object is: its class, fields, labels, size. | +| `ways_held` | Every way an object is held, rather than the one chain. | +| `find_objects` | The object list, by class name. | +| `set_verdict`, `clear_verdict` | The pencil, with the reason required the same way. | +| `take_note` | The notes, appended to. | +| `show` | Opens a tab in your window and brings it to the front. | +| `conclude` | The root cause, and the only way to finish. | + +**And the tools refuse.** That is the part worth knowing about, because it is what an agent's confidence +cannot argue with: + +* **Every call has to say why it was made.** A call with none is refused — *describe_object needs `reason`, + and it was not given* — and so is one whose reason is blank. What that buys is the log below. +* **A verdict needs a reason another reader can check**, exactly like one you typed, and it is kept with the + verdict in the same file as yours. A verdict that contradicts one already recorded is refused with the list + of what it disagrees with, the same way the window asks you. +* **`conclude` is refused until the heap dump agrees that one reference is at fault** — one object above it + recorded as expected, the object below it recorded as stuck, and nothing unexplained in between. Reporting + a root cause before that gets this back: + +``` +Not concluded. 1 step(s) between the last NOT_LEAKING object and the first LEAKING one have no verdict, so the +fault is at one of them and the chain doesn't say which: 0x12e9ed60 java.util.ArrayList. Until the chain names +one reference, a root cause would be a guess about which of those steps is at fault. Read the objects in the +unexplained stretch with describe_object, check whether anything else holds them with ways_held, and record +what you can defend with set_verdict. +``` + +Nothing here judges the answer — no model is called and nothing is scored. It is the same rule the chain +draws by, held to before an answer can be written down: an agent that has narrowed a chain to three +unexplained steps cannot report a root cause, however sure it is, and what it gets instead is the three +objects to go and read. + +**What it did is in the log**, in `~/.shark-explorer/logs`, one line per call with the reason it gave followed +by the reads that call cost: + +``` +18:19:48.035 [shark-explorer-agents] An agent called chain_from_gc_root(object=0x12d368b8, window=zvphq4r3) + because: This is the one App leak: a MainActivity the app watched and whose mDestroyed is true. Getting the + chain from a GC root to see every reference holding it and where the faulty one might be. +18:19:48.038 [heap-dump-leak_asynctask_o.hprof] Reading the chain to 0x12d368b8, for an agent +18:19:48.043 [heap-dump-leak_asynctask_o.hprof] Read the chain to 0x12d368b8, for an agent in 4 ms +``` + +So an investigation is something you can follow afterwards rather than a conclusion you have to trust — which +is the other half of the point, since the path is the part a chat window throws away. + +**What it concluded is in the heap dump**, not only in your terminal. The verdicts are in +`~/.shark-explorer/leak-statuses` with everyone else's, and `conclude` writes a **Root cause** note on the +stuck object and opens that tab, so the answer is in the window beside the evidence and still there next week: + +> ## Root cause +> +> **Faulty reference:** `MainActivity$2.this$0` +> +> […] Because it is a non-static inner class, javac gives it a synthetic `this$0` field and assigns the +> enclosing activity to it in the constructor. That field is final and written once at construction — no code +> in the app or the framework can ever clear it. […] +> +> **Not checked:** I could not read the app's source. […] The "anonymous inner class" reading rests on the +> class name `MainActivity$2`, the synthetic `this$0` field and Shark's inspector label, not on a line of +> source. + +An agent's verdicts are verdicts like any other: they say `set by hand` on every chain that runs through the +object, the reason is the one it gave, and the pencil takes one off if you disagree with it. Which is the +last thing this surface is for — the disagreement is about a reason you can read, not about who said it. + ## Reporting a problem Bug reports go to the [LeakCanary issue tracker](https://github.com/square/leakcanary/issues). Every run diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index 98ea0625fe..a28f03d670 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -64,6 +64,11 @@ metro-runtime = { module = "dev.zacsweers.metro:runtime-jvm", version.ref = "met coroutines-core = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-core", version.ref = "coroutines" } +# The JSON the Shark explorer's agent surface speaks, in shark-explorer-agent and nowhere else. The +# runtime only, deliberately: the JsonElement API this uses needs no compiler plugin, and adding the +# serialization plugin to the build would be a per-module opt-in nothing else here wants. +kotlinx-serialization-json = { module = "org.jetbrains.kotlinx:kotlinx-serialization-json", version = "1.9.0" } + kotlin-stdlib = { module = "org.jetbrains.kotlin:kotlin-stdlib", version.ref = "kotlin" } kotlin-reflect = { module = "org.jetbrains.kotlin:kotlin-reflect", version.ref = "kotlin" } diff --git a/settings.gradle b/settings.gradle index a001828024..407800bb6d 100644 --- a/settings.gradle +++ b/settings.gradle @@ -29,6 +29,7 @@ include ':samples:leakcanary-android-sample' include ':shark:shark' include ':shark:shark-android' include ':shark:shark-cli' +include ':shark:shark-explorer:shark-explorer-agent' include ':shark:shark-explorer:shark-explorer-app' include ':shark:shark-explorer:shark-explorer-core' include ':shark:shark-explorer:shark-explorer-jdwp' diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index 7f4c61fdcc..09a9c909f9 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -13,6 +13,7 @@ reading the source alone — everything else is in the code. Keep it that way. | --- | --- | --- | | `shark-explorer-core` | Heap dump → dominator tree → layout model. Layout, hit testing, navigation state. | **No Compose dependency, Java 8 target.** Must stay reusable from the Android `leakcanary-app`. | | `shark-explorer-jdwp` | Attaches to a live app as a debugger to read the pixels of its bitmaps. | **Imports `com.sun.jdi`, so it needs a JDK and can't be loaded on Android.** That's the whole reason it isn't in `core`. | +| `shark-explorer-agent` | The MCP server a window answers agents through, and the `--mcp-stdio` pipe that reaches it. | **No Compose, Java 8 target, and desktop only** — it calls `ProcessHandle`. Has its own `AGENTS.md`. | | `shark-explorer-app` | Compose Desktop UI: window, the canvas each shape draws into, details panel. | **Java 17 target** — see below. | `shark/shark-explorer/` itself holds no code, matching how `shark/` and `leakcanary/` are grouping diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md new file mode 100644 index 0000000000..8ff3363eef --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -0,0 +1,126 @@ +# Shark Explorer's agent surface — agent guide + +An [MCP](https://modelcontextprotocol.io) server inside the running app, so that an agent investigates the +heap dump **in the window somebody is looking at** rather than one of its own. + +This file is scoped to `shark/shark-explorer/shark-explorer-agent/`. Its parent, +`shark/shark-explorer/AGENTS.md`, has the app-wide rules — the heap dump being read off the UI thread, a +verdict being an argument to every read — and they all apply here. This one only records what is specific to +being talked to by a program that is not this app. + +## What the pieces are + +| File | What it is | +| --- | --- | +| `AgentHeapDump.kt` | The seam: one open heap dump, as everything here sees it. The app implements it over a window; the tests implement it over a `HeapExplorer` and three fields. | +| `AgentTools.kt` | Every tool, each a name, a schema and one read. Where the refusals are. | +| `AgentMethod.kt` | The method, as prose handed to the model twice. | +| `AgentJson.kt` | The explorer's model as JSON. | +| `AgentTool.kt` | One tool, its arguments read strictly, and `AgentRefusal`. | +| `McpSession.kt` | JSON-RPC, one message per line. | +| `AgentServer.kt` | The loopback socket a run publishes, and the file that says where. | +| `AgentStdioBridge.kt` | `--mcp-stdio`: the pipe an MCP client launches. | +| `harness/start-harness.sh` | Opens a window and prints the command that throws an agent at it. | + +Nothing here is public API — the module is in `modulesWithoutPublicApi`, like the rest of the explorer — with +two deliberate exceptions, `AgentServer`/`AgentStdioBridge`/`AgentHeapDump*` because the app calls them, and +`AgentRefusal` because the app throws it. + +## The refusals are the feature + +The whole point of this being a server rather than a library is that **it can say no**, and it works with any +client because saying no is all it does — nothing here ever calls a model. + +- `set_verdict` refuses a blank reason (through `LeakStatusOverride`'s own `require`) and refuses a verdict + that contradicts one already recorded unless it is told to flip it. +- `conclude` refuses until the heap dump agrees that **one** reference is at fault, and the refusal says which + of the three reasons it is: nothing `LEAKING`, nothing `NOT_LEAKING` above it, or *these* steps in between + with no verdict. Same rule as `faultyReferenceIndexOrNull`, read off the chain rather than asked of it, + because the three ways it answers null are three different things to do next. +- Every tool takes a `reason`, and it is enforced in `AgentTool.call` rather than only asked for in the + schema: a client is free to ignore a schema. + +So a change that makes any of these easier to satisfy is a change that removes the reason this module exists. +An agent that has narrowed a chain to three unexplained steps must not be able to report a root cause, however +confident it is. `AgentToolsTest` walks that exact story — refused, then a verdict, then concluded — and it is +the test to keep working. + +The `reason` is traceability and not a quality gate. Asking a model to explain itself does not make it right, +and [the research says it can make it worse](https://arxiv.org/abs/2504.09664); what it buys is a session log +someone can follow afterwards instead of a conclusion they have to trust. + +## In `--mcp-stdio` mode, stdout is the protocol + +`main` answers `agentBridgeExitCode` **before `installLogging()`**, because that logger writes to stdout and +one log line in the middle of a JSON-RPC stream is a session the client reports as broken. So in this module: + +- Everything the bridge has to say goes to stderr, which is where an MCP client collects a server's log. +- Nothing in the bridge path may use `SharkLog`, `println`, or anything that ends up on stdout. + +The app's own side of it — a window answering an agent — logs through `SharkLog` as usual, so a session log +reads as the reason for each call followed by the reads it caused. That is the artefact to ask for when +somebody reports that an agent got it wrong. + +## The transport, and why it is two things + +**A run publishes a loopback port and a token** to `~/.shark-explorer/agents/.agent`, and `--mcp-stdio` +is a mode of the same app binary that pipes stdio to it. Two parts because an MCP client can be configured +with a command and not with a port that changes every run. + +Deliberately **not** the socket `DeepLinkPeers` listens on, though it is the same shape. A link is one line +answered in a millisecond; this is a session held open for as long as an investigation takes. One port for +both would mean a link arriving mid-investigation and an investigation ending when a link handler closed. + +The token is the whole of the authorization, and it is worth being clear about what that is: enough to keep a +web page or another machine out, and **not** a boundary between programs run by the same person — anything +that can read `~/.shark-explorer` can read any heap dump on the disk anyway. + +`AgentServer.serve` sets **no read timeout**, unlike the link socket. An agent thinking is a quiet connection. + +## An address is a string, never a JSON number + +A heap dump's addresses fill the range of `Long`, and a JSON number is a double to most clients of this +protocol: anything above 2^53 comes back rounded, which for an address means a different object, silently. So +every address on this surface is `exactHexObjectId` — `0x12d368b8`, the same spelling the app's own files use +— and `objectIdOfHex` is the only way back. The refusal for a decimal names that case, because a model that +has seen a numeric address elsewhere will write one here. + +## kotlinx-serialization without the plugin + +`kotlinx-serialization-json` is a **runtime dependency only**: `buildJsonObject`, `Json.parseToJsonElement` +and friends. There is no `kotlin("plugin.serialization")` on this module and no `@Serializable` anywhere, +because everything crossing this boundary is either the explorer's own model — which is not ours to annotate +— or a JSON-RPC envelope of a dozen fields. Adding the plugin to get `@Serializable` would be a compiler +plugin's worth of build for a saving of nothing. + +## It is a Java 8 target that cannot run on Java 8 + +`AgentServer` uses `ProcessHandle.current().pid()`, which is Java 9. The repo-wide Java 8 target sets +`targetCompatibility` and no `options.release`, so this compiles: the bytecode is Java 8 and the reference to +a Java 9 class is only resolved at runtime. Same trick `shark-explorer-jdwp` gets away with for `com.sun.jdi`, +and it is fine for the same reason — this is desktop-only code, loaded by the desktop app and by nothing on +Android. + +So don't "fix" it by moving the module out of the Java 8 target list. Do remember that anything added here is +under the same rule as the rest of the explorer: **no Compose, and nothing that assumes a display**, since +the reads happen on the heap dump's thread and the tests run headless. + +## Build and test + +```bash +./gradlew :shark:shark-explorer:shark-explorer-agent:check # test + detekt + +# The whole surface end to end, in a real window, with an agent that has never seen this repository. +shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh [heap-dump.hprof] +``` + +Every test here runs against a heap dump built with the `dump { }` DSL and no window, which is what +`AgentHeapDump` being an interface is for. `AgentStdioBridgeTest` is the one that goes through a real socket +in both directions — it swaps `System.in` and `System.out` around the bridge, over a pipe rather than a string +of input, because a real client keeps stdin open until it has its answer. + +**The harness is how the thing this module is for actually gets tested.** It builds the packaged app, opens +one heap dump in it, and writes an MCP config pinned to that run plus a prompt that says nothing but "find the +root cause" — so what the agent follows is the method the server handed it. Then read +`~/.shark-explorer/logs`: a run that went well and a run that guessed look completely different there, and +neither of them looks like anything in a unit test. diff --git a/shark/shark-explorer/shark-explorer-agent/CLAUDE.md b/shark/shark-explorer/shark-explorer-agent/CLAUDE.md new file mode 100644 index 0000000000..43c994c2d3 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/shark/shark-explorer/shark-explorer-agent/build.gradle.kts b/shark/shark-explorer/shark-explorer-agent/build.gradle.kts new file mode 100644 index 0000000000..b963c138f6 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/build.gradle.kts @@ -0,0 +1,20 @@ +plugins { + id("org.jetbrains.kotlin.jvm") +} + +dependencies { + api(projects.shark.sharkExplorer.sharkExplorerCore) + + implementation(libs.kotlin.stdlib) + // Every operation is a read of the heap dump, and a read is suspending because the app confines it to + // the heap dump's own thread. See shark.explorer.app.HeapDumpSession. + implementation(libs.coroutines.core) + // The JsonElement API only, so that no @Serializable class here needs the compiler plugin. The wire + // format is JSON-RPC, whose shape is decided by the protocol rather than by classes of ours. + implementation(libs.kotlinx.serialization.json) + + testImplementation(libs.junit) + testImplementation(libs.assertjCore) + // Builds the heap dumps the tool tests read, rather than checking binary fixtures in. + testImplementation(projects.shark.sharkHprofTest) +} diff --git a/shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh b/shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh new file mode 100755 index 0000000000..deca0ea5b6 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh @@ -0,0 +1,130 @@ +#!/bin/bash +# +# Opens one heap dump in a real Shark Explorer, then prints the command that throws an agent at it. +# +# What this is for: the tools in this module are meant to hold an investigation to a method, and whether +# they do is not a thing a unit test can answer — it takes a model that has never seen this repository, +# reading nothing but what the tools hand back. So this sets the stage and stops: a window someone can watch, +# an MCP config pinned to that window, and a prompt that says no more than "find the root cause". +# +# The packaged app rather than `./gradlew run`, for two reasons. It is what a person has installed, so the +# command in the config is the command they would write; and a Gradle build of any kind kills a window +# launched from source, which here would be every window this harness opened. See shark/shark-explorer/AGENTS.md. + +set -euo pipefail + +readonly REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)" +readonly DEFAULT_HEAP_DUMP="shark/shark-android/src/test/resources/leak_asynctask_o.hprof" +readonly APP_PATH="shark/shark-explorer/shark-explorer-app/build/compose/binaries/main/app/Shark Explorer.app" +readonly RUNS_DIRECTORY="$HOME/.shark-explorer/agents" +readonly TEMPORARY_DIRECTORY="${TMPDIR:-/tmp}" +readonly HARNESS_DIRECTORY="${SHARK_HARNESS_DIR:-${TEMPORARY_DIRECTORY%/}/shark-explorer-harness}" +readonly TITLE="${SHARK_HARNESS_TITLE:-Agent harness}" +readonly WAIT_SECONDS=90 + +main() { + local heap_dump + heap_dump="$(absolute_path "${1:-$REPO_ROOT/$DEFAULT_HEAP_DUMP}")" + if [[ ! -f "$heap_dump" ]]; then + echo "No heap dump at $heap_dump" >&2 + exit 1 + fi + + # Everything that builds happens before the window opens, because building rewrites the jars a window + # launched from source is reading. A packaged app is a copy and survives it, but the ordering costs + # nothing and one day somebody will point this at `run`. + echo "Building the app. jlink takes about a minute the first time." + (cd "$REPO_ROOT" && ./gradlew --quiet :shark:shark-explorer:shark-explorer-app:createDistributable) + + local app="$REPO_ROOT/$APP_PATH" + local before + before="$(published_runs)" + echo "Opening $(basename "$heap_dump") in a window called \"$TITLE\"." + open -n "$app" --args --title="$TITLE" "$heap_dump" + + local pid + pid="$(wait_for_new_run "$before")" + local bridge="$app/Contents/MacOS/Shark Explorer" + + mkdir -p "$HARNESS_DIRECTORY" + write_mcp_config "$bridge" "$pid" + write_prompt + + cat <"$HARNESS_DIRECTORY/mcp.json" <"$HARNESS_DIRECTORY/prompt.txt" <<'END' +A heap dump is open in Shark Explorer, which you can reach through its MCP tools. Something in it is +leaking. Find the root cause. +END +} + +published_runs() { + ls "$RUNS_DIRECTORY" 2>/dev/null | sort || true +} + +# The process id of the run that just started, which is the file that wasn't there before it did. +wait_for_new_run() { + local before="$1" waited=0 new + while ((waited < WAIT_SECONDS)); do + new="$(comm -13 <(echo "$before") <(published_runs) | head -1)" + if [[ -n "$new" ]]; then + echo "${new%.agent}" + return 0 + fi + sleep 1 + ((waited++)) + done + echo "The window did not publish itself within ${WAIT_SECONDS}s. Look at $(newest_log)." >&2 + exit 1 +} + +newest_log() { + # shellcheck disable=SC2012 + ls -t "$HOME/.shark-explorer/logs" 2>/dev/null | head -1 | sed "s|^|$HOME/.shark-explorer/logs/|" +} + +absolute_path() { + if [[ "$1" == /* ]]; then echo "$1"; else echo "$PWD/$1"; fi +} + +main "$@" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt new file mode 100644 index 0000000000..5a35421bdf --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt @@ -0,0 +1,84 @@ +package shark.explorer.agent + +import shark.explorer.HeapExplorer +import shark.explorer.LeakStatusOverride +import shark.explorer.LeakStatusOverrides +import shark.explorer.Place + +/** + * One open heap dump an agent can ask about, which is one window of the app. + * + * An interface rather than the window itself so that every tool in [AgentTools] is testable against a heap + * dump and nothing else: the app's implementation carries a `HeapDumpSession`, the statuses set by hand and + * the tabs, none of which a test of what a tool answers needs. + * + * **A window and not a heap dump file**, matching `shark.explorer.DeepLink`: the same dump is often open + * twice — that is what comparing two of them is — so a path would be ambiguous exactly when it matters, and + * a verdict set through one of two windows has to be the verdict the other one draws. + */ +interface AgentHeapDump { + + /** What a link names this window by, and what an agent addresses it by. See [AgentTools]. */ + val windowId: String + + /** Which heap dump is open here, absolute, so that an agent can check it is the one it was asked about. */ + val heapDumpPath: String + + /** + * Runs [block] against the open heap dump, wherever the implementation reads one. + * + * Suspending because the app owns one thread per heap dump and reads queue on it, so a tool call waits its + * turn behind whatever the person at the window is doing — which is the point rather than a cost: an agent + * reading the dump the human is reading must not be able to read it from a second thread and get an answer + * the window never showed. + * + * [description] names what is being read, the way `HeapDumpSession.read` takes one, so that an agent's + * reads appear in this run's log beside the window's own. + */ + suspend fun read( + description: String, + block: (HeapExplorer) -> T + ): T + + /** Every verdict set by hand on this dump so far, which every read is made through. */ + val verdicts: LeakStatusOverrides + + /** + * Sets [verdict], along with the [solved] verdicts that had to flip for it to hold, and puts the lot on + * disk. See `shark.explorer.LeakStatusConflict`. + */ + suspend fun setVerdict( + verdict: LeakStatusOverride, + solved: List + ) + + /** Takes the verdict off [objectId], so the dump says what it says about it again. */ + suspend fun clearVerdict(objectId: Long) + + /** Appends [text] to the note of [place], which is the notepad the window shows on that tab. */ + suspend fun appendToNote( + place: Place, + text: String + ) + + /** + * Opens [place] in a tab of this window and brings the window to the front, which is what makes an agent's + * work something the person at the machine can watch rather than read about afterwards. + * + * Not suspending and not answered: this is the same hand-over a `shark://` link makes — a place put where + * the tabs take it on the next frame — so there is nothing to wait for and nothing that can fail here. + */ + fun show(place: Place) +} + +/** + * The open heap dumps of this run, which is what a connection asks before anything else. + * + * Windows come and go while an agent is connected, so this is asked per call rather than captured: a tool + * naming a window that has since closed is an error message, not a stale answer. + */ +fun interface AgentHeapDumps { + + /** Every window with a heap dump open, in the order they were opened. */ + fun openHeapDumps(): List +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt new file mode 100644 index 0000000000..bf87763234 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt @@ -0,0 +1,288 @@ +package shark.explorer.agent + +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonObjectBuilder +import kotlinx.serialization.json.add +import kotlinx.serialization.json.addJsonObject +import kotlinx.serialization.json.buildJsonArray +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put +import kotlinx.serialization.json.putJsonArray +import kotlinx.serialization.json.putJsonObject +import shark.explorer.HeapLeaks +import shark.explorer.HeapObjectSummary +import shark.explorer.HeapSizes +import shark.explorer.IndependentPaths +import shark.explorer.LeakStatusConflict +import shark.explorer.LeakStatusOverrides +import shark.explorer.ObjectDominator +import shark.explorer.ObjectList +import shark.explorer.PathStep +import shark.explorer.ReachabilityStrength +import shark.explorer.RootPath +import shark.explorer.RootPathStep +import shark.explorer.exactHexObjectId + +/** + * How the explorer's own model reads as JSON, which is the whole of what an agent sees of a heap dump. + * + * Two rules run through all of it, and both are about being read by something that is not this app. + * + * **An address is a string, never a number.** A heap dump's addresses fill the whole range of `Long`, and a + * JSON number is a double to most clients of this protocol — anything above 2^53 comes back rounded, which + * for an address means a different object, silently. So every one of them is [exactHexObjectId], the same + * spelling this app's own files use, and [shark.explorer.objectIdOfHex] is the only way back. + * + * **Nothing is summarised away, and every cap says so.** A field an agent can't see is a field it will + * guess at, so the answers here are what the window shows: the labels the inspectors wrote, the verdict and + * the reason under it, the whole chain. Where an answer is capped — a list of objects, the ways one is held + * — the count that was matched and whether anything was left out go with it, because an agent counting the + * instances of a class must never be counting the page it was shown. + */ +internal object AgentJson { + + /** Which window, which dump, how big it is, and what has been concluded about it so far. */ + fun heapDump( + windowId: String, + heapDumpPath: String, + sizes: HeapSizes, + verdicts: LeakStatusOverrides + ): JsonObject = buildJsonObject { + put("window", windowId) + put("heapDumpPath", heapDumpPath) + putJsonObject("sizes") { + put("totalBytes", sizes.totalByteCount) + put("totalObjects", sizes.totalObjectCount) + // What a retained size is a share of, and the number an agent should compare one against. + put("stronglyReachableBytes", sizes.stronglyReachableByteCount) + put("unreachableBytes", sizes.unreachableByteCount) + putJsonArray("byStrength") { + ReachabilityStrength.values().forEach { strength -> + addJsonObject { + put("strength", strength.name) + put("bytes", sizes.byteCountByStrength.getValue(strength)) + put("objects", sizes.objectCountByStrength.getValue(strength)) + } + } + } + } + put("verdictsSetByHand", verdicts(verdicts)) + } + + /** + * Every verdict set by hand, so that an agent arriving at a window someone has been working in reads the + * conclusions already reached rather than starting over on top of them. + */ + fun verdicts(overrides: LeakStatusOverrides): JsonArray = buildJsonArray { + overrides.all.sortedBy { it.objectId }.forEach { override -> + addJsonObject { + put("object", exactHexObjectId(override.objectId)) + put("verdict", override.status.name) + put("reason", override.reason) + } + } + } + + /** One object: what it is, how firmly it is held, what it retains, and every field of it. */ + fun objectSummary( + summary: HeapObjectSummary, + dominator: ObjectDominator? + ): JsonObject = buildJsonObject { + put("object", exactHexObjectId(summary.objectId)) + put("label", summary.label) + put("className", summary.className) + put("kind", summary.kind?.name) + put("headline", summary.headline) + put("strength", summary.strength.name) + put("shallowBytes", summary.shallowSize) + put("retainedBytes", summary.retainedSize) + put("retainedObjects", summary.retainedCount) + put("dominatedObjects", summary.dominatedObjectCount) + put("verdict", summary.leakStatus.name) + put("verdictReason", summary.leakStatusReason) + putJsonArray("inspectorLabels") { summary.inspectorLabels.forEach { add(it) } } + // The one object releasing which would free this one, which is the answer to "what would fix this". + if (dominator != null) { + putJsonObject("dominator") { + put("node", exactHexObjectId(dominator.nodeId)) + put("label", dominator.label) + put("kind", dominator.kind.name) + put("retainedBytes", dominator.retainedSize) + } + } + putJsonArray("fields") { + summary.fields.forEach { field -> + addJsonObject { + put("name", field.name) + put("declaringClass", field.declaringClassName) + put("value", field.value) + put("valueObject", field.inspectableObjectId?.let { exactHexObjectId(it) }) + } + } + } + // Only an array reaches this, and an agent that could not see it would read a 10,000 element array as + // the handful of elements it was shown. + put("hiddenFieldCount", summary.hiddenFieldCount) + } + + /** The shortest chain from a GC root down to an object, with dominators and the faulty reference marked. */ + fun rootPath(path: RootPath): JsonObject = buildJsonObject { + put("gcRoot", path.gcRootLabel) + put("stepCount", path.steps.size) + putJsonArray("steps") { path.steps.forEach { add(rootPathStep(it)) } } + } + + /** Every way an object is held, which is what a single chain cannot say. */ + fun independentPaths(paths: IndependentPaths): JsonObject = buildJsonObject { + put("pathCount", paths.paths.size) + // The search is greedy, so this is the difference between "held these ways" and "held at least these + // ways" — and an agent concluding that one reference is all that holds an object needs to know which of + // the two it was told. + put("hasMore", paths.hasMore) + putJsonArray("paths") { + paths.paths.forEach { path -> + addJsonObject { + put("gcRoot", path.gcRootLabel) + putJsonArray("steps") { path.steps.forEach { add(pathStep(it)) } } + } + } + } + } + + /** The leaks screen: what is stuck in this dump, gathered the way the window gathers it. */ + fun leaks(leaks: HeapLeaks): JsonObject = buildJsonObject { + put("objectCount", leaks.objectCount) + put("leakingObjectCount", leaks.leakingObjectCount) + putJsonArray("sections") { + leaks.sections.forEach { section -> + addJsonObject { + put("kind", section.kind.name) + put("title", section.kind.title) + put("explanation", section.kind.explanation) + // Whether this is a leak to fix or an object the collector will take on its own, which is the + // split that makes the list actionable. See LeakKind.isOnTheWayOut. + put("isOnTheWayOut", section.kind.isOnTheWayOut) + put("objectCount", section.objectCount) + putJsonArray("groups") { + section.groups.forEach { group -> + addJsonObject { + put("leakFingerprint", group.leakFingerprint) + put("title", group.title) + put("subtitle", group.subtitle) + // The references the leak *is*, which is what a leak investigation ends at and therefore + // the thing an agent must not have to reconstruct from a chain. + putJsonArray("suspectPath") { group.suspectPath.forEach { add(it) } } + put("retainedBytes", group.retainedSize) + putJsonArray("objects") { + group.objects.forEach { leaking -> + addJsonObject { + put("object", exactHexObjectId(leaking.objectId)) + put("className", leaking.className) + put("kind", leaking.kind.name) + put("headline", leaking.headline) + put("retainedBytes", leaking.retainedSize) + put("retainedObjects", leaking.retainedCount) + put("strength", leaking.strength.name) + put("leakingReason", leaking.leakingReason) + // The strongest evidence a heap dump carries: the app itself said this object + // should be gone. See WatchedObject. + val watcher = leaking.watcher + if (watcher != null) { + putJsonObject("watchedBecause") { + put("key", watcher.key) + put("description", watcher.description) + put("retainedMillis", watcher.retainedDurationMillis) + } + } + } + } + } + } + } + } + } + } + } + } + + /** A filtered list of the dump's objects, with what the filter matched before the cap. */ + fun objectList(list: ObjectList): JsonObject = buildJsonObject { + put("matchCount", list.matchCount) + put("totalCount", list.totalCount) + // So that an agent asking "is this really a singleton?" is answered by the match count rather than by + // however many rows fitted. + put("isComplete", !list.hasMore) + putJsonArray("objects") { + list.entries.forEach { entry -> + addJsonObject { + put("object", exactHexObjectId(entry.objectId)) + put("className", entry.className) + put("kind", entry.kind.name) + put("headline", entry.headline) + put("shallowBytes", entry.shallowSize) + put("retainedBytes", entry.retainedSize) + put("strength", entry.strength.name) + } + } + } + } + + /** What setting a verdict would disagree with, and what solving it would set those objects to. */ + fun conflicts(conflicts: List): JsonArray = buildJsonArray { + conflicts.forEach { conflict -> + addJsonObject { + put("object", exactHexObjectId(conflict.existing.objectId)) + put("objectName", conflict.objectName) + put("verdict", conflict.existing.status.name) + put("reason", conflict.existing.reason) + // Which way round the two objects are, since that is what makes the disagreement one at all. + put("holdsTheObjectBeingSet", conflict.isAbove) + putJsonObject("wouldBecome") { + put("verdict", conflict.solved.status.name) + put("reason", conflict.solved.reason) + } + } + } + } + + private fun rootPathStep(step: RootPathStep): JsonObject = buildJsonObject { + pathStepInto(step.step) + // Every path from a GC root goes through each of an object's dominators, so a marked step is one that + // releasing would free the object and the rest are only on the way to it. + put("isDominator", step.isDominator) + } + + private fun pathStep(step: PathStep): JsonObject = buildJsonObject { pathStepInto(step) } + + private fun JsonObjectBuilder.pathStepInto(step: PathStep) { + put("object", exactHexObjectId(step.objectId)) + put("className", step.className) + put("kind", step.kind.name) + put("headline", step.headline) + put("strength", step.strength.name) + put("retainedBytes", step.retainedSize) + put("retainedObjects", step.retainedCount) + putJsonArray("inspectorLabels") { step.inspectorLabels.forEach { add(it) } } + put("verdict", step.leakStatus.name) + put("verdictReason", step.leakStatusReason) + put("isInspectable", step.isInspectable) + val reference = step.reference + if (reference != null) { + putJsonObject("reference") { + put("name", reference.name) + put("ownerClassName", reference.ownerClassName) + put("locationType", reference.locationType.name) + // The one thing on a chain that says where to go and change code. + put("isFaulty", reference.isFaulty) + val libraryLeak = reference.libraryLeak + if (libraryLeak != null) { + putJsonObject("libraryLeak") { + put("pattern", libraryLeak.pattern) + put("description", libraryLeak.description) + } + } + } + } + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt new file mode 100644 index 0000000000..91fe661e03 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt @@ -0,0 +1,104 @@ +package shark.explorer.agent + +/** + * The method an agent is asked to follow, which is the part of this surface that isn't data. + * + * Handed over twice on purpose: as the `instructions` of the MCP handshake, which some clients show the + * model and some drop, and again with the answer to [AgentTools.OPEN_HEAP_DUMPS], which is the call every + * investigation starts with. A method a client dropped is a method nobody followed. + * + * **It is prose because its reader is a language model**, which is the one place in this app where a + * paragraph beats a label — the window says `Verdict` in one word to someone who already knows what a + * verdict is for. What keeps the prose honest is that the tools enforce the two claims it can't make on its + * own: a verdict is refused without a reason, and [AgentTools.CONCLUDE] is refused until the heap dump + * itself says one reference is at fault. So the method describes what the tools will hold you to rather + * than asking to be trusted. + * + * Adapted from [The LeakCanary Method](https://engineering.block.xyz/blog/the-leakcanary-method), which is + * the same five phases done by hand. + */ +internal object AgentMethod { + + /** + * What to do with a heap dump, in the order it works. + * + * Kept in one string rather than assembled from the tool descriptions, because it is an argument and not + * a list: each step is worth doing because of the step before it. + */ + val INSTRUCTIONS = """ + You are reading a heap dump through Shark Explorer, a window a person may be watching. Everything you + ask is a read of that dump, and everything you conclude is written into it where the next reader — a + colleague, another agent, the same person in a month — will find it. + + ## What a leak is + + A memory leak is ONE bad reference. Not a chain, not a subsystem, not "the activity is retained": one + field of one object that should have been cleared and wasn't. Everything below that reference is in + memory because of it and is not itself at fault. Everything above it is doing its job. + + So an investigation is a search for that single reference, and the chain from a GC root to a stuck + object is where it is. Each object on the chain gets a verdict: + + - NOT_LEAKING — this object is meant to be in memory right now. + - LEAKING — this object should be gone. + - UNKNOWN — you don't know yet. Most objects, most of the time. + + Two rules turn verdicts into an answer, and the tools apply both for you: + + - Everything holding an object that is meant to be in memory is meant to be in memory too, so a + NOT_LEAKING verdict spreads upwards. + - Everything a stuck object holds is only in memory because of it, so a LEAKING verdict spreads + downwards. + + A chain therefore reads as three zones: NOT_LEAKING at the top, LEAKING at the bottom, UNKNOWN in + between. **The leak is the one reference that crosses from the last NOT_LEAKING object to the first + LEAKING one.** While the UNKNOWN zone is more than one reference wide, you have not found it — you have + narrowed it. + + ## The order to work in + + 1. **Find something that shouldn't be there.** `list_leaks` is the heap dump's own answer: objects the + app itself handed to LeakCanary and said were done with, plus what the inspectors recognised. Start + with a leak whose objects the app watched — that is the strongest evidence a heap dump carries. + 2. **Get the chain.** `chain_from_gc_root` for one stuck object. Read every step. The steps already + carry the inspectors' labels and any verdict someone has set. + 3. **Work inwards from both ends.** Top down: which of these objects is obviously meant to be here — a + running thread, a live activity, the application itself? Bottom up: which is obviously done with? + Set what you can defend with `set_verdict` and watch the UNKNOWN zone shrink. + 4. **Attack what is left.** This is the part that takes work, and it is where the tools earn their + keep: + - `describe_object` on an object in the unknown zone. Read its fields and its inspector labels. + - `ways_held` when you need to know whether a reference really is the only thing holding something. + One chain says how it is held; this says whether there is another way. + - `find_objects` on a class you have assumed something about. Two instances of a class you took for + a singleton is the answer to a surprising number of leaks: the object on the chain is not the + instance you think it is. + - Read the app's source for the field that holds the next step. A verdict you can point at a line of + code for is a verdict that survives review. + 5. **Isolating the reference is not the root cause.** When one reference is left, you know *where* the + problem is. You still do not know *how* it happened, and stopping here is the most common way an + investigation fails. Keep going: what code assigns that field, what should have cleared it, and why + didn't it? The answer is usually a sequence of events, not a line. + 6. **Say how to reproduce it**, or say that you couldn't work that out. A root cause nobody can trigger + is a hypothesis. + + ## Rules you will be held to + + - **Every verdict needs a reason another reader can check.** A field value, an inspector label, the + app's own watcher record, a line of source. Not "this is probably a cache" and not "activities are + usually leaked this way". `set_verdict` refuses a blank reason, and a reason that isn't evidence is + worse than none. + - **Set verdicts as you go, not at the end.** They are how the tools narrow the search for you, and + they are what the person at the window sees you doing. + - **`conclude` is the only way to finish**, and it will refuse you unless the heap dump agrees that one + reference is at fault. If it refuses, the investigation is not over — the message says what is + missing. Do not report a root cause you could not conclude. + - **Say what you did not check.** An answer with a stated gap is worth more than a confident one with + an unstated gap. + - **Every call takes a `reason`**: what you are trying to learn, or what you concluded from the last + answer. It goes in this run's log next to the read it caused, which is what makes an investigation + something a person can follow afterwards rather than a conclusion they have to trust. + - **`show` puts what you are looking at on screen.** Use it when you reach something that matters. The + window is how the person watching follows the work, and it costs you one call. + """.trimIndent() +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt new file mode 100644 index 0000000000..7205a2580d --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt @@ -0,0 +1,216 @@ +package shark.explorer.agent + +import java.io.BufferedReader +import java.io.Closeable +import java.io.File +import java.io.InputStreamReader +import java.io.OutputStreamWriter +import java.io.PrintWriter +import java.net.InetAddress +import java.net.ServerSocket +import java.net.Socket +import java.net.SocketException +import java.security.SecureRandom +import java.util.Properties +import kotlinx.coroutines.runBlocking +import shark.SharkLog + +/** + * Where an agent reaches this run of the app, and how it finds out where that is. + * + * A loopback socket and a file naming it, which is the same shape as `DeepLinkPeers` and deliberately not + * the same socket: a link is one line delivered to whichever run owns a window, and this is a session held + * open for as long as an agent is working. Two features with two lifetimes on one port would mean a link + * arriving while an investigation is in flight, and an investigation ending when a link handler closed. + * + * **Every run publishes itself**, like the links do, because several explorers open at once is how this app + * is used. [AgentStdioBridge] is what picks one, and a run that was killed leaves a file that nothing + * answers on, which the next reader deletes. + * + * Loopback only, and a caller has to quote the token out of the file — which proves it can read the user's + * home directory, and therefore that it is the user. Worth spelling out what that is and isn't: this is + * enough to keep a web page or another machine out, and it is not a boundary between programs run by the + * same person. Anything that can read `~/.shark-explorer` can read any heap dump on the disk anyway. + */ +object AgentServer { + + /** + * Publishes this run and answers agents until closed. + * + * Failing to listen is not a reason to refuse to start: the window works, and what stops working is + * agents being able to reach it — which the log then says, rather than a client that hangs with no + * explanation. + */ + fun listen( + heapDumps: AgentHeapDumps, + /** Which build is answering, for the handshake. */ + serverVersion: String, + /** Where the file naming this run goes, which is `~/.shark-explorer/agents` for the real app. */ + directory: File + ): Closeable { + val serverSocket = try { + ServerSocket(ANY_FREE_PORT, BACKLOG, InetAddress.getLoopbackAddress()) + } catch (throwable: Throwable) { + SharkLog.d(throwable) { "Could not listen for agents: no agent will be able to reach this run" } + return Closeable {} + } + val token = newToken() + val file = File(directory, "${ProcessHandle.current().pid()}$RUN_SUFFIX") + return try { + write(file, serverSocket.localPort, token) + SharkLog.d { "Answering agents on port ${serverSocket.localPort}, published as $file" } + val thread = Thread({ accept(serverSocket, token, heapDumps, serverVersion) }, THREAD_NAME).apply { + isDaemon = true + start() + } + Runtime.getRuntime().addShutdownHook(Thread { file.delete() }) + Closeable { + file.delete() + serverSocket.close() + thread.interrupt() + } + } catch (throwable: Throwable) { + SharkLog.d(throwable) { "Could not publish this run at $file: no agent will find it" } + serverSocket.close() + Closeable {} + } + } + + /** Every run of this app an agent could connect to, newest first, stale files cleared out on the way. */ + internal fun publishedRuns(directory: File): List { + val files = directory.listFiles { file -> file.name.endsWith(RUN_SUFFIX) }.orEmpty() + return files.sortedByDescending { it.lastModified() }.mapNotNull { file -> read(file) } + } + + private fun read(file: File): PublishedRun? { + val properties = Properties() + return try { + file.inputStream().use { properties.load(it) } + val port = properties.getProperty(PORT_PROPERTY)?.toIntOrNull() + val token = properties.getProperty(TOKEN_PROPERTY) + if (port == null || token == null) { + SharkLog.d { "$file says no port and token, so it names no run: deleting it" } + file.delete() + null + } else { + PublishedRun(file, file.name.removeSuffix(RUN_SUFFIX), port, token) + } + } catch (throwable: Throwable) { + SharkLog.d(throwable) { "Could not read $file, so no agent can be pointed at that run" } + null + } + } + + private fun write( + file: File, + port: Int, + token: String + ) { + file.parentFile.mkdirs() + val properties = Properties().apply { + setProperty(PORT_PROPERTY, port.toString()) + setProperty(TOKEN_PROPERTY, token) + } + file.outputStream().use { properties.store(it, "Where this Shark Explorer run answers agents") } + // Best effort, and only worth anything on a machine with more than one user on it: the token is what + // this is protecting, and a token nobody can read is a run no agent can reach. + file.setReadable(false, false) + file.setReadable(true, true) + } + + private fun accept( + serverSocket: ServerSocket, + token: String, + heapDumps: AgentHeapDumps, + serverVersion: String + ) { + while (!serverSocket.isClosed) { + try { + val socket = serverSocket.accept() + // A thread per agent, because a session is held open for as long as the agent is working and two + // agents on one heap dump is a thing to allow rather than to serialise: what they would queue on + // is the heap dump's own thread, which is where reads belong anyway. + Thread({ serve(socket, token, heapDumps, serverVersion) }, THREAD_NAME).apply { + isDaemon = true + start() + } + } catch (closed: SocketException) { + // Which is what closing the socket out from under accept() looks like, and it is how this ends. + SharkLog.d { "Stopped answering agents: ${closed.message}" } + return + } catch (throwable: Throwable) { + SharkLog.d(throwable) { "An agent could not be accepted, carrying on listening" } + } + } + } + + /** + * One connection: the token, then a JSON-RPC message per line until the agent goes away. + * + * **No read timeout**, unlike the link socket. An agent thinking, or waiting for the person at the + * machine, is a connection with nothing on it for minutes at a time, and a session dropped for being + * quiet is one that loses whatever it had concluded. + */ + private fun serve( + socket: Socket, + token: String, + heapDumps: AgentHeapDumps, + serverVersion: String + ) { + socket.use { + val reader = BufferedReader(InputStreamReader(socket.getInputStream(), Charsets.UTF_8)) + val writer = PrintWriter(OutputStreamWriter(socket.getOutputStream(), Charsets.UTF_8), true) + val sentToken = reader.readLine() + if (sentToken != token) { + // Loopback only, so this is a stale file being read far more often than it is anything to worry + // about. + SharkLog.d { "An agent connected quoting the wrong token, so it was not listened to" } + writer.println(DECLINED) + return + } + writer.println(ACCEPTED) + val session = McpSession(AgentTools(heapDumps), serverVersion) + while (true) { + val line = reader.readLine() ?: break + if (line.isBlank()) { + continue + } + // Blocking on this thread rather than a scope of our own: a message is answered before the next is + // read, which is what an agent sends anyway, and the reads inside suspend onto the heap dump's + // thread where they belong. + val answer = runBlocking { session.answer(line) } + if (answer != null) { + writer.println(answer) + } + } + SharkLog.d { "An agent disconnected" } + } + } + + private fun newToken(): String { + val bytes = ByteArray(TOKEN_BYTES) + SecureRandom().nextBytes(bytes) + return bytes.joinToString("") { "%02x".format(it) } + } + + /** A run of the app that has published where it answers agents. See [publishedRuns]. */ + internal class PublishedRun( + val file: File, + /** The process id, which is what the file is named after and what identifies a run to a person. */ + val pid: String, + val port: Int, + val token: String + ) + + private const val ANY_FREE_PORT = 0 + private const val BACKLOG = 8 + private const val TOKEN_BYTES = 16 + + /** Beside the runs answering links, the notes and the logs, which is everything else this app keeps. */ + internal const val RUN_SUFFIX = ".agent" + internal const val ACCEPTED = "OK" + internal const val DECLINED = "NO" + private const val PORT_PROPERTY = "port" + private const val TOKEN_PROPERTY = "token" + private const val THREAD_NAME = "shark-explorer-agents" +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt new file mode 100644 index 0000000000..3f737cfe68 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt @@ -0,0 +1,160 @@ +package shark.explorer.agent + +import java.io.BufferedReader +import java.io.File +import java.io.InputStreamReader +import java.io.OutputStreamWriter +import java.io.PrintWriter +import java.net.InetAddress +import java.net.InetSocketAddress +import java.net.Socket + +/** + * Standard input and output, wired to the run of the app an agent wants to talk to. + * + * The one thing every MCP client can be configured with is a command to run, so this is what makes the + * window reachable without asking anybody to configure a port that changes every run: the client launches + * this, this finds the window. A dozen lines of pipe against a client integration nobody has to write. + * + * It is a mode of the app rather than a program of its own so that there is one thing to install — the + * `.app` on the machine is both the window and the bridge to it. See `shark.explorer.app.main`. + * + * **Nothing is ever written to stdout but protocol**, which is why this runs before the app's logging is + * installed: that logger writes to stdout, and one line of it in the middle of a JSON-RPC stream is a + * client that reports the server as broken. Everything this has to say goes to stderr, which is where an + * MCP client collects a server's log. + */ +object AgentStdioBridge { + + /** + * Pumps until either end goes away, and returns the exit code the process should end with. + * + * A run that has gone is the interesting failure and it is reported rather than waited on: an agent + * whose client hangs at startup has no way to tell that from a machine that is slow, so this says what + * is wrong on stderr and ends. + */ + fun run( + /** Where the runs of the app publish themselves. See [AgentServer]. */ + directory: File, + /** Which run, by process id, or null for the one that started most recently. */ + pid: String? = null, + /** How long to wait for a run to appear, for a client that launched this before the app was open. */ + waitMillis: Long = DEFAULT_WAIT_MILLIS + ): Int { + val run = waitForRun(directory, pid, waitMillis) ?: return NOTHING_TO_TALK_TO + val socket = try { + Socket().apply { + connect(InetSocketAddress(InetAddress.getLoopbackAddress(), run.port), CONNECT_TIMEOUT_MILLIS) + } + } catch (throwable: Throwable) { + // Which is a run that was killed: the file is still there and nothing is on the port. + say("Shark Explorer run ${run.pid} does not answer on port ${run.port}: $throwable") + run.file.delete() + return NOTHING_TO_TALK_TO + } + return socket.use { pump(it, run) } + } + + private fun pump( + socket: Socket, + run: AgentServer.PublishedRun + ): Int { + val toApp = PrintWriter(OutputStreamWriter(socket.getOutputStream(), Charsets.UTF_8), true) + val fromApp = BufferedReader(InputStreamReader(socket.getInputStream(), Charsets.UTF_8)) + toApp.println(run.token) + if (fromApp.readLine() != AgentServer.ACCEPTED) { + say("Shark Explorer run ${run.pid} refused the token in ${run.file}, so it is not the run that wrote it") + return NOTHING_TO_TALK_TO + } + say("Talking to Shark Explorer run ${run.pid}") + // The app's answers on their own thread, because both directions are blocking reads and a client sends + // its next message without waiting to be answered. + val answers = Thread({ + val out = PrintWriter(OutputStreamWriter(System.out, Charsets.UTF_8), true) + while (true) { + val line = fromApp.readLine() ?: break + out.println(line) + } + // The window closed, which ends the session: nothing is going to answer the client's next message. + say("Shark Explorer run ${run.pid} closed the connection") + System.out.flush() + }, "shark-explorer-agent-answers").apply { + isDaemon = true + start() + } + val stdin = BufferedReader(InputStreamReader(System.`in`, Charsets.UTF_8)) + while (true) { + val line = stdin.readLine() ?: break + toApp.println(line) + if (toApp.checkError()) { + say("Shark Explorer run ${run.pid} went away") + return NOTHING_TO_TALK_TO + } + } + // The client closed its end, which is how a session normally ends. + socket.close() + answers.join(SHUTDOWN_MILLIS) + return 0 + } + + private fun waitForRun( + directory: File, + pid: String?, + waitMillis: Long + ): AgentServer.PublishedRun? { + var waited = 0L + while (true) { + val runs = AgentServer.publishedRuns(directory) + val run = if (pid == null) runs.firstOrNull() else runs.firstOrNull { it.pid == pid } + if (run != null) { + if (pid == null && runs.size > 1) { + // Which run an agent ends up in is worth saying rather than leaving to be worked out from what + // heap dump it finds open: several explorers at once is the normal way this app is used. + say( + "${runs.size} Shark Explorer runs are open; talking to ${run.pid}, the one that started most " + + "recently. Pass $PID_OPTION to pick another: " + runs.joinToString(", ") { it.pid } + ) + } + return run + } + if (waited >= waitMillis) { + say( + if (pid == null) { + "No Shark Explorer is running, so there is no heap dump to investigate. Open one — every run " + + "of the app publishes itself in $directory — and start this again." + } else { + "No Shark Explorer run is $pid. Open runs: " + + AgentServer.publishedRuns(directory).joinToString(", ") { it.pid }.ifEmpty { "none" } + } + ) + return null + } + Thread.sleep(POLL_MILLIS) + waited += POLL_MILLIS + } + } + + /** + * On stderr, always, which is where an MCP client collects what a server has to say. + * + * Not through `SharkLog`: this process deliberately never installs the app's logging, since that writes + * to stdout and stdout is the protocol. + */ + private fun say(message: String) { + System.err.println("[shark-explorer] $message") + } + + /** What the command line says to pick a run by process id. See `shark.explorer.app.ExplorerArguments`. */ + const val PID_OPTION = "--agent-run=" + + private const val DEFAULT_WAIT_MILLIS = 10_000L + private const val POLL_MILLIS = 250L + private const val CONNECT_TIMEOUT_MILLIS = 1_000 + private const val SHUTDOWN_MILLIS = 500L + + /** + * What this process ends with when it never found a window, which is a failure a client should show: an + * MCP server that exits zero having done nothing reads as one with no tools. + */ + private const val NOTHING_TO_TALK_TO = 1 +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTool.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTool.kt new file mode 100644 index 0000000000..fb5ff2c8f6 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTool.kt @@ -0,0 +1,223 @@ +package shark.explorer.agent + +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.add +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put +import kotlinx.serialization.json.putJsonArray +import kotlinx.serialization.json.putJsonObject +import shark.explorer.objectIdOfHex + +/** + * One thing an agent can do, as a client of this protocol sees it: a name, what it is for, the shape of its + * arguments, and what it answers with. + * + * The description is written for a model rather than for a person, which in practice means it says **when to + * reach for this** and what the answer is worth, not what it returns — the schema says that. A tool + * described only by its return type is one that gets called in the wrong order. + */ +internal class AgentTool( + val name: String, + val description: String, + /** JSON Schema for the arguments, which is what a client validates against before calling. */ + val schema: JsonObject, + private val handler: suspend (AgentArguments) -> JsonObject +) { + + suspend fun call(arguments: JsonObject): JsonObject { + val read = AgentArguments(name, arguments) + // Read before the handler and on every tool, so that a call with no reason is refused rather than + // logged as a call whose reason was left blank. The schema asks for it and a client is free to ignore a + // schema, so this is where "every call says why it was made" is a rule instead of a hope. + read.reason + return handler(read) + } +} + +/** + * Why a call was refused, worded for the agent that made it. + * + * **Every one of these says what to do instead**, because a refusal is the one message an agent is certain + * to read: it is where the method is enforced rather than described. "Not concluded, 3 steps have no + * verdict, here they are" turns a wrong answer into the next thing to look at, and that is the whole + * mechanism — the server refuses, so it works with any client and nothing here has to call a model back. + * + * Public because [AgentHeapDump] is: the app implements it, and the app has refusals of its own to make — + * writing over a note somebody is typing in, recording a verdict into a file that hasn't been read. + */ +class AgentRefusal(override val message: String) : Exception(message) + +/** + * The arguments of one call, read the way a schema promised them. + * + * Reading is strict and the messages name the tool, because these arrive from a model: a number where a + * string was asked for is something it can fix on the next call if it is told which argument of which tool, + * and a silent default is a call that answered about something else. + */ +internal class AgentArguments( + private val toolName: String, + private val arguments: JsonObject +) { + + /** + * What the agent said it was trying to learn, which every tool takes and which goes in this run's log + * beside the reads it caused. See [AgentTools]. + */ + val reason: String get() = string(REASON) + + fun string(name: String): String { + val value = optionalString(name) + if (value.isNullOrBlank()) { + throw AgentRefusal( + "$toolName needs `$name`, and it was ${if (value == null) "not given" else "blank"}." + ) + } + return value + } + + fun optionalString(name: String): String? { + val element = arguments[name] ?: return null + val primitive = element as? JsonPrimitive ?: throw wrongType(name, "a string", element) + return primitive.content + } + + fun boolean( + name: String, + default: Boolean + ): Boolean { + val text = optionalString(name) ?: return default + return when (text.lowercase()) { + "true" -> true + "false" -> false + else -> throw wrongType(name, "true or false", text) + } + } + + fun int( + name: String, + default: Int + ): Int { + val text = optionalString(name) ?: return default + return text.toIntOrNull() ?: throw wrongType(name, "a whole number", text) + } + + fun objectId(name: String): Long = objectIdOf(name, string(name)) + + fun optionalObjectId(name: String): Long? = optionalString(name)?.let { objectIdOf(name, it) } + + fun stringList(name: String): List? { + val element = arguments[name] ?: return null + val array = element as? JsonArray ?: throw wrongType(name, "a list of strings", element) + return array.map { item -> + (item as? JsonPrimitive)?.content ?: throw wrongType(name, "a list of strings", element) + } + } + + /** + * An address as everything this surface spells one, or a refusal. + * + * The refusal names the decimal case, because that is the mistake worth catching: a model that has seen a + * JSON number for an address somewhere else will write one here, and `140234878714368` is an address this + * would otherwise have to either reject blankly or accept as something else. See [AgentJson]. + */ + fun objectIdOf( + name: String, + text: String + ): Long = objectIdOfHex(text) ?: throw AgentRefusal( + "`$name` of $toolName is \"$text\", which is no object address. An address is \"$HEX_PREFIX\" and up " + + "to 16 hexadecimal digits, exactly as this surface writes one — never a decimal number, since a " + + "64 bit address does not survive being one in JSON." + ) + + private fun wrongType( + name: String, + expected: String, + value: Any + ) = AgentRefusal("`$name` of $toolName has to be $expected, and it was \"$value\".") +} + +/** One argument of a tool: what it is, and whether a call without it is a call at all. */ +internal class AgentProperty( + val schema: JsonObject, + val isRequired: Boolean = true +) + +internal fun AgentProperty.optional(): AgentProperty = AgentProperty(schema, isRequired = false) + +internal fun string(description: String): AgentProperty = AgentProperty( + buildJsonObject { + put("type", "string") + put("description", description) + } +) + +internal fun boolean(description: String): AgentProperty = AgentProperty( + buildJsonObject { + put("type", "boolean") + put("description", description) + } +) + +internal fun integer(description: String): AgentProperty = AgentProperty( + buildJsonObject { + put("type", "integer") + put("description", description) + } +) + +internal fun enumString( + description: String, + values: List +): AgentProperty = AgentProperty( + buildJsonObject { + put("type", "string") + put("description", description) + putJsonArray("enum") { values.forEach { add(it) } } + } +) + +internal fun enumArray( + description: String, + values: List +): AgentProperty = AgentProperty( + buildJsonObject { + put("type", "array") + put("description", description) + putJsonObject("items") { + put("type", "string") + putJsonArray("enum") { values.forEach { add(it) } } + } + } +) + +/** + * The schema of a tool's arguments, with `reason` added to every one of them. + * + * Added here rather than written out eleven times, so that there is no tool it can be forgotten on: a + * command with no reason recorded beside it is the gap this surface exists to close. See [AgentTools]. + */ +internal fun schema(vararg properties: Pair): JsonObject { + val all = properties.toList() + (REASON to REASON_PROPERTY) + return buildJsonObject { + put("type", "object") + putJsonObject("properties") { + all.forEach { (name, property) -> put(name, property.schema) } + } + putJsonArray("required") { + all.filter { it.second.isRequired }.forEach { add(it.first) } + } + } +} + +private const val REASON = "reason" + +private val REASON_PROPERTY = string( + "Why you are making this call: what you are trying to learn, or what you concluded from the last " + + "answer. Logged beside the reads it causes, which is what makes this investigation something a person " + + "can follow afterwards rather than a conclusion they have to trust." +) + +/** How every address on this surface starts. See [AgentJson]. */ +internal const val HEX_PREFIX = "0x" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt new file mode 100644 index 0000000000..40659c94bc --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -0,0 +1,626 @@ +package shark.explorer.agent + +import kotlinx.serialization.json.add +import kotlinx.serialization.json.addJsonObject +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put +import kotlinx.serialization.json.putJsonArray +import shark.explorer.HeapDominatorTreemap +import shark.explorer.HeapObjectKind +import shark.explorer.LeakStatus +import shark.explorer.LeakStatusOverride +import shark.explorer.ObjectListFilter +import shark.explorer.Place +import shark.explorer.RootPath +import shark.explorer.RootPathStep +import shark.explorer.exactHexObjectId +import shark.explorer.leakStatusConflictsWith + +/** + * Everything an agent can do to an open heap dump, as MCP tools. + * + * Thin on purpose. The explorer already answers every question the method asks — a chain with its verdicts, + * every way an object is held, the leaks gathered the way LeakCanary gathers them — so a tool here is a + * name, a schema and one call into [AgentHeapDump.read]. What the tools add over an API is the two things + * that make an investigation checkable rather than assertable: + * + * - **A verdict needs a reason**, and a reason that contradicts the ones already set has to say so. Enforced + * by `shark.explorer.LeakStatusOverride` and by [SET_VERDICT] refusing conflicts it wasn't told to solve. + * - **[CONCLUDE] is refused until the heap dump agrees** that one reference is at fault. Which is the whole + * point: an agent that has narrowed a chain to three unexplained steps cannot report a root cause, however + * confident it is, because the software will not let it. + * + * Every tool also takes a mandatory `reason`, logged beside the reads it caused. That is traceability and + * not a quality gate — asking a model to explain itself does not make it right — but it is what turns a run + * into something a person can follow afterwards instead of a conclusion they have to take on trust. + */ +internal class AgentTools(private val heapDumps: AgentHeapDumps) { + + /** In the order an investigation uses them, which is the order a client lists them in. */ + val all: List = listOf( + openHeapDumps(), + listLeaks(), + describeObject(), + chainFromGcRoot(), + waysHeld(), + findObjects(), + setVerdict(), + clearVerdict(), + takeNote(), + show(), + conclude() + ) + + fun byName(name: String): AgentTool? = all.firstOrNull { it.name == name } + + private fun openHeapDumps() = AgentTool( + name = OPEN_HEAP_DUMPS, + description = "Every heap dump open in Shark Explorer right now, with the method to investigate it. " + + "Call this first: the window ids it hands back are what every other tool names a heap dump by, and " + + "the verdicts it lists are the conclusions somebody has already reached about that dump.", + schema = schema() + ) { _ -> + val dumps = heapDumps.openHeapDumps() + // Read before the JSON is built rather than inside it: a heap dump read suspends, and the JSON builders + // don't take a suspending block. + val described = dumps.map { dump -> + AgentJson.heapDump( + windowId = dump.windowId, + heapDumpPath = dump.heapDumpPath, + sizes = dump.read("its sizes, for an agent") { it.sizes }, + verdicts = dump.verdicts + ) + } + buildJsonObject { + // With the answer rather than only in the handshake, because a client that drops the handshake's + // instructions is a client whose model never saw them. See [AgentMethod]. + put("method", AgentMethod.INSTRUCTIONS) + putJsonArray("heapDumps") { described.forEach { add(it) } } + if (dumps.isEmpty()) { + put( + "problem", + "No heap dump is open. Shark Explorer is running, but every window of it is empty — open a " + + "dump in the app, or ask whoever is at the machine to." + ) + } + } + } + + private fun listLeaks() = AgentTool( + name = "list_leaks", + description = "What this heap dump says shouldn't be in memory, gathered into the leaks those objects " + + "are instances of. The heap dump's own answer and the place to start: objects the app itself handed " + + "to LeakCanary and said it was done with are the strongest evidence a dump carries. Sections marked " + + "isOnTheWayOut are objects the garbage collector will take on its own — not leaks to fix.", + schema = schema(WINDOW to window()) + ) { arguments -> + val dump = arguments.heapDump() + val leaks = dump.read("the leaks, for an agent") { it.tree.findLeaks(dump.verdicts) } + AgentJson.leaks(leaks) + } + + private fun describeObject() = AgentTool( + name = "describe_object", + description = "What one object is: its class, what the inspectors made of it, its verdict and the " + + "reason under it, what it retains, what dominates it, and every field with the address of each " + + "field's value. Reading fields is how a guess about an object becomes evidence.", + schema = schema(WINDOW to window(), OBJECT to objectId("The object to describe.")) + ) { arguments -> + val dump = arguments.heapDump() + val objectId = arguments.objectId(OBJECT) + dump.read("${exactHexObjectId(objectId)} for an agent") { explorer -> + val tree = explorer.tree + objectId.requireOneObjectOf(tree) + AgentJson.objectSummary( + summary = tree.summarize(objectId, dump.verdicts), + dominator = tree.dominatorOf(objectId) + ) + } + } + + private fun chainFromGcRoot() = AgentTool( + name = "chain_from_gc_root", + description = "The shortest chain of references from a GC root down to this object, which is where the " + + "leak is. Every step carries its verdict, the reason for it, the inspectors' labels and the field " + + "the step above points through. A reference marked isFaulty is the one the heap dump says is at " + + "fault; while none is, the chain does not yet name a single reference. Steps marked isDominator are " + + "the ones every path to the object goes through.", + schema = schema(WINDOW to window(), OBJECT to objectId("The object to walk up from.")) + ) { arguments -> + val dump = arguments.heapDump() + val objectId = arguments.objectId(OBJECT) + val path = dump.readRootPath(objectId) + buildJsonObject { + put("chain", AgentJson.rootPath(path)) + put("whatTheChainSays", path.verdictState().summary) + } + } + + private fun waysHeld() = AgentTool( + name = "ways_held", + description = "Every way an object is held, rather than the one chain. This is what answers \"is that " + + "reference really the only thing keeping it in memory?\" — a question a single chain cannot answer, " + + "and one that decides whether clearing a field would free anything at all. Give `from` to ask only " + + "about the ways between that object and this one.", + schema = schema( + WINDOW to window(), + OBJECT to objectId("The object being held."), + FROM to objectId("Optional: only the ways this object holds it, rather than from the GC roots.") + .optional() + ) + ) { arguments -> + val dump = arguments.heapDump() + val objectId = arguments.objectId(OBJECT) + val fromObjectId = arguments.optionalObjectId(FROM) + dump.read("every way ${exactHexObjectId(objectId)} is held, for an agent") { explorer -> + val tree = explorer.tree + objectId.requireOneObjectOf(tree) + val paths = if (fromObjectId == null) { + tree.independentPathsFromRoots(objectId, dump.verdicts) + } else { + fromObjectId.requireOneObjectOf(tree) + tree.independentPathsBetween(fromObjectId, objectId, dump.verdicts) + } + AgentJson.independentPaths(paths) + } + } + + private fun findObjects() = AgentTool( + name = "find_objects", + description = "The objects of this heap dump whose class name matches, largest retained size first, " + + "with how many matched in total. Use it on a class you have assumed something about: two instances " + + "of a class you took for a singleton is the answer to a surprising number of leaks, because the " + + "object on the chain then isn't the instance you thought it was.", + schema = schema( + WINDOW to window(), + CLASS_NAME to string("Matched against the class name.").optional(), + EXACT_MATCH to boolean( + "Whether className has to be the whole name — `android.graphics.Bitmap` or `Bitmap` — rather " + + "than part of it. Off by default, which finds every class containing it." + ).optional(), + KINDS to enumArray( + "Which kinds of object to list, all of them by default.", + HeapObjectKind.values().map { it.name } + ).optional(), + LIMIT to integer( + "How many to list, at most ${HeapDominatorTreemap.MAX_LISTED_OBJECTS}. The match count comes " + + "back whole whatever this is." + ).optional() + ) + ) { arguments -> + val dump = arguments.heapDump() + val filter = ObjectListFilter( + query = arguments.optionalString(CLASS_NAME).orEmpty(), + isExactMatch = arguments.boolean(EXACT_MATCH, default = false), + kinds = arguments.kinds() + ) + val limit = arguments.int(LIMIT, default = DEFAULT_LISTED_OBJECTS) + .coerceIn(1, HeapDominatorTreemap.MAX_LISTED_OBJECTS) + val list = dump.read("the objects matching $filter, for an agent") { explorer -> + explorer.tree.listObjects(filter, limit) + } + AgentJson.objectList(list) + } + + private fun setVerdict() = AgentTool( + name = SET_VERDICT, + description = "Records that an object is meant to be in memory (NOT_LEAKING) or should be gone " + + "(LEAKING), which is how the search narrows: a verdict spreads along every chain through that " + + "object, and naming the stuck object you are investigating as `chainTo` answers with what its " + + "chain says once yours is on it. The `reason` is the " + + "verdict's reason and is kept with it — make it something the next reader can check, a field value " + + "or a line of source rather than a hunch. Refuses a verdict that contradicts one already set " + + "unless solveConflicts is true, in which case the ones it disagrees with are flipped and say so.", + schema = schema( + WINDOW to window(), + OBJECT to objectId("The object to record a verdict about."), + VERDICT to enumString( + "LEAKING for an object that should be gone, NOT_LEAKING for one that is meant to be here.", + listOf(LeakStatus.LEAKING.name, LeakStatus.NOT_LEAKING.name) + ), + CHAIN_TO to objectId( + "The stuck object you are investigating, which is what the answer reads the chain to: a verdict is " + + "worth setting for what it does to that chain, and this is where you see the unexplained stretch " + + "narrow. Not the object of this verdict — one recorded as NOT_LEAKING is above the leak, so the " + + "chain ending at it has nothing stuck on it to point at." + ).optional(), + SOLVE_CONFLICTS to boolean( + "Whether to flip the verdicts this one contradicts. Ask without it first and read what they are." + ).optional() + ) + ) { arguments -> + val dump = arguments.heapDump() + val objectId = arguments.objectId(OBJECT) + val status = arguments.verdict() + val override = LeakStatusOverride(objectId, status, arguments.reason) + val conflicts = dump.read( + "what setting ${exactHexObjectId(objectId)} to $status disagrees with, for an agent" + ) { explorer -> + objectId.requireOneObjectOf(explorer.tree) + explorer.tree.leakStatusConflictsWith(override, dump.verdicts) + } + if (conflicts.isNotEmpty() && !arguments.boolean(SOLVE_CONFLICTS, default = false)) { + throw AgentRefusal( + "Not set: $status on ${exactHexObjectId(objectId)} contradicts ${conflicts.size} verdict(s) " + + "already recorded about this heap dump. Everything a stuck object holds is stuck, and " + + "everything holding an object that is meant to be here is meant to be here, so these cannot " + + "all be read off one chain. Either your verdict is wrong, or theirs is. The conflicts are " + + "${AgentJson.conflicts(conflicts)}. Call $SET_VERDICT again with $SOLVE_CONFLICTS true to keep " + + "yours and flip those, and say in your reason why." + ) + } + dump.setVerdict(override, conflicts.map { it.solved }) + // The chain again, because a verdict is only worth setting for what it does to one: this is where an + // agent sees the unexplained stretch narrow, and where it finds out that a reference is now pointed at. + val path = arguments.optionalObjectId(CHAIN_TO)?.let { dump.readRootPath(it) } + buildJsonObject { + put("set", true) + put("verdictsFlipped", conflicts.size) + if (path == null) { + put( + "next", + "Read the chain to the stuck object you are investigating again with chain_from_gc_root, since " + + "what this verdict is worth is what it did to that chain. Naming that object as `$CHAIN_TO` " + + "here answers with it." + ) + } else { + val state = path.verdictState() + put("chain", AgentJson.rootPath(path)) + put("whatTheChainSays", state.summary) + put("canConclude", state.faultyStep != null) + } + } + } + + private fun clearVerdict() = AgentTool( + name = "clear_verdict", + description = "Takes a verdict off an object, so the heap dump says what it says about it again. For " + + "a verdict of yours that the evidence turned out not to support — leaving a wrong one in place is " + + "worse than never setting it, because everything below it reads as stuck because of it.", + schema = schema(WINDOW to window(), OBJECT to objectId("The object to take the verdict off.")) + ) { arguments -> + val dump = arguments.heapDump() + val objectId = arguments.objectId(OBJECT) + val existing = dump.verdicts[objectId] + ?: throw AgentRefusal( + "Nothing to clear: no verdict has been recorded about ${exactHexObjectId(objectId)}." + ) + dump.clearVerdict(objectId) + buildJsonObject { + put("cleared", true) + put("was", existing.status.name) + put("itsReason", existing.reason) + } + } + + private fun takeNote() = AgentTool( + name = "take_note", + description = "Appends markdown to the notes of one place in this heap dump, which is where the " + + "person at the window reads them and what the next reader of this dump finds. Notes are kept " + + "between runs of the app. Write what you found and where you looked, not what you are about to do.", + schema = schema( + WINDOW to window(), + PLACE to place(), + TEXT to string("Markdown. `0x…` addresses in it become links to those objects.") + ) + ) { arguments -> + val dump = arguments.heapDump() + val place = arguments.place() + dump.appendToNote(place, arguments.string(TEXT)) + buildJsonObject { put("written", true) } + } + + private fun show() = AgentTool( + name = "show", + description = "Opens a place in a tab of this window and brings the window to the front, so that what " + + "you are looking at is what the person at the machine is looking at. One call, no answer to wait " + + "for. Use it when you reach something that matters rather than for every step.", + schema = schema(WINDOW to window(), PLACE to place()) + ) { arguments -> + val dump = arguments.heapDump() + val place = arguments.place() + dump.show(place) + buildJsonObject { put("shown", true) } + } + + private fun conclude() = AgentTool( + name = CONCLUDE, + description = "Reports the root cause of one leak, and the only way to finish an investigation. " + + "**Refused unless this heap dump agrees that a single reference is at fault**: one object above it " + + "recorded as NOT_LEAKING, the object below it recorded as LEAKING, and nothing unexplained in " + + "between. If it refuses, the message says what is missing and the investigation is not over. " + + "Isolating the reference is not the root cause — rootCause is how the field came to still be set, " + + "which is a sequence of events rather than a line. The conclusion is written into the notes of the " + + "object and shown in the window.", + schema = schema( + WINDOW to window(), + OBJECT to objectId("The stuck object whose being in memory is being explained."), + ROOT_CAUSE to string( + "How the faulty reference came to still be set: what assigned it, what should have cleared it, " + + "and why it didn't." + ), + HOW_TO_REPRODUCE to string( + "The steps that trigger it, or say that you could not work them out." + ).optional(), + NOT_CHECKED to string( + "What you did not verify. An answer with a stated gap is worth more than one with an unstated gap." + ).optional() + ) + ) { arguments -> + val dump = arguments.heapDump() + val objectId = arguments.objectId(OBJECT) + val path = dump.readRootPath(objectId) + val state = path.verdictState() + val faulty = state.faultyStep + ?: throw AgentRefusal( + "Not concluded. ${state.summary} Until the chain names one reference, a root cause would be a " + + "guess about which of those steps is at fault. Read the objects in the unexplained stretch with " + + "describe_object, check whether anything else holds them with ways_held, and record what you " + + "can defend with $SET_VERDICT." + ) + val reference = requireNotNull(faulty.step.reference) + val note = conclusionNote( + reference = "${reference.ownerClassName}.${reference.name}", + rootCause = arguments.string(ROOT_CAUSE), + howToReproduce = arguments.optionalString(HOW_TO_REPRODUCE), + notChecked = arguments.optionalString(NOT_CHECKED), + reason = arguments.reason + ) + dump.appendToNote(Place.Object(objectId), note) + dump.show(Place.Object(objectId)) + buildJsonObject { + put("concluded", true) + putJsonArray("faultyReference") { + addJsonObject { + put("reference", "${reference.ownerClassName}.${reference.name}") + put("declaredIn", reference.ownerClassName) + put("field", reference.name) + put("heldObject", exactHexObjectId(faulty.step.objectId)) + put("heldClassName", faulty.step.className) + val libraryLeak = reference.libraryLeak + if (libraryLeak != null) { + put("libraryLeakPattern", libraryLeak.pattern) + } + } + } + put("writtenTo", "the notes of ${exactHexObjectId(objectId)}, and shown in window ${dump.windowId}") + } + } + + /** The chain to [objectId], read through the verdicts, refusing an address that is no object of the dump. */ + private suspend fun AgentHeapDump.readRootPath(objectId: Long): RootPath = + read("the chain to ${exactHexObjectId(objectId)}, for an agent") { explorer -> + objectId.requireOneObjectOf(explorer.tree) + explorer.tree.rootPathTo(objectId, verdicts) + } + + /** Which heap dump a call is about, or a refusal naming the ones that are open. */ + private fun AgentArguments.heapDump(): AgentHeapDump { + val open = heapDumps.openHeapDumps() + val windowId = optionalString(WINDOW) + // One open dump needs no naming, which is most sessions. Two of them always do: the same file open twice + // is how two readings of it are compared, so guessing would be answering about the wrong one. + val asked = if (windowId == null) { + open.singleOrNull() + } else { + open.firstOrNull { it.windowId == windowId } + } + if (asked != null) { + return asked + } + val windows = open.joinToString(", ") { "${it.windowId} (${it.heapDumpPath})" } + throw AgentRefusal( + when { + open.isEmpty() -> + "No heap dump is open in Shark Explorer, so there is nothing to read. Call $OPEN_HEAP_DUMPS." + windowId == null -> + "${open.size} heap dumps are open, so say which with `$WINDOW`: $windows" + else -> + "No window is called \"$windowId\". A window id names one window of one run of this app, so it " + + "stops being valid when that window is closed. Open windows: $windows. Call $OPEN_HEAP_DUMPS." + } + ) + } + + private fun AgentArguments.verdict(): LeakStatus { + val text = string(VERDICT) + val status = LeakStatus.values().firstOrNull { it.name.equals(text, ignoreCase = true) } + ?: throw AgentRefusal( + "\"$text\" is no verdict. It is ${LeakStatus.LEAKING.name} for an object that should be gone or " + + "${LeakStatus.NOT_LEAKING.name} for one that is meant to be here." + ) + if (status == LeakStatus.UNKNOWN) { + throw AgentRefusal( + "${LeakStatus.UNKNOWN.name} is what an object with no verdict already is, so setting it says " + + "nothing. To take a verdict back off an object, call clear_verdict." + ) + } + return status + } + + private fun AgentArguments.kinds(): Set { + val names = stringList(KINDS) ?: return HeapObjectKind.values().toSet() + return names.map { name -> + HeapObjectKind.values().firstOrNull { it.name.equals(name, ignoreCase = true) } + ?: throw AgentRefusal( + "\"$name\" is no object kind. They are " + + HeapObjectKind.values().joinToString(", ") { it.name } + "." + ) + }.toSet() + } + + private fun AgentArguments.place(): Place { + val text = string(PLACE) + return when { + text.startsWith(HEX_PREFIX) -> Place.Object(objectIdOf(PLACE, text)) + text == PLACE_LEAKS -> Place.Leaks() + text == PLACE_OBJECTS -> Place.Objects() + text == PLACE_STARRED -> Place.Starred + text.startsWith("$PLACE_OBJECTS:") -> Place.Objects( + ObjectListFilter(query = text.substringAfter(':')) + ) + else -> throw AgentRefusal( + "\"$text\" is no place of a heap dump. A place is an object's address, \"$PLACE_LEAKS\", " + + "\"$PLACE_OBJECTS\", \"$PLACE_OBJECTS:\" or \"$PLACE_STARRED\"." + ) + } + } + + private companion object { + + /** What every investigation starts with, named because three messages point at it. */ + const val OPEN_HEAP_DUMPS = "open_heap_dumps" + const val SET_VERDICT = "set_verdict" + const val CONCLUDE = "conclude" + + const val WINDOW = "window" + const val OBJECT = "object" + const val FROM = "from" + const val CLASS_NAME = "className" + const val EXACT_MATCH = "exactMatch" + const val KINDS = "kinds" + const val LIMIT = "limit" + const val VERDICT = "verdict" + const val CHAIN_TO = "chainTo" + const val SOLVE_CONFLICTS = "solveConflicts" + const val PLACE = "place" + const val TEXT = "text" + const val ROOT_CAUSE = "rootCause" + const val HOW_TO_REPRODUCE = "howToReproduce" + const val NOT_CHECKED = "notChecked" + + const val PLACE_LEAKS = "leaks" + const val PLACE_OBJECTS = "objects" + const val PLACE_STARRED = "starred" + + /** + * How many objects a list comes back with by default, well under + * [HeapDominatorTreemap.MAX_LISTED_OBJECTS]: an agent reads the whole answer, so 500 rows of JSON is + * mostly context spent on rows nobody asked about. The match count says what was left out. + */ + const val DEFAULT_LISTED_OBJECTS = 30 + + fun window() = string( + "Which open heap dump, from ${OPEN_HEAP_DUMPS}. Optional while only one is open." + ).optional() + + fun objectId(description: String) = + string("$description An address as ${OPEN_HEAP_DUMPS} and every chain spells one: `0x…`.") + + fun place() = string( + "Which place of the heap dump: an object's `0x…` address, \"$PLACE_LEAKS\", \"$PLACE_OBJECTS\", " + + "\"$PLACE_OBJECTS:\" or \"$PLACE_STARRED\"." + ) + } +} + +/** + * What the verdicts on a chain add up to: whether one reference is at fault, and what to say when none is. + * + * The same rule `shark.explorer.faultyReferenceIndexOrNull` applies, read off the chain rather than asked + * of it, because the three ways a chain names no reference are three different things to do next — and + * telling an agent which of them it is, is most of what [AgentTools.CONCLUDE] refusing is worth. + */ +private class ChainVerdicts( + val faultyStep: RootPathStep?, + val summary: String +) + +private fun RootPath.verdictState(): ChainVerdicts { + val steps = steps + if (steps.isEmpty()) { + return ChainVerdicts( + faultyStep = null, + summary = "Nothing this heap dump was walked from reaches that object, so there is no chain to read." + ) + } + val firstStuck = steps.indexOfFirst { it.step.leakStatus == LeakStatus.LEAKING } + val lastExpected = steps.indexOfLast { it.step.leakStatus == LeakStatus.NOT_LEAKING } + if (firstStuck == -1) { + return ChainVerdicts( + faultyStep = null, + summary = "Nothing on this chain of ${steps.size} steps is ${LeakStatus.LEAKING.name}, so it points " + + "at no reference: the rules can only name one once something below it is known not to belong." + ) + } + if (lastExpected == -1) { + return ChainVerdicts( + faultyStep = null, + summary = "The chain has a ${LeakStatus.LEAKING.name} object at step ${firstStuck + 1} of " + + "${steps.size} and nothing above it is ${LeakStatus.NOT_LEAKING.name}. So whatever holds it may " + + "be something that should have let go too, and the fault could be further up than this chain " + + "knows: find the highest object here that is meant to be in memory and record it." + ) + } + if (firstStuck != lastExpected + 1) { + val unexplained = (lastExpected + 1 until firstStuck).map { steps[it] } + return ChainVerdicts( + faultyStep = null, + summary = "${unexplained.size} step(s) between the last ${LeakStatus.NOT_LEAKING.name} object and " + + "the first ${LeakStatus.LEAKING.name} one have no verdict, so the fault is at one of them and the " + + "chain doesn't say which: " + + unexplained.joinToString(", ") { "${exactHexObjectId(it.step.objectId)} ${it.step.className}" } + + "." + ) + } + val faulty = steps[firstStuck] + val reference = faulty.step.reference + ?: return ChainVerdicts( + faultyStep = null, + summary = "One reference crosses from ${LeakStatus.NOT_LEAKING.name} to " + + "${LeakStatus.LEAKING.name} here, but reading the object above again didn't find the field it was " + + "reached through, so there is no reference to name." + ) + return ChainVerdicts( + faultyStep = faulty, + summary = "${reference.ownerClassName}.${reference.name} is the faulty reference: the one step from " + + "an object meant to be in memory to one that should be gone." + ) +} + +/** What [AgentTools.CONCLUDE] writes into the notes, which is the investigation's answer where it belongs. */ +private fun conclusionNote( + reference: String, + rootCause: String, + howToReproduce: String?, + notChecked: String?, + reason: String +): String = buildString { + appendLine("## Root cause") + appendLine() + appendLine("**Faulty reference:** `$reference`") + appendLine() + appendLine(rootCause) + if (howToReproduce != null) { + appendLine() + appendLine("**How to reproduce:** $howToReproduce") + } + if (notChecked != null) { + appendLine() + appendLine("**Not checked:** $notChecked") + } + appendLine() + appendLine("_Concluded by an agent: ${reason}_") +} + +/** + * Refuses an id that is no single object of the heap dump, which is three different mistakes. + * + * `summarize` throws on a pile id and the chain walks refuse the root, so the alternative to this is a + * message about the app's internals reaching an agent that asked a reasonable question. + */ +private fun Long.requireOneObjectOf(tree: HeapDominatorTreemap) { + val refusal = when { + this == HeapDominatorTreemap.ROOT_OBJECT_ID -> + "${exactHexObjectId(this)} is the whole heap dump rather than an object of it, so there is nothing " + + "to read about it." + HeapDominatorTreemap.isPileId(this) -> + "${exactHexObjectId(this)} stands for a pile of small objects the map had no room to draw, rather " + + "than for one object. Name one of the objects instead." + tree.objectNameOrNull(this) == null -> + "${exactHexObjectId(this)} is no object of this heap dump. An address is only an address of the dump " + + "it was read from, so one copied from another dump — or from another window — names nothing here." + else -> return + } + throw AgentRefusal(refusal) +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt new file mode 100644 index 0000000000..f54f6ec5dc --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt @@ -0,0 +1,248 @@ +package shark.explorer.agent + +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.json.JsonNull +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.addJsonObject +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.put +import kotlinx.serialization.json.putJsonArray +import kotlinx.serialization.json.putJsonObject +import shark.SharkLog + +/** + * One agent's connection to this app, spoken as [MCP](https://modelcontextprotocol.io). + * + * MCP rather than a command line or a protocol of ours, for one reason: **the app is already running and an + * agent has to reach into it**. A CLI would have to open the heap dump again — seconds and hundreds of + * megabytes per question, and answers about a dump nobody is looking at — while MCP is the one interface + * every agent already has, so a client is configured once and nothing here ever calls a model. + * + * JSON-RPC 2.0, one message per line. That framing is stdio MCP's own, which is what lets the bridge in + * [AgentStdioBridge] be a pipe and nothing more. + * + * A session holds no state of its own. What an investigation accumulates — the verdicts, the notes — lives + * in the heap dump's window, so an agent that reconnects, or a second agent, reads what the first one + * concluded rather than starting from nothing. + */ +internal class McpSession( + private val tools: AgentTools, + /** Which build of the app is answering, for a client that logs what it connected to. */ + private val serverVersion: String +) { + + /** + * Answers one message, or null for one that wants no answer. + * + * Notifications are the null case and it is not optional: JSON-RPC forbids answering a message with no + * id, and a client that gets one back for `notifications/initialized` treats the session as broken. + */ + suspend fun answer(line: String): String? { + val message = try { + JSON.parseToJsonElement(line).jsonObject + } catch (notJson: Exception) { + SharkLog.d(notJson) { "An agent sent something that is no JSON-RPC message" } + return JSON.encodeToString( + JsonElement.serializer(), + errorResponse(id = null, code = PARSE_ERROR, message = "That is not JSON: $notJson") + ) + } + val id = message["id"] + val method = (message["method"] as? JsonPrimitive)?.content + if (method == null) { + return JSON.encodeToString( + JsonElement.serializer(), + errorResponse(id, INVALID_REQUEST, "A request needs a \"method\".") + ) + } + // No id is a notification: nothing is waiting for an answer and sending one is a protocol error. + if (id == null) { + SharkLog.d { "An agent sent the notification $method" } + return null + } + val response = try { + successResponse(id, dispatch(method, message["params"]?.jsonObject ?: EMPTY_PARAMS)) + } catch (unknown: UnknownMethod) { + errorResponse(id, METHOD_NOT_FOUND, unknown.message) + } catch (throwable: Throwable) { + // A read that failed or a bug of ours, either way told to the agent rather than dropped: a client + // waiting for a response it never gets has no way to tell that from the app having gone away. + SharkLog.d(throwable) { "An agent's $method failed" } + errorResponse(id, INTERNAL_ERROR, throwable.toString()) + } + return JSON.encodeToString(JsonElement.serializer(), response) + } + + private suspend fun dispatch( + method: String, + params: JsonObject + ): JsonObject = when (method) { + "initialize" -> initialize(params) + "tools/list" -> buildJsonObject { + putJsonArray("tools") { + tools.all.forEach { tool -> + addJsonObject { + put("name", tool.name) + put("description", tool.description) + put("inputSchema", tool.schema) + } + } + } + } + "tools/call" -> callTool(params) + // Answered because clients use it to find out whether this end is still there, and this end is a + // window someone may have closed. + "ping" -> buildJsonObject { } + else -> throw UnknownMethod("This server has no \"$method\". It has tools, and nothing else.") + } + + /** + * The handshake, which is also where the method is handed over. + * + * **The client's protocol version is echoed back** rather than one of ours being asserted. The spec asks a + * server to answer with the same version when it supports it, and this server has no version-specific + * behaviour at all — it serves tools, which every revision of MCP has — so echoing is both correct and + * the thing that keeps it working against a client newer than this build. + */ + private fun initialize(params: JsonObject): JsonObject { + val clientVersion = (params["protocolVersion"] as? JsonPrimitive)?.content + val clientName = (params["clientInfo"] as? JsonObject) + ?.let { (it["name"] as? JsonPrimitive)?.content } + SharkLog.d { "An agent connected: ${clientName ?: "a client that did not say who it is"}" } + return buildJsonObject { + put("protocolVersion", clientVersion ?: FALLBACK_PROTOCOL_VERSION) + putJsonObject("capabilities") { + putJsonObject("tools") { } + } + putJsonObject("serverInfo") { + put("name", SERVER_NAME) + put("version", serverVersion) + } + // Some clients show this to the model and some drop it, which is why AgentMethod is handed over with + // the first tool answer as well. + put("instructions", AgentMethod.INSTRUCTIONS) + } + } + + private suspend fun callTool(params: JsonObject): JsonObject { + val name = (params["name"] as? JsonPrimitive)?.content + ?: return toolError("A tools/call needs the \"name\" of a tool.") + val tool = tools.byName(name) + ?: return toolError( + "There is no tool called \"$name\". This server has " + + tools.all.joinToString(", ") { it.name } + "." + ) + val arguments = params["arguments"]?.jsonObject ?: EMPTY_PARAMS + // One line per call, before the reads it causes, so that a session log reads as what the agent was + // trying to learn and then what that cost. See [AgentTools]. + SharkLog.d { "An agent called $name${arguments.logLine()}" } + return try { + toolResult(tool.call(arguments)) + } catch (refused: AgentRefusal) { + // A refusal is an answer to the agent and not a failure of the server, so it comes back as a tool + // result the model reads rather than as a JSON-RPC error the client may swallow. + SharkLog.d { "Refused $name: ${refused.message}" } + toolError(refused.message) + } + } + + private fun toolResult(result: JsonObject): JsonObject = buildJsonObject { + putJsonArray("content") { + addJsonObject { + put("type", "text") + // Indented, because the reader is a model reading a chain of twenty steps and every one of them + // matters. The newlines are escaped by being inside a JSON string, so the wire stays one line. + put("text", PRETTY_JSON.encodeToString(JsonElement.serializer(), result)) + } + } + // For the clients that read it, alongside the text for the ones that don't. + put("structuredContent", result) + } + + private fun toolError(message: String): JsonObject = buildJsonObject { + putJsonArray("content") { + addJsonObject { + put("type", "text") + put("text", message) + } + } + put("isError", true) + } + + private fun successResponse( + id: JsonElement, + result: JsonObject + ): JsonObject = buildJsonObject { + put("jsonrpc", JSONRPC_VERSION) + put("id", id) + put("result", result) + } + + private fun errorResponse( + id: JsonElement?, + code: Int, + message: String + ): JsonObject = buildJsonObject { + put("jsonrpc", JSONRPC_VERSION) + // Null when the id could not be read at all, which the spec asks for rather than leaving the key out. + put("id", id ?: JsonNull) + putJsonObject("error") { + put("code", code) + put("message", message) + } + } + + private class UnknownMethod(override val message: String) : Exception(message) + + private companion object { + + val JSON = Json { + ignoreUnknownKeys = true + // A client that leaves an optional argument out sends null for it often enough to matter, and a + // tool asking for a missing argument reads the same either way. + explicitNulls = false + } + + val PRETTY_JSON = Json(JSON) { prettyPrint = true } + + val EMPTY_PARAMS = JsonObject(emptyMap()) + + const val JSONRPC_VERSION = "2.0" + + /** What a client sees this server called, and what an agent's MCP configuration names. */ + const val SERVER_NAME = "shark-explorer" + + /** + * Answered to a client that named no version, which is not a client this has met: the field is required + * of an initialize. The revision this was written against, so that such a client gets a real answer. + */ + const val FALLBACK_PROTOCOL_VERSION = "2025-06-18" + + const val PARSE_ERROR = -32700 + const val INVALID_REQUEST = -32600 + const val METHOD_NOT_FOUND = -32601 + const val INTERNAL_ERROR = -32603 + + /** + * The arguments of a call on one line of the log, with the agent's `reason` first. + * + * Its own reason and not a paraphrase: what to read a session log for is whether the steps follow from + * each other, and that is a question about the sentences the agent wrote at the time. + */ + fun JsonObject.logLine(): String { + if (isEmpty()) { + return "" + } + val reason = (this["reason"] as? JsonPrimitive)?.content + val rest = entries.filter { it.key != "reason" } + .joinToString(", ") { (key, value) -> "$key=${(value as? JsonPrimitive)?.content ?: value}" } + return listOfNotNull( + rest.takeIf { it.isNotEmpty() }?.let { "($it)" }, + reason?.let { " because: $it" } + ).joinToString("") + } + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentHeapDumps.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentHeapDumps.kt new file mode 100644 index 0000000000..244b491edb --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentHeapDumps.kt @@ -0,0 +1,122 @@ +package shark.explorer.agent + +import java.io.Closeable +import org.junit.rules.TemporaryFolder +import shark.GcRoot.JniGlobal +import shark.HprofWriterHelper +import shark.ValueHolder.BooleanHolder +import shark.ValueHolder.IntHolder +import shark.ValueHolder.ReferenceHolder +import shark.dump +import shark.explorer.HeapExplorer + +/** + * The heap dump the tools of this module are asked about, and the addresses of the three objects in it. + * + * Shaped like the leak the method is about, which takes three objects and not two: one an inspector knows + * belongs in memory, one it knows shouldn't be there, and one in between that nothing in the heap dump can + * say either way about. That middle object is the whole point — it is what makes `conclude` refuse until + * somebody has read the code and recorded what they found, and a two object dump would name its faulty + * reference with nobody having investigated anything. + */ +internal fun TemporaryFolder.applicationHoldsActivityThroughHolder(): InvestigationHeapDump { + val file = newFile("application-holds-activity-through-holder.hprof") + var applicationObjectId = 0L + var holderObjectId = 0L + var activityObjectId = 0L + file.dump { + androidBuild() + val activity = instance(activityClass(), fields = listOf(BooleanHolder(true))) + val holder = HOLDER_CLASS_NAME instance { field["activity"] = activity } + val application = instance( + clazz( + className = APPLICATION_CLASS_NAME, + superclassId = clazz(className = "android.app.Application"), + fields = listOf(HOLDER_FIELD_NAME to ReferenceHolder::class) + ), + fields = listOf(holder) + ) + gcRoot(JniGlobal(id = application.value, jniGlobalRefId = 0)) + applicationObjectId = application.value + holderObjectId = holder.value + activityObjectId = activity.value + } + return InvestigationHeapDump( + explorer = HeapExplorer.open(file), + applicationObjectId = applicationObjectId, + holderObjectId = holderObjectId, + activityObjectId = activityObjectId + ) +} + +/** + * An open heap dump and the addresses a test names its objects by. + * + * The addresses come from the fixture rather than from a search of the dump so that a test that fails is a + * test about the tool it names: finding the activity by class name first would make every one of them also a + * test of `find_objects`. + */ +internal class InvestigationHeapDump( + val explorer: HeapExplorer, + /** The app's own `Application`, which an inspector marks as belonging in memory. */ + val applicationObjectId: Long, + /** The object in between, which nothing in the heap dump knows anything about. */ + val holderObjectId: Long, + /** The destroyed activity, which an inspector marks as one that should be gone. */ + val activityObjectId: Long +) : Closeable { + + override fun close() { + explorer.close() + } +} + +/** + * `android.app.Activity` and the app's own subclass of it, which is what the object inspectors look for: an + * instance of the subclass whose inherited `mDestroyed` is true is a destroyed activity. + * + * Field values are written most derived class first, and the subclass declares none, so an instance of it is + * written with the one field the superclass has. + */ +private fun HprofWriterHelper.activityClass(): Long = clazz( + className = ACTIVITY_CLASS_NAME, + superclassId = clazz( + className = "android.app.Activity", + fields = listOf("mDestroyed" to BooleanHolder::class) + ) +) + +/** + * What `android.os.Build` looks like in a dump, which is what Shark matches its library leak patterns + * against — and a dump with the class but not these three fields makes it throw a bare NPE from under + * everything. See `shark/shark-explorer/AGENTS.md`. + * + * Duplicated from the other modules' tests rather than shared, since a test helper is not worth a module's + * public API. + */ +private fun HprofWriterHelper.androidBuild() { + "android.os.Build" clazz { + staticField["MANUFACTURER"] = string("Google") + staticField["ID"] = string("BP31.250610.004") + } + "android.os.Build\$VERSION" clazz { + // Recent enough that none of Shark's known library leaks is in this dump, so that the references a + // chain through it names are the app's own. + staticField["SDK_INT"] = IntHolder(34) + } +} + +internal const val ACTIVITY_CLASS_NAME = "com.example.MainActivity" + +internal const val HOLDER_CLASS_NAME = "com.example.Holder" + +internal const val APPLICATION_CLASS_NAME = "com.example.ExampleApplication" + +/** The field of the holder that keeps the activity, which is the faulty reference of this dump. */ +internal const val ACTIVITY_FIELD_NAME = "activity" + +/** And the field above it, which is the one a chain names while the holder has no verdict. */ +internal const val HOLDER_FIELD_NAME = "holder" + +/** How a chain spells the reference at fault once the holder is known to belong in memory. */ +internal const val FAULTY_REFERENCE = "Holder.$ACTIVITY_FIELD_NAME" diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt new file mode 100644 index 0000000000..6ae4f1dc65 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt @@ -0,0 +1,158 @@ +package shark.explorer.agent + +import java.io.BufferedReader +import java.io.Closeable +import java.io.File +import java.io.InputStreamReader +import java.io.OutputStreamWriter +import java.io.PrintWriter +import java.net.InetAddress +import java.net.Socket +import org.assertj.core.api.Assertions.assertThat +import org.junit.After +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder + +/** + * How an agent finds this run of the app and gets served by it, over a real socket. + * + * What is being tested is the half of this that isn't the protocol: a run publishing where it answers, a + * token being the whole of who may talk to it, and a file left behind by a run that is gone being cleared out + * by whoever reads it next. [McpSessionTest] covers what is said once a connection is up. + */ +class AgentServerTest { + + @get:Rule + val temporaryFolder = TemporaryFolder() + + @get:Rule + val log = RecordedLog() + + private lateinit var directory: File + private lateinit var heapDump: InvestigationHeapDump + private lateinit var window: FakeAgentHeapDump + private val closeables = mutableListOf() + + @Before + fun setUp() { + directory = temporaryFolder.newFolder("agents") + heapDump = temporaryFolder.applicationHoldsActivityThroughHolder() + window = FakeAgentHeapDump(heapDump.explorer) + } + + @After + fun tearDown() { + closeables.forEach { it.close() } + heapDump.close() + } + + @Test + fun `a run publishes where it answers, and answers there`() { + listen() + + val run = AgentServer.publishedRuns(directory).single() + assertThat(run.pid).isEqualTo(ProcessHandle.current().pid().toString()) + assertThat(run.port).isGreaterThan(0) + assertThat(run.token).hasSize(32) + + connect(run).use { client -> + assertThat(client.accepted).isTrue() + val answer = client.ask( + """{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"open_heap_dumps",""" + + """"arguments":{"reason":"Finding out what is open."}}}""" + ) + assertThat(answer).contains(heapDump.explorer.heapDumpFile.name).contains(window.windowId) + } + } + + @Test + fun `a client that quotes the wrong token is not listened to`() { + listen() + val run = AgentServer.publishedRuns(directory).single() + + connect(run, token = "0".repeat(32)).use { client -> + assertThat(client.accepted).isFalse() + } + + // And the run is still there for a client that has the right one, since a wrong token is a stale file + // being read far more often than it is anything to worry about. + connect(run).use { client -> assertThat(client.accepted).isTrue() } + } + + @Test + fun `two agents at once are two sessions of one run`() { + listen() + val run = AgentServer.publishedRuns(directory).single() + + connect(run).use { first -> + connect(run).use { second -> + assertThat(first.ask(PING)).contains("\"id\":1") + assertThat(second.ask(PING)).contains("\"id\":1") + } + } + } + + @Test + fun `closing a run takes it off the list`() { + val listening = listen() + + listening.close() + + assertThat(AgentServer.publishedRuns(directory)).isEmpty() + } + + @Test + fun `a file that names no run is deleted by whoever reads it`() { + val nonsense = File(directory, "1234${AgentServer.RUN_SUFFIX}") + nonsense.writeText("this file is not a published run") + + assertThat(AgentServer.publishedRuns(directory)).isEmpty() + assertThat(nonsense).doesNotExist() + assertThat(log).anyMatch { it.contains("names no run") } + } + + private fun listen(): Closeable = AgentServer.listen( + heapDumps = { listOf(window) }, + serverVersion = "1.2.3", + directory = directory + ).also { closeables += it } + + private fun connect( + run: AgentServer.PublishedRun, + token: String = run.token + ): TestClient = TestClient(run.port, token) + + /** An agent's end of the connection, as far as this test needs one: a token, then a line at a time. */ + private class TestClient( + port: Int, + token: String + ) : Closeable { + + private val socket = Socket(InetAddress.getLoopbackAddress(), port) + private val toApp = PrintWriter(OutputStreamWriter(socket.getOutputStream(), Charsets.UTF_8), true) + private val fromApp = BufferedReader(InputStreamReader(socket.getInputStream(), Charsets.UTF_8)) + + val accepted: Boolean + + init { + toApp.println(token) + accepted = fromApp.readLine() == AgentServer.ACCEPTED + } + + fun ask(message: String): String { + toApp.println(message) + return requireNotNull(fromApp.readLine()) { "The run answered nothing to $message" } + } + + override fun close() { + socket.close() + } + } + + private companion object { + + const val PING = """{"jsonrpc":"2.0","id":1,"method":"ping"}""" + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt new file mode 100644 index 0000000000..f0826402e0 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt @@ -0,0 +1,164 @@ +package shark.explorer.agent + +import java.io.ByteArrayOutputStream +import java.io.Closeable +import java.io.File +import java.io.PipedInputStream +import java.io.PipedOutputStream +import java.io.PrintStream +import java.net.ServerSocket +import java.util.Properties +import org.assertj.core.api.Assertions.assertThat +import org.junit.After +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import shark.explorer.exactHexObjectId + +/** + * The pipe an MCP client actually launches, end to end: stdin to a window and its answers back to stdout. + * + * Worth testing as a whole rather than in parts, because what it is for is the one thing a client can be + * configured with — a command — reaching a port that changes every run. Anything between the two ends being + * wrong is a client reporting a server with no tools. + */ +class AgentStdioBridgeTest { + + @get:Rule + val temporaryFolder = TemporaryFolder() + + @get:Rule + val log = RecordedLog() + + private lateinit var directory: File + private lateinit var heapDump: InvestigationHeapDump + private lateinit var window: FakeAgentHeapDump + private val closeables = mutableListOf() + + @Before + fun setUp() { + directory = temporaryFolder.newFolder("agents") + heapDump = temporaryFolder.applicationHoldsActivityThroughHolder() + window = FakeAgentHeapDump(heapDump.explorer) + } + + @After + fun tearDown() { + closeables.forEach { it.close() } + heapDump.close() + } + + @Test + fun `a client's messages reach the window and its answers come back`() { + closeables += AgentServer.listen( + heapDumps = { listOf(window) }, + serverVersion = "1.2.3", + directory = directory + ) + + val answers = bridge { send -> + send("""{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18"}}""") + send( + """{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"describe_object",""" + + """"arguments":{"object":"${exactHexObjectId(heapDump.holderObjectId)}",""" + + """"reason":"Reading the holder's fields through the bridge."}}}""" + ) + } + + assertThat(answers).hasSize(2) + assertThat(answers[0]).contains("\"id\":1").contains("shark-explorer") + assertThat(answers[1]).contains("\"id\":2").contains(HOLDER_CLASS_NAME) + assertThat(window.reads).isNotEmpty + } + + @Test + fun `no run to talk to is reported rather than waited for`() { + val exitCode = runBridge() + + assertThat(exitCode).isEqualTo(NOTHING_TO_TALK_TO) + } + + @Test + fun `a run that no longer answers on its port has its file cleared out`() { + val port = ServerSocket(0).use { it.localPort } + val stale = File(directory, "999999${AgentServer.RUN_SUFFIX}") + stale.outputStream().use { output -> + Properties().apply { + setProperty("port", port.toString()) + setProperty("token", "0".repeat(32)) + }.store(output, null) + } + + val exitCode = runBridge() + + assertThat(exitCode).isEqualTo(NOTHING_TO_TALK_TO) + assertThat(stale).doesNotExist() + } + + /** + * Runs the bridge over a pipe, sends what [session] sends, and hands back the lines that came out. + * + * A pipe rather than a string of input, because a real client keeps stdin open until it has what it asked + * for: closing it the moment the last message is written would be a race with the answer coming back, and a + * test that lost it would be reporting the timing rather than the wiring. + */ + private fun bridge(session: (send: (String) -> Unit) -> Unit): List { + val stdin = PipedOutputStream() + val stdout = ByteArrayOutputStream() + val previousIn = System.`in` + val previousOut = System.out + System.setIn(PipedInputStream(stdin)) + System.setOut(PrintStream(stdout, true, Charsets.UTF_8.name())) + var sent = 0 + try { + val bridge = Thread({ runBridge() }, "bridge under test").apply { + isDaemon = true + start() + } + session { message -> + stdin.write("$message\n".toByteArray(Charsets.UTF_8)) + stdin.flush() + sent++ + // One answer per message, waited for before the next goes out, which is what makes closing stdin at + // the end of the session safe. + awaitLines(stdout, sent) + } + stdin.close() + bridge.join(JOIN_MILLIS) + } finally { + System.setIn(previousIn) + System.setOut(previousOut) + } + return stdout.toString(Charsets.UTF_8.name()).lines().filter { it.isNotBlank() } + } + + /** Nothing waited for, since the run these tests are about is either already published or never will be. */ + private fun runBridge(): Int = AgentStdioBridge.run(directory, pid = null, waitMillis = 0L) + + private fun awaitLines( + stdout: ByteArrayOutputStream, + count: Int + ) { + val giveUpAt = System.currentTimeMillis() + AWAIT_MILLIS + while (System.currentTimeMillis() < giveUpAt) { + if (stdout.toString(Charsets.UTF_8.name()).lines().count { it.isNotBlank() } >= count) { + return + } + Thread.sleep(POLL_MILLIS) + } + throw AssertionError( + "Waited ${AWAIT_MILLIS}ms for $count answers and got: ${stdout.toString(Charsets.UTF_8.name())}" + ) + } + + private companion object { + + /** What the bridge ends with when it found no window, which a client shows as a server that failed. */ + const val NOTHING_TO_TALK_TO = 1 + + const val AWAIT_MILLIS = 10_000L + const val POLL_MILLIS = 20L + const val JOIN_MILLIS = 5_000L + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt new file mode 100644 index 0000000000..974fb5d226 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -0,0 +1,515 @@ +package shark.explorer.agent + +import kotlinx.coroutines.runBlocking +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.add +import kotlinx.serialization.json.buildJsonArray +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.jsonArray +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import kotlinx.serialization.json.put +import org.assertj.core.api.Assertions.assertThat +import org.assertj.core.api.Assertions.assertThatThrownBy +import org.junit.After +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import shark.explorer.HeapObjectKind +import shark.explorer.LeakStatus +import shark.explorer.ObjectListFilter +import shark.explorer.Place +import shark.explorer.exactHexObjectId + +/** + * What an agent gets back from each tool, and what it gets refused for. + * + * The story these run through is the one the whole surface exists for: a heap dump that says a destroyed + * activity shouldn't be there, a chain that will not name a single reference while the object above it has no + * verdict, a refusal to conclude that says which step is unexplained, and then — a verdict later — the same + * chain naming `Holder.activity` and a conclusion the software agreed to. + */ +class AgentToolsTest { + + @get:Rule + val temporaryFolder = TemporaryFolder() + + private lateinit var heapDump: InvestigationHeapDump + private lateinit var window: FakeAgentHeapDump + private lateinit var tools: AgentTools + + @Before + fun setUp() { + heapDump = temporaryFolder.applicationHoldsActivityThroughHolder() + window = FakeAgentHeapDump(heapDump.explorer) + tools = AgentTools { listOf(window) } + } + + @After + fun tearDown() { + heapDump.close() + } + + @Test + fun `open heap dumps hands over the method with the dump`() { + val answer = call(OPEN_HEAP_DUMPS) + + assertThat(answer.text("method")).isEqualTo(AgentMethod.INSTRUCTIONS) + val dumps = answer.array("heapDumps") + assertThat(dumps).hasSize(1) + assertThat(dumps.first().jsonObject.text("window")).isEqualTo(window.windowId) + assertThat(dumps.first().jsonObject.text("heapDumpPath")) + .isEqualTo(heapDump.explorer.heapDumpFile.absolutePath) + } + + @Test + fun `sizes come back with what a retained size is a share of`() { + val sizes = call(OPEN_HEAP_DUMPS).array("heapDumps").first().jsonObject.obj("sizes") + + assertThat(sizes.text("totalBytes").toLong()).isGreaterThan(0) + assertThat(sizes.text("stronglyReachableBytes").toLong()).isGreaterThan(0) + assertThat(sizes.array("byStrength")).isNotEmpty + } + + @Test + fun `a run with no heap dump open says so rather than answering`() { + tools = AgentTools { emptyList() } + + assertThat(call(OPEN_HEAP_DUMPS).text("problem")).contains("No heap dump is open") + assertThatThrownBy { call("list_leaks") } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining(OPEN_HEAP_DUMPS) + } + + @Test + fun `two heap dumps open have to be named`() { + val other = FakeAgentHeapDump(heapDump.explorer, windowId = "otherwindow") + tools = AgentTools { listOf(window, other) } + + assertThatThrownBy { call("list_leaks") } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("2 heap dumps are open") + .hasMessageContaining(window.windowId) + .hasMessageContaining(other.windowId) + + assertThat(call("list_leaks", "window" to other.windowId).text("objectCount")).isNotEmpty() + } + + @Test + fun `a window that is not open is refused by name`() { + assertThatThrownBy { call("list_leaks", "window" to "closedwindow") } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("No window is called \"closedwindow\"") + .hasMessageContaining(window.windowId) + } + + @Test + fun `every call needs a reason`() { + assertThatThrownBy { callWith("list_leaks", buildJsonObject { }) } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("list_leaks needs `reason`") + } + + @Test + fun `the leaks are what the heap dump says shouldn't be there`() { + val leaks = call("list_leaks") + + val objects = leaks.array("sections").flatMap { section -> + section.jsonObject.array("groups").flatMap { it.jsonObject.array("objects") } + } + assertThat(objects.map { it.jsonObject.text("object") }) + .contains(exactHexObjectId(heapDump.activityObjectId)) + assertThat(objects.map { it.jsonObject.text("className") }).contains(ACTIVITY_CLASS_NAME) + } + + @Test + fun `describing an object reads its fields with the address of each value`() { + val holder = call("describe_object", OBJECT to hex(heapDump.holderObjectId)) + + assertThat(holder.text("className")).isEqualTo(HOLDER_CLASS_NAME) + assertThat(holder.text("verdict")).isEqualTo(LeakStatus.UNKNOWN.name) + val activityField = holder.array("fields") + .single { it.jsonObject.text("name") == ACTIVITY_FIELD_NAME } + .jsonObject + assertThat(activityField.text("valueObject")).isEqualTo(hex(heapDump.activityObjectId)) + } + + @Test + fun `an object the inspectors know about comes back with their verdict and their words`() { + val activity = call("describe_object", OBJECT to hex(heapDump.activityObjectId)) + + assertThat(activity.text("verdict")).isEqualTo(LeakStatus.LEAKING.name) + assertThat(activity.text("verdictReason")).contains("mDestroyed") + } + + @Test + fun `the chain names no reference while a step in it has no verdict`() { + val answer = call("chain_from_gc_root", OBJECT to hex(heapDump.activityObjectId)) + + val steps = answer.obj("chain").array("steps").map { it.jsonObject } + assertThat(steps.map { it.text("object") }).containsExactly( + hex(heapDump.applicationObjectId), + hex(heapDump.holderObjectId), + hex(heapDump.activityObjectId) + ) + assertThat(steps.mapNotNull { it["reference"]?.jsonObject?.text("isFaulty") }) + .containsOnly("false") + assertThat(answer.text("whatTheChainSays")) + .contains("1 step(s)") + .contains(hex(heapDump.holderObjectId)) + .contains(HOLDER_CLASS_NAME) + } + + @Test + fun `a chain with nothing stuck on it says that is why it names nothing`() { + val answer = call("chain_from_gc_root", OBJECT to hex(heapDump.applicationObjectId)) + + assertThat(answer.text("whatTheChainSays")).contains("Nothing on this chain") + } + + @Test + fun `concluding is refused while a step of the chain has no verdict`() { + assertThatThrownBy { + call( + CONCLUDE, + OBJECT to hex(heapDump.activityObjectId), + "rootCause" to "The holder is a singleton that never lets go of the activity." + ) + } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("Not concluded") + .hasMessageContaining("1 step(s)") + .hasMessageContaining(HOLDER_CLASS_NAME) + .hasMessageContaining("describe_object") + + assertThat(window.notes).isEmpty() + } + + @Test + fun `a verdict narrows the chain to the stuck object to one reference`() { + val answer = call( + SET_VERDICT, + OBJECT to hex(heapDump.holderObjectId), + "verdict" to LeakStatus.NOT_LEAKING.name, + "chainTo" to hex(heapDump.activityObjectId), + "reason" to "Holder.INSTANCE is a static singleton, so it is meant to be in memory." + ) + + assertThat(answer.text("set")).isEqualTo("true") + assertThat(answer.text("verdictsFlipped")).isEqualTo("0") + assertThat(answer.text("canConclude")).isEqualTo("true") + assertThat(answer.text("whatTheChainSays")).contains("$FAULTY_REFERENCE is the faulty reference") + val faulty = answer.obj("chain").array("steps") + .single { it.jsonObject["reference"]?.jsonObject?.text("isFaulty") == "true" } + .jsonObject + assertThat(faulty.text("object")).isEqualTo(hex(heapDump.activityObjectId)) + } + + @Test + fun `a verdict set without naming the stuck object says to read that chain again`() { + val answer = call( + SET_VERDICT, + OBJECT to hex(heapDump.holderObjectId), + "verdict" to LeakStatus.NOT_LEAKING.name, + "reason" to "Holder.INSTANCE is a static singleton, so it is meant to be in memory." + ) + + assertThat(answer.text("set")).isEqualTo("true") + assertThat(answer.text("next")).contains("chain_from_gc_root").contains("chainTo") + assertThat(answer["chain"]).isNull() + } + + @Test + fun `the reason of a call is the reason kept with the verdict`() { + call( + SET_VERDICT, + OBJECT to hex(heapDump.holderObjectId), + "verdict" to LeakStatus.NOT_LEAKING.name, + "reason" to "Holder.INSTANCE is a static singleton." + ) + + val verdict = window.verdicts[heapDump.holderObjectId] + assertThat(verdict?.status).isEqualTo(LeakStatus.NOT_LEAKING) + assertThat(verdict?.reason).isEqualTo("Holder.INSTANCE is a static singleton.") + } + + @Test + fun `concluding names the faulty reference and writes it where the window shows it`() { + setHolderExpected() + + val answer = call( + CONCLUDE, + OBJECT to hex(heapDump.activityObjectId), + "rootCause" to "Holder.activity is assigned in onCreate and nothing clears it in onDestroy.", + "howToReproduce" to "Open the screen, rotate, press back.", + "notChecked" to "Whether the second instance of the holder is reached the same way.", + "reason" to "The chain names one reference and the code says why it is still set." + ) + + assertThat(answer.text("concluded")).isEqualTo("true") + val faulty = answer.array("faultyReference").single().jsonObject + assertThat(faulty.text("reference")).isEqualTo(FAULTY_REFERENCE) + assertThat(faulty.text("field")).isEqualTo(ACTIVITY_FIELD_NAME) + assertThat(faulty.text("heldObject")).isEqualTo(hex(heapDump.activityObjectId)) + assertThat(faulty.text("heldClassName")).isEqualTo(ACTIVITY_CLASS_NAME) + } + + @Test + fun `the conclusion is written into the notes of the object it explains`() { + setHolderExpected() + + call( + CONCLUDE, + OBJECT to hex(heapDump.activityObjectId), + "rootCause" to "Nothing clears Holder.activity in onDestroy.", + "notChecked" to "Whether anything else holds the holder.", + "reason" to "One reference, and the code says why it is still set." + ) + + val place = Place.Object(heapDump.activityObjectId) + assertThat(window.notes[place]?.single()) + .contains("## Root cause") + .contains("`$FAULTY_REFERENCE`") + .contains("Nothing clears Holder.activity in onDestroy.") + .contains("**Not checked:** Whether anything else holds the holder.") + .contains("One reference, and the code says why it is still set.") + assertThat(window.shown).contains(place) + } + + @Test + fun `a verdict that contradicts one already recorded is refused until it is told to flip it`() { + setHolderExpected() + + assertThatThrownBy { + call( + SET_VERDICT, + OBJECT to hex(heapDump.applicationObjectId), + "verdict" to LeakStatus.LEAKING.name, + "reason" to "This isn't the real Application, it is a copy left over from a test." + ) + } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("contradicts 1 verdict(s)") + .hasMessageContaining(hex(heapDump.holderObjectId)) + .hasMessageContaining("solveConflicts") + + val answer = call( + SET_VERDICT, + OBJECT to hex(heapDump.applicationObjectId), + "verdict" to LeakStatus.LEAKING.name, + "solveConflicts" to "true", + "reason" to "This isn't the real Application, it is a copy left over from a test." + ) + + assertThat(answer.text("verdictsFlipped")).isEqualTo("1") + assertThat(window.verdicts[heapDump.holderObjectId]?.status).isEqualTo(LeakStatus.LEAKING) + assertThat(window.verdicts[heapDump.holderObjectId]?.reason) + .contains("Holder.INSTANCE is a static singleton") + } + + @Test + fun `unknown is refused as a verdict, because it is what no verdict already is`() { + assertThatThrownBy { + call( + SET_VERDICT, + OBJECT to hex(heapDump.holderObjectId), + "verdict" to LeakStatus.UNKNOWN.name, + "reason" to "I could not work out what this is." + ) + } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("clear_verdict") + } + + @Test + fun `clearing a verdict says what it was, and clearing nothing is refused`() { + setHolderExpected() + + val answer = call("clear_verdict", OBJECT to hex(heapDump.holderObjectId)) + + assertThat(answer.text("was")).isEqualTo(LeakStatus.NOT_LEAKING.name) + assertThat(answer.text("itsReason")).contains("static singleton") + assertThat(window.verdicts.isEmpty).isTrue() + + assertThatThrownBy { call("clear_verdict", OBJECT to hex(heapDump.holderObjectId)) } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("Nothing to clear") + } + + @Test + fun `every way an object is held comes back with whether that is all of them`() { + val answer = call("ways_held", OBJECT to hex(heapDump.activityObjectId)) + + assertThat(answer.text("pathCount")).isEqualTo("1") + assertThat(answer.text("hasMore")).isEqualTo("false") + + val between = call( + "ways_held", + OBJECT to hex(heapDump.activityObjectId), + "from" to hex(heapDump.applicationObjectId) + ) + assertThat(between.text("pathCount")).isEqualTo("1") + } + + @Test + fun `finding objects counts every match rather than the rows it showed`() { + val capped = call("find_objects", "className" to "com.example", "limit" to "1") + + assertThat(capped.array("objects")).hasSize(1) + assertThat(capped.text("matchCount").toInt()).isGreaterThan(1) + assertThat(capped.text("isComplete")).isEqualTo("false") + } + + @Test + fun `one class can be asked for the instances of it and nothing else`() { + val answer = callWith( + "find_objects", + buildJsonObject { + put("className", HOLDER_CLASS_NAME) + put("exactMatch", true) + put("kinds", jsonArrayOf(HeapObjectKind.INSTANCE.name)) + put("reason", "Checking whether the holder is the singleton it looks like.") + } + ) + + assertThat(answer.text("matchCount")).isEqualTo("1") + assertThat(answer.text("isComplete")).isEqualTo("true") + assertThat(answer.array("objects").single().jsonObject.text("object")) + .isEqualTo(hex(heapDump.holderObjectId)) + } + + @Test + fun `an object kind that does not exist is refused by name`() { + assertThatThrownBy { + callWith( + "find_objects", + buildJsonObject { + put("kinds", jsonArrayOf("BITMAPS")) + put("reason", "Looking for the bitmaps.") + } + ) + } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("\"BITMAPS\" is no object kind") + } + + @Test + fun `a note is appended to the place it is about`() { + val answer = call( + "take_note", + "place" to hex(heapDump.holderObjectId), + "text" to "Holder.INSTANCE is assigned in ExampleApplication.onCreate." + ) + + assertThat(answer.text("written")).isEqualTo("true") + assertThat(window.notes[Place.Object(heapDump.holderObjectId)]) + .containsExactly("Holder.INSTANCE is assigned in ExampleApplication.onCreate.") + } + + @Test + fun `every kind of place can be shown, and nothing else can`() { + call("show", "place" to PLACE_LEAKS) + call("show", "place" to "objects") + call("show", "place" to "objects:$HOLDER_CLASS_NAME") + call("show", "place" to "starred") + call("show", "place" to hex(heapDump.activityObjectId)) + + assertThat(window.shown).containsExactly( + Place.Leaks(), + Place.Objects(), + Place.Objects(ObjectListFilter(query = HOLDER_CLASS_NAME)), + Place.Starred, + Place.Object(heapDump.activityObjectId) + ) + + assertThatThrownBy { call("show", "place" to "the leak") } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("is no place of a heap dump") + } + + @Test + fun `an address written as a decimal number is refused as one`() { + assertThatThrownBy { + call("describe_object", OBJECT to heapDump.activityObjectId.toString()) + } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("never a decimal number") + } + + @Test + fun `an address of no object of this heap dump is refused as one`() { + assertThatThrownBy { call("describe_object", OBJECT to "0x1") } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("is no object of this heap dump") + } + + @Test + fun `every read of the heap dump says what it was for`() { + call("describe_object", OBJECT to hex(heapDump.activityObjectId)) + + assertThat(window.reads).containsExactly("${hex(heapDump.activityObjectId)} for an agent") + } + + /** What the whole investigation turns on: the object in between is one somebody read the code about. */ + private fun setHolderExpected() { + call( + SET_VERDICT, + OBJECT to hex(heapDump.holderObjectId), + "verdict" to LeakStatus.NOT_LEAKING.name, + "reason" to "Holder.INSTANCE is a static singleton, so it is meant to be in memory." + ) + } + + private fun call( + name: String, + vararg arguments: Pair + ): JsonObject = callWith( + name, + buildJsonObject { + arguments.forEach { (key, value) -> put(key, value) } + if (arguments.none { it.first == "reason" }) { + put("reason", "Testing $name") + } + } + ) + + private fun callWith( + name: String, + arguments: JsonObject + ): JsonObject = runBlocking { + val tool = requireNotNull(tools.byName(name)) { "There is no tool called $name" } + tool.call(arguments) + } + + private fun hex(objectId: Long) = exactHexObjectId(objectId) + + private companion object { + + const val OPEN_HEAP_DUMPS = "open_heap_dumps" + const val SET_VERDICT = "set_verdict" + const val CONCLUDE = "conclude" + const val OBJECT = "object" + const val PLACE_LEAKS = "leaks" + + /** + * One value of a field of the answer, whatever it is, as text. + * + * As text because that is what a client of this protocol reads a JSON value as at the far end of a + * socket, and because an assertion that has to say which of `jsonPrimitive`, `boolean` and `long` a + * field is, is an assertion about kotlinx rather than about the answer. + */ + fun JsonObject.text(name: String): String = + requireNotNull(this[name]) { "$name is not in $this" }.jsonPrimitive.content + + fun JsonObject.obj(name: String): JsonObject = + requireNotNull(this[name]) { "$name is not in $this" }.jsonObject + + fun JsonObject.array(name: String): JsonArray = + requireNotNull(this[name]) { "$name is not in $this" }.jsonArray + + fun jsonArrayOf(vararg values: String): JsonArray = + buildJsonArray { values.forEach { add(it) } } + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt new file mode 100644 index 0000000000..e23f7e53d5 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt @@ -0,0 +1,73 @@ +package shark.explorer.agent + +import java.io.Closeable +import shark.SharkLog +import shark.explorer.HeapExplorer +import shark.explorer.LeakStatusOverride +import shark.explorer.LeakStatusOverrides +import shark.explorer.Place + +/** + * A heap dump open the way a window has one, without a window. + * + * Which is the whole reason [AgentHeapDump] is an interface: every tool is a read of a heap dump and a write + * of a verdict or a note, so a test of what a tool answers needs a dump and three fields, and none of + * Compose, the session or the tabs. + */ +internal class FakeAgentHeapDump( + private val explorer: HeapExplorer, + override val windowId: String = "testwindow" +) : AgentHeapDump, Closeable { + + override val heapDumpPath: String get() = explorer.heapDumpFile.absolutePath + + override var verdicts: LeakStatusOverrides = LeakStatusOverrides.NONE + private set + + /** What was written about each place, in the order it was written, so a test can read it back. */ + val notes = mutableMapOf>() + + /** The places an agent asked the window to show, in order. */ + val shown = mutableListOf() + + /** What each read was described as, which is what a session log would have said. */ + val reads = mutableListOf() + + override suspend fun read( + description: String, + block: (HeapExplorer) -> T + ): T { + reads += description + // Logged as well as recorded, because the window's own `HeapDumpSession.read` logs every read: what a + // session log has to show is the reason for a call and then the reads it caused, in that order, and a + // fake that logged nothing would leave that assertion with only half of what it is about. + SharkLog.d { description } + return block(explorer) + } + + override suspend fun setVerdict( + verdict: LeakStatusOverride, + solved: List + ) { + verdicts = verdicts.with(listOf(verdict) + solved) + } + + override suspend fun clearVerdict(objectId: Long) { + verdicts = verdicts.without(objectId) + } + + override suspend fun appendToNote( + place: Place, + text: String + ) { + notes.getOrPut(place) { mutableListOf() } += text + } + + override fun show(place: Place) { + shown += place + } + + override fun close() { + explorer.close() + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt new file mode 100644 index 0000000000..36d6261bdd --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -0,0 +1,211 @@ +package shark.explorer.agent + +import kotlinx.coroutines.runBlocking +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.jsonArray +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import org.assertj.core.api.Assertions.assertThat +import org.junit.After +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import shark.explorer.exactHexObjectId + +/** + * What a client of this server gets back, as JSON-RPC rather than as Kotlin. + * + * The tools are tested against a heap dump in [AgentToolsTest]; what is left here is everything about being + * spoken to over a socket by a program that is not this one — the handshake, a notification that must not be + * answered, a refusal arriving as something the model reads rather than as an error the client swallows, and + * the line the session log gets for every call. + */ +class McpSessionTest { + + @get:Rule + val temporaryFolder = TemporaryFolder() + + @get:Rule + val log = RecordedLog() + + private lateinit var heapDump: InvestigationHeapDump + private lateinit var window: FakeAgentHeapDump + private lateinit var session: McpSession + + @Before + fun setUp() { + heapDump = temporaryFolder.applicationHoldsActivityThroughHolder() + window = FakeAgentHeapDump(heapDump.explorer) + session = McpSession(AgentTools { listOf(window) }, serverVersion = SERVER_VERSION) + } + + @After + fun tearDown() { + heapDump.close() + } + + @Test + fun `the handshake echoes the version the client asked for and hands over the method`() { + val result = answer( + """{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2099-01-01",""" + + """"clientInfo":{"name":"a client from the future"},"capabilities":{}}}""" + ).result() + + assertThat(result.text("protocolVersion")).isEqualTo("2099-01-01") + assertThat(result.text("instructions")).isEqualTo(AgentMethod.INSTRUCTIONS) + assertThat(result.obj("serverInfo").text("name")).isEqualTo("shark-explorer") + assertThat(result.obj("serverInfo").text("version")).isEqualTo(SERVER_VERSION) + assertThat(result.obj("capabilities")["tools"]).isNotNull + } + + @Test + fun `a client that named no version is answered with the one this was written against`() { + val result = answer("""{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}""").result() + + assertThat(result.text("protocolVersion")).isEqualTo("2025-06-18") + } + + @Test + fun `every tool is listed with a schema that asks for a reason`() { + val tools = answer("""{"jsonrpc":"2.0","id":2,"method":"tools/list"}""") + .result().array("tools").map { it.jsonObject } + + assertThat(tools.map { it.text("name") }).containsExactly( + "open_heap_dumps", + "list_leaks", + "describe_object", + "chain_from_gc_root", + "ways_held", + "find_objects", + "set_verdict", + "clear_verdict", + "take_note", + "show", + "conclude" + ) + tools.forEach { tool -> + assertThat(tool.text("description")).isNotEmpty() + val schema = tool.obj("inputSchema") + assertThat(schema.obj("properties")["reason"]).describedAs(tool.text("name")).isNotNull + assertThat(schema.array("required").map { it.jsonPrimitive.content }) + .describedAs(tool.text("name")) + .contains("reason") + } + } + + @Test + fun `a notification is not answered at all`() { + assertThat(answerOrNull("""{"jsonrpc":"2.0","method":"notifications/initialized"}""")).isNull() + } + + @Test + fun `the id of a request comes back as it was sent, whatever it was`() { + val answered = answer("""{"jsonrpc":"2.0","id":"a string id","method":"ping"}""") + + assertThat(answered.text("id")).isEqualTo("a string id") + assertThat(answered.text("jsonrpc")).isEqualTo("2.0") + } + + @Test + fun `a method this server does not have says what it does have`() { + val error = answer("""{"jsonrpc":"2.0","id":3,"method":"resources/list"}""").obj("error") + + assertThat(error.text("code")).isEqualTo("-32601") + assertThat(error.text("message")).contains("resources/list").contains("tools") + } + + @Test + fun `something that is not a JSON-RPC message is answered rather than dropped`() { + val notJson = answer("this is not JSON").obj("error") + assertThat(notJson.text("code")).isEqualTo("-32700") + + val noMethod = answer("""{"jsonrpc":"2.0","id":4}""").obj("error") + assertThat(noMethod.text("code")).isEqualTo("-32600") + } + + @Test + fun `a tool answers with the same JSON as text and as structured content`() { + val result = callTool( + """{"name":"describe_object","arguments":{"object":"${hex(heapDump.holderObjectId)}",""" + + """"reason":"Reading the holder's fields."}}""" + ) + + val text = result.array("content").single().jsonObject + assertThat(text.text("type")).isEqualTo("text") + assertThat(JSON.parseToJsonElement(text.text("text")).jsonObject) + .isEqualTo(result.obj("structuredContent")) + assertThat(result.obj("structuredContent").text("className")).isEqualTo(HOLDER_CLASS_NAME) + assertThat(result["isError"]).isNull() + } + + @Test + fun `a refusal is something the model reads rather than an error the client swallows`() { + val result = callTool( + """{"name":"conclude","arguments":{"object":"${hex(heapDump.activityObjectId)}",""" + + """"rootCause":"The holder never lets go.","reason":"I know what this is."}}""" + ) + + assertThat(result.text("isError")).isEqualTo("true") + assertThat(result.array("content").single().jsonObject.text("text")) + .contains("Not concluded") + .contains(HOLDER_CLASS_NAME) + assertThat(window.notes).isEmpty() + } + + @Test + fun `a tool this server does not have is a refusal naming the ones it has`() { + val result = callTool("""{"name":"solve_the_leak","arguments":{"reason":"Trying my luck."}}""") + + assertThat(result.text("isError")).isEqualTo("true") + assertThat(result.array("content").single().jsonObject.text("text")) + .contains("solve_the_leak") + .contains("describe_object") + } + + @Test + fun `the reason an agent gave is logged before the reads it caused`() { + callTool( + """{"name":"describe_object","arguments":{"object":"${hex(heapDump.holderObjectId)}",""" + + """"reason":"Checking whether the holder is the singleton it looks like."}}""" + ) + + val called = log.indexOfFirst { it.startsWith("An agent called describe_object") } + assertThat(log[called]) + .contains("object=${hex(heapDump.holderObjectId)}") + .contains("because: Checking whether the holder is the singleton it looks like.") + assertThat(log.subList(called + 1, log.size)).contains("${hex(heapDump.holderObjectId)} for an agent") + } + + private fun callTool(params: String): JsonObject = + answer("""{"jsonrpc":"2.0","id":9,"method":"tools/call","params":$params}""").result() + + private fun answer(line: String): JsonObject = requireNotNull(answerOrNull(line)) { + "Nothing was answered to $line" + } + + private fun answerOrNull(line: String): JsonObject? = runBlocking { + session.answer(line)?.let { JSON.parseToJsonElement(it).jsonObject } + } + + private fun hex(objectId: Long) = exactHexObjectId(objectId) + + private companion object { + + const val SERVER_VERSION = "1.2.3" + + val JSON = Json + + fun JsonObject.result(): JsonObject = obj("result") + + fun JsonObject.text(name: String): String = + requireNotNull(this[name]) { "$name is not in $this" }.jsonPrimitive.content + + fun JsonObject.obj(name: String): JsonObject = + requireNotNull(this[name]) { "$name is not in $this" }.jsonObject + + fun JsonObject.array(name: String) = + requireNotNull(this[name]) { "$name is not in $this" }.jsonArray + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/RecordedLog.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/RecordedLog.kt new file mode 100644 index 0000000000..13bd2b0052 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/RecordedLog.kt @@ -0,0 +1,50 @@ +package shark.explorer.agent + +import java.util.concurrent.CopyOnWriteArrayList +import org.junit.rules.ExternalResource +import shark.SharkLog + +/** + * Everything Shark logged during one test, a line per log, read as the list of lines it is. + * + * What a session log is for here is being able to follow an investigation afterwards — the reason an agent + * gave for a call, then the reads that call cost — so the lines and the order they are in are the thing + * under test rather than a side effect of it. + * + * A rule rather than a `@Before`, because putting the logger back is the part that isn't optional: a test + * that leaves [SharkLog.logger] set breaks every test after it, whichever class those are in. A concurrent + * list because a connection is served on a thread of its own. + * + * Duplicated from the app's tests rather than shared, since a test helper is not worth a module's public + * API. + */ +// Public rather than internal because JUnit reaches a `@Rule` through a public getter, which a property of +// an internal type can't have. +class RecordedLog private constructor( + private val lines: CopyOnWriteArrayList +) : ExternalResource(), List by lines { + + constructor() : this(CopyOnWriteArrayList()) + + private var previousLogger: SharkLog.Logger? = null + + override fun before() { + previousLogger = SharkLog.logger + SharkLog.logger = object : SharkLog.Logger { + override fun d(message: String) { + lines += message + } + + override fun d( + throwable: Throwable, + message: String + ) { + lines += "$message: $throwable" + } + } + } + + override fun after() { + SharkLog.logger = previousLogger + } +} diff --git a/shark/shark-explorer/shark-explorer-app/build.gradle.kts b/shark/shark-explorer/shark-explorer-app/build.gradle.kts index 5308af4de2..cc92beb54b 100644 --- a/shark/shark-explorer/shark-explorer-app/build.gradle.kts +++ b/shark/shark-explorer/shark-explorer-app/build.gradle.kts @@ -42,6 +42,8 @@ dependencies { implementation(projects.shark.sharkExplorer.sharkExplorerCore) // Reads the bitmaps of a live process off the Android versions whose heap dumps can't carry them. implementation(projects.shark.sharkExplorer.sharkExplorerJdwp) + // What an agent asks this app's windows through, and the pipe that reaches them. See ExplorerAgents.kt. + implementation(projects.shark.sharkExplorer.sharkExplorerAgent) implementation(compose.desktop.currentOs) implementation(compose.material3) diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt new file mode 100644 index 0000000000..52976fbfd7 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt @@ -0,0 +1,170 @@ +package shark.explorer.app + +import java.io.File +import shark.SharkLog +import shark.explorer.HeapExplorer +import shark.explorer.LeakStatusOverride +import shark.explorer.LeakStatusOverrides +import shark.explorer.Place +import shark.explorer.agent.AgentHeapDump +import shark.explorer.agent.AgentHeapDumps +import shark.explorer.agent.AgentRefusal +import shark.explorer.agent.AgentServer +import shark.explorer.agent.AgentStdioBridge + +/** + * How an agent reaches the windows of this run: the app's side of `shark-explorer-agent`. + * + * The one thing worth knowing here is **why an agent is given the window and not the heap dump file**. Every + * read goes through that window's own [HeapDumpSession], so an agent's question queues on the thread the + * person at the machine is already reading with, and the verdicts and notes it writes are the ones that + * window is drawing. An agent that opened the dump itself would be answering about a heap dump nobody is + * looking at, and its conclusions would land in a file the open window would then overwrite. + */ +internal fun explorerAgentHeapDumps(windows: ExplorerWindows): AgentHeapDumps = AgentHeapDumps { + // Asked per call rather than captured, because windows come and go while an agent is connected: a tool + // naming a window that has since closed has to be an error message and not a stale answer. + windows.mapNotNull { window -> + window.openHeapDump?.let { open -> WindowAgentHeapDump(window, open) } + } +} + +/** + * Publishes this run so that agents can find it, or does nothing if it can't. See [AgentServer]. + * + * Its own socket rather than the one `DeepLinkPeers` listens on, because the two have nothing in common but + * being loopback: a link is one line answered in a millisecond, and this is a session held open for as long + * as an investigation takes. + */ +internal fun listenForAgents(windows: ExplorerWindows) = AgentServer.listen( + heapDumps = explorerAgentHeapDumps(windows), + serverVersion = SharkExplorerVersion.current, + directory = AGENT_RUNS_DIRECTORY +) + +/** + * Whether this process was started to be a pipe between an agent and another run of the app, and what to + * exit with if it was. Null for every other command line. + * + * Answered before anything else in `main` and before any logging is installed, because the app's logger + * writes to stdout and in this mode stdout is the protocol. See [AgentStdioBridge]. + */ +internal fun agentBridgeExitCode(args: Array): Int? { + if (MCP_STDIO_OPTION !in args) { + return null + } + val pid = args.firstOrNull { it.startsWith(AgentStdioBridge.PID_OPTION) } + ?.removePrefix(AgentStdioBridge.PID_OPTION) + return AgentStdioBridge.run(directory = AGENT_RUNS_DIRECTORY, pid = pid) +} + +/** + * What a window has open, for everything that isn't drawing it. + * + * The heap dump's session plus the two things an investigation writes into, gathered because they are all + * per heap dump and are all reached the same way — through the window rather than through the file. See + * [ExplorerWindow.openHeapDump]. + */ +internal class WindowHeapDump( + val session: HeapDumpSession, + val notes: HeapDumpNotes, + val leakStatuses: HeapDumpLeakStatuses +) + +/** One window's heap dump, as the agent surface sees it. */ +private class WindowAgentHeapDump( + private val window: ExplorerWindow, + private val open: WindowHeapDump +) : AgentHeapDump { + + override val windowId: String get() = window.deepLinkId + + override val heapDumpPath: String get() = open.session.heapDumpFile.absolutePath + + override suspend fun read( + description: String, + block: (HeapExplorer) -> T + ): T = open.session.read(description, block) + + override val verdicts: LeakStatusOverrides get() = open.leakStatuses.overrides + + override suspend fun setVerdict( + verdict: LeakStatusOverride, + solved: List + ) { + requireStatusesRead() + open.leakStatuses.set(verdict, solved) + } + + override suspend fun clearVerdict(objectId: Long) { + requireStatusesRead() + open.leakStatuses.clear(objectId) + } + + /** + * Appends to what has been written about [place], leaving whatever was there. + * + * **Refuses while somebody is typing in that note**, which is the one case where writing would cost + * something that exists nowhere else: a draft is unsaved text, and saving over it with the draft plus an + * agent's paragraph would put half a sentence of theirs on disk under an answer of ours. + */ + override suspend fun appendToNote( + place: Place, + text: String + ) { + val notepad = open.notes.of(place) + notepad.read() + if (!notepad.isRead) { + throw AgentRefusal( + "The notes of that place could not be read, so writing would overwrite whatever is in them: " + + (notepad.problem ?: "reading ${notepad.file} did not finish.") + ) + } + if (notepad.draft != null) { + throw AgentRefusal( + "Somebody is writing in the notes of that place right now, so there is nothing to append to yet. " + + "Say what you found in your answer instead, or try again once they have saved." + ) + } + notepad.edit() + notepad.edited(listOf(notepad.text, text).filter { it.isNotBlank() }.joinToString(PARAGRAPH_BREAK)) + notepad.save() + if (notepad.problem != null) { + throw AgentRefusal("The notes could not be saved: ${notepad.problem}") + } + } + + override fun show(place: Place) { + SharkLog.d { "An agent asked window ${window.deepLinkId} for $place" } + // The same two steps following a link takes, which is what makes an agent showing something and a + // person clicking a link land in the same place. See [ExplorerWindows.open]. + window.goToLinked(place) + window.bringToFront() + } + + /** + * Refuses until the file of statuses set by hand has been read. + * + * [HeapDumpLeakStatuses.set] declines to write before then and says so in the log, which is right for the + * button it was written for — it is disabled — and silent for an agent, which would read "no error" as + * "recorded". Saving over an unread file would delete every conclusion in it. + */ + private fun requireStatusesRead() { + if (!open.leakStatuses.isRead) { + throw AgentRefusal( + "The verdicts already recorded about this heap dump have not been read yet, so recording one now " + + "could delete them: " + (open.leakStatuses.problem ?: "reading ${open.leakStatuses.file} " + + "has not finished. Try again in a moment.") + ) + } + } +} + +/** Between what was already written about a place and what an agent has to add, which is markdown. */ +private const val PARAGRAPH_BREAK = "\n\n" + +/** Beside the runs answering links, the notes, the statuses and the logs. See [AgentServer]. */ +private val AGENT_RUNS_DIRECTORY = File(SHARK_EXPLORER_DIRECTORY, "agents") + +/** What a command line says to be a pipe rather than a window. See [AgentStdioBridge]. */ +internal const val MCP_STDIO_OPTION = "--mcp-stdio" diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt index 0cfa948432..6d73989afc 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt @@ -55,6 +55,16 @@ internal class ExplorerWindow( */ var deepLinkProblem: String? by mutableStateOf(deepLinkProblem) + /** + * The heap dump this window has open, once it is open, for everything that isn't drawing the window. + * + * Which today is one thing: an agent reaching in from outside the app. Here for the same reason + * [linkedPlaces] is — a socket thread has to find a window, and what the window has open is a + * composable's state — and set by [ExplorerApp] as the session opens and closes. Null while a heap dump is + * being opened, for a window that has none, and for one whose dump failed to open. + */ + var openHeapDump: WindowHeapDump? by mutableStateOf(null) + /** * Places a link has asked this window for and whose tabs are not open yet, oldest first. * diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt index dac2c869db..0fd2b81450 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt @@ -33,6 +33,7 @@ import java.awt.FileDialog import java.awt.Frame import java.awt.GraphicsEnvironment import java.io.File +import kotlin.system.exitProcess import kotlinx.coroutines.CancellationException import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.withContext @@ -57,6 +58,10 @@ fun main(args: Array) { if (deliveredToAnotherRun(args)) { return } + // And a run asked to be a pipe between an agent and another run opens no window and, more to the point, + // installs no logging: stdout is the protocol in that mode, and one log line in the middle of it is a + // session the agent's client reports as broken. See [AgentStdioBridge]. + agentBridgeExitCode(args)?.let { exitProcess(it) } // Launched from a terminal, so Shark's own diagnostics and any failure to open a heap dump belong on // stdout as well as in the window — and in a file, so that a session someone reports on can be read // back after it. See [installLogging]. @@ -81,11 +86,15 @@ fun main(args: Array) { DeepLinkScheme.takeUrisFromTheOs(windows) DeepLinkScheme.registerWithTheOs() DeepLinkPeers.listen(windows).use { - // Whatever no other run claimed, which for a link naming a window that has gone is an empty window - // saying so. Ours to answer for now: nobody else is going to. - DeepLinkPeers.deliver(arguments.deepLinks).forEach { windows.open(it) } - // Heap dump paths on the command line open straight away, which is how this is usually run. - explorerApplication(windows) + // Published before the first window too, so that an agent whose client started it while the heap + // dumps were still opening finds this run and waits for a dump rather than finding nothing. + listenForAgents(windows).use { + // Whatever no other run claimed, which for a link naming a window that has gone is an empty window + // saying so. Ours to answer for now: nobody else is going to. + DeepLinkPeers.deliver(arguments.deepLinks).forEach { windows.open(it) } + // Heap dump paths on the command line open straight away, which is how this is usually run. + explorerApplication(windows) + } } } } @@ -187,6 +196,9 @@ private fun explorerApplication(windows: ExplorerWindows) = application { updateNotice = updateNotice, notes = notes, leakStatuses = leakStatuses, + // What this window has open, for the agent surface: a socket thread has to be able to find it, + // and it is a composable's state. See [ExplorerWindow.openHeapDump]. + onHeapDumpOpen = { open -> window.openHeapDump = open }, deepLinkId = window.deepLinkId, // The same way a link arriving from the OS is followed, which is what makes a `shark://` link // written in a note work wherever it is read from. @@ -229,6 +241,12 @@ internal fun ExplorerApp( * default for the same reason: a test that took whoever is running it would rewrite their conclusions. */ leakStatuses: ExplorerLeakStatuses = remember { ExplorerLeakStatuses() }, + /** + * Where what this window has open is published, for everything that isn't drawing it — which today is an + * agent reaching in from outside the app. Called with the heap dump as it opens and with null as it + * closes. See [ExplorerWindow.openHeapDump]. + */ + onHeapDumpOpen: (WindowHeapDump?) -> Unit = {}, /** What a link to a place in this window names it by. See [shark.explorer.DeepLink]. */ deepLinkId: String = remember { DeepLink.newWindowId() }, /** Places a link has asked this window for, which its tabs open. See [ExplorerWindow.linkedPlaces]. */ @@ -292,8 +310,25 @@ internal fun ExplorerApp( val currentState = state // Closing the window is what ends the session: it's the only thing that takes this heap dump off // screen, since another one opens in a window of its own. + // + // And what is open is published here rather than from the effect that opens it, because this is the one + // place that also runs when it closes: a window whose dump has gone must stop being a window an agent can + // ask about, and it has to stop being one before the session is closed under it. DisposableEffect(currentState) { - onDispose { (currentState as? HeapDumpState.Open)?.session?.close() } + val open = currentState as? HeapDumpState.Open + onHeapDumpOpen( + open?.let { + WindowHeapDump( + session = it.session, + notes = notes.of(it.session.heapDumpFile), + leakStatuses = leakStatuses.of(it.session.heapDumpFile) + ) + } + ) + onDispose { + onHeapDumpOpen(null) + open?.session?.close() + } } if (takesHeapDump) { diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NodeIds.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NodeIds.kt index ce9fe96ff1..14df5256a9 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NodeIds.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NodeIds.kt @@ -26,11 +26,16 @@ fun hexObjectId(objectId: Long): String { * is read back against another dump — or the same dump on a machine that reads it the other way — would name * an object nobody chose. This is all 16 digits of it for such an id, and identical to [hexObjectId] for every * other, which is every object of every 64 bit dump below the 8 exabyte mark. + * + * Public rather than internal because everything outside this app that names an object has to spell it this + * way and read it back with [objectIdOfHex]: the files this app keeps, and the agent surface, which hands + * addresses to a caller that will send them back. Two spellings of one address across those is the bug this + * function exists to prevent. */ -internal fun exactHexObjectId(objectId: Long): String = "0x${java.lang.Long.toHexString(objectId)}" +fun exactHexObjectId(objectId: Long): String = "0x${java.lang.Long.toHexString(objectId)}" /** And back, or null for text that is no address at all. See [exactHexObjectId]. */ -internal fun objectIdOfHex(text: String): Long? { +fun objectIdOfHex(text: String): Long? { if (!text.startsWith(HEX_PREFIX)) { return null } From ab707a78853b362c06733c36271fc5dc98f5b09d Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Mon, 24 Aug 2026 11:59:31 +0200 Subject: [PATCH 02/27] Name a harness window after the task it is for MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The dock reads the file name of the bundle a process was launched from, so every window of the packaged app is a tile called "Shark Explorer" and two harness runs are indistinguishable on screen. So clone the app image, name the copy after the title, and set the two plist keys that name the menu bar — measured, all three names come from different places, which is now in the guide. The copy also has to live outside `build/compose`: another Compose task deletes that app image, and a window whose bundle went away under it dies the way a window launched from source does. `cp -c` clones, so 240 MB costs 80 ms and no disk. And two notes on what this surface should be: what MCP costs a client here (3,300 tokens of tool definitions and 1,240 of method, measured rather than guessed, against the 17,600 of GitHub's server), why a CLI and a skill are adapters over one registry rather than second implementations of the rules, and how to score whether an agent can actually solve a leak without a model doing the scoring. Co-Authored-By: Claude Opus 5 --- shark/shark-explorer/AGENTS.md | 19 ++++ shark/shark-explorer/notes/agent-eval.md | 93 +++++++++++++++++++ shark/shark-explorer/notes/agent-surface.md | 65 +++++++++++++ .../harness/start-harness.sh | 33 ++++++- 4 files changed, 207 insertions(+), 3 deletions(-) create mode 100644 shark/shark-explorer/notes/agent-eval.md create mode 100644 shark/shark-explorer/notes/agent-surface.md diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index 09a9c909f9..07eb035f99 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -307,6 +307,21 @@ bundle, because jpackage gives it a real bundle. A run from Gradle has no bundle bundle id `net.java.openjdk.java` — which is why it is called after whatever launched it until something names it. +**And a bundle has three names, from three different places**, which is why one of them being right is no +evidence about the others. Measured on a copy of the packaged app renamed and re-plisted a key at a time: + +| What a person sees | Where it comes from | +| --- | --- | +| The dock tile | The **file name** of the `.app`, and nothing else — see the next section. | +| The menu bar next to the Apple logo | `CFBundleDisplayName`, else `CFBundleName`. | +| `lsappinfo`, the app switcher | `CFBundleExecutable` — so it says `Shark_Explorer` for a renamed copy whose other two say the title. | + +So a packaged app can be renamed for a person to navigate by without being rebuilt: `cp -Rc` the app image +(a clone, 80 ms and no disk on APFS), rename the `.app`, set those two plist keys. +`shark-explorer-agent/harness/start-harness.sh` does exactly that, and **copies out of `build/compose` +first** — another Compose task deletes that app image, and a window whose bundle was deleted under it dies +the way a window launched from source does. + ## The dock only reads a bundle's file name, so `runNamed` gives it one `-Xdock:name` has not named the dock since around macOS 10.9 — [JDK-8173753][dock-bug], still open, @@ -547,6 +562,10 @@ Design decisions and findings, kept current as the work proceeds: the ones that don't are fetched off the device - `notes/dependency-injection.md` — what Dagger and Metro leave in a heap dump, why the owner rule is about the provider rather than the component, and how to dump really generated code +- `notes/agent-surface.md` — what the MCP surface costs a client in tokens, measured, and why a CLI and a + skill are adapters over the same registry rather than second implementations +- `notes/agent-eval.md` — the plan for scoring how well an agent solves a leak, with no model doing the + scoring Update these in the same change that makes them stale. They're for agents, so keep them short and skip anything derivable from the code. diff --git a/shark/shark-explorer/notes/agent-eval.md b/shark/shark-explorer/notes/agent-eval.md new file mode 100644 index 0000000000..1592a20226 --- /dev/null +++ b/shark/shark-explorer/notes/agent-eval.md @@ -0,0 +1,93 @@ +# Measuring whether an agent can solve a leak + +The plan for an eval of the agent surface. Not built yet; this is what to build and why it is shaped this way. + +## What it is for + +Every change to a tool description, a refusal or the method is a change to a prompt, and a prompt change is +not something anyone can review by reading it. [JProfiler measured +theirs](https://www.ej-technologies.com/blog/2026/07/making-the-jprofiler-mcp-server-robust-for-weaker-models/) +and found one model going from 38/55 to 55/55 scenarios and from $13.21 to $3.13 a run on the same tools with +better descriptions and harder refusals — a change nobody would have predicted from the diff. That is the +reason to have numbers rather than an opinion, and their headline finding is the one to design for: **the +weak models are where a surface is measured**, since a strong one papers over a bad description. + +## The rule: no model scores this + +An LLM judging an answer is a second unverified opinion. Everything below is decided by string comparison or +by counting, off artefacts the app already writes. + +**The answer key is the faulty reference**, `OwnerClass.field`, per heap dump. Two sources for it, both +independent of what the tools would answer: + +- **Synthetic dumps built with the `dump { }` DSL**, where the fixture *writes* the leak, so the key is + known by construction. `AgentHeapDumps.applicationHoldsActivityThroughHolder` is the first one: + `Holder.activity`, by construction. This is where the interesting variants live — see the families below. +- **The repository's real Android dumps**, whose key is written down once by hand and checked against + LeakCanary's own leak trace for the same dump. `leak_asynctask_o.hprof` is `MainActivity$2.this$0`, and + `LegacyHprofTest` already pins the same dump's leaking object and its 211,038 retained bytes, so a key that + drifts from the library's reading is a key that fails a test. + +## What one run is scored on + +| Signal | How it is read | +| --- | --- | +| Concluded at all | A `conclude` that was not refused | +| **Right reference** | Exact match of the concluded `OwnerClass.field` against the key | +| Wrong reference | Concluded, but on another step of the chain — the failure that matters most, since it is a confident wrong answer | +| Stopped short | Text answer produced with no `conclude` — the failure mode of [the shark-cli draft](https://github.com/square/leakcanary/pull/2796) | +| Verdicts against the key | A `NOT_LEAKING` on the object the key says is stuck, or the reverse | +| Rounds | Tool calls, and refusals among them | +| Cost | Wall clock, and the client's own token and dollar report where it has one | + +Rounds and refusals are the interesting secondary numbers rather than pass/fail: a change that keeps the pass +rate and halves the calls is a better surface, and a rise in refusals with the same pass rate says a refusal +message is not telling an agent what to do next. + +## Where the numbers come from + +**A machine-readable session record, one file per agent session**, written beside the human log: the client +that connected, and per call the tool, its arguments, the reason, whether it was refused, and how long the +read took. The eval reads that rather than scraping prose, and the same file is what the window's *Agent +logs* screen draws. One artefact, two readers — build it once. + +## The scenario families + +Start with two dumps to get the harness working, then grow the synthetic side, because the whole point is +cases a real dump doesn't happen to contain: + +- **Two apart** — one unexplained step between the verdicts, which is `conclude`'s refusal made real. +- **A long unknown zone** — five or six steps with nothing known, so the agent has to work inwards. +- **A decoy** — an object that reads like a leak (destroyed activity in a cache that is meant to hold it) + above the real one, where the key is the reference below. +- **Two candidates** — two references that both cross into stuck, so the answer depends on a verdict the + agent has to defend rather than on the shape of the chain. +- **A loop** — objects holding each other, where the chain's order is arbitrary and the conflict machinery + reports nothing (see `LeakStatusOverrides.isAbove`). +- **A library leak** — the fault is in the framework, and the right answer says so rather than naming app + code. + +## The runner + +``` +harness/eval/run-eval.sh --scenarios all --model --repetitions 5 +``` + +Per scenario × model × repetition: open the dump (a window, or headless once that exists), run the client +non-interactively with the same one-line prompt the harness uses today, then score from the session record. +Five repetitions because a model is not deterministic, reported as `x/5` rather than averaged. + +**One adapter per client**, each a few lines: `claude -p --output-format json` reports turns and usage, +`codex exec` and `opencode run` have their own. The prompt stays identical across clients — what is being +measured is the surface, and a prompt tuned per client measures the prompt. + +**Not in CI.** It costs money and needs the network. Run it before and after a change to the method or a +refusal, and commit the table to this file with the date and the versions, so the next change has a baseline +to beat. + +## What to do with a result + +A scenario that fails the same way across models is a bug in this surface, not in the model, and the fix is +one of the four things that JProfiler's numbers moved: a more prescriptive description, a refusal that says +what to do next, a tool that cannot be called out of order, or a piece of the method that has to be in the +tool's own description because the method was skipped. diff --git a/shark/shark-explorer/notes/agent-surface.md b/shark/shark-explorer/notes/agent-surface.md new file mode 100644 index 0000000000..85216449f0 --- /dev/null +++ b/shark/shark-explorer/notes/agent-surface.md @@ -0,0 +1,65 @@ +# The shape of the agent surface + +Why the explorer is talked to over MCP today, what that costs, and what the other shapes would buy. Numbers +measured on this branch, not estimated. + +## What MCP costs here + +Measured off `AgentTools.all` and `AgentMethod.INSTRUCTIONS`, one `tools/list` entry per tool: + +| | Characters | ≈ tokens | Paid | +| --- | --- | --- | --- | +| Eleven tool definitions | 13,116 | 3,300 | Every turn, while the server is connected | +| The method | 4,970 | 1,240 | Handshake, and again with `open_heap_dumps` | + +So the standing cost of this surface is **4 to 6 k tokens**, 2 to 3% of a 200 k window. The published +horror stories are an order of magnitude worse — GitHub's server is ~17.6 k tokens of definitions, three +servers together have been measured at 143 k — and the mitigations that shipped in 2026 (Anthropic's tool +search, code execution over MCP) are aimed at that scale. **This surface is not where a context window goes +to die**, and a per-tool cost of ~300 tokens is what buys descriptions that say when to reach for a tool. +Worth re-measuring when the tool count doubles, which the parity work will do. + +## What each shape is actually good at + +- **MCP** is the only one of the three that gets a *session*: a process already holding a parsed heap dump, + its indexes, the window a person is watching, and the verdicts set so far. Reopening `large-dump.hprof` + costs seconds and hundreds of megabytes, so a stateless call per question is not a smaller version of + this, it is a different and much slower tool. It is also the only shape a client discovers on its own. +- **A CLI** is what an agent reaches for without being told, costs nothing until it is run, and pipes into + `grep`. Two things it would buy that MCP can't: the **no window open** case, and clients that speak no + MCP. What it must not be is a second implementation — see below. +- **A skill** ([the open standard](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview), + now read by Claude, Codex, Gemini CLI, Cursor and others) is the right home for *the method*, because + progressive disclosure is exactly what the method wants: ~80 tokens of name and description at rest, the + 1,240-token body loaded only for a session that is actually investigating a heap dump. Today every session + pays for it at the handshake whether it is investigating anything or not. + +## So: one core, several adapters + +The thing worth protecting is that **the enforcement is not in the transport**. `AgentTools` is a registry of +(name, schema, handler) and every refusal is thrown from a handler, so a second adapter is a translation of +arguments in and JSON out, not a second copy of the rules: + +- `McpSession` — JSON-RPC over the socket. Exists. +- A CLI adapter — one subcommand that names a tool and its arguments, printing the answer or the refusal, and + a `--agent-help` that prints the same descriptions the schema carries so nothing has to be written twice. + Talks to a published run when there is one, and opens a heap dump itself when there isn't. +- The skill — the method as `SKILL.md`, plus how to reach either adapter. Prose, not generated, and it points + at `--agent-help` rather than listing tools that would go stale. + +What that leaves duplicated is argument parsing per adapter, which is tens of lines. What it must never +become is two places that decide whether an investigation may conclude. + +## The judgement, in one line + +Keep MCP for the window somebody is watching, add the CLI for the window that isn't open yet, and move the +method into a skill so it costs nothing until it is needed. The criticism of MCP is about surfaces ten times +this size and about servers whose tools are one HTTP call each; ours is a session against a live process, +which is the case that criticism still concedes. + +Sources worth reading before changing this: the [Milvus comparison of the three +shapes](https://milvus.io/blog/is-mcp-dead-cli-and-skills-for-ai-agents.md), Anthropic's +[skill authoring practices](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview), and +[JProfiler's account of making their MCP server survive weaker +models](https://www.ej-technologies.com/blog/2026/07/making-the-jprofiler-mcp-server-robust-for-weaker-models/), +which is the same problem as ours and is what `agent-eval.md` is about. diff --git a/shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh b/shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh index deca0ea5b6..5a654e9196 100755 --- a/shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh +++ b/shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh @@ -9,7 +9,8 @@ # # The packaged app rather than `./gradlew run`, for two reasons. It is what a person has installed, so the # command in the config is the command they would write; and a Gradle build of any kind kills a window -# launched from source, which here would be every window this harness opened. See shark/shark-explorer/AGENTS.md. +# launched from source, which here would be every window this harness opened. A copy of it, named after the +# title, for two more — see `bundle_named_after_the_title` and shark/shark-explorer/AGENTS.md. set -euo pipefail @@ -36,7 +37,9 @@ main() { echo "Building the app. jlink takes about a minute the first time." (cd "$REPO_ROOT" && ./gradlew --quiet :shark:shark-explorer:shark-explorer-app:createDistributable) - local app="$REPO_ROOT/$APP_PATH" + mkdir -p "$HARNESS_DIRECTORY" + local app + app="$(bundle_named_after_the_title)" local before before="$(published_runs)" echo "Opening $(basename "$heap_dump") in a window called \"$TITLE\"." @@ -46,7 +49,6 @@ main() { pid="$(wait_for_new_run "$before")" local bridge="$app/Contents/MacOS/Shark Explorer" - mkdir -p "$HARNESS_DIRECTORY" write_mcp_config "$bridge" "$pid" write_prompt @@ -72,6 +74,31 @@ it — then the reads that call cost. That log is the point of the exercise as m END } +# The app to launch: a copy of the packaged one, named after the title, and it prints where it put it. +# +# Two things a copy fixes. **The dock reads the file name of the bundle a process was launched from** and +# nothing else — so every window of the installed app is a tile called "Shark Explorer", and several +# harness windows at once are indistinguishable on screen. Renaming the copy names the tile; the two plist +# keys below name the menu bar, which for a real bundle comes from the plist rather than from `--title`. +# Measured, all three names — see shark/shark-explorer/AGENTS.md. +# +# **And `build/compose` is not a safe place to launch from**: another Compose task deletes the app image, +# and a window whose bundle has been deleted under it dies the way a window launched from source does. A +# copy outside the build directory survives every build after it. +# +# `cp -c` clones rather than copies, so 240 MB of jlinked runtime costs 80 ms and no disk on APFS. +bundle_named_after_the_title() { + local built="$REPO_ROOT/$APP_PATH" + local copy="$HARNESS_DIRECTORY/$TITLE.app" + rm -rf "$copy" + cp -Rc "$built" "$copy" 2>/dev/null || cp -R "$built" "$copy" + local plist="$copy/Contents/Info.plist" + /usr/libexec/PlistBuddy -c "Set :CFBundleName $TITLE" "$plist" >/dev/null + /usr/libexec/PlistBuddy -c "Add :CFBundleDisplayName string $TITLE" "$plist" >/dev/null 2>&1 || + /usr/libexec/PlistBuddy -c "Set :CFBundleDisplayName $TITLE" "$plist" >/dev/null + echo "$copy" +} + write_mcp_config() { local bridge="$1" pid="$2" # Pinned to this run rather than left to pick the most recent, because whoever is running this has other From c22ab2852db0ec2189214df72bdea936183cad35 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 07:43:30 +0200 Subject: [PATCH 03/27] Say STUCK and EXPECTED everywhere, and Shark's words nowhere An agent reported NOT_LEAKING about an object the window called Expected, which is one tool with two vocabularies: the person watching and the agent working cannot check each other if the words change at the edge of the process. So the enum is STUCK, EXPECTED and UNKNOWN now, and those are what the window shows, what the files keep and what the agent surface takes and answers with. Shark's LEAKING and NOT_LEAKING stop at the door. LeakFingerprint is the one place that maps to LeakTraceObject.LeakingStatus, because a fingerprint has to be the string LeakCanary computes. A verdict file written before this has Shark's words in it, and its rows are skipped with a line in the log saying which. That is deliberate rather than a compatibility shim: this app is an alpha and the file is three columns of text. Its header now says "verdict" too, since that is the word the window uses. Co-Authored-By: Claude Opus 5 --- docs/shark-explorer.md | 4 +- shark/shark-explorer/AGENTS.md | 21 +++-- shark/shark-explorer/notes/agent-eval.md | 2 +- shark/shark-explorer/notes/decisions.md | 2 +- .../shark-explorer-agent/AGENTS.md | 2 +- .../java/shark/explorer/agent/AgentMethod.kt | 20 ++--- .../java/shark/explorer/agent/AgentTools.kt | 34 ++++---- .../shark/explorer/agent/AgentToolsTest.kt | 20 ++--- .../shark/explorer/app/LeakStatusSection.kt | 4 +- .../java/shark/explorer/app/PathDrawing.kt | 8 +- .../java/shark/explorer/app/ViewControls.kt | 6 +- .../explorer/app/LeakStatusSectionTest.kt | 44 +++++------ .../shark/explorer/app/LeaksScreenTest.kt | 6 +- .../shark/explorer/HeapDominatorTreemap.kt | 6 +- .../java/shark/explorer/LeakFingerprint.kt | 4 +- .../main/java/shark/explorer/LeakStatus.kt | 78 ++++++++++--------- .../java/shark/explorer/LeakStatusFile.kt | 4 +- .../shark/explorer/LeakStatusOverrides.kt | 12 +-- .../java/shark/explorer/HeapLeakStatusTest.kt | 78 +++++++++---------- .../test/java/shark/explorer/HeapLeaksTest.kt | 4 +- .../java/shark/explorer/LeakStatusFileTest.kt | 8 +- .../java/shark/explorer/LeakStatusTest.kt | 30 +++---- 22 files changed, 207 insertions(+), 190 deletions(-) diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index a2092a3f6f..38e3eb9bbf 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -271,11 +271,11 @@ cannot argue with: verdict in the same file as yours. A verdict that contradicts one already recorded is refused with the list of what it disagrees with, the same way the window asks you. * **`conclude` is refused until the heap dump agrees that one reference is at fault** — one object above it - recorded as expected, the object below it recorded as stuck, and nothing unexplained in between. Reporting + recorded as `Expected`, the object below it recorded as `Stuck`, and nothing unexplained in between. Reporting a root cause before that gets this back: ``` -Not concluded. 1 step(s) between the last NOT_LEAKING object and the first LEAKING one have no verdict, so the +Not concluded. 1 step(s) between the last EXPECTED object and the first STUCK one have no verdict, so the fault is at one of them and the chain doesn't say which: 0x12e9ed60 java.util.ArrayList. Until the chain names one reference, a root cause would be a guess about which of those steps is at fault. Read the objects in the unexplained stretch with describe_object, check whether anything else holds them with ways_held, and record diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index 07eb035f99..e673fb68aa 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -70,12 +70,21 @@ numbers on the biggest dump in the repo, including how long a chain gets there. ## A verdict set by hand is an argument to every read, never state of the tree -**The window says `Verdict`, `Stuck`, `Expected` and "faulty reference"; the code says `LeakStatus`, -`LEAKING`, `NOT_LEAKING` and `suspectPath`.** That's deliberate — the identifiers match -`shark.LeakTraceObject.LeakingStatus` because they have to agree with Shark, and the words on screen -deliberately avoid "leak" on an object, since a leak is one faulty reference and calling everything under it -leaking points readers at the wrong thing. `LeakStatus.statusText` is the only place the two meet, so change -a word there and nowhere else. `notes/decisions.md` has why each word won. +**`STUCK`, `EXPECTED` and `UNKNOWN` are the only words for this, everywhere** — the enum, the window, the +`leak-statuses` files and the agent surface. Shark's `LEAKING`/`NOT_LEAKING` stops at the door: a person +watching an agent work has to be able to say the same thing about the same object as the agent, and a +vocabulary that changes at the edge of the process is one nobody can check across it. So don't reintroduce +either word, in an enum, a JSON value or a message — `LeakFingerprint` is the single place that maps to +`shark.LeakTraceObject.LeakingStatus`, and only because a fingerprint has to be the string LeakCanary +computes. `LeakStatus.statusText` is the case-only difference between the constant and a sentence. + +None of the three is built on "leak" for a reason worth keeping: a leak is one faulty reference, and calling +everything under it leaking points readers at the wrong thing. The code still says `suspectPath` where Shark +does. `notes/decisions.md` has why each word won. + +**An older `leak-statuses` file will have Shark's words in it** and its rows are skipped with a line in the +log saying which, since `LeakStatusFile` matches a status by name. That is the intended cost of having one +vocabulary; this app is an alpha and the files are three columns of text anybody can fix with `sed`. **Which reference the leak is, is decided once, over the whole path** — `faultyReferenceIndexOrNull`, called from `withLeakStatuses` — and carried on `PathReference.isFaulty` for the drawing to read. Working it out in diff --git a/shark/shark-explorer/notes/agent-eval.md b/shark/shark-explorer/notes/agent-eval.md index 1592a20226..a025c7afa7 100644 --- a/shark/shark-explorer/notes/agent-eval.md +++ b/shark/shark-explorer/notes/agent-eval.md @@ -36,7 +36,7 @@ independent of what the tools would answer: | **Right reference** | Exact match of the concluded `OwnerClass.field` against the key | | Wrong reference | Concluded, but on another step of the chain — the failure that matters most, since it is a confident wrong answer | | Stopped short | Text answer produced with no `conclude` — the failure mode of [the shark-cli draft](https://github.com/square/leakcanary/pull/2796) | -| Verdicts against the key | A `NOT_LEAKING` on the object the key says is stuck, or the reverse | +| Verdicts against the key | An `EXPECTED` on the object the key says is stuck, or the reverse | | Rounds | Tool calls, and refusals among them | | Cost | Wall clock, and the client's own token and dollar report where it has one | diff --git a/shark/shark-explorer/notes/decisions.md b/shark/shark-explorer/notes/decisions.md index 6bbd6f3d5b..5736126f01 100644 --- a/shark/shark-explorer/notes/decisions.md +++ b/shark/shark-explorer/notes/decisions.md @@ -1006,7 +1006,7 @@ who can weigh the two. The chain still says so wherever it does put one above the other, which is a reason reading `Conflicts with`. - **Flipping to the opposite status always resolves it**, which is why solving a conflict is one button. - `NOT_LEAKING` propagates upwards only and `LEAKING` downwards only, so the pair that can disagree is + `EXPECTED` propagates upwards only and `STUCK` downwards only, so the pair that can disagree is always those two, and agreeing with the new status is the same as being flipped. - **Flipped, not taken off**, so that what somebody typed is still in the file: the solved reason says which status it was, what it said, and that this is why it changed. diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index 8ff3363eef..3623b44ff8 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -34,7 +34,7 @@ client because saying no is all it does — nothing here ever calls a model. - `set_verdict` refuses a blank reason (through `LeakStatusOverride`'s own `require`) and refuses a verdict that contradicts one already recorded unless it is told to flip it. - `conclude` refuses until the heap dump agrees that **one** reference is at fault, and the refusal says which - of the three reasons it is: nothing `LEAKING`, nothing `NOT_LEAKING` above it, or *these* steps in between + of the three reasons it is: nothing `STUCK`, nothing `EXPECTED` above it, or *these* steps in between with no verdict. Same rule as `faultyReferenceIndexOrNull`, read off the chain rather than asked of it, because the three ways it answers null are three different things to do next. - Every tool takes a `reason`, and it is enforced in `AgentTool.call` rather than only asked for in the diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt index 91fe661e03..c4ec2f8880 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt @@ -39,21 +39,21 @@ internal object AgentMethod { So an investigation is a search for that single reference, and the chain from a GC root to a stuck object is where it is. Each object on the chain gets a verdict: - - NOT_LEAKING — this object is meant to be in memory right now. - - LEAKING — this object should be gone. + - EXPECTED — this object is meant to be in memory right now. + - STUCK — this object should be gone. - UNKNOWN — you don't know yet. Most objects, most of the time. - Two rules turn verdicts into an answer, and the tools apply both for you: + Those are the three words the window shows the person watching, so they are the three words to think in + and to write. Two rules turn them into an answer, and the tools apply both for you: - - Everything holding an object that is meant to be in memory is meant to be in memory too, so a - NOT_LEAKING verdict spreads upwards. - - Everything a stuck object holds is only in memory because of it, so a LEAKING verdict spreads + - Everything holding an object that is meant to be in memory is meant to be in memory too, so an + EXPECTED verdict spreads upwards. + - Everything a stuck object holds is only in memory because of it, so a STUCK verdict spreads downwards. - A chain therefore reads as three zones: NOT_LEAKING at the top, LEAKING at the bottom, UNKNOWN in - between. **The leak is the one reference that crosses from the last NOT_LEAKING object to the first - LEAKING one.** While the UNKNOWN zone is more than one reference wide, you have not found it — you have - narrowed it. + A chain therefore reads as three zones: EXPECTED at the top, STUCK at the bottom, UNKNOWN in between. + **The leak is the one reference that crosses from the last EXPECTED object to the first STUCK one.** + While the UNKNOWN zone is more than one reference wide, you have not found it — you have narrowed it. ## The order to work in diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index 40659c94bc..0a538d56b7 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -204,8 +204,8 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { private fun setVerdict() = AgentTool( name = SET_VERDICT, - description = "Records that an object is meant to be in memory (NOT_LEAKING) or should be gone " + - "(LEAKING), which is how the search narrows: a verdict spreads along every chain through that " + + description = "Records that an object is meant to be in memory (EXPECTED) or should be gone " + + "(STUCK), which is how the search narrows: a verdict spreads along every chain through that " + "object, and naming the stuck object you are investigating as `chainTo` answers with what its " + "chain says once yours is on it. The `reason` is the " + "verdict's reason and is kept with it — make it something the next reader can check, a field value " + @@ -215,13 +215,13 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { WINDOW to window(), OBJECT to objectId("The object to record a verdict about."), VERDICT to enumString( - "LEAKING for an object that should be gone, NOT_LEAKING for one that is meant to be here.", - listOf(LeakStatus.LEAKING.name, LeakStatus.NOT_LEAKING.name) + "STUCK for an object that should be gone, EXPECTED for one that is meant to be here.", + listOf(LeakStatus.STUCK.name, LeakStatus.EXPECTED.name) ), CHAIN_TO to objectId( "The stuck object you are investigating, which is what the answer reads the chain to: a verdict is " + "worth setting for what it does to that chain, and this is where you see the unexplained stretch " + - "narrow. Not the object of this verdict — one recorded as NOT_LEAKING is above the leak, so the " + + "narrow. Not the object of this verdict — one recorded as EXPECTED is above the leak, so the " + "chain ending at it has nothing stuck on it to point at." ).optional(), SOLVE_CONFLICTS to boolean( @@ -327,7 +327,7 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { name = CONCLUDE, description = "Reports the root cause of one leak, and the only way to finish an investigation. " + "**Refused unless this heap dump agrees that a single reference is at fault**: one object above it " + - "recorded as NOT_LEAKING, the object below it recorded as LEAKING, and nothing unexplained in " + + "recorded as EXPECTED, the object below it recorded as STUCK, and nothing unexplained in " + "between. If it refuses, the message says what is missing and the investigation is not over. " + "Isolating the reference is not the root cause — rootCause is how the field came to still be set, " + "which is a sequence of events rather than a line. The conclusion is written into the notes of the " + @@ -426,8 +426,8 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { val text = string(VERDICT) val status = LeakStatus.values().firstOrNull { it.name.equals(text, ignoreCase = true) } ?: throw AgentRefusal( - "\"$text\" is no verdict. It is ${LeakStatus.LEAKING.name} for an object that should be gone or " + - "${LeakStatus.NOT_LEAKING.name} for one that is meant to be here." + "\"$text\" is no verdict. It is ${LeakStatus.STUCK.name} for an object that should be gone or " + + "${LeakStatus.EXPECTED.name} for one that is meant to be here." ) if (status == LeakStatus.UNKNOWN) { throw AgentRefusal( @@ -534,20 +534,20 @@ private fun RootPath.verdictState(): ChainVerdicts { summary = "Nothing this heap dump was walked from reaches that object, so there is no chain to read." ) } - val firstStuck = steps.indexOfFirst { it.step.leakStatus == LeakStatus.LEAKING } - val lastExpected = steps.indexOfLast { it.step.leakStatus == LeakStatus.NOT_LEAKING } + val firstStuck = steps.indexOfFirst { it.step.leakStatus == LeakStatus.STUCK } + val lastExpected = steps.indexOfLast { it.step.leakStatus == LeakStatus.EXPECTED } if (firstStuck == -1) { return ChainVerdicts( faultyStep = null, - summary = "Nothing on this chain of ${steps.size} steps is ${LeakStatus.LEAKING.name}, so it points " + + summary = "Nothing on this chain of ${steps.size} steps is ${LeakStatus.STUCK.name}, so it points " + "at no reference: the rules can only name one once something below it is known not to belong." ) } if (lastExpected == -1) { return ChainVerdicts( faultyStep = null, - summary = "The chain has a ${LeakStatus.LEAKING.name} object at step ${firstStuck + 1} of " + - "${steps.size} and nothing above it is ${LeakStatus.NOT_LEAKING.name}. So whatever holds it may " + + summary = "The chain has a ${LeakStatus.STUCK.name} object at step ${firstStuck + 1} of " + + "${steps.size} and nothing above it is ${LeakStatus.EXPECTED.name}. So whatever holds it may " + "be something that should have let go too, and the fault could be further up than this chain " + "knows: find the highest object here that is meant to be in memory and record it." ) @@ -556,8 +556,8 @@ private fun RootPath.verdictState(): ChainVerdicts { val unexplained = (lastExpected + 1 until firstStuck).map { steps[it] } return ChainVerdicts( faultyStep = null, - summary = "${unexplained.size} step(s) between the last ${LeakStatus.NOT_LEAKING.name} object and " + - "the first ${LeakStatus.LEAKING.name} one have no verdict, so the fault is at one of them and the " + + summary = "${unexplained.size} step(s) between the last ${LeakStatus.EXPECTED.name} object and " + + "the first ${LeakStatus.STUCK.name} one have no verdict, so the fault is at one of them and the " + "chain doesn't say which: " + unexplained.joinToString(", ") { "${exactHexObjectId(it.step.objectId)} ${it.step.className}" } + "." @@ -567,8 +567,8 @@ private fun RootPath.verdictState(): ChainVerdicts { val reference = faulty.step.reference ?: return ChainVerdicts( faultyStep = null, - summary = "One reference crosses from ${LeakStatus.NOT_LEAKING.name} to " + - "${LeakStatus.LEAKING.name} here, but reading the object above again didn't find the field it was " + + summary = "One reference crosses from ${LeakStatus.EXPECTED.name} to " + + "${LeakStatus.STUCK.name} here, but reading the object above again didn't find the field it was " + "reached through, so there is no reference to name." ) return ChainVerdicts( diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index 974fb5d226..51fa5bfa24 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -140,7 +140,7 @@ class AgentToolsTest { fun `an object the inspectors know about comes back with their verdict and their words`() { val activity = call("describe_object", OBJECT to hex(heapDump.activityObjectId)) - assertThat(activity.text("verdict")).isEqualTo(LeakStatus.LEAKING.name) + assertThat(activity.text("verdict")).isEqualTo(LeakStatus.STUCK.name) assertThat(activity.text("verdictReason")).contains("mDestroyed") } @@ -192,7 +192,7 @@ class AgentToolsTest { val answer = call( SET_VERDICT, OBJECT to hex(heapDump.holderObjectId), - "verdict" to LeakStatus.NOT_LEAKING.name, + "verdict" to LeakStatus.EXPECTED.name, "chainTo" to hex(heapDump.activityObjectId), "reason" to "Holder.INSTANCE is a static singleton, so it is meant to be in memory." ) @@ -212,7 +212,7 @@ class AgentToolsTest { val answer = call( SET_VERDICT, OBJECT to hex(heapDump.holderObjectId), - "verdict" to LeakStatus.NOT_LEAKING.name, + "verdict" to LeakStatus.EXPECTED.name, "reason" to "Holder.INSTANCE is a static singleton, so it is meant to be in memory." ) @@ -226,12 +226,12 @@ class AgentToolsTest { call( SET_VERDICT, OBJECT to hex(heapDump.holderObjectId), - "verdict" to LeakStatus.NOT_LEAKING.name, + "verdict" to LeakStatus.EXPECTED.name, "reason" to "Holder.INSTANCE is a static singleton." ) val verdict = window.verdicts[heapDump.holderObjectId] - assertThat(verdict?.status).isEqualTo(LeakStatus.NOT_LEAKING) + assertThat(verdict?.status).isEqualTo(LeakStatus.EXPECTED) assertThat(verdict?.reason).isEqualTo("Holder.INSTANCE is a static singleton.") } @@ -286,7 +286,7 @@ class AgentToolsTest { call( SET_VERDICT, OBJECT to hex(heapDump.applicationObjectId), - "verdict" to LeakStatus.LEAKING.name, + "verdict" to LeakStatus.STUCK.name, "reason" to "This isn't the real Application, it is a copy left over from a test." ) } @@ -298,13 +298,13 @@ class AgentToolsTest { val answer = call( SET_VERDICT, OBJECT to hex(heapDump.applicationObjectId), - "verdict" to LeakStatus.LEAKING.name, + "verdict" to LeakStatus.STUCK.name, "solveConflicts" to "true", "reason" to "This isn't the real Application, it is a copy left over from a test." ) assertThat(answer.text("verdictsFlipped")).isEqualTo("1") - assertThat(window.verdicts[heapDump.holderObjectId]?.status).isEqualTo(LeakStatus.LEAKING) + assertThat(window.verdicts[heapDump.holderObjectId]?.status).isEqualTo(LeakStatus.STUCK) assertThat(window.verdicts[heapDump.holderObjectId]?.reason) .contains("Holder.INSTANCE is a static singleton") } @@ -329,7 +329,7 @@ class AgentToolsTest { val answer = call("clear_verdict", OBJECT to hex(heapDump.holderObjectId)) - assertThat(answer.text("was")).isEqualTo(LeakStatus.NOT_LEAKING.name) + assertThat(answer.text("was")).isEqualTo(LeakStatus.EXPECTED.name) assertThat(answer.text("itsReason")).contains("static singleton") assertThat(window.verdicts.isEmpty).isTrue() @@ -457,7 +457,7 @@ class AgentToolsTest { call( SET_VERDICT, OBJECT to hex(heapDump.holderObjectId), - "verdict" to LeakStatus.NOT_LEAKING.name, + "verdict" to LeakStatus.EXPECTED.name, "reason" to "Holder.INSTANCE is a static singleton, so it is meant to be in memory." ) } diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/LeakStatusSection.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/LeakStatusSection.kt index e72504f17f..3abc029d3e 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/LeakStatusSection.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/LeakStatusSection.kt @@ -398,9 +398,9 @@ internal class ObjectLeakStatus( /** A mark beside the words, so that which status this is doesn't rest on the colour alone. */ private val LeakStatus.glyph: String get() = when (this) { - LeakStatus.NOT_LEAKING -> "✓" + LeakStatus.EXPECTED -> "✓" LeakStatus.UNKNOWN -> "?" - LeakStatus.LEAKING -> "✗" + LeakStatus.STUCK -> "✗" } /** diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/PathDrawing.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/PathDrawing.kt index fc44bf6a0a..12fdf04683 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/PathDrawing.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/PathDrawing.kt @@ -699,16 +699,16 @@ private val BADGE_LETTER_COLOR = Color.White */ internal val LeakStatus.background: Color? get() = when (this) { - LeakStatus.NOT_LEAKING -> ALIVE_BACKGROUND + LeakStatus.EXPECTED -> ALIVE_BACKGROUND LeakStatus.UNKNOWN -> null - LeakStatus.LEAKING -> LEAKING_BACKGROUND + LeakStatus.STUCK -> LEAKING_BACKGROUND } internal val LeakStatus.textColor: Color get() = when (this) { - LeakStatus.NOT_LEAKING -> ALIVE_TEXT + LeakStatus.EXPECTED -> ALIVE_TEXT LeakStatus.UNKNOWN -> MUTED_TEXT - LeakStatus.LEAKING -> LEAKING_TEXT + LeakStatus.STUCK -> LEAKING_TEXT } /** Green for an object something knows is still needed. */ diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ViewControls.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ViewControls.kt index 2d89697913..ac3d86ff23 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ViewControls.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ViewControls.kt @@ -179,8 +179,8 @@ private fun StrengthLegend( swatch = LEAK_COLOR, label = when { isFindingLeaks -> FINDING_LEAKS - leakCount != null -> "$LEAKING $leakCount" - else -> LEAKING + leakCount != null -> "$STUCK $leakCount" + else -> STUCK } ) } @@ -238,7 +238,7 @@ private val REFERENCE_STRENGTHS = ReachabilityStrength.values().toList() - setOf * that: two spellings of one verdict, one of them over the map and the other beside it, would read as two * different things being shaded. */ -internal const val LEAKING = "Stuck" +internal const val STUCK = "Stuck" /** And while the pass over the heap dump that finds them runs, which is what ticking it starts. */ internal const val FINDING_LEAKS = "Stuck: looking…" diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeakStatusSectionTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeakStatusSectionTest.kt index 8747619512..daa64dbba8 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeakStatusSectionTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeakStatusSectionTest.kt @@ -66,7 +66,7 @@ class LeakStatusSectionTest { openHeapDump { it.activityObjectId } onNodeWithText(STATUS_LABEL).assertIsDisplayed() - onNode(shows(LeakStatus.LEAKING)).assertIsDisplayed() + onNode(shows(LeakStatus.STUCK)).assertIsDisplayed() // And why, because a status is a conclusion and half of them are about another object. The reason as // the panel has it, which is what the chain beside it prefixes with the status. onNodeWithText(DESTROYED_REASON).assertIsDisplayed() @@ -97,16 +97,16 @@ class LeakStatusSectionTest { openHeapDump { it.activityObjectId } changeStatus() - choose(LeakStatus.NOT_LEAKING) + choose(LeakStatus.EXPECTED) write(TYPED_REASON) set() - waitUntilAtLeastOneExists(shows(LeakStatus.NOT_LEAKING), SAVE_TIMEOUT_MILLIS) + waitUntilAtLeastOneExists(shows(LeakStatus.EXPECTED), SAVE_TIMEOUT_MILLIS) // Marked as somebody's rather than the heap dump's, which is the difference between reading the dump // and reading a conclusion about it, and with what it overruled after it. onNodeWithText("$SET_BY_HAND$TYPED_REASON. Conflicts with $DESTROYED_REASON").assertIsDisplayed() waitUntil(timeoutMillis = SAVE_TIMEOUT_MILLIS) { - statusFile().read()[heapDump.activityObjectId]?.status == LeakStatus.NOT_LEAKING + statusFile().read()[heapDump.activityObjectId]?.status == LeakStatus.EXPECTED } assertThat(statusFile().read()[heapDump.activityObjectId]!!.reason).isEqualTo(TYPED_REASON) } @@ -118,7 +118,7 @@ class LeakStatusSectionTest { openHeapDump { it.activityObjectId } changeStatus() - choose(LeakStatus.NOT_LEAKING) + choose(LeakStatus.EXPECTED) setButton().assertIsNotEnabled() write("because I read the code") @@ -130,17 +130,17 @@ class LeakStatusSectionTest { explorerUiTest { openHeapDump { it.activityObjectId } changeStatus() - choose(LeakStatus.NOT_LEAKING) + choose(LeakStatus.EXPECTED) write("this screen is deliberately kept") set() - waitUntilAtLeastOneExists(shows(LeakStatus.NOT_LEAKING), SAVE_TIMEOUT_MILLIS) + waitUntilAtLeastOneExists(shows(LeakStatus.EXPECTED), SAVE_TIMEOUT_MILLIS) // Which is the one thing only the dialog of a status already set offers. changeStatus() onNode(hasText(CLEAR_STATUS) and isButton()).performClick() // And the heap dump says what it said about the object again. - waitUntilAtLeastOneExists(shows(LeakStatus.LEAKING), SAVE_TIMEOUT_MILLIS) + waitUntilAtLeastOneExists(shows(LeakStatus.STUCK), SAVE_TIMEOUT_MILLIS) waitUntil(timeoutMillis = SAVE_TIMEOUT_MILLIS) { statusFile().read().isEmpty } } } @@ -151,18 +151,18 @@ class LeakStatusSectionTest { openHeapDump(setAlready = { holderIsLeaking() }) { it.activityObjectId } changeStatus() - choose(LeakStatus.NOT_LEAKING) + choose(LeakStatus.EXPECTED) write("this screen is deliberately kept") set() // The one it disagrees with, by name, with what it was given as its reason: whoever is about to // overrule it is the only person who can weigh the two, and only if they can read it. waitUntilAtLeastOneExists(hasText("$HOLDER_NAME $CONFLICT_ABOVE"), SAVE_TIMEOUT_MILLIS) - onNodeWithText("${LeakStatus.LEAKING.statusText}: $HOLDER_REASON").assertIsDisplayed() - onNodeWithText("$CONFLICT_BECOMES ${LeakStatus.NOT_LEAKING.statusText}") + onNodeWithText("${LeakStatus.STUCK.statusText}: $HOLDER_REASON").assertIsDisplayed() + onNodeWithText("$CONFLICT_BECOMES ${LeakStatus.EXPECTED.statusText}") .assertIsDisplayed() // And nothing written while the question is open, which is what makes undoing it free. - assertThat(statusFile().read().all.map { it.status }).containsExactly(LeakStatus.LEAKING) + assertThat(statusFile().read().all.map { it.status }).containsExactly(LeakStatus.STUCK) } } @@ -170,7 +170,7 @@ class LeakStatusSectionTest { explorerUiTest { openHeapDump(setAlready = { holderIsLeaking() }) { it.activityObjectId } changeStatus() - choose(LeakStatus.NOT_LEAKING) + choose(LeakStatus.EXPECTED) write("this screen is deliberately kept") set() waitUntilAtLeastOneExists(hasText(SOLVE_CONFLICTS), SAVE_TIMEOUT_MILLIS) @@ -181,9 +181,9 @@ class LeakStatusSectionTest { statusFile().read()[heapDump.activityObjectId] != null } val overrides = statusFile().read() - assertThat(overrides[heapDump.activityObjectId]!!.status).isEqualTo(LeakStatus.NOT_LEAKING) + assertThat(overrides[heapDump.activityObjectId]!!.status).isEqualTo(LeakStatus.EXPECTED) val flipped = overrides[heapDump.holderObjectId]!! - assertThat(flipped.status).isEqualTo(LeakStatus.NOT_LEAKING) + assertThat(flipped.status).isEqualTo(LeakStatus.EXPECTED) // Flipped rather than taken off, so that what was typed about it is still in the file. assertThat(flipped.reason).contains(HOLDER_REASON) } @@ -193,7 +193,7 @@ class LeakStatusSectionTest { explorerUiTest { openHeapDump(setAlready = { holderIsLeaking() }) { it.activityObjectId } changeStatus() - choose(LeakStatus.NOT_LEAKING) + choose(LeakStatus.EXPECTED) write("this screen is deliberately kept") set() waitUntilAtLeastOneExists(hasText(UNDO_STATUS), SAVE_TIMEOUT_MILLIS) @@ -203,7 +203,7 @@ class LeakStatusSectionTest { onNodeWithText(SOLVE_CONFLICTS).assertDoesNotExist() val overrides = statusFile().read() assertThat(overrides.all.map { it.objectId }).containsExactly(heapDump.holderObjectId) - assertThat(overrides[heapDump.holderObjectId]!!.status).isEqualTo(LeakStatus.LEAKING) + assertThat(overrides[heapDump.holderObjectId]!!.status).isEqualTo(LeakStatus.STUCK) assertThat(overrides[heapDump.activityObjectId]).isNull() } } @@ -224,7 +224,7 @@ class LeakStatusSectionTest { onNodeWithText("$FAULTY_STEP $FAULTY_REFERENCE").assertIsDisplayed() changeStatus() - choose(LeakStatus.NOT_LEAKING) + choose(LeakStatus.EXPECTED) write(TYPED_REASON) set() @@ -325,16 +325,16 @@ class LeakStatusSectionTest { /** Repeated from the section rather than shared: a glyph is one of the words the window says. */ private fun LeakStatus.glyphOf() = when (this) { - LeakStatus.NOT_LEAKING -> "✓" + LeakStatus.EXPECTED -> "✓" LeakStatus.UNKNOWN -> "?" - LeakStatus.LEAKING -> "✗" + LeakStatus.STUCK -> "✗" } /** A status set on the holder in a run before the one under test, which is the file being there. */ - private fun holderIsLeaking() = holderWasSetTo(LeakStatus.LEAKING, HOLDER_REASON) + private fun holderIsLeaking() = holderWasSetTo(LeakStatus.STUCK, HOLDER_REASON) /** And the other way: a holder that belongs in memory, with the activity below it still stuck. */ - private fun holderIsExpected() = holderWasSetTo(LeakStatus.NOT_LEAKING, HOLDER_EXPECTED_REASON) + private fun holderIsExpected() = holderWasSetTo(LeakStatus.EXPECTED, HOLDER_EXPECTED_REASON) private fun holderWasSetTo( status: LeakStatus, diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeaksScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeaksScreenTest.kt index 1409f25d43..bb5941f051 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeaksScreenTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeaksScreenTest.kt @@ -198,7 +198,7 @@ class LeaksScreenTest { // Nothing is ticked to make this happen: the leaks are looked for as the heap dump opens, and the // box says how many there are once they are found. leakToggle().assertIsOn() - waitUntilAtLeastOneExists(hasText("$LEAKING $LEAKING_OBJECT_COUNT"), OPEN_TIMEOUT_MILLIS) + waitUntilAtLeastOneExists(hasText("$STUCK $LEAKING_OBJECT_COUNT"), OPEN_TIMEOUT_MILLIS) leakToggle().performClick() @@ -235,7 +235,7 @@ class LeaksScreenTest { onAllNodesWithText(listed)[0].performClick() - val leaking = LeakStatus.LEAKING.statusText + val leaking = LeakStatus.STUCK.statusText waitUntilAtLeastOneExists(hasText("$leaking: ", substring = true), OPEN_TIMEOUT_MILLIS) assertThat(onAllNodesWithText("mDestroyed", substring = true).fetchSemanticsNodes()).isNotEmpty() } @@ -318,7 +318,7 @@ class LeaksScreenTest { /** The checkbox above the view that shades the objects that shouldn't be in memory. */ private fun ComposeUiTest.leakToggle(): SemanticsNodeInteraction = - onNode(hasText(LEAKING, substring = true) and isToggleable()) + onNode(hasText(STUCK, substring = true) and isToggleable()) /** And one of the boxes beside it that colour the map by how firmly an object is held. */ private fun ComposeUiTest.strengthToggle(): SemanticsNodeInteraction = diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDominatorTreemap.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDominatorTreemap.kt index 58f9ae9845..eefb2fe3d1 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDominatorTreemap.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDominatorTreemap.kt @@ -162,7 +162,7 @@ class HeapDominatorTreemap internal constructor( leakingIndexes.markLeaking(override.objectId, override.objectId in leakingCandidateIds) } overrides.all.forEach { override -> - leakingIndexes.markLeaking(override.objectId, override.status == LeakStatus.LEAKING) + leakingIndexes.markLeaking(override.objectId, override.status == LeakStatus.STUCK) } indexedOverrides = overrides } @@ -987,7 +987,7 @@ class HeapDominatorTreemap internal constructor( } val ids = LinkedHashSet(leakingCandidateIds) overrides.all.forEach { override -> - if (override.status == LeakStatus.LEAKING) { + if (override.status == LeakStatus.STUCK) { ids += override.objectId } else { ids -= override.objectId @@ -1077,7 +1077,7 @@ class HeapDominatorTreemap internal constructor( retainedSize = nodes[objectId]?.retainedSize ?: 0L, retainedCount = nodes[objectId]?.retainedCount ?: 0, strength = strength, - leakingReason = target?.leakStatusReason?.takeIf { target.leakStatus == LeakStatus.LEAKING }, + leakingReason = target?.leakStatusReason?.takeIf { target.leakStatus == LeakStatus.STUCK }, watcher = watcher ) val simpleClassName = leakingObject.className.substringAfterLast('.') diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakFingerprint.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakFingerprint.kt index ea00e9f5f7..cc12be1042 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakFingerprint.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakFingerprint.kt @@ -51,9 +51,9 @@ private fun PathStep.toLeakTraceObject() = LeakTraceObject( className = className, labels = inspectorLabels.toSet(), leakingStatus = when (leakStatus) { - LeakStatus.NOT_LEAKING -> LeakingStatus.NOT_LEAKING + LeakStatus.EXPECTED -> LeakingStatus.NOT_LEAKING LeakStatus.UNKNOWN -> LeakingStatus.UNKNOWN - LeakStatus.LEAKING -> LeakingStatus.LEAKING + LeakStatus.STUCK -> LeakingStatus.LEAKING }, leakingStatusReason = leakStatusReason.orEmpty(), retainedHeapByteSize = null, diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatus.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatus.kt index f0b6dc0889..7511232c4c 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatus.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatus.kt @@ -6,13 +6,24 @@ package shark.explorer * Every chain the explorer draws carries these, not only the ones that turn out to be leaks: a chain from * a GC root down to a bitmap runs through a dozen objects, and which of them are supposed to be alive is * what says where along it something went wrong. The reason for the leak is between the last - * [NOT_LEAKING] object and the first [LEAKING] one, because everything above the first is doing its job + * [EXPECTED] object and the first [STUCK] one, because everything above the first is doing its job * and everything below the last is being kept alive by it. + * + * **These are the words the window shows, the files keep and an agent reads**, deliberately the same three + * everywhere rather than Shark's `LEAKING` and `NOT_LEAKING` translated per surface. A person watching an + * agent work and the agent itself have to be able to say the same thing about the same object, and a + * vocabulary that changes at the edge of the process is one nobody can check across it. [LeakFingerprint] is + * the one place that maps to `shark.LeakTraceObject.LeakingStatus`, because a fingerprint has to be the same + * string LeakCanary computes. + * + * **And none of the three is built on "leak".** A leak is one faulty reference that should have been + * cleared, and everything under it is retained by that one mistake — so a word like `Leaking` on twenty + * objects points a reader at the twenty rather than at the one thing to fix. */ enum class LeakStatus { /** Something knows this object is still needed: a live activity, a class, a running thread. */ - NOT_LEAKING, + EXPECTED, /** Nothing knows either way, which is most of a heap dump. */ UNKNOWN, @@ -25,23 +36,20 @@ enum class LeakStatus { * garbage collector hasn't run — so this is the same verdict, said the same way, and where an object sits * on that scale is what the leaks screen is for rather than what this says. */ - LEAKING + STUCK } /** - * How a status is named where it is read: on a chain, in the reason another object gives, in the row above - * the panes. + * The same word as a sentence reads it: on a chain, in the reason another object gives, in the row above the + * panes. Only the case differs from the constant, which is what this exists for. * * In this module rather than in the window, because the reasons worked out here are sentences that name * statuses — a status set by hand says which status it was set from — and two spellings of one status * would show up in one line of one window. * - * **One word each, and neither of the two built on "leak".** A leak is one faulty reference that should - * have been cleared, and everything under it is retained by that one mistake — so a word like `Leaking` or - * `Leaked` on twenty objects points a reader at the twenty rather than at the one thing to fix. `Stuck` - * says what is true of the object without accusing it: it should be gone and something is holding it, - * which is the question worth asking. `Expected` says its being in memory is legitimate at this point in - * the app's life, which is what an inspector actually recognizes. + * `Stuck` says what is true of the object without accusing it: it should be gone and something is holding + * it, which is the question worth asking. `Expected` says its being in memory is legitimate at this point + * in the app's life, which is what an inspector actually recognizes. * * No heap analyser has a verdict like this to borrow words from — JProfiler classifies objects by * reference type and by age, YourKit by reachability scope, and both leave the judgement to the reader, @@ -51,9 +59,9 @@ enum class LeakStatus { */ val LeakStatus.statusText: String get() = when (this) { - LeakStatus.NOT_LEAKING -> "Expected" + LeakStatus.EXPECTED -> "Expected" LeakStatus.UNKNOWN -> "Unknown" - LeakStatus.LEAKING -> "Stuck" + LeakStatus.STUCK -> "Stuck" } /** What one object of a path is, and why. See [LeakStatus]. */ @@ -87,8 +95,8 @@ internal class InspectedPathObject( * leaking is not leaking either, because it is holding something that is still needed; and everything * below a leaking object is leaking, because the only thing keeping it in memory is an object that * shouldn't be there. So the inspectors have to recognize one object of a chain for the whole chain to - * read, and what's left in the middle — between the last [LeakStatus.NOT_LEAKING] and the first - * [LeakStatus.LEAKING] — is where the **faulty reference** is: the one reference that should have been + * read, and what's left in the middle — between the last [LeakStatus.EXPECTED] and the first + * [LeakStatus.STUCK] — is where the **faulty reference** is: the one reference that should have been * cleared, and the whole of what there is to fix. * * This is [shark.RealLeakTracerFactory]'s algorithm, kept in step with it deliberately: a chain here and @@ -115,44 +123,44 @@ internal fun leakStatusesOf(objects: List): List - if (status.status == LeakStatus.NOT_LEAKING) { + if (status.status == LeakStatus.EXPECTED) { lastNotLeakingIndex = index // So that the first leaking object is never above the last one that isn't: an object that is // leaking and is held by something that isn't means the leak starts below it. firstLeakingIndex = lastIndex - } else if (status.status == LeakStatus.LEAKING && firstLeakingIndex == lastIndex) { + } else if (status.status == LeakStatus.STUCK && firstLeakingIndex == lastIndex) { firstLeakingIndex = index } } for (index in 0 until lastNotLeakingIndex) { val nextNotLeakingIndex = (index + 1..lastNotLeakingIndex) - .first { statuses[it].status == LeakStatus.NOT_LEAKING } + .first { statuses[it].status == LeakStatus.EXPECTED } val nextNotLeakingName = "${objects[nextNotLeakingIndex].simpleClassName}↓" val reason = statuses[index].reason statuses[index] = LeakStatusAndReason( - status = LeakStatus.NOT_LEAKING, + status = LeakStatus.EXPECTED, reason = when (statuses[index].status) { // With a reason of its own only when a hand gave it one, which the path is then overruling: an // object someone said nothing is known about is one of the two statuses this can disagree with. LeakStatus.UNKNOWN -> "$nextNotLeakingName is expected".conflicting(reason) - LeakStatus.NOT_LEAKING -> "$nextNotLeakingName is expected and $reason" - LeakStatus.LEAKING -> "$nextNotLeakingName is expected. Conflicts with $reason" + LeakStatus.EXPECTED -> "$nextNotLeakingName is expected and $reason" + LeakStatus.STUCK -> "$nextNotLeakingName is expected. Conflicts with $reason" } ) } for (index in lastIndex downTo firstLeakingIndex + 1) { val previousLeakingIndex = (index - 1 downTo firstLeakingIndex) - .first { statuses[it].status == LeakStatus.LEAKING } + .first { statuses[it].status == LeakStatus.STUCK } val previousLeakingName = "${objects[previousLeakingIndex].simpleClassName}↑" val reason = statuses[index].reason statuses[index] = LeakStatusAndReason( - status = LeakStatus.LEAKING, + status = LeakStatus.STUCK, reason = when (statuses[index].status) { LeakStatus.UNKNOWN -> "$previousLeakingName is stuck".conflicting(reason) - LeakStatus.LEAKING -> "$previousLeakingName is stuck and $reason" + LeakStatus.STUCK -> "$previousLeakingName is stuck and $reason" // No object below the first leaking one is left not leaking: the first leaking index is reset // past every object that isn't, and the loop above turned the rest into not leaking already. - LeakStatus.NOT_LEAKING -> error( + LeakStatus.EXPECTED -> error( "${objects[index].simpleClassName} at $index is expected, below " + "${objects[previousLeakingIndex].simpleClassName} at $previousLeakingIndex, which is stuck" ) @@ -178,13 +186,13 @@ internal fun leakStatusesOf(objects: List): List.suspectReferenceIndexes(): List { - val firstStuck = indexOfFirst { it.leakStatus == LeakStatus.LEAKING } + val firstStuck = indexOfFirst { it.leakStatus == LeakStatus.STUCK } if (firstStuck == -1) { return emptyList() } // Never below the first stuck object: [leakStatusesOf] pushes that one past every object expected to be // in memory, so the stretch between the two ends is never empty and never runs backwards. - val lastExpected = indexOfLast { it.leakStatus == LeakStatus.NOT_LEAKING } + val lastExpected = indexOfLast { it.leakStatus == LeakStatus.EXPECTED } return (lastExpected + 1..firstStuck).filter { this[it].reference != null } } @@ -209,8 +217,8 @@ internal fun List.suspectReferenceIndexes(): List { * that stretch into a single reference. */ internal fun List.faultyReferenceIndexOrNull(): Int? { - val firstStuck = indexOfFirst { it.leakStatus == LeakStatus.LEAKING } - val lastExpected = indexOfLast { it.leakStatus == LeakStatus.NOT_LEAKING } + val firstStuck = indexOfFirst { it.leakStatus == LeakStatus.STUCK } + val lastExpected = indexOfLast { it.leakStatus == LeakStatus.EXPECTED } if (firstStuck == -1 || lastExpected == -1 || firstStuck != lastExpected + 1) { return null } @@ -238,12 +246,12 @@ private fun InspectedPathObject.ownStatus(leakingWins: Boolean): LeakStatusAndRe } return when { leaking != null && notLeaking != null -> if (leakingWins) { - LeakStatusAndReason(LeakStatus.LEAKING, "$leaking. Conflicts with $notLeaking") + LeakStatusAndReason(LeakStatus.STUCK, "$leaking. Conflicts with $notLeaking") } else { - LeakStatusAndReason(LeakStatus.NOT_LEAKING, "$notLeaking. Conflicts with $leaking") + LeakStatusAndReason(LeakStatus.EXPECTED, "$notLeaking. Conflicts with $leaking") } - leaking != null -> LeakStatusAndReason(LeakStatus.LEAKING, leaking) - notLeaking != null -> LeakStatusAndReason(LeakStatus.NOT_LEAKING, notLeaking) + leaking != null -> LeakStatusAndReason(LeakStatus.STUCK, leaking) + notLeaking != null -> LeakStatusAndReason(LeakStatus.EXPECTED, notLeaking) else -> LeakStatusAndReason(LeakStatus.UNKNOWN, null) } } @@ -266,8 +274,8 @@ private fun setByHandStatus( notLeaking: String? ): LeakStatusAndReason { val overruled = when (setByHand.status) { - LeakStatus.LEAKING -> notLeaking - LeakStatus.NOT_LEAKING -> leaking + LeakStatus.STUCK -> notLeaking + LeakStatus.EXPECTED -> leaking // Both of them, since saying nothing is known about an object overrules anything that claimed to know. LeakStatus.UNKNOWN -> listOfNotNull(notLeaking, leaking).joinToString(" and ").takeIf { it.isNotEmpty() } } diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatusFile.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatusFile.kt index 9cb2761a7b..706febfca4 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatusFile.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatusFile.kt @@ -107,8 +107,8 @@ class LeakStatusFile( /** What the columns are, for whoever opens this file without the app that wrote it. */ private const val HEADER = - "# Leaking statuses set by hand in Shark Explorer.\n" + - "# object\tstatus\treason, with \\n \\t \\\\ escaped" + "# Verdicts set by hand in Shark Explorer.\n" + + "# object\tverdict\treason, with \\n \\t \\\\ escaped" } } diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatusOverrides.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatusOverrides.kt index 14d1e409a0..92b426e42b 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatusOverrides.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/LeakStatusOverrides.kt @@ -131,13 +131,13 @@ fun HeapDominatorTreemap.leakStatusConflictsWith( } // A leaking object above forces everything it holds to be leaking, so it disagrees with anything else // down here. - val holdsIt = existing.status == LeakStatus.LEAKING && - override.status != LeakStatus.LEAKING && + val holdsIt = existing.status == LeakStatus.STUCK && + override.status != LeakStatus.STUCK && isAbove(aboveObjectId = existing.objectId, belowObjectId = override.objectId) // And an object below that is still needed forces everything holding it to be needed too, so it // disagrees with anything else up here. - val heldByIt = existing.status == LeakStatus.NOT_LEAKING && - override.status != LeakStatus.NOT_LEAKING && + val heldByIt = existing.status == LeakStatus.EXPECTED && + override.status != LeakStatus.EXPECTED && isAbove(aboveObjectId = override.objectId, belowObjectId = existing.objectId) if (!holdsIt && !heldByIt) { return@mapNotNull null @@ -205,8 +205,8 @@ private fun LeakStatusOverride.solvedBy( ): LeakStatusOverride = LeakStatusOverride( objectId = objectId, status = when (status) { - LeakStatus.LEAKING -> LeakStatus.NOT_LEAKING - LeakStatus.NOT_LEAKING -> LeakStatus.LEAKING + LeakStatus.STUCK -> LeakStatus.EXPECTED + LeakStatus.EXPECTED -> LeakStatus.STUCK // Nothing to flip: an object nobody claims to know about overrules nothing, so it is never one of the // statuses a new one has to be solved against. LeakStatus.UNKNOWN -> error( diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeakStatusTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeakStatusTest.kt index 1d5fe565ec..e1c659ec1e 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeakStatusTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeakStatusTest.kt @@ -4,8 +4,8 @@ import org.assertj.core.api.Assertions.assertThat import org.junit.Rule import org.junit.Test import org.junit.rules.TemporaryFolder -import shark.explorer.LeakStatus.LEAKING -import shark.explorer.LeakStatus.NOT_LEAKING +import shark.explorer.LeakStatus.STUCK +import shark.explorer.LeakStatus.EXPECTED import shark.explorer.LeakStatus.UNKNOWN /** @@ -23,7 +23,7 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val summary = explorer.tree.summarize(dump.activityObjectId) - assertThat(summary.leakStatus).isEqualTo(LEAKING) + assertThat(summary.leakStatus).isEqualTo(STUCK) assertThat(summary.leakStatusReason).contains("mDestroyed") } } @@ -34,10 +34,10 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val summary = explorer.tree.summarize( objectId = dump.activityObjectId, - overrides = overrides(dump.activityObjectId, NOT_LEAKING, "kept for one more frame on purpose") + overrides = overrides(dump.activityObjectId, EXPECTED, "kept for one more frame on purpose") ) - assertThat(summary.leakStatus).isEqualTo(NOT_LEAKING) + assertThat(summary.leakStatus).isEqualTo(EXPECTED) // What the inspector said is kept as the record of what the hand overruled. assertThat(summary.leakStatusReason) .isEqualTo("set by hand — kept for one more frame on purpose. Conflicts with Activity#mDestroyed is true") @@ -51,10 +51,10 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val summary = explorer.tree.summarize( objectId = dump.windowObjectId, - overrides = overrides(dump.activityObjectId, NOT_LEAKING, "kept for one more frame on purpose") + overrides = overrides(dump.activityObjectId, EXPECTED, "kept for one more frame on purpose") ) - assertThat(summary.leakStatus).isEqualTo(LEAKING) + assertThat(summary.leakStatus).isEqualTo(STUCK) } } @@ -64,14 +64,14 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val path = explorer.tree.rootPathTo( objectId = dump.windowObjectId, - overrides = overrides(dump.activityObjectId, NOT_LEAKING, "kept for one more frame on purpose") + overrides = overrides(dump.activityObjectId, EXPECTED, "kept for one more frame on purpose") ) val activity = path.steps.single { it.step.objectId == dump.activityObjectId }.step - assertThat(activity.leakStatus).isEqualTo(NOT_LEAKING) + assertThat(activity.leakStatus).isEqualTo(EXPECTED) assertThat(activity.leakStatusReason).contains(SET_BY_HAND) // And what a chain reads off it: the object above the activity is holding something still needed. - assertThat(path.steps.first().step.leakStatus).isEqualTo(NOT_LEAKING) + assertThat(path.steps.first().step.leakStatus).isEqualTo(EXPECTED) } } @@ -84,7 +84,7 @@ class HeapLeakStatusTest { // The holder above the destroyed activity is where the leak starts, so it is left unknown. assertThat(plain.steps.first().step.leakStatus).isEqualTo(UNKNOWN) assertThat(plain.steps.single { it.step.objectId == dump.activityObjectId }.step.leakStatus) - .isEqualTo(LEAKING) + .isEqualTo(STUCK) val read = explorer.tree.rootPathTo( objectId = dump.windowObjectId, @@ -93,7 +93,7 @@ class HeapLeakStatusTest { // The window is still leaking on its own account — an inspector recognized it — so what the activity // no longer being leaking changes is the activity, not the object the chain leads to. - assertThat(read.steps.last().step.leakStatus).isEqualTo(LEAKING) + assertThat(read.steps.last().step.leakStatus).isEqualTo(STUCK) assertThat(read.steps.single { it.step.objectId == dump.activityObjectId }.step.leakStatus) .isEqualTo(UNKNOWN) } @@ -113,7 +113,7 @@ class HeapLeakStatusTest { val path = explorer.tree.rootPathTo( objectId = dump.windowObjectId, - overrides = overrides(dump.activityObjectId, NOT_LEAKING, "kept for one more frame on purpose") + overrides = overrides(dump.activityObjectId, EXPECTED, "kept for one more frame on purpose") ) // And saying the activity belongs there is what leaves one step between the two verdicts: the window @@ -154,8 +154,8 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val conflicts = explorer.tree.leakStatusConflictsWith( - override = override(dump.windowObjectId, NOT_LEAKING, "this window is reused deliberately"), - overrides = overrides(dump.activityObjectId, LEAKING, "this screen was closed") + override = override(dump.windowObjectId, EXPECTED, "this window is reused deliberately"), + overrides = overrides(dump.activityObjectId, STUCK, "this screen was closed") ) val conflict = conflicts.single() @@ -163,7 +163,7 @@ class HeapLeakStatusTest { assertThat(conflict.objectName).contains(ACTIVITY_CLASS_NAME.substringAfterLast('.')) assertThat(conflict.isAbove).isTrue() // Solving it flips the one already set, and the reason says that this is why. - assertThat(conflict.solved.status).isEqualTo(NOT_LEAKING) + assertThat(conflict.solved.status).isEqualTo(EXPECTED) assertThat(conflict.solved.reason) .contains("below this can be \"Expected\"", "Was \"Stuck\": this screen was closed") } @@ -175,14 +175,14 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val conflicts = explorer.tree.leakStatusConflictsWith( - override = override(dump.activityObjectId, LEAKING, "this screen was closed"), - overrides = overrides(dump.windowObjectId, NOT_LEAKING, "this window is reused deliberately") + override = override(dump.activityObjectId, STUCK, "this screen was closed"), + overrides = overrides(dump.windowObjectId, EXPECTED, "this window is reused deliberately") ) val conflict = conflicts.single() assertThat(conflict.existing.objectId).isEqualTo(dump.windowObjectId) assertThat(conflict.isAbove).isFalse() - assertThat(conflict.solved.status).isEqualTo(LEAKING) + assertThat(conflict.solved.status).isEqualTo(STUCK) assertThat(conflict.solved.reason).contains("above this can be \"Stuck\"") } } @@ -192,8 +192,8 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val conflicts = explorer.tree.leakStatusConflictsWith( - override = override(dump.windowObjectId, LEAKING, "and so is the window it holds"), - overrides = overrides(dump.activityObjectId, LEAKING, "this screen was closed") + override = override(dump.windowObjectId, STUCK, "and so is the window it holds"), + overrides = overrides(dump.activityObjectId, STUCK, "this screen was closed") ) assertThat(conflicts).isEmpty() @@ -207,8 +207,8 @@ class HeapLeakStatusTest { val (one, other) = explorer.tree.findLeaks().leakingObjectIds.toList() val conflicts = explorer.tree.leakStatusConflictsWith( - override = override(one, NOT_LEAKING, "this screen is meant to be kept"), - overrides = overrides(other, LEAKING, "and that one is not") + override = override(one, EXPECTED, "this screen is meant to be kept"), + overrides = overrides(other, STUCK, "and that one is not") ) assertThat(conflicts).isEmpty() @@ -238,8 +238,8 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val conflicts = explorer.tree.leakStatusConflictsWith( - override = override(dump.taskObjectId, LEAKING, "this task should have been cancelled"), - overrides = overrides(dump.wrapperObjectId, NOT_LEAKING, "the executor is running this") + override = override(dump.taskObjectId, STUCK, "this task should have been cancelled"), + overrides = overrides(dump.wrapperObjectId, EXPECTED, "the executor is running this") ) assertThat(conflicts).isEmpty() @@ -252,8 +252,8 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val conflicts = explorer.tree.leakStatusConflictsWith( - override = override(dump.wrapperObjectId, NOT_LEAKING, "the executor is running this"), - overrides = overrides(dump.taskObjectId, LEAKING, "this task should have been cancelled") + override = override(dump.wrapperObjectId, EXPECTED, "the executor is running this"), + overrides = overrides(dump.taskObjectId, STUCK, "this task should have been cancelled") ) assertThat(conflicts).isEmpty() @@ -272,8 +272,8 @@ class HeapLeakStatusTest { objectId = dump.activityObjectId, overrides = LeakStatusOverrides.of( listOf( - override(dump.wrapperObjectId, NOT_LEAKING, "the executor is running this"), - override(dump.taskObjectId, LEAKING, "this task should have been cancelled") + override(dump.wrapperObjectId, EXPECTED, "the executor is running this"), + override(dump.taskObjectId, STUCK, "this task should have been cancelled") ) ) ) @@ -292,8 +292,8 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val conflicts = explorer.tree.leakStatusConflictsWith( - override = override(dump.activityObjectId, NOT_LEAKING, "this screen is coming back"), - overrides = overrides(dump.taskObjectId, LEAKING, "this task should have been cancelled") + override = override(dump.activityObjectId, EXPECTED, "this screen is coming back"), + overrides = overrides(dump.taskObjectId, STUCK, "this task should have been cancelled") ) val conflict = conflicts.single() @@ -307,8 +307,8 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val conflicts = explorer.tree.leakStatusConflictsWith( - override = override(dump.activityObjectId, NOT_LEAKING, "changed my mind"), - overrides = overrides(dump.activityObjectId, LEAKING, "this screen was closed") + override = override(dump.activityObjectId, EXPECTED, "changed my mind"), + overrides = overrides(dump.activityObjectId, STUCK, "this screen was closed") ) assertThat(conflicts).isEmpty() @@ -321,7 +321,7 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val conflicts = explorer.tree.leakStatusConflictsWith( - override = override(dump.windowObjectId, LEAKING, "the window is the problem"), + override = override(dump.windowObjectId, STUCK, "the window is the problem"), overrides = overrides(dump.activityObjectId, UNKNOWN, "no idea about this activity") ) @@ -339,7 +339,7 @@ class HeapLeakStatusTest { assertThat(explorer.tree.findLeaks().leakingObjectIds).containsExactly(dump.activityObjectId) val leaks = explorer.tree.findLeaks( - overrides(holderObjectId, LEAKING, "this cache is never emptied") + overrides(holderObjectId, STUCK, "this cache is never emptied") ) // The activity is now only in memory because of an object that shouldn't be there, which is the one @@ -353,7 +353,7 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val leaks = explorer.tree.findLeaks( - overrides(dump.activityObjectId, NOT_LEAKING, "kept for one more frame on purpose") + overrides(dump.activityObjectId, EXPECTED, "kept for one more frame on purpose") ) // What the activity was holding is still a destroyed window, and nothing above it is a leak any @@ -367,7 +367,7 @@ class HeapLeakStatusTest { val dump = testFolder.nestedLeaksHeapDump() HeapExplorer.open(dump.file).use { explorer -> - explorer.tree.findLeaks(overrides(dump.activityObjectId, NOT_LEAKING, "kept on purpose")) + explorer.tree.findLeaks(overrides(dump.activityObjectId, EXPECTED, "kept on purpose")) assertThat(explorer.tree.findLeaks().leakingObjectIds).containsExactly(dump.activityObjectId) } @@ -385,7 +385,7 @@ class HeapLeakStatusTest { val path = explorer.tree.rootPathTo( objectId = dump.windowObjectId, - overrides = overrides(dump.activityObjectId, NOT_LEAKING, "this screen is coming back") + overrides = overrides(dump.activityObjectId, EXPECTED, "this screen is coming back") ) // And takes the short way through it once there is nothing to avoid, which is the chain and the @@ -405,7 +405,7 @@ class HeapLeakStatusTest { HeapExplorer.open(dump.file).use { explorer -> val path = explorer.tree.rootPathTo( objectId = dump.activityObjectId, - overrides = overrides(dump.taskObjectId, LEAKING, "this task should have been cancelled") + overrides = overrides(dump.taskObjectId, STUCK, "this task should have been cancelled") ) // The frame holding the activity is two steps from the thread where the executor's field is four, and diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeaksTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeaksTest.kt index b09d99ea25..fa5b8f8773 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeaksTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeaksTest.kt @@ -270,7 +270,7 @@ class HeapLeaksTest { val leaking = tree.findLeaks().objectsOf(APPLICATION).first() val steps = tree.rootPathTo(leaking.objectId).steps.map { it.step } - assertThat(steps.last().leakStatus).isEqualTo(LeakStatus.LEAKING) + assertThat(steps.last().leakStatus).isEqualTo(LeakStatus.STUCK) assertThat(steps.last().leakStatusReason).contains("mDestroyed") } } @@ -330,7 +330,7 @@ class HeapLeaksTest { // it and says it is leaking, which is that chain being read as a leak trace. val steps = tree.rootPathTo(heapDump.windowObjectId).steps.map { it.step } assertThat(steps.map { it.objectId }).contains(heapDump.activityObjectId) - assertThat(steps.last().leakStatus).isEqualTo(LeakStatus.LEAKING) + assertThat(steps.last().leakStatus).isEqualTo(LeakStatus.STUCK) } } diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/LeakStatusFileTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/LeakStatusFileTest.kt index 5d998b6ef9..2c47a44e10 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/LeakStatusFileTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/LeakStatusFileTest.kt @@ -19,10 +19,10 @@ class LeakStatusFileTest { @Test fun `what was set is read back`() { val file = statusFile("heap.hprof") - file.write(LeakStatusOverrides.of(listOf(override(HOLDER_ID, LeakStatus.LEAKING, "the screen is gone")))) + file.write(LeakStatusOverrides.of(listOf(override(HOLDER_ID, LeakStatus.STUCK, "the screen is gone")))) val read = file.read()[HOLDER_ID]!! - assertThat(read.status).isEqualTo(LeakStatus.LEAKING) + assertThat(read.status).isEqualTo(LeakStatus.STUCK) assertThat(read.reason).isEqualTo("the screen is gone") } @@ -103,7 +103,7 @@ class LeakStatusFileTest { @Test fun `a status with no reason is not a status`() { assertThatIllegalArgumentException().isThrownBy { - LeakStatusOverride(objectId = HOLDER_ID, status = LeakStatus.LEAKING, reason = " ") + LeakStatusOverride(objectId = HOLDER_ID, status = LeakStatus.STUCK, reason = " ") }.withMessageContaining("no reason") } @@ -119,7 +119,7 @@ class LeakStatusFileTest { private fun override( objectId: Long, - status: LeakStatus = LeakStatus.LEAKING, + status: LeakStatus = LeakStatus.STUCK, reason: String = "because I read the code" ) = LeakStatusOverride(objectId = objectId, status = status, reason = reason) diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/LeakStatusTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/LeakStatusTest.kt index d2a32a8a97..dc53b68fc4 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/LeakStatusTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/LeakStatusTest.kt @@ -2,8 +2,8 @@ package shark.explorer import org.assertj.core.api.Assertions.assertThat import org.junit.Test -import shark.explorer.LeakStatus.LEAKING -import shark.explorer.LeakStatus.NOT_LEAKING +import shark.explorer.LeakStatus.STUCK +import shark.explorer.LeakStatus.EXPECTED import shark.explorer.LeakStatus.UNKNOWN class LeakStatusTest { @@ -21,7 +21,7 @@ class LeakStatusTest { ) assertThat(statuses.map { it.status }) - .containsExactly(NOT_LEAKING, NOT_LEAKING, NOT_LEAKING, UNKNOWN) + .containsExactly(EXPECTED, EXPECTED, EXPECTED, UNKNOWN) // Named after the object that decided it, and which way along the path it is. assertThat(statuses[0].reason).isEqualTo("Activity↓ is expected") assertThat(statuses[2].reason).isEqualTo("Activity#mDestroyed is false") @@ -32,7 +32,7 @@ class LeakStatusTest { listOf(unknown("Holder"), leaking("Activity"), unknown("View"), unknown("Payload")) ) - assertThat(statuses.map { it.status }).containsExactly(UNKNOWN, LEAKING, LEAKING, LEAKING) + assertThat(statuses.map { it.status }).containsExactly(UNKNOWN, STUCK, STUCK, STUCK) assertThat(statuses[2].reason).isEqualTo("Activity↑ is stuck") } @@ -41,7 +41,7 @@ class LeakStatusTest { listOf(notLeaking("Thread"), unknown("Holder"), unknown("Cache"), leaking("Activity")) ) - assertThat(statuses.map { it.status }).containsExactly(NOT_LEAKING, UNKNOWN, UNKNOWN, LEAKING) + assertThat(statuses.map { it.status }).containsExactly(EXPECTED, UNKNOWN, UNKNOWN, STUCK) } @Test fun `the object a path ends at is not made to be leaking`() { @@ -55,7 +55,7 @@ class LeakStatusTest { @Test fun `an object both sides recognize is taken to be still needed`() { val statuses = leakStatusesOf(listOf(conflicted("Activity"), unknown("Payload"))) - assertThat(statuses.first().status).isEqualTo(NOT_LEAKING) + assertThat(statuses.first().status).isEqualTo(EXPECTED) assertThat(statuses.first().reason) .isEqualTo("Activity#mDestroyed is false. Conflicts with Activity#mDestroyed is true") } @@ -63,7 +63,7 @@ class LeakStatusTest { @Test fun `except at the end of the path, where it is the object being asked about`() { val statuses = leakStatusesOf(listOf(unknown("Holder"), conflicted("Activity"))) - assertThat(statuses.last().status).isEqualTo(LEAKING) + assertThat(statuses.last().status).isEqualTo(STUCK) assertThat(statuses.last().reason) .isEqualTo("Activity#mDestroyed is true. Conflicts with Activity#mDestroyed is false") } @@ -75,7 +75,7 @@ class LeakStatusTest { listOf(leaking("Cache", reason = "Cache#entry is stale"), notLeaking("Activity"), unknown("Payload")) ) - assertThat(statuses.map { it.status }).containsExactly(NOT_LEAKING, NOT_LEAKING, UNKNOWN) + assertThat(statuses.map { it.status }).containsExactly(EXPECTED, EXPECTED, UNKNOWN) assertThat(statuses.first().reason) .isEqualTo("Activity↓ is expected. Conflicts with Cache#entry is stale") } @@ -86,18 +86,18 @@ class LeakStatusTest { @Test fun `a status set by hand wins over the inspector that disagreed with it`() { val statuses = leakStatusesOf( - listOf(unknown("Holder"), setByHand(leaking("Activity"), NOT_LEAKING, "kept for one more frame")) + listOf(unknown("Holder"), setByHand(leaking("Activity"), EXPECTED, "kept for one more frame")) ) - assertThat(statuses.last().status).isEqualTo(NOT_LEAKING) + assertThat(statuses.last().status).isEqualTo(EXPECTED) assertThat(statuses.last().reason) .isEqualTo("set by hand — kept for one more frame. Conflicts with Activity#mDestroyed is true") } @Test fun `a status set by hand on an object nothing knew about has only its own reason`() { - val statuses = leakStatusesOf(listOf(setByHand(unknown("Cache"), LEAKING, "this cache is unbounded"))) + val statuses = leakStatusesOf(listOf(setByHand(unknown("Cache"), STUCK, "this cache is unbounded"))) - assertThat(statuses.single().status).isEqualTo(LEAKING) + assertThat(statuses.single().status).isEqualTo(STUCK) assertThat(statuses.single().reason).isEqualTo("set by hand — this cache is unbounded") } @@ -117,12 +117,12 @@ class LeakStatusTest { val statuses = leakStatusesOf( listOf( unknown("Thread"), - setByHand(unknown("Presenter"), LEAKING, "this screen was closed"), + setByHand(unknown("Presenter"), STUCK, "this screen was closed"), unknown("View") ) ) - assertThat(statuses.map { it.status }).containsExactly(UNKNOWN, LEAKING, LEAKING) + assertThat(statuses.map { it.status }).containsExactly(UNKNOWN, STUCK, STUCK) assertThat(statuses.last().reason).isEqualTo("Presenter↑ is stuck") } @@ -133,7 +133,7 @@ class LeakStatusTest { listOf(leaking("Activity"), setByHand(unknown("View"), UNKNOWN, "no idea what this is")) ) - assertThat(statuses.last().status).isEqualTo(LEAKING) + assertThat(statuses.last().status).isEqualTo(STUCK) assertThat(statuses.last().reason) .isEqualTo("Activity↑ is stuck. Conflicts with set by hand — no idea what this is") } From 7682d4b48d05456a6f2ce3d14fada8c8b176f4a9 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 08:25:05 +0200 Subject: [PATCH 04/27] Show what an agent did, in the window it did it in MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An investigation an agent ran and one a person ran are the same investigation: the same tree, the same verdicts, the same notes. So what an agent did belongs in the window, and in words — the *Agent logs* screen is one row per agent that has connected, and opening one is every call it made with the sentence it gave for making it. A row leads where the call went, so reading what an agent did and going to look at it are one move. A call about another heap dump is drawn and leads nowhere: an address is an address of one dump. The rows are a file, `~/.shark-explorer/agents/sessions/*.jsonl`, one per connection and the newest hundred kept. One artefact with two readers, which is why the reading half sits beside the writing half: this screen, and the eval in notes/agent-eval.md, which now has a session record to score a run from rather than prose to scrape. The description of a call is worked out before the call is answered, so a refused call still records what it was asking about — the refusals are the half of a session worth reading afterwards, and one nobody can follow up on is a dead end on the screen. Which is also why the refusal for a contradicted verdict is prose now rather than a JSON array of the verdicts it disagrees with: a refusal is the one answer on this surface a person reads, and that one was three verdicts and their reasons as raw protocol on the screen that exists to not show it. --- docs/shark-explorer-changelog.md | 6 + docs/shark-explorer.md | 36 +- shark/shark-explorer/AGENTS.md | 3 + shark/shark-explorer/notes/agent-eval.md | 6 + .../shark-explorer-agent/AGENTS.md | 28 ++ .../java/shark/explorer/agent/AgentJson.kt | 19 - .../java/shark/explorer/agent/AgentServer.kt | 32 +- .../shark/explorer/agent/AgentSessionFile.kt | 415 ++++++++++++++++++ .../java/shark/explorer/agent/AgentTools.kt | 117 ++++- .../java/shark/explorer/agent/McpSession.kt | 81 +++- .../explorer/agent/AgentSessionFileTest.kt | 149 +++++++ .../shark/explorer/agent/AgentToolsTest.kt | 6 + .../shark/explorer/agent/McpSessionTest.kt | 53 ++- .../shark/explorer/app/AgentLogsScreen.kt | 226 ++++++++++ .../java/shark/explorer/app/ExplorerAgents.kt | 12 + .../shark/explorer/app/HeapDumpExplorer.kt | 66 +++ .../src/main/java/shark/explorer/app/Main.kt | 9 + .../shark/explorer/app/AgentLogsScreenTest.kt | 180 ++++++++ .../src/main/java/shark/explorer/DeepLink.kt | 22 +- .../src/main/java/shark/explorer/NoteFile.kt | 3 + .../src/main/java/shark/explorer/Place.kt | 30 ++ .../test/java/shark/explorer/DeepLinkTest.kt | 24 +- 22 files changed, 1474 insertions(+), 49 deletions(-) create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt create mode 100644 shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt create mode 100644 shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index 5713eeb1c5..eccef4bec6 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -45,6 +45,12 @@ uses, without the one for a newly recognized library leak: does, and reporting a root cause is refused until the chain names one faulty reference. Point any MCP client at the installed app with `--mcp-stdio`. See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). +* ✨ **Agent logs**: every agent that has connected to the app is a row on a screen of its own, and opening + one is everything that agent did — what each call did, which object it did it to, and the sentence it gave + for making it, with the refusals in red. A row leads where the call went, so reading what an agent did and + going to look at it are one move. Kept in `~/.shark-explorer/agents/sessions`, one file per session and the + newest hundred kept, so a session outlives the window it was worked in. + See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). * ✨ **The chain marks the faulty reference**: the one step going from an `Expected` object straight to a `Stuck` one reads `Holder.activity · faulty reference`, which is the leak itself rather than one of the objects it left behind, and the same reference the **Leaks** screen names that leak after. A chain whose diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 38e3eb9bbf..c7fc9adbaa 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -287,8 +287,40 @@ draws by, held to before an answer can be written down: an agent that has narrow unexplained steps cannot report a root cause, however sure it is, and what it gets instead is the three objects to go and read. -**What it did is in the log**, in `~/.shark-explorer/logs`, one line per call with the reason it gave followed -by the reads that call cost: +**What it did is on the *Agent logs* screen**, one row per agent that has connected to the app. Open a row +and there is every call that agent made, in order and in words — what it did, which object it did it to, and +the sentence it gave for doing it: + +``` +08:23:11 Listed the leaks + because: Starting from what the heap dump already says shouldn't be here. +08:23:18 Read the chain to 0x12d368b8 + because: This is the one App leak: a MainActivity the app watched and whose mDestroyed is + true. Reading the chain from a GC root. +08:23:27 Described 0x12d00c30 + because: The FutureTask in the middle of the chain: checking whether it is really running. +08:23:34 Looked for every way of holding 0x12d368b8 + because: Checking whether anything else holds the activity, or only this one chain. +``` + +**A row leads where the call went**: click *Read the chain to 0x12d368b8* and the window opens that object, +so reading what an agent did and going to look at it are one move. + +**A refused call is a row too**, in red, under the reason the agent gave for making it — and those are the +half of a session worth reading, since a refusal is where the method sent an agent back to the heap dump +rather than on to an answer: + +``` +08:23:45 Concluded about 0x12d00c30 + because: […] + Refused: Not concluded. Nothing on this chain of 4 steps is STUCK, so it points at no + reference: the rules can only name one once something below it is known not to belong. […] +``` + +A session is kept in `~/.shark-explorer/agents/sessions`, one file per agent that connected and the newest +hundred kept, a line of JSON per call — so it outlives the window and can be read by something other than +this app. **And the reads each call cost are in the run's log**, in `~/.shark-explorer/logs`, where the reason +it gave is followed by the work it caused: ``` 18:19:48.035 [shark-explorer-agents] An agent called chain_from_gc_root(object=0x12d368b8, window=zvphq4r3) diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index e673fb68aa..e6027525e9 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -548,6 +548,9 @@ numbers belong in `notes/bitmaps.md`. default keeps notes in `~/.shark-explorer/notes`, so a test taking it writes into the notes of whoever is running it. A window that never opens one only lists that directory to see which tabs to mark, which is why the tests that don't touch notes need no directory of their own. +- **A UI test that opens the *Agent logs* screen must pass its own `agentSessions`.** The default reads + `~/.shark-explorer/agents/sessions`, so a test taking it draws whatever the person running it last handed + to an agent — and asserts on rows it didn't write. `AgentLogsScreenTest` builds the sessions it expects. - **A UI test must pass a `DeviceHeapDumps` built on a fake `Adb`.** `ExplorerApp`'s default shells out to the machine's `adb`, so a test that takes it has whatever device is plugged in to answer for — and the window can dump the heap of a real process. `FakeAdb` matches command prefixes, because the remote dump diff --git a/shark/shark-explorer/notes/agent-eval.md b/shark/shark-explorer/notes/agent-eval.md index a025c7afa7..96ad4bf1c7 100644 --- a/shark/shark-explorer/notes/agent-eval.md +++ b/shark/shark-explorer/notes/agent-eval.md @@ -51,6 +51,12 @@ that connected, and per call the tool, its arguments, the reason, whether it was read took. The eval reads that rather than scraping prose, and the same file is what the window's *Agent logs* screen draws. One artefact, two readers — build it once. +**That part exists**: `AgentSessionFile` writes `~/.shark-explorer/agents/sessions/*.jsonl` and reads it back, +so a scorer is a walk over `AgentSessionFile.sessionsIn(…)`. Every signal in the table above is on it except +the two the client reports — an answer written with no `conclude` at all, and the cost — which come from the +adapter's own output. Which session belongs to which scenario run is the file the connection was given: +`AgentServer` logs it as the connection opens, and one run of the eval is one connection. + ## The scenario families Start with two dumps to get the harness working, then grow the synthetic side, because the whole point is diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index 3623b44ff8..e60f97b704 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -18,6 +18,7 @@ being talked to by a program that is not this app. | `AgentJson.kt` | The explorer's model as JSON. | | `AgentTool.kt` | One tool, its arguments read strictly, and `AgentRefusal`. | | `McpSession.kt` | JSON-RPC, one message per line. | +| `AgentSessionFile.kt` | One session on disk, both ways: what a call is written as, and what it reads back as. | | `AgentServer.kt` | The loopback socket a run publishes, and the file that says where. | | `AgentStdioBridge.kt` | `--mcp-stdio`: the pipe an MCP client launches. | | `harness/start-harness.sh` | Opens a window and prints the command that throws an agent at it. | @@ -49,6 +50,33 @@ The `reason` is traceability and not a quality gate. Asking a model to explain i and [the research says it can make it worse](https://arxiv.org/abs/2504.09664); what it buys is a session log someone can follow afterwards instead of a conclusion they have to trust. +## A session file has two readers, and neither is in this process + +`AgentSessionFile` writes `~/.shark-explorer/agents/sessions/agent--.jsonl`, one file per +connection, a JSON object per line, the newest `KEEP_SESSION_COUNT` kept. What reads it back is **the window's +*Agent logs* screen and the eval in `notes/agent-eval.md`** — one artefact, two readers, which is why the +reading half lives here beside the writing half and is tested with it. A field written and never read back is +a row of that screen saying nothing. + +Three things follow that reading the code won't tell you. + +**A call is described before it is answered, not after.** `McpSession.callTool` asks `AgentTools.target` what +the call is about and only then invokes the handler, so **a refused call still records its place** and its row +is still clickable. That is deliberate: the refusals are the half of a session worth reading afterwards, and +a refusal nobody can follow up on is a dead end on the screen. `target` derives the place from the argument +*names* rather than from a second list of tool names — one exception, `list_leaks`, which takes no argument +saying where it is. + +**The verbs are here rather than in the app.** `verbOfTool` is beside the tool names, so that a screen never +spells them itself and drift is one list rather than two. `AgentSessionFileTest` asserts every tool in the +registry has one; a tool added without a verb reads as its own name, which is the protocol showing through on +the screen that exists to not show it. + +**Writing a session never throws and never blocks the answer.** A bad line is skipped on read with a +`SharkLog.d` saying which, a file whose header is missing falls back to the id in its name, and a truncated +last line — an app killed mid-write — keeps every call before it. An agent's call must not fail because the +record of it couldn't be written. + ## In `--mcp-stdio` mode, stdout is the protocol `main` answers `agentBridgeExitCode` **before `installLogging()`**, because that logger writes to stdout and diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt index bf87763234..15afd8ed1d 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt @@ -14,7 +14,6 @@ import shark.explorer.HeapLeaks import shark.explorer.HeapObjectSummary import shark.explorer.HeapSizes import shark.explorer.IndependentPaths -import shark.explorer.LeakStatusConflict import shark.explorer.LeakStatusOverrides import shark.explorer.ObjectDominator import shark.explorer.ObjectList @@ -228,24 +227,6 @@ internal object AgentJson { } } - /** What setting a verdict would disagree with, and what solving it would set those objects to. */ - fun conflicts(conflicts: List): JsonArray = buildJsonArray { - conflicts.forEach { conflict -> - addJsonObject { - put("object", exactHexObjectId(conflict.existing.objectId)) - put("objectName", conflict.objectName) - put("verdict", conflict.existing.status.name) - put("reason", conflict.existing.reason) - // Which way round the two objects are, since that is what makes the disagreement one at all. - put("holdsTheObjectBeingSet", conflict.isAbove) - putJsonObject("wouldBecome") { - put("verdict", conflict.solved.status.name) - put("reason", conflict.solved.reason) - } - } - } - } - private fun rootPathStep(step: RootPathStep): JsonObject = buildJsonObject { pathStepInto(step.step) // Every path from a GC root goes through each of an object's dominators, so a marked step is one that diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt index 7205a2580d..8f673ce06c 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt @@ -59,7 +59,11 @@ object AgentServer { return try { write(file, serverSocket.localPort, token) SharkLog.d { "Answering agents on port ${serverSocket.localPort}, published as $file" } - val thread = Thread({ accept(serverSocket, token, heapDumps, serverVersion) }, THREAD_NAME).apply { + val sessions = sessionsDirectory(directory) + val thread = Thread( + { accept(serverSocket, token, heapDumps, serverVersion, sessions) }, + THREAD_NAME + ).apply { isDaemon = true start() } @@ -76,6 +80,14 @@ object AgentServer { } } + /** + * Where the sessions of the agents that connect to a run published in [directory] are written down. + * + * One function rather than a path spelled in two modules: the app reads these to draw them, and a screen + * looking in the wrong directory is a screen that says no agent has ever been here. See [AgentSessionFile]. + */ + fun sessionsDirectory(directory: File): File = File(directory, SESSIONS_DIRECTORY) + /** Every run of this app an agent could connect to, newest first, stale files cleared out on the way. */ internal fun publishedRuns(directory: File): List { val files = directory.listFiles { file -> file.name.endsWith(RUN_SUFFIX) }.orEmpty() @@ -122,7 +134,8 @@ object AgentServer { serverSocket: ServerSocket, token: String, heapDumps: AgentHeapDumps, - serverVersion: String + serverVersion: String, + sessions: File ) { while (!serverSocket.isClosed) { try { @@ -130,7 +143,7 @@ object AgentServer { // A thread per agent, because a session is held open for as long as the agent is working and two // agents on one heap dump is a thing to allow rather than to serialise: what they would queue on // is the heap dump's own thread, which is where reads belong anyway. - Thread({ serve(socket, token, heapDumps, serverVersion) }, THREAD_NAME).apply { + Thread({ serve(socket, token, heapDumps, serverVersion, sessions) }, THREAD_NAME).apply { isDaemon = true start() } @@ -155,7 +168,8 @@ object AgentServer { socket: Socket, token: String, heapDumps: AgentHeapDumps, - serverVersion: String + serverVersion: String, + sessions: File ) { socket.use { val reader = BufferedReader(InputStreamReader(socket.getInputStream(), Charsets.UTF_8)) @@ -169,7 +183,12 @@ object AgentServer { return } writer.println(ACCEPTED) - val session = McpSession(AgentTools(heapDumps), serverVersion) + // A file per accepted connection, so that two agents at one heap dump are two sessions to read rather + // than one file with both of their reasoning in it. Named before the handshake, since a client that + // connects and says nothing is itself worth a line on that screen. + val sessionFile = AgentSessionFile.starting(sessions, serverVersion) + SharkLog.d { "An agent's session is being written to ${sessionFile.file}" } + val session = McpSession(AgentTools(heapDumps), serverVersion, sessionFile) while (true) { val line = reader.readLine() ?: break if (line.isBlank()) { @@ -208,6 +227,9 @@ object AgentServer { /** Beside the runs answering links, the notes and the logs, which is everything else this app keeps. */ internal const val RUN_SUFFIX = ".agent" + + /** Under the directory the runs publish themselves in, since a session is a run being talked to. */ + private const val SESSIONS_DIRECTORY = "sessions" internal const val ACCEPTED = "OK" internal const val DECLINED = "NO" private const val PORT_PROPERTY = "port" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt new file mode 100644 index 0000000000..39331b6b26 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt @@ -0,0 +1,415 @@ +package shark.explorer.agent + +import java.io.File +import java.security.SecureRandom +import java.time.Instant +import java.time.ZoneId +import java.time.format.DateTimeFormatter +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.put +import kotlinx.serialization.json.putJsonObject +import shark.SharkLog +import shark.explorer.DeepLink +import shark.explorer.Place + +/** + * What one agent did, written down as it does it: the client that connected, then a line per call. + * + * **One artefact, two readers.** The window draws this as the *Agent logs* screen, so that the person at the + * machine can follow an investigation they didn't watch — every call as a verb, the address it was about, and + * the reason the agent gave for making it. And the eval reads the same file to score a run, because the + * numbers worth having about this surface are counts of what happened: whether it concluded, on which + * reference, in how many calls, how many of them refused. See `notes/agent-eval.md`. + * + * Which is why it is machine readable and appended to rather than the run log reworded: the run log is prose + * about everything the app did, and this is the one agent's calls with nothing else in the file. + * + * JSON, one object per line, flushed per line, because the session worth reading is often the one that ended + * by the agent giving up or the app being killed. A header line naming the session, then a line per call. + * Beside the notes and the verdicts under `~/.shark-explorer`, since it is the same kind of thing: what + * somebody concluded about a heap dump, kept where the next reader will find it. + */ +class AgentSessionFile private constructor( + /** The file itself, shown in the window so that a session can be read without this app. */ + val file: File, + /** What this session is called, in the window and in the file. See [newSessionId]. */ + val sessionId: String, + private val startedAt: Instant, + private val serverVersion: String +) { + + private var isHeaderWritten = false + + /** + * Says who connected, which is the handshake and therefore the first thing to land in the file. + * + * Written here rather than at construction because the client only says its name in `initialize`, and a + * session file that exists before anyone has spoken would be a session nobody had. + */ + fun opened( + client: String?, + protocolVersion: String? + ) { + writeHeader(client, protocolVersion) + } + + /** + * Adds one call, whether it was answered or refused. + * + * A refusal is a line like any other and not an error to leave out: what a session is read for is what the + * agent tried, and the refusals are where it was made to go back and look again. + */ + fun called(call: AgentSessionCall) { + // A client that calls a tool before the handshake is one this has not met, and its calls still belong in + // a file that says which session they were. + writeHeader(client = null, protocolVersion = null) + append(call.asJson()) + } + + private fun writeHeader( + client: String?, + protocolVersion: String? + ) { + if (isHeaderWritten) { + return + } + isHeaderWritten = true + append( + buildJsonObject { + put(SESSION_KEY, sessionId) + put(STARTED_AT_KEY, startedAt.toString()) + client?.let { put(CLIENT_KEY, it) } + protocolVersion?.let { put(PROTOCOL_KEY, it) } + put(SERVER_KEY, serverVersion) + } + ) + } + + /** + * One line on disk, or a line in the run log saying why not. + * + * Never thrown: an agent's session must not end because the disk it was being written to filled up, and + * this is the app's side of the connection, so the run log is where anyone would look. + */ + private fun append(line: JsonObject) { + try { + file.appendText(JSON.encodeToString(JsonElement.serializer(), line) + "\n") + } catch (throwable: Throwable) { + SharkLog.d(throwable) { "Could not write to the agent session log $file" } + } + } + + companion object { + + /** + * Starts a session's file in [directory], and deletes all but the newest [keepSessionCount] of them. + * + * The same housekeeping `shark.explorer.SessionLog` does for the run logs, for the same reason: a + * directory that grows for ever is one nobody opens. A session is a few kilobytes, so this keeps more of + * them than there are runs — the question "what did that agent do last week" is one people ask. + */ + fun starting( + directory: File, + serverVersion: String, + startedAt: Instant = Instant.now(), + sessionId: String = newSessionId(), + keepSessionCount: Int = KEEP_SESSION_COUNT + ): AgentSessionFile { + directory.mkdirs() + val name = FILE_NAME_PREFIX + FILE_NAME_TIME.format(startedAt) + "-$sessionId$FILE_NAME_SUFFIX" + deleteOlderSessions(directory, keepSessionCount - 1) + return AgentSessionFile(File(directory, name), sessionId, startedAt, serverVersion) + } + + /** + * Every session written in [directory], newest first, with the calls of each in the order they were + * made. + * + * A line that can't be read is skipped and says so in the run log, like the file of verdicts beside it: + * this is evidence, and one truncated line — a session whose app was killed mid-write — must not be a + * session that reads as empty. + */ + fun sessionsIn(directory: File): List { + val files = directory.listFiles { file: File -> file.name.isSessionFile() }.orEmpty() + // Named after the time the session started, so newest first is a sort by name. + return files.sortedByDescending { it.name }.map { file -> sessionIn(file) } + } + + private fun sessionIn(file: File): AgentSession { + val lines = try { + file.readLines() + } catch (throwable: Throwable) { + SharkLog.d(throwable) { "Could not read the agent session log $file" } + emptyList() + } + var header: JsonObject? = null + val calls = mutableListOf() + lines.forEachIndexed { index, line -> + val read = line.asJsonOrNull(file, index + 1) + when { + read == null -> Unit + read[TOOL_KEY] != null -> read.asCallOrNull(file, index + 1)?.let { calls += it } + read[SESSION_KEY] != null -> header = header ?: read + else -> SharkLog.d { "Skipping line ${index + 1} of $file: it is neither a session nor a call" } + } + } + return AgentSession( + // From the file name for a session whose header never landed, so that a row still has a name to be + // opened by: the id is in the name, which is what makes that recoverable. + sessionId = header?.text(SESSION_KEY) ?: file.name.sessionIdOfName(), + startedAt = header?.instant(STARTED_AT_KEY) ?: calls.firstOrNull()?.at, + client = header?.text(CLIENT_KEY), + serverVersion = header?.text(SERVER_KEY), + file = file, + calls = calls + ) + } + + /** + * Eight hexadecimal characters, from [SecureRandom] like the token beside it. + * + * Random rather than counted, for the reason `DeepLink.newWindowId` is: ids handed out in order repeat + * across runs of the app, and a session log named the same as one from yesterday is two investigations + * that read as one. + */ + fun newSessionId(): String { + val bytes = ByteArray(SESSION_ID_BYTES) + SecureRandom().nextBytes(bytes) + return bytes.joinToString("") { "%02x".format(it) } + } + + private fun deleteOlderSessions( + directory: File, + keepCount: Int + ) { + val files = directory.listFiles { file: File -> file.name.isSessionFile() }.orEmpty() + files.sortedBy { it.name }.dropLast(keepCount).forEach { older -> + if (!older.delete()) { + SharkLog.d { "Could not delete the log of an older agent session, $older" } + } + } + } + + private fun AgentSessionCall.asJson(): JsonObject = buildJsonObject { + put(AT_KEY, at.toString()) + put(TOOL_KEY, tool) + reason?.let { put(REASON_KEY, it) } + windowId?.let { put(WINDOW_KEY, it) } + heapDumpPath?.let { put(HEAP_DUMP_KEY, it) } + // As the link the window hands out for that place, which is the whole of what a row has to be + // clickable: the place to go to, and a line the agent's human can paste anywhere. See [DeepLink]. + link()?.let { put(LINK_KEY, it) } + refusal?.let { put(REFUSAL_KEY, it) } + put(MILLIS_KEY, millis) + if (arguments.isNotEmpty()) { + putJsonObject(ARGUMENTS_KEY) { + arguments.forEach { (name, value) -> put(name, value) } + } + } + } + + private fun JsonObject.asCallOrNull( + file: File, + lineNumber: Int + ): AgentSessionCall? { + val tool = text(TOOL_KEY) + val at = instant(AT_KEY) + if (tool == null || at == null) { + SharkLog.d { "Skipping line $lineNumber of $file: it says no tool, or no time it was called" } + return null + } + val link = text(LINK_KEY) + return AgentSessionCall( + at = at, + tool = tool, + reason = text(REASON_KEY), + windowId = text(WINDOW_KEY), + heapDumpPath = text(HEAP_DUMP_KEY), + place = link?.let { placeOfLinkOrNull(it, file, lineNumber) }, + arguments = this[ARGUMENTS_KEY]?.asStringMap().orEmpty(), + refusal = text(REFUSAL_KEY), + millis = text(MILLIS_KEY)?.toLongOrNull() ?: 0L + ) + } + + private fun placeOfLinkOrNull( + link: String, + file: File, + lineNumber: Int + ): Place? = try { + DeepLink.parse(link).place + } catch (noSuchPlace: IllegalArgumentException) { + // A link written by a build that spelled a place differently, which is a row that leads nowhere + // rather than a session that fails to open. + SharkLog.d(noSuchPlace) { "Line $lineNumber of $file links to no place of a heap dump" } + null + } + + private fun String.asJsonOrNull( + file: File, + lineNumber: Int + ): JsonObject? { + if (isBlank()) { + return null + } + return try { + JSON.parseToJsonElement(this).jsonObject + } catch (notJson: Exception) { + // Which is what the last line of a session whose app was killed mid-write looks like. + SharkLog.d(notJson) { "Skipping line $lineNumber of $file: it is not one JSON object" } + null + } + } + + private fun JsonElement.asStringMap(): Map = + (this as? JsonObject)?.mapValues { (_, value) -> + (value as? JsonPrimitive)?.content ?: value.toString() + }.orEmpty() + + private fun JsonObject.text(name: String): String? = (this[name] as? JsonPrimitive)?.content + + private fun JsonObject.instant(name: String): Instant? = text(name)?.let { text -> + try { + Instant.parse(text) + } catch (notATime: Exception) { + SharkLog.d(notATime) { "\"$text\" is no time an agent session was written at" } + null + } + } + + private fun String.isSessionFile(): Boolean = + startsWith(FILE_NAME_PREFIX) && endsWith(FILE_NAME_SUFFIX) + + private fun String.sessionIdOfName(): String = + removeSuffix(FILE_NAME_SUFFIX).substringAfterLast('-') + + private val JSON = Json { ignoreUnknownKeys = true } + + /** + * How many sessions are kept. More than the run logs beside them, because these are a few kilobytes + * each and because "what did that agent do" is asked about an investigation rather than about a run. + */ + const val KEEP_SESSION_COUNT = 100 + + private const val SESSION_ID_BYTES = 4 + + private const val FILE_NAME_PREFIX = "agent-" + private const val FILE_NAME_SUFFIX = ".jsonl" + + /** Sorts oldest first as text, which is what makes ordering these a sort by name. */ + private val FILE_NAME_TIME: DateTimeFormatter = + DateTimeFormatter.ofPattern("yyyy-MM-dd_HH-mm-ss_SSS").withZone(ZoneId.systemDefault()) + + private const val SESSION_KEY = "agentSession" + private const val STARTED_AT_KEY = "startedAt" + private const val CLIENT_KEY = "client" + private const val PROTOCOL_KEY = "protocol" + private const val SERVER_KEY = "sharkExplorer" + + private const val AT_KEY = "at" + private const val TOOL_KEY = "tool" + private const val REASON_KEY = "reason" + private const val WINDOW_KEY = "window" + private const val HEAP_DUMP_KEY = "heapDump" + private const val LINK_KEY = "link" + private const val REFUSAL_KEY = "refused" + private const val MILLIS_KEY = "millis" + private const val ARGUMENTS_KEY = "arguments" + } +} + +/** One agent's session, read back off disk. See [AgentSessionFile]. */ +class AgentSession( + val sessionId: String, + /** When the client connected, or when it first called something for a session with no header. */ + val startedAt: Instant?, + /** What the client called itself in the handshake, and null for one that didn't say. */ + val client: String?, + /** Which build of the app answered it. */ + val serverVersion: String?, + val file: File, + val calls: List +) { + + /** How many of the calls were refused, which is the one number a list of sessions is worth showing. */ + val refusedCount: Int get() = calls.count { it.refusal != null } +} + +/** + * One call an agent made, and what it did. + * + * The place is what makes a row of the *Agent logs* screen clickable: it is where the window goes when the + * row is clicked, so that reading what an agent did and going to look at it are the same move. Null for a + * call about no place of a heap dump — the first one of every session is, since asking which dumps are open + * is asking about the app rather than about a dump. + */ +class AgentSessionCall( + val at: Instant, + val tool: String, + /** Why the agent said it was making the call, and null for one refused for not saying. */ + val reason: String?, + val windowId: String?, + val heapDumpPath: String?, + val place: Place?, + /** The rest of the arguments, by name, with `reason` and `window` left out: they have fields of their own. */ + val arguments: Map, + /** Why the call was refused, and null for one that was answered. See [AgentRefusal]. */ + val refusal: String?, + /** How long the app took to answer, which is mostly how long the heap dump read took. */ + val millis: Long +) { + + /** The link to [place] in the window the call was made against, for a call that was about one. */ + fun link(): String? { + val place = place ?: return null + val windowId = windowId ?: return null + return DeepLink(windowId, place).toUri() + } +} + +/** + * What the call did, as a couple of words. + * + * Here rather than in the window that draws it because this is where the tool names are: a screen spelling + * them itself would be a second list of them to keep in step. Every tool has one, which `AgentSessionFileTest` + * is what keeps true — a tool added without a verb reads as its own name, which is the raw protocol showing + * through on a screen that exists to not show it. + */ +val AgentSessionCall.verb: String get() = verbOfTool(tool, arguments) ?: tool.replace('_', ' ') + +/** + * What the call was about, in the words the window uses for it: an address, a class name, a place. + * + * Null for a call whose subject is the whole heap dump or the app itself, where the verb says all of it. + */ +val AgentSessionCall.subject: String? + get() = arguments[SUBJECT_OBJECT] ?: arguments[SUBJECT_PLACE] ?: arguments[SUBJECT_CLASS_NAME] + +/** Null for a tool this build has no verb for, which is what a test asserts never happens. */ +internal fun verbOfTool( + tool: String, + arguments: Map +): String? = when (tool) { + "open_heap_dumps" -> "Asked which heap dumps are open" + "list_leaks" -> "Listed the leaks" + "describe_object" -> "Described" + "chain_from_gc_root" -> "Read the chain to" + "ways_held" -> "Looked for every way of holding" + "find_objects" -> "Searched for" + "set_verdict" -> "Recorded ${arguments[SUBJECT_VERDICT] ?: "a verdict"} on" + "clear_verdict" -> "Took the verdict off" + "take_note" -> "Wrote a note on" + "show" -> "Showed" + "conclude" -> "Concluded about" + else -> null +} + +private const val SUBJECT_OBJECT = "object" +private const val SUBJECT_PLACE = "place" +private const val SUBJECT_CLASS_NAME = "className" +private const val SUBJECT_VERDICT = "verdict" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index 0a538d56b7..fc0d2ccf09 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -1,5 +1,6 @@ package shark.explorer.agent +import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.add import kotlinx.serialization.json.addJsonObject import kotlinx.serialization.json.buildJsonObject @@ -8,6 +9,7 @@ import kotlinx.serialization.json.putJsonArray import shark.explorer.HeapDominatorTreemap import shark.explorer.HeapObjectKind import shark.explorer.LeakStatus +import shark.explorer.LeakStatusConflict import shark.explorer.LeakStatusOverride import shark.explorer.ObjectListFilter import shark.explorer.Place @@ -87,7 +89,7 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { } private fun listLeaks() = AgentTool( - name = "list_leaks", + name = LIST_LEAKS, description = "What this heap dump says shouldn't be in memory, gathered into the leaks those objects " + "are instances of. The heap dump's own answer and the place to start: objects the app itself handed " + "to LeakCanary and said it was done with are the strongest evidence a dump carries. Sections marked " + @@ -244,9 +246,10 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { "Not set: $status on ${exactHexObjectId(objectId)} contradicts ${conflicts.size} verdict(s) " + "already recorded about this heap dump. Everything a stuck object holds is stuck, and " + "everything holding an object that is meant to be here is meant to be here, so these cannot " + - "all be read off one chain. Either your verdict is wrong, or theirs is. The conflicts are " + - "${AgentJson.conflicts(conflicts)}. Call $SET_VERDICT again with $SOLVE_CONFLICTS true to keep " + - "yours and flip those, and say in your reason why." + "all be read off one chain. Either your verdict is wrong, or theirs is:\n" + + conflicts.joinToString("\n") { it.asSentence() } + + "\nCall $SET_VERDICT again with $SOLVE_CONFLICTS true to keep yours and flip those, and say in " + + "your reason why." ) } dump.setVerdict(override, conflicts.map { it.solved }) @@ -394,20 +397,51 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { explorer.tree.rootPathTo(objectId, verdicts) } + /** + * Which window and which place a call was about, for the log of the session it was made in. + * + * Read off the arguments rather than out of the handler, so that a call that was refused is recorded + * pointing at whatever it was asking about — which is most of what makes a refusal worth reading + * afterwards. Nothing here refuses: this is a description of a call, and a call with an argument this + * can't make sense of is one the handler is about to refuse with a message of its own. + */ + fun target( + name: String, + arguments: JsonObject + ): AgentTarget { + val read = AgentArguments(name, arguments) + val dump = read.orNull { resolvedDump(optionalString(WINDOW)) } + val place = read.orNull { placeOrNull(name) } + return AgentTarget( + windowId = dump?.windowId, + heapDumpPath = dump?.heapDumpPath, + place = place + ) + } + + /** + * Which place of the heap dump a call is about, from what it was given rather than from which tool it is. + * + * By argument name, so that a tool added here is described by this without being listed in it: everything + * about an object takes `object`, everything about a place takes `place`, and the search takes a class + * name. The one tool whose subject is in neither is the list of leaks, which takes nothing at all. + */ + private fun AgentArguments.placeOrNull(name: String): Place? = when { + optionalString(PLACE) != null -> place() + optionalString(OBJECT) != null -> Place.Object(objectId(OBJECT)) + optionalString(CLASS_NAME) != null -> Place.Objects(ObjectListFilter(query = string(CLASS_NAME))) + name == LIST_LEAKS -> Place.Leaks() + else -> null + } + /** Which heap dump a call is about, or a refusal naming the ones that are open. */ private fun AgentArguments.heapDump(): AgentHeapDump { - val open = heapDumps.openHeapDumps() val windowId = optionalString(WINDOW) - // One open dump needs no naming, which is most sessions. Two of them always do: the same file open twice - // is how two readings of it are compared, so guessing would be answering about the wrong one. - val asked = if (windowId == null) { - open.singleOrNull() - } else { - open.firstOrNull { it.windowId == windowId } - } + val asked = resolvedDump(windowId) if (asked != null) { return asked } + val open = heapDumps.openHeapDumps() val windows = open.joinToString(", ") { "${it.windowId} (${it.heapDumpPath})" } throw AgentRefusal( when { @@ -422,6 +456,34 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { ) } + /** + * The window a call names, and null for one that names none of the open ones. + * + * One open dump needs no naming, which is most sessions. Two of them always do: the same file open twice is + * how two readings of it are compared, so guessing would be answering about the wrong one. + */ + private fun resolvedDump(windowId: String?): AgentHeapDump? { + val open = heapDumps.openHeapDumps() + return if (windowId == null) { + open.singleOrNull() + } else { + open.firstOrNull { it.windowId == windowId } + } + } + + /** + * Whatever [block] reads, or null if the arguments wouldn't answer it. + * + * Only for [target], and that is the whole of why it exists: describing a call must not refuse one. The + * handler reads the same arguments a moment later and refuses with a message written for the agent, which + * is where a bad address belongs. + */ + private fun AgentArguments.orNull(block: AgentArguments.() -> T?): T? = try { + block() + } catch (refused: AgentRefusal) { + null + } + private fun AgentArguments.verdict(): LeakStatus { val text = string(VERDICT) val status = LeakStatus.values().firstOrNull { it.name.equals(text, ignoreCase = true) } @@ -473,6 +535,9 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { const val SET_VERDICT = "set_verdict" const val CONCLUDE = "conclude" + /** Named because [placeOrNull] is the one description of a call that has to know which tool it is. */ + const val LIST_LEAKS = "list_leaks" + const val WINDOW = "window" const val OBJECT = "object" const val FROM = "from" @@ -514,6 +579,18 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { } } +/** + * What a call was about: which window, which heap dump, and which place of it. + * + * Only for the session log, which is the one reader that needs this without needing the answer: a row of the + * *Agent logs* screen is a verb, a subject and somewhere to go when it is clicked. See [AgentSessionCall]. + */ +internal class AgentTarget( + val windowId: String?, + val heapDumpPath: String?, + val place: Place? +) + /** * What the verdicts on a chain add up to: whether one reference is at fault, and what to say when none is. * @@ -603,6 +680,22 @@ private fun conclusionNote( appendLine("_Concluded by an agent: ${reason}_") } +/** + * One verdict a new one disagrees with, as a line of the refusal that says so. + * + * A sentence rather than the JSON this used to be. Not for the model's sake — it reads either — but because + * a refusal is the one answer on this surface that is also read by a person: it is what the window's *Agent + * logs* screen draws under the call it refused, and a JSON array of three verdicts with their reasons in it + * is the raw protocol on a screen that exists to not show it. + * + * Which way round the two objects are is in it, since that is what makes the disagreement one at all. + */ +private fun LeakStatusConflict.asSentence(): String { + val side = if (isAbove) "which holds it" else "which it holds" + return "- ${exactHexObjectId(existing.objectId)} $objectName, $side, is ${existing.status}: " + + "${existing.reason} Keeping yours makes it ${solved.status}." +} + /** * Refuses an id that is no single object of the heap dump, which is three different mistakes. * diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt index f54f6ec5dc..235859869c 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt @@ -1,5 +1,6 @@ package shark.explorer.agent +import java.time.Instant import kotlinx.serialization.json.Json import kotlinx.serialization.json.JsonElement import kotlinx.serialization.json.JsonNull @@ -31,7 +32,13 @@ import shark.SharkLog internal class McpSession( private val tools: AgentTools, /** Which build of the app is answering, for a client that logs what it connected to. */ - private val serverVersion: String + private val serverVersion: String, + /** + * Where this session is written down, which is what the window's *Agent logs* screen draws and what the + * eval scores. One per connection, so that two agents at one heap dump are two files. See + * [AgentSessionFile]. + */ + private val sessionFile: AgentSessionFile ) { /** @@ -109,9 +116,13 @@ internal class McpSession( */ private fun initialize(params: JsonObject): JsonObject { val clientVersion = (params["protocolVersion"] as? JsonPrimitive)?.content - val clientName = (params["clientInfo"] as? JsonObject) - ?.let { (it["name"] as? JsonPrimitive)?.content } + val clientInfo = params["clientInfo"] as? JsonObject + val clientName = listOfNotNull( + (clientInfo?.get("name") as? JsonPrimitive)?.content, + (clientInfo?.get("version") as? JsonPrimitive)?.content + ).joinToString(" ").takeIf { it.isNotEmpty() } SharkLog.d { "An agent connected: ${clientName ?: "a client that did not say who it is"}" } + sessionFile.opened(client = clientName, protocolVersion = clientVersion) return buildJsonObject { put("protocolVersion", clientVersion ?: FALLBACK_PROTOCOL_VERSION) putJsonObject("capabilities") { @@ -139,16 +150,55 @@ internal class McpSession( // One line per call, before the reads it causes, so that a session log reads as what the agent was // trying to learn and then what that cost. See [AgentTools]. SharkLog.d { "An agent called $name${arguments.logLine()}" } + // What the call is about, read before it is made rather than after: a refused call is recorded pointing + // at whatever it was asking about, which is most of what makes a refusal worth reading afterwards. + val target = tools.target(name, arguments) + val at = Instant.now() + val startedAt = System.nanoTime() return try { - toolResult(tool.call(arguments)) + val result = toolResult(tool.call(arguments)) + record(name, arguments, target, refusal = null, at = at, startedAt = startedAt) + result } catch (refused: AgentRefusal) { // A refusal is an answer to the agent and not a failure of the server, so it comes back as a tool // result the model reads rather than as a JSON-RPC error the client may swallow. SharkLog.d { "Refused $name: ${refused.message}" } + record(name, arguments, target, refusal = refused.message, at = at, startedAt = startedAt) toolError(refused.message) } } + /** + * Writes the call down, answered or refused. + * + * Here rather than in [AgentTool] because this is the one place that has both halves of a call: what was + * asked, and what came back. Every call, in the order they were made, is what turns a session into + * something a person can follow — and the reason for each is the agent's own sentence rather than a + * paraphrase of it. + */ + private fun record( + name: String, + arguments: JsonObject, + target: AgentTarget, + refusal: String?, + at: Instant, + startedAt: Long + ) { + sessionFile.called( + AgentSessionCall( + at = at, + tool = name, + reason = (arguments[REASON_ARGUMENT] as? JsonPrimitive)?.content, + windowId = target.windowId, + heapDumpPath = target.heapDumpPath, + place = target.place, + arguments = arguments.recorded(), + refusal = refusal, + millis = (System.nanoTime() - startedAt) / NANOS_PER_MILLI + ) + ) + } + private fun toolResult(result: JsonObject): JsonObject = buildJsonObject { putJsonArray("content") { addJsonObject { @@ -226,6 +276,25 @@ internal class McpSession( const val METHOD_NOT_FOUND = -32601 const val INTERNAL_ERROR = -32603 + /** What the agent said it was after, which every tool takes. See [AgentTool]. */ + const val REASON_ARGUMENT = "reason" + + /** Which window a call names, which the session log keeps as a field of its own. */ + const val WINDOW_ARGUMENT = "window" + + const val NANOS_PER_MILLI = 1_000_000L + + /** + * The rest of the arguments, as text, for the row a session log keeps. + * + * Without the two that have fields of their own, and never as the JSON that arrived: what this is read + * back for is a screen that says what an agent did in words, so a value here is one the window can put + * beside a verb. + */ + fun JsonObject.recorded(): Map = + filterKeys { it != REASON_ARGUMENT && it != WINDOW_ARGUMENT } + .mapValues { (_, value) -> (value as? JsonPrimitive)?.content ?: value.toString() } + /** * The arguments of a call on one line of the log, with the agent's `reason` first. * @@ -236,8 +305,8 @@ internal class McpSession( if (isEmpty()) { return "" } - val reason = (this["reason"] as? JsonPrimitive)?.content - val rest = entries.filter { it.key != "reason" } + val reason = (this[REASON_ARGUMENT] as? JsonPrimitive)?.content + val rest = entries.filter { it.key != REASON_ARGUMENT } .joinToString(", ") { (key, value) -> "$key=${(value as? JsonPrimitive)?.content ?: value}" } return listOfNotNull( rest.takeIf { it.isNotEmpty() }?.let { "($it)" }, diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt new file mode 100644 index 0000000000..eaf267a07a --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt @@ -0,0 +1,149 @@ +package shark.explorer.agent + +import java.io.File +import java.time.Instant +import org.assertj.core.api.Assertions.assertThat +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import shark.explorer.Place + +/** + * The file an agent's session is written to, read back. + * + * Both halves are tested here rather than only the writing, because this file has two readers that are never + * in the same process as the writer: the window drawing the *Agent logs* screen, and the eval scoring a run. + * A field that is written and never read back is a row of that screen that says nothing. + */ +class AgentSessionFileTest { + + @get:Rule + val temporaryFolder = TemporaryFolder() + + @get:Rule + val log = RecordedLog() + + private val directory: File get() = File(temporaryFolder.root, "sessions") + + @Test + fun `a session is read back as it was written`() { + val file = AgentSessionFile.starting(directory, SERVER_VERSION, startedAt = STARTED_AT) + file.opened(client = "claude-code 9.9.9", protocolVersion = "2025-06-18") + file.called( + call( + tool = "describe_object", + reason = "Reading the holder's fields.", + place = Place.Object(OBJECT_ID), + arguments = mapOf("object" to "0x12d368b8") + ) + ) + + val session = AgentSessionFile.sessionsIn(directory).single() + assertThat(session.sessionId).isEqualTo(file.sessionId) + assertThat(session.startedAt).isEqualTo(STARTED_AT) + assertThat(session.client).isEqualTo("claude-code 9.9.9") + assertThat(session.serverVersion).isEqualTo(SERVER_VERSION) + val call = session.calls.single() + assertThat(call.tool).isEqualTo("describe_object") + assertThat(call.reason).isEqualTo("Reading the holder's fields.") + assertThat(call.place).isEqualTo(Place.Object(OBJECT_ID)) + assertThat(call.windowId).isEqualTo(WINDOW_ID) + assertThat(call.heapDumpPath).isEqualTo("/dumps/leak.hprof") + assertThat(call.arguments).containsEntry("object", "0x12d368b8") + assertThat(call.millis).isEqualTo(12L) + } + + @Test + fun `a call that was refused says so, and still says what it was about`() { + val file = AgentSessionFile.starting(directory, SERVER_VERSION) + file.called(call(tool = "conclude", place = Place.Object(OBJECT_ID), refusal = "Not concluded. 3 steps")) + + val call = AgentSessionFile.sessionsIn(directory).single().calls.single() + assertThat(call.refusal).isEqualTo("Not concluded. 3 steps") + assertThat(call.place).isEqualTo(Place.Object(OBJECT_ID)) + } + + @Test + fun `a session whose last line was cut off keeps the calls before it`() { + val file = AgentSessionFile.starting(directory, SERVER_VERSION) + file.opened(client = "a client", protocolVersion = null) + file.called(call(tool = "list_leaks", place = Place.Leaks())) + // Which is what a session whose app was killed mid-write looks like on disk. + file.file.appendText("""{"at":"2026-08-25T18:19:48.0""") + + val session = AgentSessionFile.sessionsIn(directory).single() + assertThat(session.calls.map { it.tool }).containsExactly("list_leaks") + assertThat(log).anyMatch { it.contains("is not one JSON object") } + } + + @Test + fun `newest first, whichever order the files were listed in`() { + AgentSessionFile.starting(directory, SERVER_VERSION, startedAt = STARTED_AT, sessionId = "aaaaaaaa") + .opened(client = "the older agent", protocolVersion = null) + AgentSessionFile.starting( + directory, + SERVER_VERSION, + startedAt = STARTED_AT.plusSeconds(60), + sessionId = "bbbbbbbb" + ).opened(client = "the newer agent", protocolVersion = null) + + assertThat(AgentSessionFile.sessionsIn(directory).map { it.client }) + .containsExactly("the newer agent", "the older agent") + } + + @Test + fun `only the newest sessions are kept`() { + repeat(4) { index -> + AgentSessionFile.starting( + directory, + SERVER_VERSION, + startedAt = STARTED_AT.plusSeconds(index.toLong()), + sessionId = "session$index", + keepSessionCount = 2 + ).opened(client = "agent $index", protocolVersion = null) + } + + assertThat(AgentSessionFile.sessionsIn(directory).map { it.client }) + .containsExactly("agent 3", "agent 2") + } + + @Test + fun `every tool has a verb, so that no screen ends up showing the protocol`() { + val withoutAVerb = AgentTools { emptyList() }.all + .map { it.name } + .filter { verbOfTool(it, emptyMap()) == null } + + assertThat(withoutAVerb).isEmpty() + } + + @Test + fun `a directory no agent has ever connected through is no sessions rather than a failure`() { + assertThat(AgentSessionFile.sessionsIn(File(temporaryFolder.root, "never-used"))).isEmpty() + } + + private fun call( + tool: String, + reason: String? = "Because.", + place: Place? = null, + arguments: Map = emptyMap(), + refusal: String? = null + ) = AgentSessionCall( + at = STARTED_AT, + tool = tool, + reason = reason, + windowId = WINDOW_ID, + heapDumpPath = "/dumps/leak.hprof", + place = place, + arguments = arguments, + refusal = refusal, + millis = 12L + ) + + private companion object { + const val SERVER_VERSION = "1.2.3" + const val WINDOW_ID = "zvphq4r3" + const val OBJECT_ID = 0x12d368b8L + + val STARTED_AT: Instant = Instant.parse("2026-08-25T18:19:48.035Z") + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index 51fa5bfa24..91da51be7e 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -293,6 +293,12 @@ class AgentToolsTest { .isInstanceOf(AgentRefusal::class.java) .hasMessageContaining("contradicts 1 verdict(s)") .hasMessageContaining(hex(heapDump.holderObjectId)) + // The verdict it disagrees with, in the words it was given, since that is what decides which of the + // two is wrong. As a sentence and not as JSON: this refusal is drawn on the *Agent logs* screen, which + // exists to not show the protocol. See AgentTools.asSentence. + .hasMessageContaining("Holder.INSTANCE is a static singleton") + .hasMessageContaining("Keeping yours makes it ${LeakStatus.STUCK}") + .hasMessageNotContaining("{\"") .hasMessageContaining("solveConflicts") val answer = call( diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index 36d6261bdd..0b47e1d89d 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -1,5 +1,6 @@ package shark.explorer.agent +import java.io.File import kotlinx.coroutines.runBlocking import kotlinx.serialization.json.Json import kotlinx.serialization.json.JsonObject @@ -12,6 +13,7 @@ import org.junit.Before import org.junit.Rule import org.junit.Test import org.junit.rules.TemporaryFolder +import shark.explorer.Place import shark.explorer.exactHexObjectId /** @@ -33,12 +35,18 @@ class McpSessionTest { private lateinit var heapDump: InvestigationHeapDump private lateinit var window: FakeAgentHeapDump private lateinit var session: McpSession + private lateinit var sessionsDirectory: File @Before fun setUp() { heapDump = temporaryFolder.applicationHoldsActivityThroughHolder() window = FakeAgentHeapDump(heapDump.explorer) - session = McpSession(AgentTools { listOf(window) }, serverVersion = SERVER_VERSION) + sessionsDirectory = File(temporaryFolder.root, "sessions") + session = McpSession( + tools = AgentTools { listOf(window) }, + serverVersion = SERVER_VERSION, + sessionFile = AgentSessionFile.starting(sessionsDirectory, SERVER_VERSION) + ) } @After @@ -178,6 +186,49 @@ class McpSessionTest { assertThat(log.subList(called + 1, log.size)).contains("${hex(heapDump.holderObjectId)} for an agent") } + @Test + fun `every call is written down with its reason and somewhere to go`() { + answer( + """{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18",""" + + """"clientInfo":{"name":"claude-code","version":"9.9.9"},"capabilities":{}}}""" + ) + callTool( + """{"name":"describe_object","arguments":{"object":"${hex(heapDump.holderObjectId)}",""" + + """"reason":"Checking whether the holder is the singleton it looks like."}}""" + ) + + val session = sessions().single() + assertThat(session.client).isEqualTo("claude-code 9.9.9") + assertThat(session.serverVersion).isEqualTo(SERVER_VERSION) + val call = session.calls.single() + assertThat(call.verb).isEqualTo("Described") + assertThat(call.subject).isEqualTo(hex(heapDump.holderObjectId)) + assertThat(call.reason).isEqualTo("Checking whether the holder is the singleton it looks like.") + assertThat(call.refusal).isNull() + // Which is what makes the row clickable: the place, in the window the call was made against. + assertThat(call.place).isEqualTo(Place.Object(heapDump.holderObjectId)) + assertThat(call.link()).isEqualTo("shark://${window.windowId}/object?id=${hex(heapDump.holderObjectId)}") + assertThat(call.heapDumpPath).isEqualTo(window.heapDumpPath) + } + + @Test + fun `a refused call is written down with the refusal and what it was asking about`() { + callTool( + """{"name":"conclude","arguments":{"object":"${hex(heapDump.activityObjectId)}",""" + + """"rootCause":"The holder never lets go.","reason":"I know what this is."}}""" + ) + + val call = sessions().single().calls.single() + assertThat(call.verb).isEqualTo("Concluded about") + assertThat(call.refusal).contains("Not concluded") + assertThat(call.reason).isEqualTo("I know what this is.") + // Refused, and still pointing at the object it was refused about: a refusal nobody can follow up on is + // the half of a session that is worth reading afterwards. + assertThat(call.place).isEqualTo(Place.Object(heapDump.activityObjectId)) + } + + private fun sessions(): List = AgentSessionFile.sessionsIn(sessionsDirectory) + private fun callTool(params: String): JsonObject = answer("""{"jsonrpc":"2.0","id":9,"method":"tools/call","params":$params}""").result() diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt new file mode 100644 index 0000000000..24e5e3075b --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt @@ -0,0 +1,226 @@ +package shark.explorer.app + +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.width +import androidx.compose.foundation.rememberScrollState +import androidx.compose.foundation.text.selection.SelectionContainer +import androidx.compose.foundation.verticalScroll +import androidx.compose.material3.HorizontalDivider +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.Surface +import androidx.compose.material3.Text +import androidx.compose.runtime.Composable +import androidx.compose.ui.Modifier +import androidx.compose.ui.unit.dp +import java.io.File +import java.time.Instant +import java.time.ZoneId +import java.time.format.DateTimeFormatter +import shark.explorer.Place +import shark.explorer.agent.AgentSession +import shark.explorer.agent.AgentSessionCall +import shark.explorer.agent.subject +import shark.explorer.agent.verb + +/** + * Every agent that has worked on a heap dump through this app, one row each. + * + * Because an agent works in this window: it reads the dump the person at the machine is reading, sets the + * verdicts they see and writes into the same notes. So what it did has to be here, in words, rather than in + * a JSON stream a client happens to have kept — and a row of it has to lead where it went, which is what + * makes the two of them one investigation instead of two. + * + * Not per heap dump, unlike the notes and the verdicts: a session is one agent's connection to this app and + * can read whichever dumps were open. Whether a row is about *this* window's dump is what decides whether + * clicking it goes anywhere. See [AgentLogScreen]. + */ +@Composable +internal fun AgentLogsScreen( + sessions: List, + onOpen: (Place, OpenIn) -> Unit, + onCopyLink: (Place) -> Unit, + modifier: Modifier = Modifier +) { + Surface(modifier, color = MaterialTheme.colorScheme.surface) { + Column( + Modifier.verticalScroll(rememberScrollState()).padding(16.dp), + verticalArrangement = Arrangement.spacedBy(8.dp) + ) { + Text(Place.AGENT_LOGS_LABEL, style = MaterialTheme.typography.titleMedium) + if (sessions.isEmpty()) { + Text(NO_SESSIONS, style = MaterialTheme.typography.bodyMedium) + } + sessions.forEach { session -> + val place = Place.AgentLog(session.sessionId) + val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } + OpenTarget(open, { onCopyLink(place) }) { + Column(Modifier.openable(open)) { + Text(session.title(), style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) + Text(session.summary(), style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) + } + } + } + } + } +} + +/** + * What one agent did, call by call, in the order it made them. + * + * **Verbs and addresses rather than the protocol.** What is worth reading here is whether the steps follow + * from each other, and that is a question about what was asked and why — a screen of JSON is the same + * information in the one form nobody reads. So a row is what the call did, what it was about, and the + * sentence the agent gave for making it, which is its own words and not a paraphrase. + * + * A row about an object of the heap dump this window has open leads to it, like every other way to an + * object here. One about another dump says which, and leads nowhere: a session can span windows, and + * silently landing on the wrong dump's object at the same address would be worse than not moving. + */ +@Composable +internal fun AgentLogScreen( + session: AgentSession?, + /** Which heap dump this window has open, which is what decides whether a row leads anywhere. */ + heapDumpFile: File, + onOpen: (Place, OpenIn) -> Unit, + onCopyLink: (Place) -> Unit, + modifier: Modifier = Modifier +) { + Surface(modifier, color = MaterialTheme.colorScheme.surface) { + Column( + Modifier.verticalScroll(rememberScrollState()).padding(16.dp), + verticalArrangement = Arrangement.spacedBy(8.dp) + ) { + if (session == null) { + Text(NO_SUCH_SESSION, style = MaterialTheme.typography.bodyMedium) + return@Column + } + Text(session.title(), style = MaterialTheme.typography.titleMedium) + Text(session.summary(), style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) + // The file, because a session outlives this window: it gets read by a script, pasted into an issue, + // or opened in an editor months later, and none of that can happen if only this screen knows where it + // is. Selectable for the same reason the addresses are. + SelectionContainer { + Text(session.file.path, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) + } + HorizontalDivider() + if (session.calls.isEmpty()) { + Text(NOTHING_ASKED, style = MaterialTheme.typography.bodyMedium) + } + session.calls.forEach { call -> + AgentCallRow( + call = call, + heapDumpFile = heapDumpFile, + onOpen = onOpen, + onCopyLink = onCopyLink + ) + } + } + } +} + +/** One call: when, what it did, and why the agent said it was doing it. */ +@Composable +private fun AgentCallRow( + call: AgentSessionCall, + heapDumpFile: File, + onOpen: (Place, OpenIn) -> Unit, + onCopyLink: (Place) -> Unit +) { + // A row leads somewhere only when the place it names is a place of the dump this window has open. An + // address is an address of one heap dump, so the same one in another dump is a different object. + val place = call.place?.takeIf { call.isAbout(heapDumpFile) } + Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) { + Text( + call.at.clockTime(), + Modifier.width(TIME_WIDTH), + style = MaterialTheme.typography.bodySmall, + color = MUTED_TEXT + ) + Column { + if (place == null) { + Text(call.line(), style = MaterialTheme.typography.bodyMedium) + } else { + val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } + OpenTarget(open, { onCopyLink(place) }) { + Text( + call.line(), + Modifier.openable(open), + style = MaterialTheme.typography.bodyMedium, + color = LINK_COLOR + ) + } + } + call.reason?.let { reason -> + // The agent's own sentence, indented under what it did: read down the column of these and a session + // either follows from itself or doesn't, which is the whole of what this screen is for. + Text("$BECAUSE $reason", style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) + } + call.refusal?.let { refusal -> + Text( + "$REFUSED $refusal", + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.error + ) + } + } + } +} + +/** Whether the call was about the heap dump this window has open. See [AgentLogScreen]. */ +private fun AgentSessionCall.isAbout(heapDumpFile: File): Boolean = + heapDumpPath == null || heapDumpPath == heapDumpFile.absolutePath + +/** What the call did and what it was about, as one line: "Described 0x12d368b8". */ +private fun AgentSessionCall.line(): String = listOfNotNull(verb, subject).joinToString(" ") + +/** What a session is called: who connected, and when. */ +private fun AgentSession.title(): String = listOfNotNull( + client ?: A_CLIENT_THAT_DID_NOT_SAY, + startedAt?.let { "at ${it.clockTime()}" } +).joinToString(" ") + +/** + * What it did, in numbers: how many calls, how many of those were refused, and which heap dumps it read. + * + * The refusals are here rather than only in the session because they are the number worth seeing before + * opening one: a session that was refused half its calls is a session where the method was being enforced, + * which is either an agent that was made to go back and look, or a refusal message that isn't landing. + */ +private fun AgentSession.summary(): String { + val dumps = calls.mapNotNull { it.heapDumpPath }.distinct().map { File(it).name } + return listOfNotNull( + "${calls.size} call(s)", + "$refusedCount refused".takeIf { refusedCount > 0 }, + dumps.joinToString(", ").takeIf { it.isNotEmpty() }, + sessionId + ).joinToString(" · ") +} + +/** The time of day, as every line of this app's own log is stamped. See `shark.explorer.SessionLog`. */ +private fun Instant.clockTime(): String = CLOCK_TIME.format(this) + +private val CLOCK_TIME: DateTimeFormatter = + DateTimeFormatter.ofPattern("HH:mm:ss").withZone(ZoneId.systemDefault()) + +/** Wide enough for the clock and no wider, so that the verbs line up down the screen. */ +private val TIME_WIDTH = 60.dp + +private const val BECAUSE = "because:" + +private const val REFUSED = "Refused:" + +private const val A_CLIENT_THAT_DID_NOT_SAY = "An agent" + +private const val NO_SESSIONS = + "No agent has connected to this app yet. Hand a heap dump to one by pointing its MCP client at Shark " + + "Explorer, and everything it does lands here." + +private const val NOTHING_ASKED = + "This agent connected and asked nothing before it went away." + +private const val NO_SUCH_SESSION = + "There is no session with that name. A link to one leads to the file it was written to, which is kept " + + "until a hundred newer sessions have pushed it out." diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt index 52976fbfd7..18cfdd3156 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt @@ -10,6 +10,8 @@ import shark.explorer.agent.AgentHeapDump import shark.explorer.agent.AgentHeapDumps import shark.explorer.agent.AgentRefusal import shark.explorer.agent.AgentServer +import shark.explorer.agent.AgentSession +import shark.explorer.agent.AgentSessionFile import shark.explorer.agent.AgentStdioBridge /** @@ -163,6 +165,16 @@ private class WindowAgentHeapDump( /** Between what was already written about a place and what an agent has to add, which is markdown. */ private const val PARAGRAPH_BREAK = "\n\n" +/** + * What every agent that has connected to this app did, newest session first. + * + * Read off disk rather than kept in memory, and not only this run's: the question the *Agent logs* screen + * answers is "what has an agent done to this heap dump", and the answer to that outlives the run it happened + * in. A directory of small files, so re-reading it is what keeps the screen live while an agent works. + */ +internal fun agentSessions(): List = + AgentSessionFile.sessionsIn(AgentServer.sessionsDirectory(AGENT_RUNS_DIRECTORY)) + /** Beside the runs answering links, the notes, the statuses and the logs. See [AgentServer]. */ private val AGENT_RUNS_DIRECTORY = File(SHARK_EXPLORER_DIRECTORY, "agents") diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt index 1f28311a31..73b5a456f1 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt @@ -43,6 +43,7 @@ import androidx.compose.ui.unit.IntSize import androidx.compose.ui.unit.dp import java.awt.Toolkit import java.awt.datatransfer.StringSelection +import java.io.File import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.delay import kotlinx.coroutines.flow.collectLatest @@ -76,6 +77,7 @@ import shark.explorer.Tabs import shark.explorer.TreemapLayout import shark.explorer.TreemapPresentation import shark.explorer.TreemapRect +import shark.explorer.agent.AgentSession import shark.explorer.detours import shark.explorer.formatObjectCount import shark.explorer.hexObjectId @@ -129,6 +131,14 @@ internal fun HeapDumpExplorer( * window of this run, another run of the app, or nowhere. See [DeepLinkPeers.follow]. */ followDeepLink: (DeepLink) -> Unit = { link -> SharkLog.d { "Nothing here to follow $link with" } }, + /** + * What every agent that has connected to this app did, read whenever the screen showing them is open. + * + * A function rather than state, because this is a directory of files an agent is appending to while the + * screen is being read: re-reading it is what makes the screen live. Overridden by tests, which have + * sessions of their own rather than the ones under this machine's home directory. + */ + agentSessions: () -> List = ::agentSessions, /** Overridden by tests, which have no browser. */ openUrl: (String) -> Unit = ::openInBrowser, /** Overridden by tests, which have no system clipboard and want to read what would have been copied. */ @@ -178,6 +188,8 @@ internal fun HeapDumpExplorer( var showsBitmapsFromDevice by remember { mutableStateOf(false) } /** The objects starred so far, with everything the list shows about them read once. */ var favourites by remember { mutableStateOf(emptyList()) } + /** What the agents that have worked through this app did, while a screen showing them is open. */ + var sessions by remember { mutableStateOf(emptyList()) } /** What each tab is called, by the place it is on. Only grows: a place is named once and stays named. */ var placeTitles by remember { mutableStateOf(emptyMap()) } /** @@ -492,6 +504,21 @@ internal fun HeapDumpExplorer( // opened per tab, since what it answers is a question about the whole strip. See [HeapDumpNotes.list]. LaunchedEffect(notes) { notes.list() } + // What the agents that have connected to this app did, for as long as a screen showing them is open. Read + // again on a timer rather than watched, because an agent appends to its session file while somebody is + // reading it — being able to watch an investigation happen is the point — and a handful of small files is + // cheaper to read again than a file watcher is to set up and take down per tab. + val showsAgentLogs = place is Place.AgentLogs || place is Place.AgentLog + LaunchedEffect(showsAgentLogs) { + if (!showsAgentLogs) { + return@LaunchedEffect + } + while (true) { + sessions = withContext(Dispatchers.IO) { agentSessions() } + delay(AGENT_LOGS_REFRESH_MILLIS) + } + } + // And what has been decided about this heap dump's objects by hand, also once per run: one small file, // read before anything is drawn from it, because a chain read without it would be the heap dump's own // answer where someone has already recorded another. See [HeapDumpLeakStatuses]. @@ -699,9 +726,13 @@ internal fun HeapDumpExplorer( leaks = leaks, isFindingLeaks = isFindingLeaks, favourites = favourites, + sessions = sessions, + heapDumpFile = session.heapDumpFile, sizes = sizes, onOpen = openObject, onCopyLink = copyObjectLink, + onOpenPlace = open, + onCopyPlaceLink = copyLink, onReplacePlace = { tabs = tabs.replacingCurrent(it) }, onRemoveStar = { objectId -> favourites = favourites.filterNot { it.objectId == objectId } }, modifier = Modifier.fillMaxSize() @@ -1034,9 +1065,16 @@ private fun ListPlace( leaks: HeapLeaks?, isFindingLeaks: Boolean, favourites: List, + /** What the agents that have worked through this app did, for the screens that draw them. */ + sessions: List, + /** Which heap dump this window has open, which is what decides where an agent's row leads. */ + heapDumpFile: File, sizes: HeapSizes, onOpen: (Long, OpenIn) -> Unit, onCopyLink: (Long) -> Unit, + /** Where a row leading to something that is not an object goes. See [AgentLogsScreen]. */ + onOpenPlace: (Place, OpenIn) -> Unit, + onCopyPlaceLink: (Place) -> Unit, onReplacePlace: (Place) -> Unit, onRemoveStar: (Long) -> Unit, modifier: Modifier = Modifier @@ -1079,6 +1117,20 @@ private fun ListPlace( onRemove = onRemoveStar, modifier = modifier ) + is Place.AgentLogs -> AgentLogsScreen( + sessions = sessions, + onOpen = onOpenPlace, + onCopyLink = onCopyPlaceLink, + modifier = modifier + ) + is Place.AgentLog -> AgentLogScreen( + // Null for a session that has been pushed out by newer ones, or one from another machine's link. + session = sessions.firstOrNull { it.sessionId == place.sessionId }, + heapDumpFile = heapDumpFile, + onOpen = onOpenPlace, + onCopyLink = onCopyPlaceLink, + modifier = modifier + ) // The places with a view of their own are drawn by the panes, not here. is Place.Object, is Place.SmallerObjects -> Unit } @@ -1155,6 +1207,10 @@ private fun ScreenBar( onCopyLink = onCopyLink, isEnabled = starredCount > 0 ) + // Beside the reader's own trail through the heap dump, because it is the same kind of thing: what has + // been looked at, by whoever was looking. An agent works in this window rather than in one of its own, + // so what it did belongs on this bar and not in a file somebody has to be told about. + ScreenButton(Place.AgentLogs, Place.AGENT_LOGS_LABEL, onOpen, onCopyLink) // Only when there are bitmaps the dump has no pixels for, because that's the only thing a device can // add: pixels the dump carries are already on the map by the time this bar is read. if (bitmapCounts.withoutImageCount > 0) { @@ -1569,6 +1625,16 @@ private const val FILTER_SETTLE_MILLIS = 250L */ private const val HOVER_SETTLE_MILLIS = 100L +/** + * How often the sessions of the agents that have connected are read again, while a screen showing them is + * open. + * + * A second, because what this is for is watching an agent work: a row appearing as the call it stands for is + * answered is the difference between following an investigation and reading a report of one. Off entirely + * while no such screen is open, which is nearly always. + */ +private const val AGENT_LOGS_REFRESH_MILLIS = 1_000L + /** What a tab is called for the beat between it being opened and the heap dump having named it. */ private const val NAMING_TAB = "…" diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt index 0fd2b81450..3fec8a3c68 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt @@ -45,6 +45,7 @@ import shark.explorer.HeapSizes import shark.explorer.NativeBitmapPixels import shark.explorer.Place import shark.explorer.ReachabilityStrength +import shark.explorer.agent.AgentSession import shark.explorer.formatByteSize import shark.explorer.formatObjectCount import shark.explorer.jdwp.JdwpBitmaps @@ -247,6 +248,13 @@ internal fun ExplorerApp( * closes. See [ExplorerWindow.openHeapDump]. */ onHeapDumpOpen: (WindowHeapDump?) -> Unit = {}, + /** + * What every agent that has connected did, for the screens that draw them. + * + * Its own by default, and overridden by tests for the reason the notes are: a test reading the sessions + * under whoever is running it would be a test of their investigations rather than of this window. + */ + agentSessions: () -> List = ::agentSessions, /** What a link to a place in this window names it by. See [shark.explorer.DeepLink]. */ deepLinkId: String = remember { DeepLink.newWindowId() }, /** Places a link has asked this window for, which its tabs open. See [ExplorerWindow.linkedPlaces]. */ @@ -373,6 +381,7 @@ internal fun ExplorerApp( fetchedBitmapPixels = currentState.bitmapPixels, notes = notes.of(currentState.session.heapDumpFile), leakStatuses = leakStatuses.of(currentState.session.heapDumpFile), + agentSessions = agentSessions, deepLinkId = deepLinkId, linkedPlaces = linkedPlaces, onLinkedPlaceOpened = onLinkedPlaceOpened, diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt new file mode 100644 index 0000000000..b5203e82d3 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt @@ -0,0 +1,180 @@ +package shark.explorer.app + +import androidx.compose.material3.MaterialTheme +import androidx.compose.ui.semantics.Role +import androidx.compose.ui.semantics.SemanticsProperties +import androidx.compose.ui.test.ComposeUiTest +import androidx.compose.ui.test.ExperimentalTestApi +import androidx.compose.ui.test.SemanticsMatcher +import androidx.compose.ui.test.assertHasNoClickAction +import androidx.compose.ui.test.assertIsDisplayed +import androidx.compose.ui.test.hasText +import androidx.compose.ui.test.onNodeWithText +import androidx.compose.ui.test.performClick +import androidx.compose.ui.test.waitUntilAtLeastOneExists +import java.io.File +import java.time.Instant +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import shark.explorer.Adb +import shark.explorer.AdbOutput +import shark.explorer.DeviceHeapDumps +import shark.explorer.Place +import shark.explorer.agent.AgentSession +import shark.explorer.agent.AgentSessionCall +import shark.explorer.exactHexObjectId + +/** + * What an agent did, in the window it did it in. + * + * An investigation an agent ran and one a person ran are the same investigation: it reads this heap dump, + * sets the verdicts they see and writes the same notes. So what it did is read here in words, and **a row + * leads where the call went** — which is what these tests are about, along with the one case where it must + * not: a call about another heap dump, whose addresses mean nothing here. + */ +@OptIn(ExperimentalTestApi::class) +class AgentLogsScreenTest { + + @get:Rule + val testFolder = TemporaryFolder() + + /** Every UI test here records what Shark logged. See [RecordedLog]. */ + @get:Rule val logged = RecordedLog() + + private lateinit var heapDump: LeakyHeapDump + + @Before fun setUp() { + heapDump = testFolder.leakyHeapDump() + } + + @Test fun `the button leads to the sessions, and a session to what the agent did`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(call())))) + + onNodeWithText(CLIENT, substring = true).assertIsDisplayed() + onNodeWithText(CLIENT, substring = true).performClick() + + // The verb, the address, and the agent's own sentence for why it asked: no JSON on any of it. + waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) + onNodeWithText("Described ${hex(activityObjectId())}").assertIsDisplayed() + } + } + + @Test fun `a refused call says so, and still says what it was about`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(call(tool = "conclude", refusal = REFUSAL))))) + onNodeWithText(CLIENT, substring = true).performClick() + + waitUntilAtLeastOneExists(hasText(REFUSAL, substring = true), OPEN_TIMEOUT_MILLIS) + onNodeWithText("Concluded about ${hex(activityObjectId())}").assertIsDisplayed() + } + } + + @Test fun `a row leads to the object the call was about`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(call())))) + onNodeWithText(CLIENT, substring = true).performClick() + waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) + + onNodeWithText("Described ${hex(activityObjectId())}").performClick() + + // What the inspectors made of the object the agent was reading, which is the whole promise of the + // screen: reading what it did and going to look at it are one move. + waitUntilAtLeastOneExists(hasText("mDestroyed", substring = true), OPEN_TIMEOUT_MILLIS) + } + } + + @Test fun `a call about another heap dump is read here and leads nowhere`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(call(heapDumpPath = "/dumps/another.hprof"))))) + onNodeWithText(CLIENT, substring = true).performClick() + waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) + + // An address is an address of one heap dump, so the same one here is a different object — or no + // object at all. Read it, don't follow it. + onNodeWithText("Described ${hex(activityObjectId())}").assertHasNoClickAction() + } + } + + @Test fun `a window no agent has connected to says what would put something here`() { + explorerUiTest { + openAgentLogs(emptyList()) + + onNodeWithText(NO_AGENT_YET, substring = true).assertIsDisplayed() + } + } + + /** Opens the window on [leakyHeapDump] with [sessions] as the agents that have worked through it. */ + private fun ComposeUiTest.openAgentLogs(sessions: List) { + setContent { + MaterialTheme { + ExplorerApp( + heapDumpFile = heapDump.file, + onHeapDumpChosen = { _, _ -> }, + // Given rather than read off this machine: the sessions under whoever is running the tests are + // their investigations, and none of this window's business. + agentSessions = { sessions }, + deviceHeapDumps = DeviceHeapDumps(NO_DEVICE_ADB) + ) + } + } + waitForTheTree(OPEN_TIMEOUT_MILLIS) + screenButton(Place.AGENT_LOGS_LABEL).performClick() + } + + private fun session(calls: List) = AgentSession( + sessionId = SESSION_ID, + startedAt = STARTED_AT, + client = CLIENT, + serverVersion = "1.2.3", + file = File(testFolder.root, "sessions/agent-$SESSION_ID.jsonl"), + calls = calls + ) + + /** The call every test here is about: the agent reading one of the destroyed activities. */ + private fun call( + tool: String = "describe_object", + heapDumpPath: String = heapDump.file.absolutePath, + refusal: String? = null + ) = AgentSessionCall( + at = STARTED_AT, + tool = tool, + reason = REASON, + windowId = "zvphq4r3", + heapDumpPath = heapDumpPath, + place = Place.Object(activityObjectId()), + arguments = mapOf("object" to hex(activityObjectId())), + refusal = refusal, + millis = 12L + ) + + private fun activityObjectId() = heapDump.activityObjectIds.first() + + private fun hex(objectId: Long) = exactHexObjectId(objectId) + + /** + * A button on the row of screens an open heap dump can be read through, as against the tab of the same + * name that clicking it opens. See [ExplorerAppTest] for why the role is what tells them apart. + */ + private fun ComposeUiTest.screenButton(label: String) = onNode(hasText(label) and isButton()) + + private fun isButton(): SemanticsMatcher = + SemanticsMatcher.expectValue(SemanticsProperties.Role, Role.Button) + + private companion object { + + const val SESSION_ID = "1a2b3c4d" + const val CLIENT = "claude-code 9.9.9" + const val REASON = "Checking whether this activity is really destroyed." + const val REFUSAL = "3 step(s) have no verdict" + const val NO_AGENT_YET = "No agent has connected" + + val STARTED_AT: Instant = Instant.parse("2026-08-25T18:19:48.035Z") + + private const val OPEN_TIMEOUT_MILLIS = 10_000L + + private val NO_DEVICE_ADB = Adb { AdbOutput(exitCode = 0, text = "List of devices attached\n") } + } +} diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt index 633b0577c3..a7c7320782 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt @@ -104,6 +104,8 @@ data class DeepLink( ) LEAKS_PATH -> Place.Leaks(parameters.all(EXPANDED_PARAMETER).toSet()) STARRED_PATH -> Place.Starred + AGENT_LOGS_PATH -> Place.AgentLogs + AGENT_LOG_PATH -> Place.AgentLog(parameters.required(SESSION_PARAMETER, uri)) else -> throw IllegalArgumentException("\"$path\" is no place of \"$uri\". ${usage()}") } @@ -200,9 +202,18 @@ data class DeepLink( internal const val OBJECTS_PATH = "objects" internal const val LEAKS_PATH = "leaks" internal const val STARRED_PATH = "starred" + internal const val AGENT_LOGS_PATH = "agent-logs" + internal const val AGENT_LOG_PATH = "agent-log" - private val PLACE_PATHS = - listOf(OBJECT_PATH, SMALLER_OBJECTS_PATH, OBJECTS_PATH, LEAKS_PATH, STARRED_PATH) + private val PLACE_PATHS = listOf( + OBJECT_PATH, + SMALLER_OBJECTS_PATH, + OBJECTS_PATH, + LEAKS_PATH, + STARRED_PATH, + AGENT_LOGS_PATH, + AGENT_LOG_PATH + ) internal const val ID_PARAMETER = "id" internal const val PARENT_PARAMETER = "parent" @@ -212,19 +223,22 @@ data class DeepLink( internal const val EXACT_PARAMETER = "exact" internal const val KINDS_PARAMETER = "kinds" internal const val EXPANDED_PARAMETER = "expanded" + internal const val SESSION_PARAMETER = "session" internal const val HEX_PREFIX = "0x" private const val HEX_RADIX = 16 } } -/** Which of the five a place is written as. The other half of `DeepLink.placeOf`, and it has to stay so. */ +/** Which place this is written as. The other half of `DeepLink.placeOf`, and it has to stay so. */ private fun Place.linkPath(): String = when (this) { is Place.Object -> DeepLink.OBJECT_PATH is Place.SmallerObjects -> DeepLink.SMALLER_OBJECTS_PATH is Place.Objects -> DeepLink.OBJECTS_PATH is Place.Leaks -> DeepLink.LEAKS_PATH is Place.Starred -> DeepLink.STARRED_PATH + is Place.AgentLogs -> DeepLink.AGENT_LOGS_PATH + is Place.AgentLog -> DeepLink.AGENT_LOG_PATH } /** @@ -255,6 +269,8 @@ private fun Place.linkParameters(): List> = when (this) { } is Place.Leaks -> expandedGroups.sorted().map { DeepLink.EXPANDED_PARAMETER to it } is Place.Starred -> emptyList() + is Place.AgentLogs -> emptyList() + is Place.AgentLog -> listOf(DeepLink.SESSION_PARAMETER to sessionId) } /** The exact 64 bits, unsigned, which is what `DeepLink.nodeId` reads back. */ diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NoteFile.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NoteFile.kt index 0330a052fe..17c8c95a70 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NoteFile.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NoteFile.kt @@ -74,6 +74,9 @@ fun Place.noteKey(): String = when (this) { is Place.Objects -> "object-list" is Place.Leaks -> "leaks" is Place.Starred -> "starred" + is Place.AgentLogs -> "agent-logs" + // Per session, because a note about what one agent did is about that investigation and not about agents. + is Place.AgentLog -> "agent-log-$sessionId" } /** The note about the heap dump as a whole, which is the place its first tab opens on. */ diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/Place.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/Place.kt index 4353bbbc56..f4ef184c1d 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/Place.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/Place.kt @@ -102,6 +102,31 @@ sealed interface Place { override val viewRootObjectId: Long? get() = null } + /** Every agent that has worked on a heap dump of this app, one row each. See `AgentSessionFile`. */ + data object AgentLogs : Place { + + override val title: String get() = AGENT_LOGS_LABEL + + override val viewRootObjectId: Long? get() = null + } + + /** + * What one agent did, call by call. + * + * A place like any other, so that it is a tab, a link, and somewhere the back arrow returns to — which is + * the whole point: reading what an agent did and going to look at what it was looking at have to be the + * same kind of move, or following an investigation means keeping a list of addresses on paper. + */ + data class AgentLog( + /** Which session, as [DeepLink] and the file it was written to name it. */ + val sessionId: String + ) : Place { + + override val title: String get() = "$AGENT_LOGS_LABEL $sessionId" + + override val viewRootObjectId: Long? get() = null + } + companion object { /** @@ -132,6 +157,9 @@ sealed interface Place { /** And to the objects starred so far. */ const val STARRED_LABEL = "Starred" + + /** And to what the agents that have worked on a heap dump of this app did. */ + const val AGENT_LOGS_LABEL = "Agent logs" } } @@ -157,6 +185,8 @@ fun HeapDominatorTreemap.titleOf(place: Place): String = when (place) { is Place.Objects -> place.title is Place.Leaks -> place.title is Place.Starred -> place.title + is Place.AgentLogs -> place.title + is Place.AgentLog -> place.title } /** diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt index a7f994f812..7c0cbcdb6e 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt @@ -134,6 +134,26 @@ class DeepLinkTest { assertThat(DeepLink.parse("shark://abcd2345/starred").place).isEqualTo(Place.Starred) } + /** + * Which is what an agent's own session is handed over by: the human it is working for gets a link to what + * it did, rather than a path to a file and instructions for finding the row. + */ + @Test + fun `one agent's session is named by the session`() { + val place = Place.AgentLog("1a2b3c4d") + + assertThat(DeepLink("abcd2345", place).toUri()) + .isEqualTo("shark://abcd2345/agent-log?session=1a2b3c4d") + assertThat(DeepLink.parse("shark://abcd2345/agent-log?session=1a2b3c4d").place).isEqualTo(place) + } + + @Test + fun `an agent log link with no session says what is missing`() { + assertThatThrownBy { DeepLink.parse("shark://abcd2345/agent-log") } + .isInstanceOf(IllegalArgumentException::class.java) + .hasMessageContaining("needs a \"session\"") + } + /** * The one that has to keep being true as places are added: every [Place] is reachable by link, which is * the whole promise. A place with no spelling here fails this rather than being found out by clicking one. @@ -148,7 +168,9 @@ class DeepLinkTest { Place.Objects(ObjectListFilter(query = "Bitmap", isExactMatch = true)), Place.Leaks(), Place.Leaks(expandedGroups = setOf("APPLICATION 12ab")), - Place.Starred + Place.Starred, + Place.AgentLogs, + Place.AgentLog("1a2b3c4d") ) places.forEach { place -> From acf1974b75a86f55f33aa42359539c9a4891fe4b Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 08:41:58 +0200 Subject: [PATCH 05/27] Name the objects an agent asked about the way a tab does MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A row of a session read `Described 0x12d368b8`, which is the address the agent wrote and not what this window calls that object anywhere else. Now it reads `Described MainActivity 0x12d368b8`, from the same titleOf the tabs are named by, so that a row and the tab clicking it opens are recognisably one object. Resolved in the window rather than recorded in the session file: an address is what the agent said, and what it stands for is a read of the heap dump this window has open — the read that names a tab. Which leaves a call about another dump as the address it was, since naming it would mean reading a dump nobody here has open. An agent can also name an address this dump has no object at, and that call is a row too, so the address is asked about before it is named: titleOf throws on an object the graph hasn't got. --- .../shark/explorer/app/AgentLogsScreen.kt | 30 +++++++++--- .../shark/explorer/app/HeapDumpExplorer.kt | 49 +++++++++++++++++++ .../shark/explorer/app/AgentLogsScreenTest.kt | 22 +++++++-- 3 files changed, 90 insertions(+), 11 deletions(-) diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt index 24e5e3075b..c7c1234b44 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt @@ -70,10 +70,12 @@ internal fun AgentLogsScreen( /** * What one agent did, call by call, in the order it made them. * - * **Verbs and addresses rather than the protocol.** What is worth reading here is whether the steps follow + * **Verbs and object names rather than the protocol.** What is worth reading here is whether the steps follow * from each other, and that is a question about what was asked and why — a screen of JSON is the same * information in the one form nobody reads. So a row is what the call did, what it was about, and the - * sentence the agent gave for making it, which is its own words and not a paraphrase. + * sentence the agent gave for making it, which is its own words and not a paraphrase. An agent names objects + * by address, and this names them the way the rest of the window does, so that a row and the tab it opens are + * recognisably the same object. * * A row about an object of the heap dump this window has open leads to it, like every other way to an * object here. One about another dump says which, and leads nowhere: a session can span windows, and @@ -84,6 +86,12 @@ internal fun AgentLogScreen( session: AgentSession?, /** Which heap dump this window has open, which is what decides whether a row leads anywhere. */ heapDumpFile: File, + /** + * What this window calls each place a call was about — `MainActivity 0x12d368b8` — for the places it has + * been asked about yet. A place that isn't in here is drawn as the address the agent wrote, which is what + * a call about another heap dump stays as: naming it would mean reading a dump this window doesn't have. + */ + placeTitles: Map, onOpen: (Place, OpenIn) -> Unit, onCopyLink: (Place) -> Unit, modifier: Modifier = Modifier @@ -113,6 +121,7 @@ internal fun AgentLogScreen( AgentCallRow( call = call, heapDumpFile = heapDumpFile, + title = call.place?.let { placeTitles[it] }, onOpen = onOpen, onCopyLink = onCopyLink ) @@ -126,6 +135,8 @@ internal fun AgentLogScreen( private fun AgentCallRow( call: AgentSessionCall, heapDumpFile: File, + /** What this window calls what the call was about, and null while it hasn't been read or can't be. */ + title: String?, onOpen: (Place, OpenIn) -> Unit, onCopyLink: (Place) -> Unit ) { @@ -141,12 +152,12 @@ private fun AgentCallRow( ) Column { if (place == null) { - Text(call.line(), style = MaterialTheme.typography.bodyMedium) + Text(call.line(title), style = MaterialTheme.typography.bodyMedium) } else { val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } OpenTarget(open, { onCopyLink(place) }) { Text( - call.line(), + call.line(title), Modifier.openable(open), style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR @@ -173,8 +184,15 @@ private fun AgentCallRow( private fun AgentSessionCall.isAbout(heapDumpFile: File): Boolean = heapDumpPath == null || heapDumpPath == heapDumpFile.absolutePath -/** What the call did and what it was about, as one line: "Described 0x12d368b8". */ -private fun AgentSessionCall.line(): String = listOfNotNull(verb, subject).joinToString(" ") +/** + * What the call did and what it was about, as one line: "Described MainActivity 0x12d368b8". + * + * [title] is what this window calls that object, which is what a tab on it is called too — the row and the + * tab it opens have to read the same. Without one, the address the agent wrote: a call about another heap + * dump, or one this window hasn't read yet. + */ +private fun AgentSessionCall.line(title: String?): String = + listOfNotNull(verb, title ?: subject).joinToString(" ") /** What a session is called: who connected, and when. */ private fun AgentSession.title(): String = listOfNotNull( diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt index 73b5a456f1..c9540a75b3 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt @@ -79,6 +79,7 @@ import shark.explorer.TreemapPresentation import shark.explorer.TreemapRect import shark.explorer.agent.AgentSession import shark.explorer.detours +import shark.explorer.exactHexObjectId import shark.explorer.formatObjectCount import shark.explorer.hexObjectId import shark.explorer.leakStatusConflictsWith @@ -192,6 +193,12 @@ internal fun HeapDumpExplorer( var sessions by remember { mutableStateOf(emptyList()) } /** What each tab is called, by the place it is on. Only grows: a place is named once and stays named. */ var placeTitles by remember { mutableStateOf(emptyMap()) } + /** + * And what to call the places an agent asked about, which is the same question with one difference: an + * agent can name an address this heap dump has no object at, so this map answers for a place a tab could + * not be opened on. See [agentPlaceTitle]. + */ + var agentPlaceTitles by remember { mutableStateOf(emptyMap()) } /** * The note about the tab on screen, and null once the last tab has been closed — which is the one state * with no tab to write about. @@ -519,6 +526,27 @@ internal fun HeapDumpExplorer( } } + // And what to call the objects those agents asked about, so that a row of a session names an object the + // way the tab it opens does — `MainActivity 0x12d368b8` — rather than as the bare address the agent wrote. + // The session file holds addresses on purpose: an address is what an agent said, and what it stands for is + // a read of the heap dump this window has open, which is the same read that names a tab. + val unnamedAgentPlaces = (place as? Place.AgentLog) + ?.let { open -> sessions.firstOrNull { it.sessionId == open.sessionId } } + ?.calls.orEmpty() + .filter { it.heapDumpPath == null || it.heapDumpPath == session.heapDumpFile.absolutePath } + .mapNotNull { it.place } + .filter { it !in agentPlaceTitles } + .distinct() + LaunchedEffect(session, unnamedAgentPlaces) { + if (unnamedAgentPlaces.isEmpty()) { + return@LaunchedEffect + } + val named = session.read("what to call ${unnamedAgentPlaces.size} places an agent asked about") { explorer -> + unnamedAgentPlaces.associateWith { explorer.tree.agentPlaceTitle(it) } + } + agentPlaceTitles = agentPlaceTitles + named + } + // And what has been decided about this heap dump's objects by hand, also once per run: one small file, // read before anything is drawn from it, because a chain read without it would be the heap dump's own // answer where someone has already recorded another. See [HeapDumpLeakStatuses]. @@ -728,6 +756,7 @@ internal fun HeapDumpExplorer( favourites = favourites, sessions = sessions, heapDumpFile = session.heapDumpFile, + agentPlaceTitles = agentPlaceTitles, sizes = sizes, onOpen = openObject, onCopyLink = copyObjectLink, @@ -1069,6 +1098,8 @@ private fun ListPlace( sessions: List, /** Which heap dump this window has open, which is what decides where an agent's row leads. */ heapDumpFile: File, + /** What this window calls the places those agents asked about. See [agentPlaceTitle]. */ + agentPlaceTitles: Map, sizes: HeapSizes, onOpen: (Long, OpenIn) -> Unit, onCopyLink: (Long) -> Unit, @@ -1127,6 +1158,7 @@ private fun ListPlace( // Null for a session that has been pushed out by newer ones, or one from another machine's link. session = sessions.firstOrNull { it.sessionId == place.sessionId }, heapDumpFile = heapDumpFile, + placeTitles = agentPlaceTitles, onOpen = onOpenPlace, onCopyLink = onCopyPlaceLink, modifier = modifier @@ -1607,6 +1639,23 @@ private suspend fun HeapDumpSession.describing( ) } +/** + * What this window calls a place an agent asked about: the title a tab on it would have. + * + * The same [titleOf] the tabs are named by, so that a row of a session and the tab clicking it opens read the + * same — an agent and the person watching it are looking at one object, and two spellings of it would be two + * objects to them. + * + * With the one difference that makes this a function of its own: an agent can name an address this heap dump + * has no object at, which is a call it was refused and still a row worth reading. [titleOf] would throw on + * it, so the address is asked about first and stands for itself when it is nothing here. + */ +private fun HeapDominatorTreemap.agentPlaceTitle(place: Place): String = when (place) { + is Place.Object -> + if (objectNameOrNull(place.objectId) == null) exactHexObjectId(place.objectId) else titleOf(place) + else -> titleOf(place) +} + /** What the panes are being filled in for, for the log. See [HeapDumpSession.read]. */ private fun Place.description(): String = when (this) { is Place.Object -> "what ${nodeIdText(objectId)} is" diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt index b5203e82d3..8deb01c54c 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt @@ -25,6 +25,7 @@ import shark.explorer.Place import shark.explorer.agent.AgentSession import shark.explorer.agent.AgentSessionCall import shark.explorer.exactHexObjectId +import shark.explorer.hexObjectId /** * What an agent did, in the window it did it in. @@ -56,9 +57,10 @@ class AgentLogsScreenTest { onNodeWithText(CLIENT, substring = true).assertIsDisplayed() onNodeWithText(CLIENT, substring = true).performClick() - // The verb, the address, and the agent's own sentence for why it asked: no JSON on any of it. + // The verb, the object named the way a tab on it is named, and the agent's own sentence for why it + // asked: no JSON and no bare address on any of it. waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) - onNodeWithText("Described ${hex(activityObjectId())}").assertIsDisplayed() + waitUntilAtLeastOneExists(hasText(describedRow()), OPEN_TIMEOUT_MILLIS) } } @@ -68,7 +70,10 @@ class AgentLogsScreenTest { onNodeWithText(CLIENT, substring = true).performClick() waitUntilAtLeastOneExists(hasText(REFUSAL, substring = true), OPEN_TIMEOUT_MILLIS) - onNodeWithText("Concluded about ${hex(activityObjectId())}").assertIsDisplayed() + waitUntilAtLeastOneExists( + hasText("Concluded about ${activityName()}"), + OPEN_TIMEOUT_MILLIS + ) } } @@ -78,7 +83,8 @@ class AgentLogsScreenTest { onNodeWithText(CLIENT, substring = true).performClick() waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) - onNodeWithText("Described ${hex(activityObjectId())}").performClick() + waitUntilAtLeastOneExists(hasText(describedRow()), OPEN_TIMEOUT_MILLIS) + onNodeWithText(describedRow()).performClick() // What the inspectors made of the object the agent was reading, which is the whole promise of the // screen: reading what it did and going to look at it are one move. @@ -93,7 +99,7 @@ class AgentLogsScreenTest { waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) // An address is an address of one heap dump, so the same one here is a different object — or no - // object at all. Read it, don't follow it. + // object at all. Read it as the agent wrote it, and don't follow it. onNodeWithText("Described ${hex(activityObjectId())}").assertHasNoClickAction() } } @@ -152,6 +158,12 @@ class AgentLogsScreenTest { private fun activityObjectId() = heapDump.activityObjectIds.first() + /** How the window names the activity: the same title the tab a row opens carries. */ + private fun activityName() = + "${LEAKING_ACTIVITY_CLASS_NAME.substringAfterLast('.')} ${hexObjectId(activityObjectId())}" + + private fun describedRow() = "Described ${activityName()}" + private fun hex(objectId: Long) = exactHexObjectId(objectId) /** From 2f96751de42f505c5832e8ed2e03a4638516976b Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 08:54:14 +0200 Subject: [PATCH 06/27] Name the solved leak above the chain, and to an agent MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A chain that names a faulty reference is an investigation that is over, and until now the only way to see that was to find the marked step: a real chain is tens of steps, and the pane is scrolled to the last of them. So the answer is also said where the eye starts, as a two line `Leak solved` section above the list, and handed to an agent as `faultyReference` at the top of the chain rather than left to be found by scanning the steps for isFaulty. Both read the same spelling, `PathReference.leakLabel()`, which the leaks screen and the note `conclude` writes now use too — four surfaces naming one leak four ways is four leaks to whoever is grepping. Co-Authored-By: Claude Opus 5 --- docs/shark-explorer-changelog.md | 4 +- docs/shark-explorer.md | 12 ++ shark/shark-explorer/AGENTS.md | 9 ++ .../java/shark/explorer/agent/AgentJson.kt | 7 + .../java/shark/explorer/agent/AgentTools.kt | 17 ++- .../shark/explorer/agent/AgentToolsTest.kt | 7 + .../java/shark/explorer/app/RootPathPanel.kt | 120 ++++++++++++------ .../explorer/app/LeakStatusSectionTest.kt | 13 +- .../java/shark/explorer/DominatorPaths.kt | 19 +++ .../shark/explorer/HeapDominatorTreemap.kt | 25 +--- .../src/main/java/shark/explorer/RootPath.kt | 14 ++ .../java/shark/explorer/HeapLeakStatusTest.kt | 13 +- 12 files changed, 186 insertions(+), 74 deletions(-) diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index eccef4bec6..44bcc4377e 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -55,5 +55,7 @@ uses, without the one for a newly recognized library leak: `Stuck` one reads `Holder.activity · faulty reference`, which is the leak itself rather than one of the objects it left behind, and the same reference the **Leaks** screen names that leak after. A chain whose two verdicts are further apart than one step carries no mark, since which reference in between is at fault - is what isn't known — overrule a verdict in between and the mark appears. + is what isn't known — overrule a verdict in between and the mark appears. Once there is a mark, a + `Leak solved` line above **What holds it** names that reference where the eye starts, rather than leaving + it to be found tens of steps down a chain, and an agent reading the chain is answered with the same name. See [The verdict](shark-explorer.md#the-verdict). diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index c7fc9adbaa..7cc3b83232 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -173,6 +173,18 @@ where to go and change code — the shades on the objects are what the leak left it is the same reference the Leaks screen names that leak after, so a row there and the chain you open from it name one thing. +**And when it has one, `Leak solved` says so above the chain**, with the reference under it and nothing else: + +``` +Leak solved +Holder.activity +``` + +Because a real chain is tens of steps and **What holds it** is scrolled to the last of them, so a mark +somewhere in the middle is an answer you have to go looking for. The name is the one to go and grep for, and +it is the same string the Leaks screen, a note written by `conclude`, and an agent's `faultyReference` all +use. + **A chain with no such step carries no mark**, which is deliberate: what would be marked would be a guess drawn as an answer. With objects nothing knows either way about between the two verdicts, the fault is at one of those steps and nothing on the chain says which. With nothing `Expected` above the stuck object at all, diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index e6027525e9..82138c612b 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -99,6 +99,15 @@ fingerprint — so it is tempting to mark the top of it, which is what the first for being where the walk started. A longer stretch is a fault at one of several steps with nothing saying which, and nothing drawn is the answer for that. +**And it is named in one place, `PathReference.leakLabel()`.** Four surfaces say which reference a leak is — +the row of the leaks screen, the `Leak solved` section above the chain, the `faultyReference` an agent is +answered with, and the note `conclude` writes — and a leak named `Holder.activity` by one of them and +`Holder#activity` by another is two leaks to whoever is reading, or grepping. `Owner.field` is not the whole +of it either: an array entry loses its index (`Object[][x]`, since which slot a leak was in is no part of +what it is) and a reference from a running method has no name of its own. `RootPath.faultyReference()` is the +matching single answer to "does this chain name one?", which is what both the window's section and the agent's +field are. + Someone reading a heap dump can overrule what the inspectors made of an object, and the statuses they set are a `LeakStatusOverrides` **passed into every question whose answer they change** — `summarize`, `rootPathTo`, `independentPathsBetween`, `independentPathsFromRoots`, `findLeaks`, `isBelowLeakingObject` — diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt index 15afd8ed1d..0feaf46da6 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt @@ -22,6 +22,8 @@ import shark.explorer.ReachabilityStrength import shark.explorer.RootPath import shark.explorer.RootPathStep import shark.explorer.exactHexObjectId +import shark.explorer.faultyReference +import shark.explorer.leakLabel /** * How the explorer's own model reads as JSON, which is the whole of what an agent sees of a heap dump. @@ -129,6 +131,11 @@ internal object AgentJson { fun rootPath(path: RootPath): JsonObject = buildJsonObject { put("gcRoot", path.gcRootLabel) put("stepCount", path.steps.size) + // What the chain is for, said once at the top rather than left to be found by scanning the steps for + // isFaulty — and in the words the window names the leak with, so that an answer handed to a person + // matches the section they are reading it under. Null until the verdicts either side of one reference + // are both set, which is the state an investigation is working towards. + put("faultyReference", path.faultyReference()?.leakLabel()) putJsonArray("steps") { path.steps.forEach { add(rootPathStep(it)) } } } diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index fc0d2ccf09..6401d52443 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -16,6 +16,7 @@ import shark.explorer.Place import shark.explorer.RootPath import shark.explorer.RootPathStep import shark.explorer.exactHexObjectId +import shark.explorer.leakLabel import shark.explorer.leakStatusConflictsWith /** @@ -124,9 +125,11 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { name = "chain_from_gc_root", description = "The shortest chain of references from a GC root down to this object, which is where the " + "leak is. Every step carries its verdict, the reason for it, the inspectors' labels and the field " + - "the step above points through. A reference marked isFaulty is the one the heap dump says is at " + - "fault; while none is, the chain does not yet name a single reference. Steps marked isDominator are " + - "the ones every path to the object goes through.", + "the step above points through. faultyReference is what the chain names the leak — the one reference " + + "to go and change, also marked isFaulty on the step it reaches, and shown as `Leak solved` above the " + + "chain in the window. It is null while the verdicts don't yet cross from EXPECTED to STUCK at a " + + "single reference, which is the state an investigation works towards and what conclude requires. " + + "Steps marked isDominator are the ones every path to the object goes through.", schema = schema(WINDOW to window(), OBJECT to objectId("The object to walk up from.")) ) { arguments -> val dump = arguments.heapDump() @@ -363,7 +366,7 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { ) val reference = requireNotNull(faulty.step.reference) val note = conclusionNote( - reference = "${reference.ownerClassName}.${reference.name}", + reference = reference.leakLabel(), rootCause = arguments.string(ROOT_CAUSE), howToReproduce = arguments.optionalString(HOW_TO_REPRODUCE), notChecked = arguments.optionalString(NOT_CHECKED), @@ -375,7 +378,9 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { put("concluded", true) putJsonArray("faultyReference") { addJsonObject { - put("reference", "${reference.ownerClassName}.${reference.name}") + // The words the window names this leak with, so that an answer an agent gives its human and the + // section at the top of the chain they are looking at are the same string. + put("reference", reference.leakLabel()) put("declaredIn", reference.ownerClassName) put("field", reference.name) put("heldObject", exactHexObjectId(faulty.step.objectId)) @@ -650,7 +655,7 @@ private fun RootPath.verdictState(): ChainVerdicts { ) return ChainVerdicts( faultyStep = faulty, - summary = "${reference.ownerClassName}.${reference.name} is the faulty reference: the one step from " + + summary = "${reference.leakLabel()} is the faulty reference: the one step from " + "an object meant to be in memory to one that should be gone." ) } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index 91da51be7e..d4df8747d7 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -2,6 +2,7 @@ package shark.explorer.agent import kotlinx.coroutines.runBlocking import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonNull import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.add import kotlinx.serialization.json.buildJsonArray @@ -156,6 +157,9 @@ class AgentToolsTest { ) assertThat(steps.mapNotNull { it["reference"]?.jsonObject?.text("isFaulty") }) .containsOnly("false") + // Which is the field an agent reads to know whether it is done, so an unsolved chain has to leave it + // out rather than answer with something that could be mistaken for a name. + assertThat(answer.obj("chain")["faultyReference"]).isEqualTo(JsonNull) assertThat(answer.text("whatTheChainSays")) .contains("1 step(s)") .contains(hex(heapDump.holderObjectId)) @@ -205,6 +209,9 @@ class AgentToolsTest { .single { it.jsonObject["reference"]?.jsonObject?.text("isFaulty") == "true" } .jsonObject assertThat(faulty.text("object")).isEqualTo(hex(heapDump.activityObjectId)) + // And named at the top of the chain in the same words the window's `Leak solved` section uses, so that + // an agent quoting it to its human names what the human is looking at. + assertThat(answer.obj("chain").text("faultyReference")).isEqualTo(FAULTY_REFERENCE) } @Test diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/RootPathPanel.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/RootPathPanel.kt index c7613198a1..167cf34b59 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/RootPathPanel.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/RootPathPanel.kt @@ -13,6 +13,7 @@ import androidx.compose.foundation.lazy.LazyListScope import androidx.compose.foundation.lazy.itemsIndexed import androidx.compose.foundation.lazy.rememberLazyListState import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.material3.HorizontalDivider import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Surface import androidx.compose.material3.Text @@ -27,11 +28,15 @@ import shark.explorer.DrawnRootPath import shark.explorer.HEAD_INDEX import shark.explorer.HeapDominatorTreemap import shark.explorer.HeapObjectSummary +import shark.explorer.LeakStatus +import shark.explorer.PathReference import shark.explorer.RootPath import shark.explorer.RootPathStep import shark.explorer.RootPathWay import shark.explorer.detours import shark.explorer.drawnWith +import shark.explorer.faultyReference +import shark.explorer.leakLabel import shark.explorer.stepsAfter import shark.explorer.stepsBelow @@ -117,51 +122,80 @@ internal fun RootPathPanel( } } Surface(modifier, color = MaterialTheme.colorScheme.surface) { - // A row per object, drawn only where the pane has the room for it: a chain is as long as the heap dump - // makes it, and the linked structures of a real one run to hundreds of steps — a pane that composed all - // of them would take a minute to draw a chain nobody has scrolled to yet. - LazyColumn(state = listState, contentPadding = PaddingValues(12.dp)) { - item { - // Where every chain starts, and the way back to the screen the window opens on. - Column { - PathRootRow( - nextStrength = drawn?.path?.steps?.firstOrNull()?.step?.strength - ?: cutTail?.firstOrNull()?.step?.strength, + Column { + drawn?.path?.faultyReference()?.let { SolvedLeak(it) } + // A row per object, drawn only where the pane has the room for it: a chain is as long as the heap dump + // makes it, and the linked structures of a real one run to hundreds of steps — a pane that composed all + // of them would take a minute to draw a chain nobody has scrolled to yet. + LazyColumn(state = listState, contentPadding = PaddingValues(12.dp)) { + item { + // Where every chain starts, and the way back to the screen the window opens on. + Column { + PathRootRow( + nextStrength = drawn?.path?.steps?.firstOrNull()?.step?.strength + ?: cutTail?.firstOrNull()?.step?.strength, + onOpen = onOpen, + onCopyLink = onCopyLink + ) + Spacer(Modifier.height(BLOCK_SPACING)) + } + } + if (drawn != null) { + rootPathTrace( + drawn = drawn, + stronglyReachableByteCount = stronglyReachableByteCount, + ways = ways, + chosenWays = chosenWays, + onChooseWay = onChooseWay, onOpen = onOpen, onCopyLink = onCopyLink ) - Spacer(Modifier.height(BLOCK_SPACING)) + } else { + noChainText(selection, summary, isWholeHeapDump, rootPath, hasTail = cutTail != null)?.let { + item { Text(it, style = MaterialTheme.typography.bodySmall) } + } } - } - if (drawn != null) { - rootPathTrace( - drawn = drawn, - stronglyReachableByteCount = stronglyReachableByteCount, - ways = ways, - chosenWays = chosenWays, - onChooseWay = onChooseWay, - onOpen = onOpen, - onCopyLink = onCopyLink - ) - } else { - noChainText(selection, summary, isWholeHeapDump, rootPath, hasTail = cutTail != null)?.let { - item { Text(it, style = MaterialTheme.typography.bodySmall) } + if (tail != null) { + hoveredTail( + steps = tail, + isCut = false, + stronglyReachableByteCount = stronglyReachableByteCount + ) + } else if (cutTail != null) { + // Nothing above it on screen is what holds it, so the end of it is the object being described here. + hoveredTail( + steps = cutTail, + isCut = true, + stronglyReachableByteCount = stronglyReachableByteCount + ) } } - if (tail != null) { - hoveredTail( - steps = tail, - isCut = false, - stronglyReachableByteCount = stronglyReachableByteCount - ) - } else if (cutTail != null) { - // Nothing above it on screen is what holds it, so the end of it is the object being described here. - hoveredTail( - steps = cutTail, - isCut = true, - stronglyReachableByteCount = stronglyReachableByteCount - ) - } + } + } +} + +/** + * That the chain below is solved, and which reference solved it: two lines above everything else. + * + * The chain already marks that reference where it sits, and a reader still has to find it: a real chain is + * tens of steps, this pane is scrolled to the bottom of it, and the answer is somewhere in the middle. So the + * answer is also said where the eye starts, in the words the leaks screen names the leak with — a reader who + * has got this far is looking for a name to go and grep for, not for another paragraph. + * + * Above the list rather than as its first row, because the list scrolls itself to the end every time the + * pointer moves: a row at the top of it is a row that scrolls away. + */ +@Composable +private fun SolvedLeak(faultyReference: PathReference) { + Hint(FAULTY_REFERENCE_HINT) { + Column(Modifier.padding(start = 12.dp, top = 12.dp, end = 12.dp)) { + Text(LEAK_SOLVED, style = MaterialTheme.typography.labelSmall, color = MUTED_TEXT) + Text( + faultyReference.leakLabel(), + style = MaterialTheme.typography.bodyMedium, + color = LeakStatus.STUCK.textColor + ) + HorizontalDivider(Modifier.padding(top = 8.dp)) } } } @@ -318,6 +352,14 @@ private fun WaysOfDetour( } } +/** + * What the section above the chain is called, when there is a reference to name under it. + * + * Past tense, and about the leak rather than about the reader: a chain with a faulty reference on it is + * solved by the heap dump and the verdicts together, whoever set them and whenever. + */ +internal const val LEAK_SOLVED = "Leak solved" + /** Shown until a rectangle has been clicked, which is what this pane draws a chain for. */ internal const val NO_ROOT_PATH_YET = "Click a rectangle to see what holds it." diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeakStatusSectionTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeakStatusSectionTest.kt index daa64dbba8..a50dbb5ba5 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeakStatusSectionTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LeakStatusSectionTest.kt @@ -215,23 +215,30 @@ class LeakStatusSectionTest { * Both halves in one window, because they are one answer: the reference is marked from the verdicts either * side of it, so a verdict set by hand is what puts the mark on the chain and what takes it off again. */ - @Test fun `the chain marks which reference the leak is, and a hand can take the mark off`() { + @Test fun `the chain names which reference the leak is, twice, and a hand can take that off`() { explorerUiTest { // Set in a run before this one, and what leaves a single reference below it: with nothing on this // chain known to belong in memory, the fault is at either of its two steps and neither is marked. openHeapDump(setAlready = { holderIsExpected() }) { it.activityObjectId } onNodeWithText("$FAULTY_STEP $FAULTY_REFERENCE").assertIsDisplayed() + // And said again above the chain, which is not a duplicate: a real chain is tens of steps, this pane + // is scrolled to the last of them, and a mark somewhere in the middle is an answer to go looking for. + // The exact text is the section's, the mark on the chain having the words above after it. + onNodeWithText(LEAK_SOLVED).assertIsDisplayed() + onNodeWithText(FAULTY_STEP).assertIsDisplayed() changeStatus() choose(LeakStatus.EXPECTED) write(TYPED_REASON) set() - // Nothing on this chain is stuck any more, so there is no reference to point at — and the step is - // still drawn, which is the mark being about the leak rather than about the reference. + // Nothing on this chain is stuck any more, so there is no reference to point at and nothing is solved + // — and the step is still drawn, which is both of those being about the leak rather than the reference. waitUntilAtLeastOneExists(hasText(TYPED_REASON, substring = true), SAVE_TIMEOUT_MILLIS) onNodeWithText(FAULTY_REFERENCE, substring = true).assertDoesNotExist() + onNodeWithText(LEAK_SOLVED).assertDoesNotExist() + // Which now matches the step on the chain rather than the section that was above it. onNodeWithText(FAULTY_STEP).assertIsDisplayed() } } diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DominatorPaths.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DominatorPaths.kt index 5dfb8e3cd5..8cac6adb43 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DominatorPaths.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DominatorPaths.kt @@ -144,6 +144,25 @@ data class PathReference( val isFaulty: Boolean = false ) +/** + * A reference the way a leak is named after it: `MainActivity$2.this$0`, `Object[][x]`. + * + * By the class that *declares* the field rather than by the referrer's own class, and with an array index + * erased, since which slot an object sits in is no part of what makes a leak that leak. + * + * One spelling in one place because three surfaces say it and they have to agree: the row of the leaks + * screen, the section that names a solved leak at the top of the chain, and the `faultyReference` an agent + * is answered with. A leak named one way here and another way there is two leaks to whoever is reading. + */ +fun PathReference.leakLabel(): String = when (locationType) { + ReferenceLocationType.ARRAY_ENTRY -> "$ownerClassName[x]" + // The same words the chain pane draws for a reference from a running method, spelled again here rather + // than shared with it: what a leak is named after has to read the way the chain reads, and that is a + // string rather than a module's API. + ReferenceLocationType.LOCAL -> "$ownerClassName." + ReferenceLocationType.INSTANCE_FIELD, ReferenceLocationType.STATIC_FIELD -> "$ownerClassName.$name" +} + /** * A reference Shark recognizes as one that leaks in code the app doesn't control. See * [PathReference.libraryLeak]. diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDominatorTreemap.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDominatorTreemap.kt index eefb2fe3d1..bd91a5aa95 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDominatorTreemap.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDominatorTreemap.kt @@ -42,7 +42,6 @@ import shark.HprofRecord.HeapDumpRecord.ObjectRecord.PrimitiveArrayDumpRecord.Sh import shark.ObjectDominators.DominatorNode import shark.ObjectReporter import shark.AndroidObjectInspectors -import shark.ReferenceLocationType import shark.SharkLog import shark.ValueHolder.BooleanHolder import shark.ValueHolder.ByteHolder @@ -1105,7 +1104,7 @@ class HeapDominatorTreemap internal constructor( // // Null when nothing holds the object at all, which is what leaves an unreachable leak named after its // class: there is no reference left to name it after. - val weakenedBy = steps.firstOrNull { it.strength >= strength }?.reference?.genericLabel() + val weakenedBy = steps.firstOrNull { it.strength >= strength }?.reference?.leakLabel() if (weakenedBy == null && steps.isNotEmpty()) { SharkLog.d { "Nothing on the chain to ${hexObjectId(objectId)} holds it as weakly as $strength does, so it is " + @@ -1189,20 +1188,7 @@ class HeapDominatorTreemap internal constructor( * marks nothing, since which of those references is at fault is exactly what isn't known. */ private fun suspectSubpath(steps: List): List = - steps.suspectReferenceIndexes().map { steps[it].reference!!.genericLabel() } - - /** - * A reference spelled the way the chain pane spells it — `Tile.view`, `Object[][x]` — with an array index - * erased, since which slot an object is in is no part of what makes a leak that leak. - * - * Spelled that way because a leak is named after one of these, and the name is only worth anything if it - * is the same string as the step someone then goes looking for on the chain. - */ - private fun PathReference.genericLabel(): String = when (locationType) { - ReferenceLocationType.ARRAY_ENTRY -> "$ownerClassName[x]" - ReferenceLocationType.LOCAL -> "$ownerClassName.$LOCAL_VARIABLE" - ReferenceLocationType.INSTANCE_FIELD, ReferenceLocationType.STATIC_FIELD -> "$ownerClassName.$name" - } + steps.suspectReferenceIndexes().map { steps[it].reference!!.leakLabel() } /** One leaking object and which leak it is an instance of, before the instances are gathered. */ private class FoundLeak( @@ -1836,13 +1822,6 @@ class HeapDominatorTreemap internal constructor( */ private const val MAX_LEAKING_OBJECTS = 500 - /** - * What a reference from a running method is called, since it has no name of its own. The same words - * the chain pane uses, duplicated rather than shared: a leak named after a reference has to be named - * the way the chain spells it, and that is a string, not a module's API. - */ - private const val LOCAL_VARIABLE = "" - /** * How many ways of holding an object [independentPathsBetween] spells out. Six chains is already more * than fits in a panel, and an object held from more places than that is held by a data structure diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/RootPath.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/RootPath.kt index 5f0e882feb..6e9c36a7e0 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/RootPath.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/RootPath.kt @@ -53,6 +53,20 @@ fun RootPath.stepsBelow(rootNodeId: Long): List { return steps.subList(fromIndex, steps.size) } +/** + * The one reference this chain is the leak *of*, and null for a chain that isn't a solved leak. + * + * Which is what a chain is read for: the objects on it say what is still in memory, and this says what to + * go and change. A reader who has set the two verdicts either side of one reference has finished — the heap + * dump has no more to add — so this being non-null is the same fact as the investigation being over. + * + * At most one, because [PathReference.isFaulty] is set for the single crossing from expected to stuck and + * for nothing else, so there is no need for a caller to decide between two of them. See + * [faultyReferenceIndexOrNull] for the rule and for the three ways a chain has none. + */ +fun RootPath.faultyReference(): PathReference? = + steps.firstNotNullOfOrNull { step -> step.step.reference?.takeIf { it.isFaulty } } + /** * The part of this chain below [objectId], or null when no step of it is that object. * diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeakStatusTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeakStatusTest.kt index e1c659ec1e..ce4783519b 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeakStatusTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapLeakStatusTest.kt @@ -110,6 +110,7 @@ class HeapLeakStatusTest { // Nothing on this chain is known to belong in memory — a holder nothing knows either way about, then // two destroyed objects — so the fault is at one of two steps and the chain marks neither. assertThat(explorer.tree.rootPathTo(dump.windowObjectId).faultyReferences()).isEmpty() + assertThat(explorer.tree.rootPathTo(dump.windowObjectId).faultyReference()).isNull() val path = explorer.tree.rootPathTo( objectId = dump.windowObjectId, @@ -120,6 +121,9 @@ class HeapLeakStatusTest { // it is still holding is the leak, and there is now nothing else it could be. Named after the class // that declares the field, which is the framework's `Activity` rather than the app's subclass of it. assertThat(path.faultyReferences()).containsExactly("Activity.mWindow") + // The same reference, through the one call the window's `Leak solved` section and an agent's + // `faultyReference` both go through: a chain either names the leak or it doesn't. + assertThat(path.faultyReference()?.leakLabel()).isEqualTo("Activity.mWindow") } } @@ -428,11 +432,16 @@ class HeapLeakStatusTest { reason: String ) = LeakStatusOverrides.of(listOf(override(objectId, status, reason))) - /** The references of a chain marked as the leak, spelled the way a leak of the leaks screen is named. */ + /** + * The references of a chain marked as the leak, spelled the way a leak of the leaks screen is named. + * + * Every step rather than [RootPath.faultyReference], which stops at the first: a chain marking two of them + * would leave the window and the agent naming one leak each, and this is what would notice. + */ private fun RootPath.faultyReferences(): List = steps.mapNotNull { it.step.reference } .filter { it.isFaulty } - .map { "${it.ownerClassName}.${it.name}" } + .map { it.leakLabel() } companion object { /** An address no dump these tests write has an object at, since they start at 1. */ From 2873d2a74a2aedc924563d5632bb3ff732ca5047 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 09:24:29 +0200 Subject: [PATCH 07/27] Let an agent reach every screen and press every button MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The agent surface answered questions about a heap dump somebody else had opened. Which meant an agent that needed the treemap, or a second dump, or a dump off a device had one answer available: ask your human to click something. So the five tools that were missing: - `dominator_tree`, the treemap without the pixels. Bounded in both directions, because the answer is a tree and every node is a read. - `read_notes`, and `take_note` gaining `replace`. A surface that could only append leaves a wrong conclusion in the notes of the object it is wrong about, under a correction — and the next reader finds both. - `open_heap_dump` and `list_devices`/`dump_heap`, which are the two buttons above the map. The two that make a window answer once the dump can be *read*, not once the window exists: everything else here is a read, so a window id handed over mid-index refuses every call made with it. That needs a failed open to be visible outside the composition, hence `ExplorerWindow.openProblem` — and without it a file that was never a heap dump is a call that never comes back. `AgentHeapDumps` stops being a `fun interface`; the tests get a fake of it rather than a lambda. The place vocabulary moves to `AgentPlace.kt`, both directions in one file, which is also what keeps `AgentTools` under detekt's class size. Re-measured what the surface costs a client: 16 tools, 18,880 characters of definitions, up from 13,116 for eleven. --- docs/shark-explorer-changelog.md | 7 +- docs/shark-explorer.md | 19 +- shark/shark-explorer/notes/agent-surface.md | 18 +- .../shark-explorer-agent/AGENTS.md | 21 ++ .../shark/explorer/agent/AgentHeapDump.kt | 67 ++++- .../java/shark/explorer/agent/AgentJson.kt | 53 ++++ .../java/shark/explorer/agent/AgentPlace.kt | 74 ++++++ .../shark/explorer/agent/AgentSessionFile.kt | 20 +- .../java/shark/explorer/agent/AgentTools.kt | 231 +++++++++++++++--- .../shark/explorer/agent/AgentServerTest.kt | 2 +- .../explorer/agent/AgentSessionFileTest.kt | 2 +- .../explorer/agent/AgentStdioBridgeTest.kt | 2 +- .../shark/explorer/agent/AgentToolsTest.kt | 173 ++++++++++++- .../shark/explorer/agent/FakeAgentHeapDump.kt | 61 +++++ .../shark/explorer/agent/McpSessionTest.kt | 9 +- .../java/shark/explorer/app/ExplorerAgents.kt | 211 +++++++++++++--- .../java/shark/explorer/app/ExplorerWindow.kt | 33 ++- .../src/main/java/shark/explorer/app/Main.kt | 47 +++- .../java/shark/explorer/DominatorOutline.kt | 89 +++++++ .../src/main/java/shark/explorer/NoteFile.kt | 53 +++- 20 files changed, 1078 insertions(+), 114 deletions(-) create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentPlace.kt create mode 100644 shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DominatorOutline.kt diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index 44bcc4377e..aa866718f3 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -42,8 +42,11 @@ uses, without the one for a newly recognized library leak: you have open — the same tree, the same verdicts, the same notes — and `show` puts what it is looking at on your screen. What it can be held to is the point: every call has to say why it was made and lands in the run's log beside the reads it caused, a verdict needs a reason another reader can check exactly as yours - does, and reporting a root cause is refused until the chain names one faulty reference. Point any MCP client - at the installed app with `--mcp-stdio`. + does, and reporting a root cause is refused until the chain names one faulty reference. There is no screen + it can't reach and no button it can't press — the treemap as a tree of retained sizes, the notes read and + rewritten as well as added to, `Open heap dump…` for a file nobody has open, and `Take heap dump…` down to + picking the process off a device — because a surface with less than that answers "ask your human to click + something". Point any MCP client at the installed app with `--mcp-stdio`. See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). * ✨ **Agent logs**: every agent that has connected to the app is a row on a screen of its own, and opening one is everything that agent did — what each call did, which object it did it to, and the sentence it gave diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 7cc3b83232..11fe7e388b 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -248,8 +248,8 @@ Point your client at the app itself: That is the app's own launcher, and `--mcp-stdio` makes this copy of it a pipe to the window already open rather than a second window. Nothing else to install and no port to configure: it talks to the run that started most recently, says which one that was, and takes `--agent-run=` when several explorers are -open. Open a heap dump before you start — with no window there is nothing to investigate, and it says so -rather than waiting. +open. A heap dump open before you start is one less thing for it to do, but not a requirement: with no +window it says so, and `open_heap_dump` and `dump_heap` are how it gets one. Then ask for what you actually want. This is the whole prompt the session below was given: @@ -261,18 +261,29 @@ leak is — one bad reference, the three zones of a chain, the rules that spread the order that finds it, which is [the LeakCanary method](https://engineering.block.xyz/blog/the-leakcanary-method) as the tools enforce it. +**Everything the window can do, it can do** — there is no screen an agent can't reach and no button it can't +press, because a surface with less than that is one whose answer is "ask your human to click something": + | Tool | What it is | | --- | --- | | `open_heap_dumps` | Every window and what is open in it, with the method to follow. | | `list_leaks` | The **Leaks** screen: what this heap dump says shouldn't be there. | | `chain_from_gc_root` | One chain, every step with its labels and its verdict. | | `describe_object` | What an object is: its class, fields, labels, size. | -| `ways_held` | Every way an object is held, rather than the one chain. | +| `ways_held` | Every way an object is held, rather than the one chain — the *X ways from here* list. | | `find_objects` | The object list, by class name. | +| `dominator_tree` | The treemap, without the pixels: where the memory has gone, a level at a time. | | `set_verdict`, `clear_verdict` | The pencil, with the reason required the same way. | -| `take_note` | The notes, appended to. | +| `read_notes`, `take_note` | The notes: where somebody has been, what they wrote, and adding to or replacing it. | | `show` | Opens a tab in your window and brings it to the front. | | `conclude` | The root cause, and the only way to finish. | +| `open_heap_dump` | **Open heap dump…**, for a file nobody has open yet. | +| `list_devices`, `dump_heap` | **Take heap dump…**: which device, which process, and the dump itself. | + +The last three are what make an agent useful when there is nothing open yet: point it at a dump a bug report +came with, or at a process on a device, and the window it lands in is one you can look over its shoulder in. +`dump_heap` takes minutes on a large app and answers once the dump can be read — the steps are in the run's +log while it works. **And the tools refuse.** That is the part worth knowing about, because it is what an agent's confidence cannot argue with: diff --git a/shark/shark-explorer/notes/agent-surface.md b/shark/shark-explorer/notes/agent-surface.md index 85216449f0..b8ff23bc55 100644 --- a/shark/shark-explorer/notes/agent-surface.md +++ b/shark/shark-explorer/notes/agent-surface.md @@ -9,15 +9,17 @@ Measured off `AgentTools.all` and `AgentMethod.INSTRUCTIONS`, one `tools/list` e | | Characters | ≈ tokens | Paid | | --- | --- | --- | --- | -| Eleven tool definitions | 13,116 | 3,300 | Every turn, while the server is connected | -| The method | 4,970 | 1,240 | Handshake, and again with `open_heap_dumps` | +| Sixteen tool definitions | 18,880 | 4,720 | Every turn, while the server is connected | +| The method | 5,065 | 1,270 | Handshake, and again with `open_heap_dumps` | -So the standing cost of this surface is **4 to 6 k tokens**, 2 to 3% of a 200 k window. The published -horror stories are an order of magnitude worse — GitHub's server is ~17.6 k tokens of definitions, three -servers together have been measured at 143 k — and the mitigations that shipped in 2026 (Anthropic's tool -search, code execution over MCP) are aimed at that scale. **This surface is not where a context window goes -to die**, and a per-tool cost of ~300 tokens is what buys descriptions that say when to reach for a tool. -Worth re-measuring when the tool count doubles, which the parity work will do. +So the standing cost of this surface is **5 to 6 k tokens**, around 3% of a 200 k window. Parity took the +tool count from eleven to sixteen and the definitions from 13,116 characters to 18,880 — **a fifth of the +window's budget for the five tools that mean an agent never has to ask its human to click something**, which +is the trade this surface exists to make. The published horror stories are still an order of magnitude worse: +GitHub's server is ~17.6 k tokens of definitions, and three servers together have been measured at 143 k. The +mitigations that shipped in 2026 (Anthropic's tool search, code execution over MCP) are aimed at that scale. +**This surface is not where a context window goes to die**, and a per-tool cost of ~300 tokens is what buys +descriptions that say when to reach for a tool. Re-measure it if the count doubles again. ## What each shape is actually good at diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index e60f97b704..b89860349a 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -14,6 +14,7 @@ being talked to by a program that is not this app. | --- | --- | | `AgentHeapDump.kt` | The seam: one open heap dump, as everything here sees it. The app implements it over a window; the tests implement it over a `HeapExplorer` and three fields. | | `AgentTools.kt` | Every tool, each a name, a schema and one read. Where the refusals are. | +| `AgentPlace.kt` | Where a tab is, as one string an agent can be answered with and hand back. Both directions. | | `AgentMethod.kt` | The method, as prose handed to the model twice. | | `AgentJson.kt` | The explorer's model as JSON. | | `AgentTool.kt` | One tool, its arguments read strictly, and `AgentRefusal`. | @@ -105,6 +106,26 @@ that can read `~/.shark-explorer` can read any heap dump on the disk anyway. `AgentServer.serve` sets **no read timeout**, unlike the link socket. An agent thinking is a quiet connection. +## Everything the window can do, this can do + +`AgentTools` covers every screen and every button, `Take heap dump…` included, and that is a rule rather than +how far it happened to get. A surface that can read a heap dump but not open one answers "ask your human to +click something", which is the opposite of the point — so a capability added to the window is a tool added +here, and the same the other way round. + +Two consequences worth knowing before adding one. + +**A tool that makes a window is answered once the dump is *readable*.** `AgentHeapDumps.open` and `dumpHeap` +hand back an `AgentHeapDump`, not a path or a window id, because everything else on this surface is a read: an +id handed over while the dump is still being indexed is one that refuses every call made with it. The app's +side waits on three outcomes — open, failed to open, window closed — which is why `ExplorerWindow` publishes +`openProblem` beside `openHeapDump`. Waiting on "opened" alone means a file that was never a heap dump is a +call that never comes back. + +**A tool that reaches `adb` is minutes, and says so in the log rather than in the answer.** There is nothing to +stream progress through — an agent is waiting on one JSON object — so `~/.shark-explorer/logs` is where a dump +that is still being pulled says how far it has got. + ## An address is a string, never a JSON number A heap dump's addresses fill the range of `Long`, and a JSON number is a double to most clients of this diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt index 5a35421bdf..6d2f9ee7c1 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt @@ -1,5 +1,8 @@ package shark.explorer.agent +import java.io.File +import shark.explorer.AndroidDevice +import shark.explorer.DeviceProcess import shark.explorer.HeapExplorer import shark.explorer.LeakStatusOverride import shark.explorer.LeakStatusOverrides @@ -61,6 +64,31 @@ interface AgentHeapDump { text: String ) + /** + * Puts [text] in the note of [place] instead of what is there, which is what editing one is. + * + * Beside appending rather than instead of it, because the two are different acts: an agent adding what it + * found is appending, and an agent that has been told its own paragraph was wrong is editing. A surface + * that could only append would leave a wrong conclusion in the notes of the object it is wrong about, + * under a correction — and the next reader finds both. + */ + suspend fun replaceNote( + place: Place, + text: String + ) + + /** What is in the note of [place], and empty for a place nobody has written about. */ + suspend fun readNote(place: Place): String + + /** + * Every place of this heap dump with a note, which is what the window marks a tab for. + * + * The listing rather than the notes, like the tab strip: what it answers is where somebody has been, and + * reading what they wrote is a call per place. Places from a newer version of the app, or from a screen + * a caller cannot be sent to, are left out — see `shark.explorer.placeOfNoteKeyOrNull`. + */ + suspend fun notedPlaces(): List + /** * Opens [place] in a tab of this window and brings the window to the front, which is what makes an agent's * work something the person at the machine can watch rather than read about afterwards. @@ -72,13 +100,44 @@ interface AgentHeapDump { } /** - * The open heap dumps of this run, which is what a connection asks before anything else. + * The heap dumps of this run, open and openable, which is what a connection asks before anything else. * - * Windows come and go while an agent is connected, so this is asked per call rather than captured: a tool - * naming a window that has since closed is an error message, not a stale answer. + * Windows come and go while an agent is connected, so [openHeapDumps] is asked per call rather than + * captured: a tool naming a window that has since closed is an error message, not a stale answer. + * + * The rest of it is everything the app can be asked for that isn't about a dump it already has open — + * opening another one, and taking one off a device — which is the same thing as **everything the buttons + * above the map can do**. An agent that can read a heap dump but not open one is an agent that has to ask + * its human to click something, which is the opposite of what this surface is for. */ -fun interface AgentHeapDumps { +interface AgentHeapDumps { /** Every window with a heap dump open, in the order they were opened. */ fun openHeapDumps(): List + + /** + * Opens [file] in a window of this app and answers once it can be read. + * + * Once it can be *read*, rather than once the window exists: everything else here is a read, so an answer + * handed over before the dump is open would be a window id that refuses every call made with it. Which + * makes this the one call that takes as long as opening a heap dump takes. + */ + suspend fun open(file: File): AgentHeapDump + + /** Every device `adb` is connected to, whether or not a heap dump could be taken off it. */ + suspend fun devices(): List + + /** The processes of one device that belong to an installed app, which are the ones worth dumping. */ + suspend fun processesOf(serialNumber: String): List + + /** + * Takes a heap dump of a process, opens it in a window of this app, and answers once it can be read. + * + * Minutes, on a large app: a dump is written on the device, waited for, pulled, and then opened. The + * steps land in this run's log as they happen, which is where to look while this hasn't come back. + */ + suspend fun dumpHeap( + serialNumber: String, + processName: String + ): AgentHeapDump } diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt index 0feaf46da6..616a32fa57 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt @@ -10,6 +10,9 @@ import kotlinx.serialization.json.buildJsonObject import kotlinx.serialization.json.put import kotlinx.serialization.json.putJsonArray import kotlinx.serialization.json.putJsonObject +import shark.explorer.AndroidDevice +import shark.explorer.DeviceProcess +import shark.explorer.DominatorOutline import shark.explorer.HeapLeaks import shark.explorer.HeapObjectSummary import shark.explorer.HeapSizes @@ -127,6 +130,56 @@ internal object AgentJson { put("hiddenFieldCount", summary.hiddenFieldCount) } + /** + * The dominator tree in outline: what holds the most memory, and what holds the most of that. + * + * The treemap, without pixels. Which is the answer to "where has the memory gone" rather than to "why is + * this object still here" — an agent that starts from a leak never needs this, and one asked why an app is + * using 400 MB has nowhere else to start. + */ + fun dominatorOutline(outline: DominatorOutline): JsonObject = buildJsonObject { + put("node", exactHexObjectId(outline.nodeId)) + put("label", outline.label) + put("retainedBytes", outline.retainedSize) + put("strength", outline.strength.name) + // A pile of objects rather than one of them, which is what the top of every tree is mostly made of. + put("objectCount", outline.objectCount) + put("className", outline.className) + // Against the children handed back, so that "this is all of it" and "this is the biggest few of it" are + // never the same answer. + put("dominatedNodeCount", outline.childCount) + putJsonArray("dominates") { outline.children.forEach { add(dominatorOutline(it)) } } + } + + /** Every device `adb` is connected to, and whether a heap dump could be taken off each. */ + fun devices(devices: List): JsonArray = buildJsonArray { + devices.forEach { device -> + addJsonObject { + put("device", device.serialNumber) + put("description", device.description) + put("state", device.state) + put("sdkInt", device.sdkInt) + put("model", device.model) + put("fingerprint", device.fingerprint) + // The difference between a device with two dumpable processes and one with all of them, and not a + // question about the app: a release build on a `userdebug` device can be dumped. + put("dumpsAnyProcess", device.dumpsAnyProcess) + } + } + } + + /** The processes of one device that belong to an installed app. */ + fun processes(processes: List): JsonArray = buildJsonArray { + processes.forEach { process -> + addJsonObject { + put("process", process.name) + put("processId", process.processId) + // The system's own apps are dumpable only on a debuggable build, and are thirty of these. + put("isSystemApp", process.isSystemApp) + } + } + } + /** The shortest chain from a GC root down to an object, with dominators and the faulty reference marked. */ fun rootPath(path: RootPath): JsonObject = buildJsonObject { put("gcRoot", path.gcRootLabel) diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentPlace.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentPlace.kt new file mode 100644 index 0000000000..0c7d788117 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentPlace.kt @@ -0,0 +1,74 @@ +package shark.explorer.agent + +import shark.explorer.ObjectListFilter +import shark.explorer.Place +import shark.explorer.exactHexObjectId + +/** + * How a place of a heap dump is spelled to an agent, and read back off one. + * + * `shark.explorer.Place` is where a tab is, and this is its whole vocabulary as a single string an agent can + * be answered with and then hand back: `0x12d368b8`, `leaks`, `objects:android.graphics.Bitmap`. Both + * directions in one file because they are one spelling — a place written one way and read another is a place + * an agent can be told about and then cannot go to. + * + * Deliberately not `shark://` links, which name a place too: a link names a *window* as well, so it is the + * thing to hand to a person rather than the thing to pass back over this protocol. See + * [AgentTools] `show`. + */ +internal fun AgentArguments.place(): Place { + val text = string(PLACE) + return when { + text.startsWith(HEX_PREFIX) -> Place.Object(objectIdOf(PLACE, text)) + text == PLACE_LEAKS -> Place.Leaks() + text == PLACE_OBJECTS -> Place.Objects() + text == PLACE_STARRED -> Place.Starred + text == PLACE_AGENT_LOGS -> Place.AgentLogs + text.startsWith("$PLACE_AGENT_LOGS$PLACE_SEPARATOR") -> + Place.AgentLog(text.substringAfter(PLACE_SEPARATOR)) + text.startsWith("$PLACE_OBJECTS$PLACE_SEPARATOR") -> Place.Objects( + ObjectListFilter(query = text.substringAfter(PLACE_SEPARATOR)) + ) + else -> throw AgentRefusal("\"$text\" is no place of a heap dump. $PLACES_ARE") + } +} + +/** + * The same place written back, and null for one an agent has no way of naming. + * + * The one that can't be is the pile of objects a rectangle had no room for: which objects are in it follows + * from how wide the window is, so there is nothing here that would name it again on a screen of another size. + */ +internal fun placeText(place: Place): String? = when (place) { + is Place.Object -> exactHexObjectId(place.objectId) + is Place.Objects -> + if (place.filter.query.isEmpty()) { + PLACE_OBJECTS + } else { + "$PLACE_OBJECTS$PLACE_SEPARATOR${place.filter.query}" + } + is Place.Leaks -> PLACE_LEAKS + is Place.Starred -> PLACE_STARRED + is Place.AgentLogs -> PLACE_AGENT_LOGS + is Place.AgentLog -> "$PLACE_AGENT_LOGS$PLACE_SEPARATOR${place.sessionId}" + is Place.SmallerObjects -> null +} + +/** What every tool taking one says, so that a place is described the one way. */ +internal fun place() = string("Which place of the heap dump. $PLACES_ARE") + +internal const val PLACE = "place" + +private const val PLACE_LEAKS = "leaks" +private const val PLACE_OBJECTS = "objects" +private const val PLACE_STARRED = "starred" +private const val PLACE_AGENT_LOGS = "agent-logs" + +/** Between a screen and which of it, since three of these take one. */ +private const val PLACE_SEPARATOR = ":" + +/** Every place there is, said the one way, since a schema and a refusal both have to list them. */ +private const val PLACES_ARE = + "A place is an object's `0x…` address, \"$PLACE_LEAKS\", \"$PLACE_OBJECTS\", " + + "\"$PLACE_OBJECTS$PLACE_SEPARATOR\", \"$PLACE_STARRED\", \"$PLACE_AGENT_LOGS\" or " + + "\"$PLACE_AGENT_LOGS$PLACE_SEPARATOR\"." diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt index 39331b6b26..9d6c82887a 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt @@ -401,11 +401,25 @@ internal fun verbOfTool( "chain_from_gc_root" -> "Read the chain to" "ways_held" -> "Looked for every way of holding" "find_objects" -> "Searched for" + // Both of these are about the whole heap dump when they name nothing, so the verb has to stand on its + // own: a row reads as the verb and then the subject, and "Read the notes on" alone says nothing. + "dominator_tree" -> + if (SUBJECT_OBJECT in arguments) "Read the dominator tree under" else "Read the dominator tree" "set_verdict" -> "Recorded ${arguments[SUBJECT_VERDICT] ?: "a verdict"} on" "clear_verdict" -> "Took the verdict off" - "take_note" -> "Wrote a note on" + "read_notes" -> if (SUBJECT_PLACE in arguments) "Read the notes on" else "Read what has been written" + // Worth the difference on the screen: a note replaced is a paragraph that was there and isn't any more, + // which is the one thing an agent does here that a reader can't get back. + "take_note" -> if (arguments[SUBJECT_REPLACE] == "true") "Rewrote the note on" else "Wrote a note on" "show" -> "Showed" "conclude" -> "Concluded about" + // The app rather than a heap dump, so each of these says the whole of what it did: there is no subject + // to put after it, the heap dump it opens not existing as a place until it is open. + "open_heap_dump" -> "Opened ${arguments[SUBJECT_PATH] ?: "a heap dump"}" + "list_devices" -> arguments[SUBJECT_DEVICE] + ?.let { "Listed the processes of $it" } + ?: "Asked which devices are connected" + "dump_heap" -> "Dumped the heap of ${arguments[SUBJECT_PROCESS] ?: "a process"}" else -> null } @@ -413,3 +427,7 @@ private const val SUBJECT_OBJECT = "object" private const val SUBJECT_PLACE = "place" private const val SUBJECT_CLASS_NAME = "className" private const val SUBJECT_VERDICT = "verdict" +private const val SUBJECT_REPLACE = "replace" +private const val SUBJECT_PATH = "path" +private const val SUBJECT_DEVICE = "device" +private const val SUBJECT_PROCESS = "process" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index 6401d52443..c9f92a5794 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -1,11 +1,14 @@ package shark.explorer.agent +import java.io.File import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.add import kotlinx.serialization.json.addJsonObject import kotlinx.serialization.json.buildJsonObject import kotlinx.serialization.json.put import kotlinx.serialization.json.putJsonArray +import shark.explorer.DEFAULT_OUTLINE_CHILDREN +import shark.explorer.DEFAULT_OUTLINE_DEPTH import shark.explorer.HeapDominatorTreemap import shark.explorer.HeapObjectKind import shark.explorer.LeakStatus @@ -18,6 +21,8 @@ import shark.explorer.RootPathStep import shark.explorer.exactHexObjectId import shark.explorer.leakLabel import shark.explorer.leakStatusConflictsWith +import shark.explorer.nodeIdText +import shark.explorer.outlineOf /** * Everything an agent can do to an open heap dump, as MCP tools. @@ -47,11 +52,16 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { chainFromGcRoot(), waysHeld(), findObjects(), + dominatorTree(), setVerdict(), clearVerdict(), + readNotes(), takeNote(), show(), - conclude() + conclude(), + openHeapDump(), + listDevices(), + dumpHeap() ) fun byName(name: String): AgentTool? = all.firstOrNull { it.name == name } @@ -207,6 +217,43 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { AgentJson.objectList(list) } + private fun dominatorTree() = AgentTool( + name = "dominator_tree", + description = "Where the memory has gone: what holds the most of it, what holds the most of that, and " + + "so on. The tree the window draws as a treemap, without the pixels. Start at the whole heap dump and " + + "give `object` to walk down from one node. This answers \"why is this app using 400 MB\" — for " + + "\"why is this object still here\", read its chain instead.", + schema = schema( + WINDOW to window(), + OBJECT to objectId("Optional: the node to walk down from, the whole heap dump by default.") + .optional(), + MAX_DEPTH to integer( + "How many levels down, at most $MAX_OUTLINE_DEPTH. $DEFAULT_OUTLINE_DEPTH by default." + ).optional(), + MAX_CHILDREN to integer( + "How many of each node's biggest children to walk into, at most $MAX_OUTLINE_CHILDREN. " + + "$DEFAULT_OUTLINE_CHILDREN by default." + ).optional() + ) + ) { arguments -> + val dump = arguments.heapDump() + val nodeId = arguments.optionalObjectId(OBJECT) ?: HeapDominatorTreemap.ROOT_OBJECT_ID + val maxDepth = arguments.int(MAX_DEPTH, default = DEFAULT_OUTLINE_DEPTH) + .coerceIn(0, MAX_OUTLINE_DEPTH) + val maxChildren = arguments.int(MAX_CHILDREN, default = DEFAULT_OUTLINE_CHILDREN) + .coerceIn(1, MAX_OUTLINE_CHILDREN) + dump.read("the dominator tree under ${nodeIdText(nodeId)}, for an agent") { explorer -> + val tree = explorer.tree + if (nodeId != HeapDominatorTreemap.ROOT_OBJECT_ID && nodeId !in tree) { + throw AgentRefusal( + "${exactHexObjectId(nodeId)} is no node of this heap dump's dominator tree, so there is nothing " + + "under it to walk. Leave `$OBJECT` out for the whole heap dump." + ) + } + AgentJson.dominatorOutline(tree.outlineOf(nodeId, maxDepth, maxChildren)) + } + } + private fun setVerdict() = AgentTool( name = SET_VERDICT, description = "Records that an object is meant to be in memory (EXPECTED) or should be gone " + @@ -299,21 +346,65 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { } } + private fun readNotes() = AgentTool( + name = "read_notes", + description = "What has already been written about this heap dump — by the person at the window, by " + + "you earlier, or by whoever read it last. Without `$PLACE`, every place that has a note, so that an " + + "investigation starts from what is known rather than on top of it. With one, that note in full. " + + "Notes outlive the window and are where a conclusion is kept.", + schema = schema(WINDOW to window(), PLACE to place().optional()) + ) { arguments -> + val dump = arguments.heapDump() + val place = arguments.optionalString(PLACE)?.let { arguments.place() } + if (place != null) { + val text = dump.readNote(place) + return@AgentTool buildJsonObject { + put("place", arguments.string(PLACE)) + put("characters", text.length) + put("text", text) + } + } + // Every note of the dump would be a screenful of markdown per place, so this is the listing the tab + // strip is: where somebody has been, and one call each to read what they wrote. + val spellings = dump.notedPlaces().mapNotNull { placeText(it) } + buildJsonObject { + put("placeCount", spellings.size) + putJsonArray("places") { spellings.forEach { add(it) } } + if (spellings.isEmpty()) { + put("nothingWritten", "Nobody has written anything about this heap dump yet.") + } + } + } + private fun takeNote() = AgentTool( name = "take_note", - description = "Appends markdown to the notes of one place in this heap dump, which is where the " + - "person at the window reads them and what the next reader of this dump finds. Notes are kept " + - "between runs of the app. Write what you found and where you looked, not what you are about to do.", + description = "Writes markdown into the notes of one place in this heap dump, which is where the " + + "person at the window reads them and what the next reader of this dump finds. Appends by default, " + + "leaving whatever was there; `$REPLACE` true puts yours in place of it, which is what correcting " + + "something you wrote earlier is — read it first with read_notes. Notes are kept between runs of the " + + "app. Write what you found and where you looked, not what you are about to do.", schema = schema( WINDOW to window(), PLACE to place(), - TEXT to string("Markdown. `0x…` addresses in it become links to those objects.") + TEXT to string("Markdown. `0x…` addresses in it become links to those objects."), + REPLACE to boolean( + "Whether to replace the note rather than add to the end of it. Off by default." + ).optional() ) ) { arguments -> val dump = arguments.heapDump() val place = arguments.place() - dump.appendToNote(place, arguments.string(TEXT)) - buildJsonObject { put("written", true) } + val text = arguments.string(TEXT) + val replaces = arguments.boolean(REPLACE, default = false) + if (replaces) { + dump.replaceNote(place, text) + } else { + dump.appendToNote(place, text) + } + buildJsonObject { + put("written", true) + put("replaced", replaces) + } } private fun show() = AgentTool( @@ -395,6 +486,88 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { } } + private fun openHeapDump() = AgentTool( + name = OPEN_HEAP_DUMP, + description = "Opens a heap dump file in a window of Shark Explorer and answers once it can be read, " + + "which is the same thing as somebody clicking `Open heap dump…`. For a dump nobody has open yet: a " + + "file a bug report came with, one you took with dump_heap, or a second dump of the same app to " + + "compare against. Opening a large dump takes a while, and this waits for it.", + schema = schema( + PATH to string("The absolute path of an `.hprof` file on this machine.") + ) + ) { arguments -> + val path = arguments.string(PATH) + val file = File(path) + if (!file.isFile) { + throw AgentRefusal( + "There is no file at $path. A path here is a path on the machine Shark Explorer is running on, " + + "absolute, and it has to exist before this can open it." + ) + } + val dump = heapDumps.open(file) + buildJsonObject { + put("window", dump.windowId) + put("heapDumpPath", dump.heapDumpPath) + put("opened", true) + put("next", "Call $LIST_LEAKS with this window to see what the dump says about itself.") + } + } + + private fun listDevices() = AgentTool( + name = "list_devices", + description = "The Android devices `adb` is connected to, and with `$DEVICE`, the app processes of one " + + "of them. What the window's `Take heap dump` button asks. A process can only be dumped if the app " + + "was built debuggable or the whole build is (dumpsAnyProcess), and `adb` is the only thing here that " + + "reaches outside this machine.", + schema = schema( + DEVICE to string("Optional: a device's serial number, to list its app processes.").optional() + ) + ) { arguments -> + val serialNumber = arguments.optionalString(DEVICE) + if (serialNumber == null) { + val devices = heapDumps.devices() + return@AgentTool buildJsonObject { + putJsonArray("devices") { AgentJson.devices(devices).forEach { add(it) } } + if (devices.isEmpty()) { + put( + "problem", + "`adb` is connected to no device. Plug one in, start an emulator, or ask whoever is at the " + + "machine to." + ) + } + } + } + val processes = heapDumps.processesOf(serialNumber) + buildJsonObject { + put("device", serialNumber) + putJsonArray("processes") { AgentJson.processes(processes).forEach { add(it) } } + } + } + + private fun dumpHeap() = AgentTool( + name = "dump_heap", + description = "Takes a heap dump of a running process, opens it in a window of Shark Explorer and " + + "answers once it can be read — the whole of what the window's `Take heap dump` button does. " + + "**Minutes, on a large app**: the device writes the dump, it is pulled over `adb`, and then opened. " + + "The garbage is collected first where the device is new enough, so what is in the dump is what is " + + "really still held. Ask list_devices first for the device and the process.", + schema = schema( + DEVICE to string("The serial number of the device, from list_devices."), + PROCESS to string("The name of the process to dump, from list_devices.") + ) + ) { arguments -> + val dump = heapDumps.dumpHeap( + serialNumber = arguments.string(DEVICE), + processName = arguments.string(PROCESS) + ) + buildJsonObject { + put("window", dump.windowId) + put("heapDumpPath", dump.heapDumpPath) + put("dumped", true) + put("next", "Call $LIST_LEAKS with this window to see what the dump says about itself.") + } + } + /** The chain to [objectId], read through the verdicts, refusing an address that is no object of the dump. */ private suspend fun AgentHeapDump.readRootPath(objectId: Long): RootPath = read("the chain to ${exactHexObjectId(objectId)}, for an agent") { explorer -> @@ -516,23 +689,6 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { }.toSet() } - private fun AgentArguments.place(): Place { - val text = string(PLACE) - return when { - text.startsWith(HEX_PREFIX) -> Place.Object(objectIdOf(PLACE, text)) - text == PLACE_LEAKS -> Place.Leaks() - text == PLACE_OBJECTS -> Place.Objects() - text == PLACE_STARRED -> Place.Starred - text.startsWith("$PLACE_OBJECTS:") -> Place.Objects( - ObjectListFilter(query = text.substringAfter(':')) - ) - else -> throw AgentRefusal( - "\"$text\" is no place of a heap dump. A place is an object's address, \"$PLACE_LEAKS\", " + - "\"$PLACE_OBJECTS\", \"$PLACE_OBJECTS:\" or \"$PLACE_STARRED\"." - ) - } - } - private companion object { /** What every investigation starts with, named because three messages point at it. */ @@ -543,6 +699,9 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { /** Named because [placeOrNull] is the one description of a call that has to know which tool it is. */ const val LIST_LEAKS = "list_leaks" + /** And because the refusal for a path that isn't a heap dump points at it. */ + const val OPEN_HEAP_DUMP = "open_heap_dump" + const val WINDOW = "window" const val OBJECT = "object" const val FROM = "from" @@ -553,16 +712,17 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { const val VERDICT = "verdict" const val CHAIN_TO = "chainTo" const val SOLVE_CONFLICTS = "solveConflicts" - const val PLACE = "place" const val TEXT = "text" + const val REPLACE = "replace" + const val MAX_DEPTH = "maxDepth" + const val MAX_CHILDREN = "maxChildren" + const val PATH = "path" + const val DEVICE = "device" + const val PROCESS = "process" const val ROOT_CAUSE = "rootCause" const val HOW_TO_REPRODUCE = "howToReproduce" const val NOT_CHECKED = "notChecked" - const val PLACE_LEAKS = "leaks" - const val PLACE_OBJECTS = "objects" - const val PLACE_STARRED = "starred" - /** * How many objects a list comes back with by default, well under * [HeapDominatorTreemap.MAX_LISTED_OBJECTS]: an agent reads the whole answer, so 500 rows of JSON is @@ -577,10 +737,15 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { fun objectId(description: String) = string("$description An address as ${OPEN_HEAP_DUMPS} and every chain spells one: `0x…`.") - fun place() = string( - "Which place of the heap dump: an object's `0x…` address, \"$PLACE_LEAKS\", \"$PLACE_OBJECTS\", " + - "\"$PLACE_OBJECTS:\" or \"$PLACE_STARRED\"." - ) + /** + * How far down the dominator tree one call will walk, and how wide. + * + * A cap rather than a warning because the answer is a tree: ten levels of ten children is 10^10 nodes, + * each of them a read of the heap dump, and a model that asked for it would have been waiting for the + * rest of the day. Five of fifteen is a long screenful. + */ + const val MAX_OUTLINE_DEPTH = 5 + const val MAX_OUTLINE_CHILDREN = 15 } } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt index 6ae4f1dc65..fa0cf46bc0 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt @@ -114,7 +114,7 @@ class AgentServerTest { } private fun listen(): Closeable = AgentServer.listen( - heapDumps = { listOf(window) }, + heapDumps = FakeAgentHeapDumps(listOf(window)), serverVersion = "1.2.3", directory = directory ).also { closeables += it } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt index eaf267a07a..6515df0d33 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt @@ -109,7 +109,7 @@ class AgentSessionFileTest { @Test fun `every tool has a verb, so that no screen ends up showing the protocol`() { - val withoutAVerb = AgentTools { emptyList() }.all + val withoutAVerb = AgentTools(FakeAgentHeapDumps()).all .map { it.name } .filter { verbOfTool(it, emptyMap()) == null } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt index f0826402e0..70be652c75 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt @@ -52,7 +52,7 @@ class AgentStdioBridgeTest { @Test fun `a client's messages reach the window and its answers come back`() { closeables += AgentServer.listen( - heapDumps = { listOf(window) }, + heapDumps = FakeAgentHeapDumps(listOf(window)), serverVersion = "1.2.3", directory = directory ) diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index d4df8747d7..bf64f090c2 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -18,6 +18,8 @@ import org.junit.Before import org.junit.Rule import org.junit.Test import org.junit.rules.TemporaryFolder +import shark.explorer.AndroidDevice +import shark.explorer.DeviceProcess import shark.explorer.HeapObjectKind import shark.explorer.LeakStatus import shark.explorer.ObjectListFilter @@ -45,7 +47,7 @@ class AgentToolsTest { fun setUp() { heapDump = temporaryFolder.applicationHoldsActivityThroughHolder() window = FakeAgentHeapDump(heapDump.explorer) - tools = AgentTools { listOf(window) } + tools = AgentTools(FakeAgentHeapDumps(listOf(window))) } @After @@ -76,7 +78,7 @@ class AgentToolsTest { @Test fun `a run with no heap dump open says so rather than answering`() { - tools = AgentTools { emptyList() } + tools = AgentTools(FakeAgentHeapDumps()) assertThat(call(OPEN_HEAP_DUMPS).text("problem")).contains("No heap dump is open") assertThatThrownBy { call("list_leaks") } @@ -87,7 +89,7 @@ class AgentToolsTest { @Test fun `two heap dumps open have to be named`() { val other = FakeAgentHeapDump(heapDump.explorer, windowId = "otherwindow") - tools = AgentTools { listOf(window, other) } + tools = AgentTools(FakeAgentHeapDumps(listOf(window, other))) assertThatThrownBy { call("list_leaks") } .isInstanceOf(AgentRefusal::class.java) @@ -465,6 +467,171 @@ class AgentToolsTest { assertThat(window.reads).containsExactly("${hex(heapDump.activityObjectId)} for an agent") } + @Test + fun `the dominator tree comes back as a tree, with what was left out of each level counted`() { + val answer = call("dominator_tree", "maxDepth" to "1", "maxChildren" to "2") + + // The whole heap dump, which is the node every treemap opens on and the default here. + assertThat(answer.text("retainedBytes").toLong()).isGreaterThan(0) + val children = answer.array("dominates").map { it.jsonObject } + assertThat(children).hasSizeLessThanOrEqualTo(2) + // Largest first, which is the order every list in this app is in. + val retained = children.map { it.text("retainedBytes").toLong() } + assertThat(retained).isEqualTo(retained.sortedDescending()) + // Against the children handed back, so that "this is all of it" is never mistaken for the biggest few. + assertThat(answer.text("dominatedNodeCount").toInt()) + .isGreaterThanOrEqualTo(children.size) + // One level asked for is one level answered with, so nothing under these was walked. + assertThat(children.flatMap { it.array("dominates") }).isEmpty() + } + + @Test + fun `the dominator tree under one object is the tree under that object`() { + val answer = call("dominator_tree", OBJECT to hex(heapDump.holderObjectId), "maxDepth" to "1") + + assertThat(answer.text("node")).isEqualTo(hex(heapDump.holderObjectId)) + assertThat(answer.array("dominates").map { it.jsonObject.text("node") }) + .contains(hex(heapDump.activityObjectId)) + } + + @Test + fun `an object the tree has no node for is refused rather than walked`() { + assertThatThrownBy { call("dominator_tree", OBJECT to "0x1") } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("is no node of this heap dump's dominator tree") + } + + @Test + fun `the notes say where somebody has been before they are read`() { + val empty = call("read_notes") + + assertThat(empty.text("placeCount")).isEqualTo("0") + assertThat(empty.text("nothingWritten")).contains("Nobody has written anything") + + call( + "take_note", + "place" to hex(heapDump.holderObjectId), + "text" to "Holder.INSTANCE is assigned in ExampleApplication.onCreate." + ) + + assertThat(call("read_notes").array("places").map { it.jsonPrimitive.content }) + .containsExactly(hex(heapDump.holderObjectId)) + val note = call("read_notes", "place" to hex(heapDump.holderObjectId)) + assertThat(note.text("text")).contains("assigned in ExampleApplication.onCreate") + assertThat(note.text("characters").toInt()).isGreaterThan(0) + } + + @Test + fun `a note can be replaced, which is what correcting one is`() { + val place = Place.Object(heapDump.holderObjectId) + call("take_note", "place" to hex(heapDump.holderObjectId), "text" to "A second holder holds it too.") + + val answer = call( + "take_note", + "place" to hex(heapDump.holderObjectId), + "text" to "There is only one holder; I had misread the object list.", + "replace" to "true" + ) + + assertThat(answer.text("replaced")).isEqualTo("true") + // In place of what was there rather than under it, so that the wrong paragraph is not what the next + // reader finds first. + assertThat(window.notes[place]) + .containsExactly("There is only one holder; I had misread the object list.") + } + + @Test + fun `a heap dump nobody has open can be opened by its path`() { + val other = FakeAgentHeapDump(heapDump.explorer, windowId = "openedwindow") + val heapDumps = FakeAgentHeapDumps(listOf(window), opens = { other }) + tools = AgentTools(heapDumps) + + val answer = call("open_heap_dump", "path" to heapDump.explorer.heapDumpFile.absolutePath) + + assertThat(answer.text("window")).isEqualTo("openedwindow") + assertThat(answer.text("opened")).isEqualTo("true") + assertThat(heapDumps.opened).containsExactly(heapDump.explorer.heapDumpFile) + } + + @Test + fun `a path with no file at it is refused before anything is opened`() { + val heapDumps = FakeAgentHeapDumps(listOf(window)) + tools = AgentTools(heapDumps) + + assertThatThrownBy { call("open_heap_dump", "path" to "/no/such/dump.hprof") } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("There is no file at /no/such/dump.hprof") + + assertThat(heapDumps.opened).isEmpty() + } + + @Test + fun `the devices adb is connected to, and then the processes of one`() { + val device = AndroidDevice( + serialNumber = "emulator-5554", + state = "device", + fingerprint = "google/sdk_gphone64_arm64/emu64a:16/BE1A.250305.005/13103848:userdebug/dev-keys", + model = "sdk_gphone64_arm64", + sdkInt = 36, + isDebuggableBuild = true + ) + val process = DeviceProcess(processId = 4231, name = "com.example.app") + tools = AgentTools(FakeAgentHeapDumps(listOf(window), devices = mapOf(device to listOf(process)))) + + val devices = call("list_devices").array("devices").map { it.jsonObject } + assertThat(devices.single().text("device")).isEqualTo("emulator-5554") + // The difference between a device with two dumpable processes on it and one with all of them. + assertThat(devices.single().text("dumpsAnyProcess")).isEqualTo("true") + + val processes = call("list_devices", "device" to "emulator-5554").array("processes").map { it.jsonObject } + assertThat(processes.single().text("process")).isEqualTo("com.example.app") + assertThat(processes.single().text("processId")).isEqualTo("4231") + } + + @Test + fun `a machine with nothing plugged in says so rather than answering with an empty list`() { + tools = AgentTools(FakeAgentHeapDumps(listOf(window))) + + assertThat(call("list_devices").text("problem")).contains("connected to no device") + } + + @Test + fun `a heap dump taken off a device is opened in a window`() { + val device = AndroidDevice( + serialNumber = "emulator-5554", + state = "device", + fingerprint = null, + model = null, + sdkInt = 36, + isDebuggableBuild = true + ) + val process = DeviceProcess(processId = 4231, name = "com.example.app") + val dumped = FakeAgentHeapDump(heapDump.explorer, windowId = "dumpedwindow") + val heapDumps = FakeAgentHeapDumps( + open = listOf(window), + devices = mapOf(device to listOf(process)), + opens = { dumped } + ) + tools = AgentTools(heapDumps) + + val answer = call("dump_heap", "device" to "emulator-5554", "process" to "com.example.app") + + assertThat(answer.text("window")).isEqualTo("dumpedwindow") + assertThat(answer.text("dumped")).isEqualTo("true") + assertThat(heapDumps.dumped).containsExactly("emulator-5554" to "com.example.app") + } + + @Test + fun `the agent logs are a place too, since an agent can be asked what another one did`() { + call("show", "place" to "agent-logs") + call("show", "place" to "agent-logs:agent-20260825-abcdef") + + assertThat(window.shown).containsExactly( + Place.AgentLogs, + Place.AgentLog("agent-20260825-abcdef") + ) + } + /** What the whole investigation turns on: the object in between is one somebody read the code about. */ private fun setHolderExpected() { call( diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt index e23f7e53d5..0e6d00e15f 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt @@ -1,7 +1,10 @@ package shark.explorer.agent import java.io.Closeable +import java.io.File import shark.SharkLog +import shark.explorer.AndroidDevice +import shark.explorer.DeviceProcess import shark.explorer.HeapExplorer import shark.explorer.LeakStatusOverride import shark.explorer.LeakStatusOverrides @@ -63,6 +66,18 @@ internal class FakeAgentHeapDump( notes.getOrPut(place) { mutableListOf() } += text } + override suspend fun replaceNote( + place: Place, + text: String + ) { + notes[place] = mutableListOf(text) + } + + override suspend fun readNote(place: Place): String = + notes[place]?.joinToString("\n\n").orEmpty() + + override suspend fun notedPlaces(): List = notes.keys.toList() + override fun show(place: Place) { shown += place } @@ -71,3 +86,49 @@ internal class FakeAgentHeapDump( explorer.close() } } + +/** + * The heap dumps of a run, as far as a test needs them: the windows it was given, and nothing plugged in. + * + * [AgentHeapDumps] is the app's whole side of this surface — the windows open, plus the two buttons above the + * map — and a test of what a tool answers has neither a window nor a device. So the dumps are handed in, a + * dump opened from a path is whatever [opens] makes of it, and `adb` answers with [devices]: enough for a + * refusal to be a refusal about the right thing, which is what these tools mostly are. + */ +internal class FakeAgentHeapDumps( + private val open: List = emptyList(), + /** Keyed by serial number, each with the processes that device is running. */ + private val devices: Map> = emptyMap(), + /** What a file, or a dump pulled off a device, opens as. Refuses by default, since most tests open none. */ + private val opens: (File) -> AgentHeapDump = { file -> + throw AgentRefusal("This test opens no heap dump, so there is nothing to open $file as.") + } +) : AgentHeapDumps { + + /** What was asked to be opened, and what was dumped, in order, so a test can read the calls back. */ + val opened = mutableListOf() + val dumped = mutableListOf>() + + override fun openHeapDumps(): List = open + + override suspend fun open(file: File): AgentHeapDump { + opened += file + return opens(file) + } + + override suspend fun devices(): List = devices.keys.toList() + + override suspend fun processesOf(serialNumber: String): List = + devices.entries.firstOrNull { it.key.serialNumber == serialNumber }?.value + ?: throw AgentRefusal("`adb` is connected to no device called \"$serialNumber\".") + + override suspend fun dumpHeap( + serialNumber: String, + processName: String + ): AgentHeapDump { + processesOf(serialNumber).firstOrNull { it.name == processName } + ?: throw AgentRefusal("No process called \"$processName\" is running on $serialNumber.") + dumped += serialNumber to processName + return opens(File("$processName.hprof")) + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index 0b47e1d89d..acada49d8c 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -43,7 +43,7 @@ class McpSessionTest { window = FakeAgentHeapDump(heapDump.explorer) sessionsDirectory = File(temporaryFolder.root, "sessions") session = McpSession( - tools = AgentTools { listOf(window) }, + tools = AgentTools(FakeAgentHeapDumps(listOf(window))), serverVersion = SERVER_VERSION, sessionFile = AgentSessionFile.starting(sessionsDirectory, SERVER_VERSION) ) @@ -87,11 +87,16 @@ class McpSessionTest { "chain_from_gc_root", "ways_held", "find_objects", + "dominator_tree", "set_verdict", "clear_verdict", + "read_notes", "take_note", "show", - "conclude" + "conclude", + "open_heap_dump", + "list_devices", + "dump_heap" ) tools.forEach { tool -> assertThat(tool.text("description")).isNotEmpty() diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt index 18cfdd3156..17a682e9b6 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt @@ -1,7 +1,14 @@ package shark.explorer.app +import androidx.compose.runtime.snapshotFlow import java.io.File +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.flow.first +import kotlinx.coroutines.withContext import shark.SharkLog +import shark.explorer.AndroidDevice +import shark.explorer.DeviceHeapDumps +import shark.explorer.DeviceProcess import shark.explorer.HeapExplorer import shark.explorer.LeakStatusOverride import shark.explorer.LeakStatusOverrides @@ -13,23 +20,7 @@ import shark.explorer.agent.AgentServer import shark.explorer.agent.AgentSession import shark.explorer.agent.AgentSessionFile import shark.explorer.agent.AgentStdioBridge - -/** - * How an agent reaches the windows of this run: the app's side of `shark-explorer-agent`. - * - * The one thing worth knowing here is **why an agent is given the window and not the heap dump file**. Every - * read goes through that window's own [HeapDumpSession], so an agent's question queues on the thread the - * person at the machine is already reading with, and the verdicts and notes it writes are the ones that - * window is drawing. An agent that opened the dump itself would be answering about a heap dump nobody is - * looking at, and its conclusions would land in a file the open window would then overwrite. - */ -internal fun explorerAgentHeapDumps(windows: ExplorerWindows): AgentHeapDumps = AgentHeapDumps { - // Asked per call rather than captured, because windows come and go while an agent is connected: a tool - // naming a window that has since closed has to be an error message and not a stale answer. - windows.mapNotNull { window -> - window.openHeapDump?.let { open -> WindowAgentHeapDump(window, open) } - } -} +import shark.explorer.placeOfNoteKeyOrNull /** * Publishes this run so that agents can find it, or does nothing if it can't. See [AgentServer]. @@ -38,12 +29,128 @@ internal fun explorerAgentHeapDumps(windows: ExplorerWindows): AgentHeapDumps = * being loopback: a link is one line answered in a millisecond, and this is a session held open for as long * as an investigation takes. */ -internal fun listenForAgents(windows: ExplorerWindows) = AgentServer.listen( - heapDumps = explorerAgentHeapDumps(windows), +internal fun listenForAgents( + windows: ExplorerWindows, + deviceHeapDumps: DeviceHeapDumps +) = AgentServer.listen( + heapDumps = WindowAgentHeapDumps(windows, deviceHeapDumps), serverVersion = SharkExplorerVersion.current, directory = AGENT_RUNS_DIRECTORY ) +/** + * How an agent reaches this run: the app's side of `shark-explorer-agent`. + * + * The one thing worth knowing here is **why an agent is given a window and not a heap dump file**. Every read + * goes through that window's own [HeapDumpSession], so an agent's question queues on the thread the person at + * the machine is already reading with, and the verdicts and notes it writes are the ones that window is + * drawing. An agent that opened the dump itself would be answering about a heap dump nobody is looking at, and + * its conclusions would land in a file the open window would then overwrite. + * + * Which is also why opening a dump and taking one off a device end in a window here rather than in a file + * path: they are the two buttons above the map, and an agent pressing one has to end up somewhere its human + * can follow it to. + */ +private class WindowAgentHeapDumps( + private val windows: ExplorerWindows, + private val deviceHeapDumps: DeviceHeapDumps +) : AgentHeapDumps { + + override fun openHeapDumps(): List = + // Asked per call rather than captured, because windows come and go while an agent is connected: a tool + // naming a window that has since closed has to be an error message and not a stale answer. + windows.mapNotNull { window -> + window.openHeapDump?.let { open -> WindowAgentHeapDump(window, open) } + } + + override suspend fun open(file: File): AgentHeapDump { + // Edited from whichever thread the agent's connection is on, exactly as a link arriving from another run + // of this app edits it: a window is snapshot state, and the composition takes the change on the next + // frame. See [ExplorerWindow.linkedPlaces]. + val window = windows.openHeapDump(file) + SharkLog.d { "An agent opened ${file.absolutePath} in window ${window.deepLinkId}" } + return awaitHeapDump(window) + } + + override suspend fun devices(): List = onAdbThread { + deviceHeapDumps.connectedDevices() + } + + override suspend fun processesOf(serialNumber: String): List = onAdbThread { + deviceHeapDumps.appProcesses(device(serialNumber)) + } + + override suspend fun dumpHeap( + serialNumber: String, + processName: String + ): AgentHeapDump { + val heapDumpFile = onAdbThread { + val device = device(serialNumber) + val process = deviceHeapDumps.appProcesses(device).firstOrNull { it.name == processName } + ?: throw AgentRefusal( + "No process called \"$processName\" is running on ${device.description}. A process is dumped by " + + "name because a pid changes every time the app restarts, so ask list_devices again: what it " + + "answers with is what is running now." + ) + // Every step of it in this run's log, which is the only place a dump that is taking minutes says how + // far it has got — the agent is waiting for one answer and there is nothing to stream it through. + deviceHeapDumps.dumpHeap(device, process) { step -> SharkLog.d { "For an agent: $step" } } + } + // No pixels fetched to go with it, unlike the dialog's tick box: that is a second suspension of the app, + // minutes of it, and an agent reads a bitmap's size rather than looking at it. Whoever is at the window + // can still fetch them from the panel afterwards. + val window = windows.openHeapDump(heapDumpFile) + SharkLog.d { "An agent dumped $processName into window ${window.deepLinkId}" } + return awaitHeapDump(window) + } + + /** + * Waits for [window]'s heap dump to be readable, and refuses when it never will be. + * + * Three ways it ends and only one of them is an answer — the dump opens, it fails to open, or the window is + * closed under it — because a window id handed over before its dump is open is one that refuses every call + * made with it, and either of the other two would otherwise be a call that never comes back. Which is what + * [ExplorerWindow.openProblem] exists for. + */ + private suspend fun awaitHeapDump(window: ExplorerWindow): AgentHeapDump { + snapshotFlow { + window.openHeapDump != null || window.openProblem != null || window !in windows + }.first { it } + val open = window.openHeapDump + if (open != null) { + return WindowAgentHeapDump(window, open) + } + val name = window.heapDumpFile?.name + throw AgentRefusal( + window.openProblem?.let { "$name could not be opened as a heap dump: $it" } + ?: "Window ${window.deepLinkId} was closed before $name had finished opening, so there is nothing " + + "to read. Opening it again is a call away." + ) + } + + /** The device with this serial number, or a refusal listing the ones there are. */ + private fun device(serialNumber: String): AndroidDevice { + val devices = deviceHeapDumps.connectedDevices() + return devices.firstOrNull { it.serialNumber == serialNumber } + ?: throw AgentRefusal( + "`adb` is connected to no device called \"$serialNumber\". " + if (devices.isEmpty()) { + "It is connected to nothing at all." + } else { + "It is connected to " + devices.joinToString(", ") { "${it.serialNumber} (${it.description})" } + + "." + } + ) + } + + /** + * Everything `adb` blocks on, off the connection's thread. + * + * Which is not the heap dump's thread either: a dump takes minutes of shelling out, and the window it will + * open in is being read by whoever is at the machine while it does. + */ + private suspend fun onAdbThread(block: () -> T): T = withContext(Dispatchers.IO) { block() } +} + /** * Whether this process was started to be a pipe between an agent and another run of the app, and what to * exit with if it was. Null for every other command line. @@ -103,39 +210,73 @@ private class WindowAgentHeapDump( open.leakStatuses.clear(objectId) } + override suspend fun appendToNote( + place: Place, + text: String + ) = write(place) { existing -> + listOf(existing, text).filter { it.isNotBlank() }.joinToString(PARAGRAPH_BREAK) + } + + override suspend fun replaceNote( + place: Place, + text: String + ) = write(place) { text } + + override suspend fun readNote(place: Place): String = readable(place).text + + override suspend fun notedPlaces(): List { + // The same listing the tab strip is marked from, read once per run of the app either way. + open.notes.list() + return open.notes.writtenAbout.mapNotNull { key -> placeOfNoteKeyOrNull(key) } + } + /** - * Appends to what has been written about [place], leaving whatever was there. + * Puts what [newText] makes of the saved note on disk, whether that is the note plus a paragraph or + * something else entirely. * * **Refuses while somebody is typing in that note**, which is the one case where writing would cost - * something that exists nowhere else: a draft is unsaved text, and saving over it with the draft plus an - * agent's paragraph would put half a sentence of theirs on disk under an answer of ours. + * something that exists nowhere else: a draft is unsaved text, and saving over it would put half a sentence + * of theirs on disk under an answer of ours. */ - override suspend fun appendToNote( + private suspend fun write( place: Place, - text: String + newText: (String) -> String ) { - val notepad = open.notes.of(place) - notepad.read() - if (!notepad.isRead) { - throw AgentRefusal( - "The notes of that place could not be read, so writing would overwrite whatever is in them: " + - (notepad.problem ?: "reading ${notepad.file} did not finish.") - ) - } + val notepad = readable(place) if (notepad.draft != null) { throw AgentRefusal( - "Somebody is writing in the notes of that place right now, so there is nothing to append to yet. " + - "Say what you found in your answer instead, or try again once they have saved." + "Somebody is writing in the notes of that place right now, so writing there would take their " + + "unsaved words with it. Say what you found in your answer instead, or try again once they have " + + "saved." ) } notepad.edit() - notepad.edited(listOf(notepad.text, text).filter { it.isNotBlank() }.joinToString(PARAGRAPH_BREAK)) + notepad.edited(newText(notepad.text)) notepad.save() if (notepad.problem != null) { throw AgentRefusal("The notes could not be saved: ${notepad.problem}") } } + /** + * The notepad of [place] with its file read, refusing when it couldn't be. + * + * Before reading it as much as before writing it: an unread notepad's text is empty because nothing has + * been read rather than because nothing was written, so answering with it would be telling an agent that + * a note it is about to replace does not exist. + */ + private suspend fun readable(place: Place): PlaceNotes { + val notepad = open.notes.of(place) + notepad.read() + if (!notepad.isRead) { + throw AgentRefusal( + "The notes of that place could not be read, so what is in them is unknown: " + + (notepad.problem ?: "reading ${notepad.file} did not finish.") + ) + } + return notepad + } + override fun show(place: Place) { SharkLog.d { "An agent asked window ${window.deepLinkId} for $place" } // The same two steps following a link takes, which is what makes an agent showing something and a diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt index 6d73989afc..b99fd371d2 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt @@ -65,6 +65,16 @@ internal class ExplorerWindow( */ var openHeapDump: WindowHeapDump? by mutableStateOf(null) + /** + * Why the heap dump this window was given could not be opened, and null while nothing has gone wrong. + * + * The other half of [openHeapDump] for a reader outside the composition, and not its opposite: a window + * whose dump is still being indexed has neither, which is what tells something waiting for that dump that + * waiting is still the right thing to do. Set by [ExplorerApp] the same way, and cleared as another dump + * opens here. + */ + var openProblem: String? by mutableStateOf(null) + /** * Places a link has asked this window for and whose tabs are not open yet, oldest first. * @@ -138,7 +148,7 @@ internal class ExplorerWindow( */ internal class ExplorerWindows( /** Put in front of every window title of this run. See [ExplorerArguments.titlePrefix]. */ - private val titlePrefix: String? = null, + val titlePrefix: String? = null, /** One Compose window is drawn per entry, so a window opening or closing is an edit of this. */ private val windows: SnapshotStateList = mutableStateListOf() ) : MutableList by windows { @@ -229,8 +239,9 @@ internal fun ExplorerWindows.openHeapDump( } else { window.heapDumpFile = heapDumpFile window.bitmapPixels = bitmapPixels - // Whatever a link said about this window being empty is answered now that it isn't. + // Whatever a link, or a dump that failed to open, said about this window is answered now that it has one. window.deepLinkProblem = null + window.openProblem = null } // One run's log covers every window of that run, and what tells the lines apart afterwards is the // thread each was written from, so which window a heap dump went to is worth a line of its own. @@ -240,5 +251,23 @@ internal fun ExplorerWindows.openHeapDump( } } +/** + * Shows [heapDumpFile] somewhere, and says where, for a dump that arrives from **outside any window**. + * + * Which is an agent opening a file, or taking one off a device: there is no window it was asked from, so the + * rule above is asked of every window instead of one — the window showing nothing is the one with nothing to + * lose, and failing that a heap dump is a window. + */ +internal fun ExplorerWindows.openHeapDump( + heapDumpFile: File, + /** Fetched with the dump, for a device whose dump can't carry the pixels of its bitmaps. */ + bitmapPixels: NativeBitmapPixels? = null +): ExplorerWindow { + val window = firstOrNull { it.heapDumpFile == null } + ?: ExplorerWindow(cascade = freeCascade(), titlePrefix = titlePrefix).also { add(it) } + openHeapDump(window, heapDumpFile, bitmapPixels) + return window +} + /** Between what a run is called and which heap dump a window shows, as elsewhere in this window. */ private const val TITLE_SEPARATOR = " · " diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt index 3fec8a3c68..8cb685cb3a 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt @@ -82,6 +82,9 @@ fun main(args: Array) { SharkLog.d { "Read that as $arguments" } nameThisRun(arguments.titlePrefix ?: APP_NAME) val windows = explorerWindows(arguments) + // One per run rather than one per window, because an agent has no window: opening a heap dump and taking + // one off a device are what make a window, so whatever does them has to outlive every window there is. + val deviceHeapDumps = commandLineDeviceHeapDumps() // Both before the first window, so that a link arriving while the heap dumps are still opening is one // the window queues rather than one that lands on an app not listening yet. DeepLinkScheme.takeUrisFromTheOs(windows) @@ -89,12 +92,12 @@ fun main(args: Array) { DeepLinkPeers.listen(windows).use { // Published before the first window too, so that an agent whose client started it while the heap // dumps were still opening finds this run and waits for a dump rather than finding nothing. - listenForAgents(windows).use { + listenForAgents(windows, deviceHeapDumps).use { // Whatever no other run claimed, which for a link naming a window that has gone is an empty window // saying so. Ours to answer for now: nobody else is going to. DeepLinkPeers.deliver(arguments.deepLinks).forEach { windows.open(it) } // Heap dump paths on the command line open straight away, which is how this is usually run. - explorerApplication(windows) + explorerApplication(windows, deviceHeapDumps) } } } @@ -147,7 +150,10 @@ private fun nameThisRun(name: String) { } /** One window per heap dump open, which is what [openHeapDump] keeps true as more are opened. */ -private fun explorerApplication(windows: ExplorerWindows) = application { +private fun explorerApplication( + windows: ExplorerWindows, + deviceHeapDumps: DeviceHeapDumps +) = application { val updateNotice = remember { UpdateNotice() } // One notepad per place for the whole run, so that a heap dump open in two windows is one set of notes // rather than two that overwrite each other. See [ExplorerNotes]. @@ -200,13 +206,17 @@ private fun explorerApplication(windows: ExplorerWindows) = application { // What this window has open, for the agent surface: a socket thread has to be able to find it, // and it is a composable's state. See [ExplorerWindow.openHeapDump]. onHeapDumpOpen = { open -> window.openHeapDump = open }, + onHeapDumpProblem = { problem -> window.openProblem = problem }, deepLinkId = window.deepLinkId, // The same way a link arriving from the OS is followed, which is what makes a `shark://` link // written in a note work wherever it is read from. followDeepLink = { link -> DeepLinkPeers.follow(link, windows) }, linkedPlaces = window.linkedPlaces, onLinkedPlaceOpened = { place -> window.linkedPlaceOpened(place) }, - deepLinkProblem = window.deepLinkProblem + deepLinkProblem = window.deepLinkProblem, + // The run's rather than this window's, because an agent reaches the same one through no window + // at all — and because two windows asking `adb` at once is two `adb` processes. + deviceHeapDumps = deviceHeapDumps ) } } @@ -248,6 +258,14 @@ internal fun ExplorerApp( * closes. See [ExplorerWindow.openHeapDump]. */ onHeapDumpOpen: (WindowHeapDump?) -> Unit = {}, + /** + * And where a heap dump that could not be opened says so, for the same readers. + * + * Beside [onHeapDumpOpen] rather than folded into it, because the two are not opposites: a window whose + * dump is still opening has neither, and something waiting for that dump has to be able to tell "not + * yet" from "never" without waiting for a timeout to decide it. Null again once another dump opens here. + */ + onHeapDumpProblem: (String?) -> Unit = {}, /** * What every agent that has connected did, for the screens that draw them. * @@ -279,12 +297,7 @@ internal fun ExplorerApp( /** Overridden by tests, which have no display to put a file dialog on. */ chooseHeapDumpFile: () -> File? = ::showHeapDumpFileDialog, /** Overridden by tests, which have no device to go back to and no `adb` to ask. */ - deviceHeapDumps: DeviceHeapDumps = remember { - val adb = CommandLineAdb() - // A debugger is what reaches into a process for the two things `am dumpheap` can't ask it for on an - // old enough device: the pixels of a bitmap below API 35, and a collection below API 27. - DeviceHeapDumps(adb, JdwpBitmaps(adb), JdwpGc(adb)) - } + deviceHeapDumps: DeviceHeapDumps = remember { commandLineDeviceHeapDumps() } ) { var state: HeapDumpState by remember { mutableStateOf(HeapDumpState.None) } var takesHeapDump by remember { mutableStateOf(false) } @@ -333,6 +346,7 @@ internal fun ExplorerApp( ) } ) + onHeapDumpProblem((currentState as? HeapDumpState.Failed)?.message) onDispose { onHeapDumpOpen(null) open?.session?.close() @@ -535,6 +549,19 @@ private fun cascadedPosition(cascade: Int): WindowPosition { return WindowPosition(x = screen.x.dp + centredX + step, y = screen.y.dp + centredY + step) } +/** + * How this app reaches a device: the machine's own `adb`, with a debugger for what `adb` can't ask for. + * + * A function rather than a constant because it is one per run and a test's is its own — the default of a + * composable that a test takes over, and what `main` hands to everything that has no window. + */ +private fun commandLineDeviceHeapDumps(): DeviceHeapDumps { + val adb = CommandLineAdb() + // A debugger is what reaches into a process for the two things `am dumpheap` can't ask it for on an old + // enough device: the pixels of a bitmap below API 35, and a collection below API 27. + return DeviceHeapDumps(adb, JdwpBitmaps(adb), JdwpGc(adb)) +} + /** * The platform file picker, through AWT: Compose Multiplatform has none of its own, and this is the * native dialog on macOS and Windows. diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DominatorOutline.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DominatorOutline.kt new file mode 100644 index 0000000000..7b7cc65309 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DominatorOutline.kt @@ -0,0 +1,89 @@ +package shark.explorer + +/** + * The dominator tree under one node, in outline: the biggest few of what it dominates, and the biggest few of + * what each of those dominates, a fixed number of levels down. + * + * The same tree the treemap, the rings and the stack all draw, without a viewport. Which is what makes it the + * shape for a reader that has no screen — an agent — and what makes it worth being one function rather than + * three: those three shapes each spend their pixels differently on the same answer, and where the memory is + * doesn't depend on how it is drawn. + * + * Bounded in both directions because the root of a production heap dump has six figures of children and a + * chain of single dominators runs to hundreds of levels. What was left out is counted rather than dropped + * silently — [childCount] against the children handed back — so that a reader can tell "this is all of it" + * from "this is the top of it". + */ +data class DominatorOutline( + /** What to ask about this node next, and what a link to it names: an object's address, or a pile's id. */ + val nodeId: Long, + /** How the window labels this node: `MainActivity`, `42 × Bitmap`, the whole heap dump. */ + val label: String, + /** Bytes it retains, which is its own shallow size plus everything it dominates. */ + val retainedSize: Long, + val strength: ReachabilityStrength, + /** + * How many objects this node stands for, for a node that is a pile of them rather than one object — a + * class at the top of the tree, or the uncollected garbage. Null for one object, which is most nodes. + */ + val objectCount: Int?, + /** The class the pile is of, for a pile of one class. Null with [objectCount]. */ + val className: String?, + /** How many nodes this one dominates directly, of which [children] is the largest few. */ + val childCount: Int, + /** Largest retained size first, which is the order every list in this app is in. */ + val children: List +) + +/** + * The outline of the dominator tree under [nodeId], largest first. See [DominatorOutline]. + * + * Reads the heap dump once per node it names, so it belongs on the heap dump's thread like every other read + * here, and the two bounds are what keep that a bounded number of reads: [maxDepth] levels of at most + * [maxChildren] nodes each. + * + * @throws IllegalArgumentException for a node this tree hasn't got, the way every other question here does. + */ +fun HeapDominatorTreemap.outlineOf( + nodeId: Long = HeapDominatorTreemap.ROOT_OBJECT_ID, + /** How many levels below [nodeId] to walk. Zero is the node on its own, with its children counted. */ + maxDepth: Int = DEFAULT_OUTLINE_DEPTH, + /** How many children of each node to walk into, the largest first. */ + maxChildren: Int = DEFAULT_OUTLINE_CHILDREN +): DominatorOutline { + require(nodeId == HeapDominatorTreemap.ROOT_OBJECT_ID || nodeId in this) { + "${hexObjectId(nodeId)} is no node of this heap dump's dominator tree" + } + val childIds = children(nodeId) + val group = groupOrNull(nodeId) + return DominatorOutline( + nodeId = nodeId, + label = label(nodeId), + retainedSize = weight(nodeId), + strength = strengthOf(nodeId), + objectCount = group?.objectCount, + className = group?.className, + childCount = childIds.size, + children = if (maxDepth <= 0) { + emptyList() + } else { + // Sorted here rather than trusted from the tree: what a level of this is worth is that the biggest + // thing is first, and the order children come back in is the order they were found in. + childIds.sortedByDescending { weight(it) } + .take(maxChildren) + .map { outlineOf(it, maxDepth - 1, maxChildren) } + } + ) +} + +/** + * Three levels, which is what says where the memory is without saying what holds what. + * + * The top of a heap dump is a handful of classes; the level under it is the objects of one; the level under + * that is the first thing that is somebody's own code. Deeper than that is a chain to walk with a chain, not + * an outline to read. + */ +const val DEFAULT_OUTLINE_DEPTH = 3 + +/** And ten of each, which is more than the eye reads off a treemap and less than a screenful of text. */ +const val DEFAULT_OUTLINE_CHILDREN = 10 diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NoteFile.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NoteFile.kt index 17c8c95a70..7849e8301a 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NoteFile.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/NoteFile.kt @@ -68,20 +68,59 @@ fun Place.noteKey(): String = when (this) { if (objectId == HeapDominatorTreemap.ROOT_OBJECT_ID) { HEAP_DUMP_KEY } else { - "object-${hexObjectId(objectId)}" + "$OBJECT_KEY_PREFIX${hexObjectId(objectId)}" } - is Place.SmallerObjects -> "smaller-objects-${hexObjectId(parentObjectId)}" - is Place.Objects -> "object-list" - is Place.Leaks -> "leaks" - is Place.Starred -> "starred" - is Place.AgentLogs -> "agent-logs" + is Place.SmallerObjects -> "$SMALLER_OBJECTS_KEY_PREFIX${hexObjectId(parentObjectId)}" + is Place.Objects -> OBJECT_LIST_KEY + is Place.Leaks -> LEAKS_KEY + is Place.Starred -> STARRED_KEY + is Place.AgentLogs -> AGENT_LOGS_KEY // Per session, because a note about what one agent did is about that investigation and not about agents. - is Place.AgentLog -> "agent-log-$sessionId" + is Place.AgentLog -> "$AGENT_LOG_KEY_PREFIX$sessionId" +} + +/** + * The place a [noteKey] was written for, and null for a key this version of the app doesn't know. + * + * The other way round from [noteKey] and therefore missing what a key deliberately leaves out — which filter + * the object list had, which leaks were unfolded — so this answers with the plain list rather than the screen + * somebody wrote the note from. That is the same note either way, which is the whole point of a key being + * what a note is about rather than how it was arranged. + * + * **For reading a directory of notes back**, which is the one thing that has a key and wants a place: a + * listing says which places this heap dump has been written about, and that is only worth saying if each of + * them is somewhere a reader can be sent. A key from a newer version of the app, or a file somebody dropped + * in the directory by hand, is null rather than an error. + * + * One caveat, from [hexObjectId] being the recognisable spelling rather than the exact one: a 32 bit heap + * dump's sign-widened address and the positive address of the same digits share a key, so they share a note, + * and this answers with the positive one. The note is right; the tab it opens, for that one dump, may not be. + */ +fun placeOfNoteKeyOrNull(key: String): Place? = when { + key == HEAP_DUMP_KEY -> Place.wholeHeapDump() + key == OBJECT_LIST_KEY -> Place.Objects() + key == LEAKS_KEY -> Place.Leaks() + key == STARRED_KEY -> Place.Starred + key == AGENT_LOGS_KEY -> Place.AgentLogs + key.startsWith(AGENT_LOG_KEY_PREFIX) -> Place.AgentLog(key.removePrefix(AGENT_LOG_KEY_PREFIX)) + key.startsWith(OBJECT_KEY_PREFIX) -> + objectIdOfHex(key.removePrefix(OBJECT_KEY_PREFIX))?.let { Place.Object(it) } + // A pile of the objects one rectangle had no room for is drawn from the window's width, so how many + // objects it stands for and what they weigh are no part of the key and can't be answered here. + else -> null } /** The note about the heap dump as a whole, which is the place its first tab opens on. */ private const val HEAP_DUMP_KEY = "heap-dump" +private const val OBJECT_KEY_PREFIX = "object-" +private const val SMALLER_OBJECTS_KEY_PREFIX = "smaller-objects-" +private const val OBJECT_LIST_KEY = "object-list" +private const val LEAKS_KEY = "leaks" +private const val STARRED_KEY = "starred" +private const val AGENT_LOGS_KEY = "agent-logs" +private const val AGENT_LOG_KEY_PREFIX = "agent-log-" + /** * One note: a markdown file, read and written whole. * From 649106bdbdee8d2de08ef5322b1d6b9a9bdba318 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 09:53:56 +0200 Subject: [PATCH 08/27] Serve an agent that has no window to investigate in MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An agent could reach the tools only through a window somebody had already opened, so a machine with no screen — a build server, a heap dump at the end of an ssh session, anything driving an agent unattended — had no way in, and even on a desktop the first answer was "ask your human to launch Shark Explorer". Two halves. `--mcp-stdio` with nothing running now starts a window, on the heap dump its own command line named, and leaves it open for whoever comes back to it. And `--no-ui` serves the tools from that process with no window anywhere. One code path with one call swapped rather than a headless mode beside the windowed one: `AgentHeapDump.show` gained a return, and it is the only thing that differs. Which is the point — the notes and the verdicts were never on the screen, so a heap dump investigated with no window opens in one later with the verdicts, the reasons and the conclusion already on it. `show` answers with why it showed nothing rather than saying it did, because an agent telling its human where to look is worse than saying nothing when there is nowhere to look. Two things found by driving it for real rather than by reading it: A window opened for an agent publishes the run before its heap dump is readable, so the agent's first move is to open the path it was pointed at — and that used to make a second window and a second index of the same file. Opening a path some window already has now hands back that window, which is the opposite of what the button does, and on purpose: a person clicking `Open heap dump…` twice is comparing two readings, an agent naming a path is naming a heap dump. The headless side joins an open already in flight for the same reason, which is also what lets a dump named on the command line be indexed in the background instead of holding up the client's `initialize`. And every bridge session ended with a `SocketException` trace on stderr — the answers thread reading a socket this end had just closed — which is where an MCP client collects a server's log, so a clean exit read as a crash. It now knows the close was its own, and the test asserts stderr stays quiet. --- docs/shark-explorer-changelog.md | 7 + docs/shark-explorer.md | 29 +- shark/shark-explorer/AGENTS.md | 2 +- .../shark-explorer-agent/AGENTS.md | 23 +- .../shark/explorer/agent/AgentHeapDump.kt | 10 +- .../shark/explorer/agent/AgentStdioBridge.kt | 70 +++- .../shark/explorer/agent/AgentStdioServer.kt | 58 ++++ .../java/shark/explorer/agent/AgentTools.kt | 21 +- .../explorer/agent/AgentStdioBridgeTest.kt | 64 +++- .../explorer/agent/AgentStdioServerTest.kt | 157 +++++++++ .../shark/explorer/agent/FakeAgentHeapDump.kt | 5 +- .../java/shark/explorer/app/DeepLinkScheme.kt | 17 +- .../java/shark/explorer/app/ExplorerAgents.kt | 316 +++++++++++++----- .../shark/explorer/app/ExplorerArguments.kt | 10 +- .../shark/explorer/app/ExplorerLogging.kt | 32 +- .../shark/explorer/app/ExplorerProcess.kt | 66 ++++ .../java/shark/explorer/app/ExplorerWindow.kt | 2 +- .../explorer/app/HeadlessAgentHeapDumps.kt | 134 ++++++++ .../src/main/java/shark/explorer/app/Main.kt | 6 +- .../explorer/app/AgentCommandLineTest.kt | 46 +++ .../shark/explorer/app/ExplorerProcessTest.kt | 38 +++ .../shark/explorer/app/ExplorerWindowTest.kt | 47 +++ .../app/HeadlessAgentHeapDumpsTest.kt | 132 ++++++++ 23 files changed, 1144 insertions(+), 148 deletions(-) create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioServer.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioServerTest.kt create mode 100644 shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerProcess.kt create mode 100644 shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt create mode 100644 shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentCommandLineTest.kt create mode 100644 shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerProcessTest.kt create mode 100644 shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index aa866718f3..87da8b981a 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -48,6 +48,13 @@ uses, without the one for a newly recognized library leak: picking the process off a device — because a surface with less than that answers "ask your human to click something". Point any MCP client at the installed app with `--mcp-stdio`. See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). +* ✨ **An agent no longer needs a window to have been opened for it.** With nothing running, `--mcp-stdio` + opens one — on the heap dump its command line named, if it named one — and leaves it open for whoever comes + back to it. And with `--no-ui`, the tools are served from that process with no window anywhere, for a build + server or a heap dump at the end of an ssh session: everything works the same except `show`, which says it + has nowhere to put a tab rather than answering that it showed you something. Notes and verdicts were never + on the screen, so a heap dump investigated with no window opens in one later with all of it on. + See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). * ✨ **Agent logs**: every agent that has connected to the app is a row on a screen of its own, and opening one is everything that agent did — what each call did, which object it did it to, and the sentence it gave for making it, with the refusals in red. A row leads where the call went, so reading what an agent did and diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 11fe7e388b..653ac4e427 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -248,8 +248,31 @@ Point your client at the app itself: That is the app's own launcher, and `--mcp-stdio` makes this copy of it a pipe to the window already open rather than a second window. Nothing else to install and no port to configure: it talks to the run that started most recently, says which one that was, and takes `--agent-run=` when several explorers are -open. A heap dump open before you start is one less thing for it to do, but not a requirement: with no -window it says so, and `open_heap_dump` and `dump_heap` are how it gets one. +open. + +**A window open before you start is not a requirement.** If nothing is running, this opens one — and if the +command line named a heap dump, that window opens it, so the same configuration works whether or not you got +there first: + +```json +"args": ["--mcp-stdio", "--title=For an agent", "/Users/you/dumps/bug-4821.hprof"] +``` + +The window it opens outlives the agent's session, which is the point: whatever it concluded is on the tabs it +left open when you come back to it. + +**And there is a case with no screen at all** — a build server, a heap dump on the far end of an ssh session, +or something driving an agent with nobody watching. Add `--no-ui` and the tools are served from that process +instead of piped to a window: + +```json +"args": ["--mcp-stdio", "--no-ui", "/var/dumps/bug-4821.hprof"] +``` + +Everything works the same except `show`, which has nowhere to put a tab and says so rather than answering that +it showed you something. Nothing else changes, because **notes and verdicts were never on the screen** — they +are files beside the heap dump, so a dump investigated over ssh today opens in a window tomorrow with the +verdicts, the reasons and the conclusion already on it. Then ask for what you actually want. This is the whole prompt the session below was given: @@ -275,7 +298,7 @@ press, because a surface with less than that is one whose answer is "ask your hu | `dominator_tree` | The treemap, without the pixels: where the memory has gone, a level at a time. | | `set_verdict`, `clear_verdict` | The pencil, with the reason required the same way. | | `read_notes`, `take_note` | The notes: where somebody has been, what they wrote, and adding to or replacing it. | -| `show` | Opens a tab in your window and brings it to the front. | +| `show` | Opens a tab in your window and brings it to the front. The one tool a `--no-ui` run can't do. | | `conclude` | The root cause, and the only way to finish. | | `open_heap_dump` | **Open heap dump…**, for a file nobody has open yet. | | `list_devices`, `dump_heap` | **Take heap dump…**: which device, which process, and the dump itself. | diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index 82138c612b..ae8f7481b2 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -13,7 +13,7 @@ reading the source alone — everything else is in the code. Keep it that way. | --- | --- | --- | | `shark-explorer-core` | Heap dump → dominator tree → layout model. Layout, hit testing, navigation state. | **No Compose dependency, Java 8 target.** Must stay reusable from the Android `leakcanary-app`. | | `shark-explorer-jdwp` | Attaches to a live app as a debugger to read the pixels of its bitmaps. | **Imports `com.sun.jdi`, so it needs a JDK and can't be loaded on Android.** That's the whole reason it isn't in `core`. | -| `shark-explorer-agent` | The MCP server a window answers agents through, and the `--mcp-stdio` pipe that reaches it. | **No Compose, Java 8 target, and desktop only** — it calls `ProcessHandle`. Has its own `AGENTS.md`. | +| `shark-explorer-agent` | The MCP server a window answers agents through, the `--mcp-stdio` pipe that reaches it, and `--no-ui` for a run with no window at all. | **No Compose, Java 8 target, and desktop only** — it calls `ProcessHandle`. Has its own `AGENTS.md`. | | `shark-explorer-app` | Compose Desktop UI: window, the canvas each shape draws into, details panel. | **Java 17 target** — see below. | `shark/shark-explorer/` itself holds no code, matching how `shark/` and `leakcanary/` are grouping diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index b89860349a..fc8ce9ce33 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -22,6 +22,7 @@ being talked to by a program that is not this app. | `AgentSessionFile.kt` | One session on disk, both ways: what a call is written as, and what it reads back as. | | `AgentServer.kt` | The loopback socket a run publishes, and the file that says where. | | `AgentStdioBridge.kt` | `--mcp-stdio`: the pipe an MCP client launches. | +| `AgentStdioServer.kt` | And `--no-ui`: the same tools over this process's own stdio, for a run with no window. | | `harness/start-harness.sh` | Opens a window and prints the command that throws an agent at it. | Nothing here is public API — the module is in `modulesWithoutPublicApi`, like the rest of the explorer — with @@ -85,17 +86,37 @@ one log line in the middle of a JSON-RPC stream is a session the client reports - Everything the bridge has to say goes to stderr, which is where an MCP client collects a server's log. - Nothing in the bridge path may use `SharkLog`, `println`, or anything that ends up on stdout. +- `--no-ui` installs the app's logging with stderr as its stream rather than skipping it, because there the + tools run in this process and their diagnostics are worth a log file. Same rule, wider scope. The app's own side of it — a window answering an agent — logs through `SharkLog` as usual, so a session log reads as the reason for each call followed by the reads it caused. That is the artefact to ask for when somebody reports that an agent got it wrong. -## The transport, and why it is two things +## The transport, and why it is three things **A run publishes a loopback port and a token** to `~/.shark-explorer/agents/.agent`, and `--mcp-stdio` is a mode of the same app binary that pipes stdio to it. Two parts because an MCP client can be configured with a command and not with a port that changes every run. +The third is `--no-ui`, which serves [AgentTools] from the `--mcp-stdio` process itself, with no socket and no +window: a build server, or a heap dump at the end of an ssh session. **The two are one code path with one +call swapped**, `AgentHeapDump.show` — see `shark.explorer.app.HeadlessAgentHeapDumps` — and that is the rule +rather than how it happened to land: the notes and the verdicts are files, so a run with no screen is not a +reduced version of the surface, it is the same surface with nowhere to put a tab. + +Two things about the headless one that reading it won't tell you. + +**Nothing may reach stdout at all**, which is stricter than the bridge: the tools run in this process, so the +heap dump's own `SharkLog` diagnostics are in it too. `main` passes `System.err` to `installLogging` in this +mode, and a `println` anywhere under a tool breaks the session rather than only looking untidy. + +**A heap dump named on the command line is opened in the background, not before the first message.** A client +is waiting on `initialize` and a gigabyte of heap dump is minutes of indexing, so a slow dump would be a +server the client kills at startup. What makes that safe is that opening the same path twice joins the open +already in flight instead of starting a second — so an agent that calls `open_heap_dump` on the path it was +pointed at waits for the one that is already happening, and never gets a second index of the same file. + Deliberately **not** the socket `DeepLinkPeers` listens on, though it is the same shape. A link is one line answered in a millisecond; this is a session held open for as long as an investigation takes. One port for both would mean a link arriving mid-investigation and an investigation ending when a link handler closed. diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt index 6d2f9ee7c1..cc682ed04b 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt @@ -93,10 +93,14 @@ interface AgentHeapDump { * Opens [place] in a tab of this window and brings the window to the front, which is what makes an agent's * work something the person at the machine can watch rather than read about afterwards. * - * Not suspending and not answered: this is the same hand-over a `shark://` link makes — a place put where - * the tabs take it on the next frame — so there is nothing to wait for and nothing that can fail here. + * Not suspending: this is the same hand-over a `shark://` link makes — a place put where the tabs take it + * on the next frame — so there is nothing to wait for. + * + * @return null when it was shown, and why it wasn't otherwise. Which is one case, a run started with no + * window at all, and it is worth answering rather than logging: an agent told its human to look at + * something they cannot see has said the one thing worse than nothing. */ - fun show(place: Place) + fun show(place: Place): String? } /** diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt index 3f737cfe68..9fb915ee49 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt @@ -8,6 +8,7 @@ import java.io.PrintWriter import java.net.InetAddress import java.net.InetSocketAddress import java.net.Socket +import java.util.concurrent.atomic.AtomicBoolean /** * Standard input and output, wired to the run of the app an agent wants to talk to. @@ -39,9 +40,18 @@ object AgentStdioBridge { /** Which run, by process id, or null for the one that started most recently. */ pid: String? = null, /** How long to wait for a run to appear, for a client that launched this before the app was open. */ - waitMillis: Long = DEFAULT_WAIT_MILLIS + waitMillis: Long = DEFAULT_WAIT_MILLIS, + /** + * How to open a window to investigate in when no run of the app is open, and null to wait for one. + * + * Because the alternative is an agent whose only answer is "ask somebody to launch Shark Explorer", and + * a window opened here is a window the person at the machine can then watch — which is the whole reason + * this surface is a window rather than a library. Not called when a run was asked for by [pid]: that + * names a window, and opening a different one would be answering about the wrong heap dump. + */ + openAWindow: (() -> Unit)? = null ): Int { - val run = waitForRun(directory, pid, waitMillis) ?: return NOTHING_TO_TALK_TO + val run = waitForRun(directory, pid, waitMillis, openAWindow) ?: return NOTHING_TO_TALK_TO val socket = try { Socket().apply { connect(InetSocketAddress(InetAddress.getLoopbackAddress(), run.port), CONNECT_TIMEOUT_MILLIS) @@ -67,17 +77,27 @@ object AgentStdioBridge { return NOTHING_TO_TALK_TO } say("Talking to Shark Explorer run ${run.pid}") + // Set before the socket is closed from this side, so that the read it interrupts knows it was us. Which is + // how *every* session that ends normally ends, so without this each one finishes with a stack trace on + // stderr — where an MCP client collects a server's log, and reads it as the server having crashed. + val ending = AtomicBoolean(false) // The app's answers on their own thread, because both directions are blocking reads and a client sends // its next message without waiting to be answered. val answers = Thread({ val out = PrintWriter(OutputStreamWriter(System.out, Charsets.UTF_8), true) - while (true) { - val line = fromApp.readLine() ?: break - out.println(line) + try { + while (true) { + val line = fromApp.readLine() ?: break + out.println(line) + } + // The window closed, which ends the session: nothing is going to answer the client's next message. + say("Shark Explorer run ${run.pid} closed the connection") + } catch (throwable: Throwable) { + if (!ending.get()) { + say("Shark Explorer run ${run.pid} stopped answering: $throwable") + } } - // The window closed, which ends the session: nothing is going to answer the client's next message. - say("Shark Explorer run ${run.pid} closed the connection") - System.out.flush() + out.flush() }, "shark-explorer-agent-answers").apply { isDaemon = true start() @@ -92,6 +112,7 @@ object AgentStdioBridge { } } // The client closed its end, which is how a session normally ends. + ending.set(true) socket.close() answers.join(SHUTDOWN_MILLIS) return 0 @@ -100,12 +121,26 @@ object AgentStdioBridge { private fun waitForRun( directory: File, pid: String?, - waitMillis: Long + waitMillis: Long, + openAWindow: (() -> Unit)? ): AgentServer.PublishedRun? { var waited = 0L + var deadline = waitMillis + var opened = false + // Naming a run names a window and therefore a heap dump, so opening a different one would be answering + // about the wrong dump: for that command line there is nothing to open, only something to wait for. + val opensAWindow = openAWindow != null && pid == null while (true) { val runs = AgentServer.publishedRuns(directory) val run = if (pid == null) runs.firstOrNull() else runs.firstOrNull { it.pid == pid } + if (run == null && !opened && opensAWindow) { + say("No Shark Explorer is running, so one is being opened to investigate in.") + requireNotNull(openAWindow).invoke() + opened = true + // From here rather than from the start, because what is being waited for changed: a JVM starting, + // Compose coming up and a window appearing, rather than a file that may already be there. + deadline = waited + OPENING_WAIT_MILLIS + } if (run != null) { if (pid == null && runs.size > 1) { // Which run an agent ends up in is worth saying rather than leaving to be worked out from what @@ -117,9 +152,13 @@ object AgentStdioBridge { } return run } - if (waited >= waitMillis) { + if (waited >= deadline) { say( - if (pid == null) { + if (pid == null && opened) { + "A Shark Explorer was started and has not published itself in " + + "${OPENING_WAIT_MILLIS / 1000} seconds, so something went wrong opening it. Its log is in " + + "the newest file under ~/.shark-explorer/logs." + } else if (pid == null) { "No Shark Explorer is running, so there is no heap dump to investigate. Open one — every run " + "of the app publishes itself in $directory — and start this again." } else { @@ -148,6 +187,15 @@ object AgentStdioBridge { const val PID_OPTION = "--agent-run=" private const val DEFAULT_WAIT_MILLIS = 10_000L + + /** + * How long a window opened from here is given to publish itself. + * + * Longer than [DEFAULT_WAIT_MILLIS] by a lot, because it covers a cold JVM, Compose starting and jlink's + * runtime being paged in — and because the alternative to waiting is telling an agent there is no window + * while one is in the middle of appearing. + */ + private const val OPENING_WAIT_MILLIS = 60_000L private const val POLL_MILLIS = 250L private const val CONNECT_TIMEOUT_MILLIS = 1_000 private const val SHUTDOWN_MILLIS = 500L diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioServer.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioServer.kt new file mode 100644 index 0000000000..b7e8daa0e7 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioServer.kt @@ -0,0 +1,58 @@ +package shark.explorer.agent + +import java.io.BufferedReader +import java.io.File +import java.io.InputStreamReader +import java.io.OutputStreamWriter +import java.io.PrintWriter +import kotlinx.coroutines.runBlocking + +/** + * The same tools over this process's own stdin and stdout, for a run with no window. + * + * [AgentStdioBridge] is a pipe to a window somebody is watching; this is the other case, a machine with no + * screen to watch it on — someone's build server, or a heap dump on the far end of an ssh session. Both are + * `--mcp-stdio`, and which one a run is depends only on whether it was told there is no UI. + * + * There is no socket and no token here, because there is nothing to find: the client launched this process and + * talks to it down the pipe it already holds. Which also makes it the one shape of this surface with no + * authorization question at all. + * + * **Nothing may be written to stdout but protocol**, the same rule the bridge is under, and here it reaches + * further: the tools run in this process, so the heap dump's own diagnostics are in it too. `main` points + * those at stderr in this mode. See `shark.explorer.app.installLogging`. + */ +object AgentStdioServer { + + /** + * Answers a message per line until the client closes its end, and returns the exit code to end with. + * + * One message answered before the next is read, exactly as [AgentServer] serves a socket: a client sends + * its next call after it has been answered anyway, and the reads inside suspend onto whichever thread owns + * the heap dump. + */ + fun run( + heapDumps: AgentHeapDumps, + serverVersion: String, + /** Where a session is written down, the same directory the windowed runs write theirs to. */ + sessions: File + ): Int { + // Named before the handshake, like a socket session is, so that a client which connects and says nothing + // is still a row on the *Agent logs* screen of whoever reads these later. + val sessionFile = AgentSessionFile.starting(sessions, serverVersion) + val session = McpSession(AgentTools(heapDumps), serverVersion, sessionFile) + val reader = BufferedReader(InputStreamReader(System.`in`, Charsets.UTF_8)) + val writer = PrintWriter(OutputStreamWriter(System.out, Charsets.UTF_8), true) + while (true) { + val line = reader.readLine() ?: break + if (line.isBlank()) { + continue + } + val answer = runBlocking { session.answer(line) } + if (answer != null) { + writer.println(answer) + } + } + return 0 + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index c9f92a5794..1282a50e4c 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -92,8 +92,9 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { if (dumps.isEmpty()) { put( "problem", - "No heap dump is open. Shark Explorer is running, but every window of it is empty — open a " + - "dump in the app, or ask whoever is at the machine to." + "No heap dump is open yet. Call $OPEN_HEAP_DUMP with the path of an `.hprof` file, or dump_heap " + + "to take one off a device. If you were pointed at a heap dump, that path is the one to open: it " + + "may be being indexed right now, and opening it again waits for that rather than starting over." ) } } @@ -416,8 +417,12 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { ) { arguments -> val dump = arguments.heapDump() val place = arguments.place() - dump.show(place) - buildJsonObject { put("shown", true) } + val problem = dump.show(place) + buildJsonObject { + put("shown", problem == null) + // So that an agent about to tell its human where to look finds out that there is nowhere. + put("problem", problem) + } } private fun conclude() = AgentTool( @@ -464,7 +469,7 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { reason = arguments.reason ) dump.appendToNote(Place.Object(objectId), note) - dump.show(Place.Object(objectId)) + val showProblem = dump.show(Place.Object(objectId)) buildJsonObject { put("concluded", true) putJsonArray("faultyReference") { @@ -482,7 +487,11 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { } } } - put("writtenTo", "the notes of ${exactHexObjectId(objectId)}, and shown in window ${dump.windowId}") + put( + "writtenTo", + "the notes of ${exactHexObjectId(objectId)}" + + if (showProblem == null) ", and shown in window ${dump.windowId}" else ". $showProblem" + ) } } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt index 70be652c75..dfce9a5dff 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioBridgeTest.kt @@ -36,6 +36,9 @@ class AgentStdioBridgeTest { private lateinit var window: FakeAgentHeapDump private val closeables = mutableListOf() + /** What the client would collect as this server's log. See [bridge]. */ + private val said = ByteArrayOutputStream() + @Before fun setUp() { directory = temporaryFolder.newFolder("agents") @@ -70,6 +73,9 @@ class AgentStdioBridgeTest { assertThat(answers[0]).contains("\"id\":1").contains("shark-explorer") assertThat(answers[1]).contains("\"id\":2").contains(HOLDER_CLASS_NAME) assertThat(window.reads).isNotEmpty + // A session that ended the way every session ends — the client closing stdin — and stderr is where an + // MCP client collects a server's log, so a stack trace here is a client reporting a crash on every run. + assertThat(said.toString(Charsets.UTF_8.name())).doesNotContain("Exception") } @Test @@ -79,6 +85,49 @@ class AgentStdioBridgeTest { assertThat(exitCode).isEqualTo(NOTHING_TO_TALK_TO) } + @Test + fun `a window is opened when there is none, and talked to once it publishes itself`() { + var opened = 0 + + val answers = bridge( + openAWindow = { + opened++ + // The last thing a real one does as it comes up, and the only part of it this test needs: what the + // bridge is waiting for is a published run, not a display. Starting a process would be a jlink build + // and a screen, and the wait either side of it is this same wait. + closeables += AgentServer.listen( + heapDumps = FakeAgentHeapDumps(listOf(window)), + serverVersion = "1.2.3", + directory = directory + ) + } + ) { send -> + send("""{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18"}}""") + } + + // Once, however long the window takes: an agent that waited through two of these would have two windows. + assertThat(opened).isEqualTo(1) + assertThat(answers).hasSize(1) + assertThat(answers[0]).contains("\"id\":1") + } + + @Test + fun `a run asked for by pid is waited for rather than replaced with a new window`() { + var opened = 0 + + val exitCode = AgentStdioBridge.run( + directory, + pid = "999999", + waitMillis = 0L, + openAWindow = { opened++ } + ) + + // Naming a run names a window, and a heap dump: opening a different one would be answering confidently + // about the wrong dump, which is worse than saying that run has gone. + assertThat(exitCode).isEqualTo(NOTHING_TO_TALK_TO) + assertThat(opened).isZero + } + @Test fun `a run that no longer answers on its port has its file cleared out`() { val port = ServerSocket(0).use { it.localPort } @@ -103,16 +152,23 @@ class AgentStdioBridgeTest { * for: closing it the moment the last message is written would be a race with the answer coming back, and a * test that lost it would be reporting the timing rather than the wiring. */ - private fun bridge(session: (send: (String) -> Unit) -> Unit): List { + private fun bridge( + openAWindow: (() -> Unit)? = null, + session: (send: (String) -> Unit) -> Unit + ): List { val stdin = PipedOutputStream() val stdout = ByteArrayOutputStream() val previousIn = System.`in` val previousOut = System.out + val previousErr = System.err System.setIn(PipedInputStream(stdin)) System.setOut(PrintStream(stdout, true, Charsets.UTF_8.name())) + // Taken over as well as stdout, because what a server says about itself is half of what a client shows + // when something goes wrong — and because a trace printed here is the thing one of these tests is about. + System.setErr(PrintStream(said, true, Charsets.UTF_8.name())) var sent = 0 try { - val bridge = Thread({ runBridge() }, "bridge under test").apply { + val bridge = Thread({ runBridge(openAWindow) }, "bridge under test").apply { isDaemon = true start() } @@ -129,12 +185,14 @@ class AgentStdioBridgeTest { } finally { System.setIn(previousIn) System.setOut(previousOut) + System.setErr(previousErr) } return stdout.toString(Charsets.UTF_8.name()).lines().filter { it.isNotBlank() } } /** Nothing waited for, since the run these tests are about is either already published or never will be. */ - private fun runBridge(): Int = AgentStdioBridge.run(directory, pid = null, waitMillis = 0L) + private fun runBridge(openAWindow: (() -> Unit)? = null): Int = + AgentStdioBridge.run(directory, pid = null, waitMillis = 0L, openAWindow = openAWindow) private fun awaitLines( stdout: ByteArrayOutputStream, diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioServerTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioServerTest.kt new file mode 100644 index 0000000000..224b7eb01f --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentStdioServerTest.kt @@ -0,0 +1,157 @@ +package shark.explorer.agent + +import java.io.ByteArrayOutputStream +import java.io.File +import java.io.PipedInputStream +import java.io.PipedOutputStream +import java.io.PrintStream +import org.assertj.core.api.Assertions.assertThat +import org.junit.After +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import shark.explorer.exactHexObjectId + +/** + * The tools served over this process's own stdio, which is `--mcp-stdio --no-ui`. + * + * The same shape as [AgentStdioBridgeTest] and the opposite case: there the tools are in another process and + * this is a pipe to it, here there is no other process and no socket at all. Worth its own test because that + * makes it the one path where an agent's calls and the heap dump's own diagnostics share a process, and the + * rule is that only one of them may reach stdout. + */ +class AgentStdioServerTest { + + @get:Rule + val temporaryFolder = TemporaryFolder() + + @get:Rule + val log = RecordedLog() + + private lateinit var sessions: File + private lateinit var heapDump: InvestigationHeapDump + private lateinit var dump: FakeAgentHeapDump + + @Before + fun setUp() { + sessions = temporaryFolder.newFolder("sessions") + heapDump = temporaryFolder.applicationHoldsActivityThroughHolder() + dump = FakeAgentHeapDump(heapDump.explorer) + } + + @After + fun tearDown() { + heapDump.close() + } + + @Test + fun `a client's messages are answered from this process`() { + val answers = serve { send -> + send("""{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18"}}""") + send( + """{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"describe_object",""" + + """"arguments":{"object":"${exactHexObjectId(heapDump.holderObjectId)}",""" + + """"reason":"Reading the holder's fields with no window open."}}}""" + ) + } + + assertThat(answers).hasSize(2) + assertThat(answers[0]).contains("\"id\":1").contains("shark-explorer") + assertThat(answers[1]).contains("\"id\":2").contains(HOLDER_CLASS_NAME) + assertThat(dump.reads).isNotEmpty + } + + @Test + fun `a session with no window is written down like any other`() { + serve { send -> + send("""{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18"}}""") + send( + """{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"list_leaks",""" + + """"arguments":{"reason":"What the dump says about itself."}}}""" + ) + } + + // The *Agent logs* screen of whoever opens this dump in a window later is what reads these, which is the + // point of writing them from a run that has no screen at all. + val session = AgentSessionFile.sessionsIn(sessions).single() + assertThat(session.calls.map { it.tool }).contains("list_leaks") + } + + @Test + fun `a blank line is not a message`() { + val answers = serve { send -> + send("") + send("""{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18"}}""") + } + + // A client that ends its messages with a newline the reader then sees again would otherwise get an error + // for a message it never sent. + assertThat(answers).hasSize(1) + } + + /** + * Runs the server over a pipe, sends what [session] sends, and hands back the lines that came out. + * + * A pipe rather than a string of input, for the reason [AgentStdioBridgeTest] uses one: a real client keeps + * stdin open until it has been answered, so closing it after the last message would be a race with the + * answer. + */ + private fun serve(session: (send: (String) -> Unit) -> Unit): List { + val stdin = PipedOutputStream() + val stdout = ByteArrayOutputStream() + val previousIn = System.`in` + val previousOut = System.out + System.setIn(PipedInputStream(stdin)) + System.setOut(PrintStream(stdout, true, Charsets.UTF_8.name())) + var expected = 0 + try { + val server = Thread({ + AgentStdioServer.run( + heapDumps = FakeAgentHeapDumps(listOf(dump)), + serverVersion = "1.2.3", + sessions = sessions + ) + }, "stdio server under test").apply { + isDaemon = true + start() + } + session { message -> + stdin.write("$message\n".toByteArray(Charsets.UTF_8)) + stdin.flush() + if (message.isNotBlank()) { + expected++ + awaitLines(stdout, expected) + } + } + stdin.close() + server.join(JOIN_MILLIS) + } finally { + System.setIn(previousIn) + System.setOut(previousOut) + } + return stdout.toString(Charsets.UTF_8.name()).lines().filter { it.isNotBlank() } + } + + private fun awaitLines( + stdout: ByteArrayOutputStream, + count: Int + ) { + val giveUpAt = System.currentTimeMillis() + AWAIT_MILLIS + while (System.currentTimeMillis() < giveUpAt) { + if (stdout.toString(Charsets.UTF_8.name()).lines().count { it.isNotBlank() } >= count) { + return + } + Thread.sleep(POLL_MILLIS) + } + throw AssertionError( + "Waited ${AWAIT_MILLIS}ms for $count answers and got: ${stdout.toString(Charsets.UTF_8.name())}" + ) + } + + private companion object { + const val AWAIT_MILLIS = 10_000L + const val POLL_MILLIS = 20L + const val JOIN_MILLIS = 5_000L + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt index 0e6d00e15f..45c4bad06d 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt @@ -78,8 +78,11 @@ internal class FakeAgentHeapDump( override suspend fun notedPlaces(): List = notes.keys.toList() - override fun show(place: Place) { + override fun show(place: Place): String? { shown += place + // Null is "it was shown", which is what a window answers. The refusal a run with no window makes is + // `HeadlessAgentHeapDumpsTest`'s, since it is that run's one difference from this one. + return null } override fun close() { diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkScheme.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkScheme.kt index 7b7c32af1f..cb616406ab 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkScheme.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkScheme.kt @@ -83,7 +83,7 @@ internal object DeepLinkScheme { * starts to deliver one. */ fun registerWithTheOs() { - val launcher = launcherPath() + val launcher = launcherPathOrNull() if (launcher == null) { SharkLog.d { "Not registering ${DeepLink.SCHEME}:// with the OS: this run is a JVM on a classpath rather than " + @@ -108,18 +108,6 @@ internal object DeepLinkScheme { } } - /** - * The executable a link should be started with, or null when this run is not one. - * - * A packaged build is a launcher jpackage generated; everything else is `java`, and registering that - * would tell the OS to open links with a JVM and no classpath. - */ - private fun launcherPath(): String? { - val command = ProcessHandle.current().info().command().orElse(null) ?: return null - val name = File(command).name - return if (name in JVM_EXECUTABLES) null else command - } - private fun registerOnWindows(launcher: String) { val command = "\"$launcher\" \"%1\"" if (run("reg", "query", COMMAND_KEY, "/ve").contains(launcher)) { @@ -183,9 +171,6 @@ internal object DeepLinkScheme { private fun osName(): String = System.getProperty("os.name").orEmpty().lowercase() - /** A run launched as one of these is a classpath rather than an app, whatever bundle it came out of. */ - private val JVM_EXECUTABLES = setOf("java", "java.exe", "javaw.exe") - private val SCHEME_KEY = "HKCU\\Software\\Classes\\${DeepLink.SCHEME}" private val COMMAND_KEY = "$SCHEME_KEY\\shell\\open\\command" diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt index 17a682e9b6..c37719e926 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt @@ -20,6 +20,7 @@ import shark.explorer.agent.AgentServer import shark.explorer.agent.AgentSession import shark.explorer.agent.AgentSessionFile import shark.explorer.agent.AgentStdioBridge +import shark.explorer.agent.AgentStdioServer import shark.explorer.placeOfNoteKeyOrNull /** @@ -39,39 +40,17 @@ internal fun listenForAgents( ) /** - * How an agent reaches this run: the app's side of `shark-explorer-agent`. + * Everything an agent can ask this run that isn't a question about one open heap dump: which dumps are open, + * opening another, and the two `adb` questions behind taking one off a device. * - * The one thing worth knowing here is **why an agent is given a window and not a heap dump file**. Every read - * goes through that window's own [HeapDumpSession], so an agent's question queues on the thread the person at - * the machine is already reading with, and the verdicts and notes it writes are the ones that window is - * drawing. An agent that opened the dump itself would be answering about a heap dump nobody is looking at, and - * its conclusions would land in a file the open window would then overwrite. - * - * Which is also why opening a dump and taking one off a device end in a window here rather than in a file - * path: they are the two buttons above the map, and an agent pressing one has to end up somewhere its human - * can follow it to. + * Whether this run has windows shows up in exactly two places — which dumps are open, and what opening one + * means — so those are what a subclass answers and the rest is here. A device is a device either way, and a + * heap dump pulled off one is a file that then has to be opened, which is [open] again. */ -private class WindowAgentHeapDumps( - private val windows: ExplorerWindows, +internal abstract class RunAgentHeapDumps( private val deviceHeapDumps: DeviceHeapDumps ) : AgentHeapDumps { - override fun openHeapDumps(): List = - // Asked per call rather than captured, because windows come and go while an agent is connected: a tool - // naming a window that has since closed has to be an error message and not a stale answer. - windows.mapNotNull { window -> - window.openHeapDump?.let { open -> WindowAgentHeapDump(window, open) } - } - - override suspend fun open(file: File): AgentHeapDump { - // Edited from whichever thread the agent's connection is on, exactly as a link arriving from another run - // of this app edits it: a window is snapshot state, and the composition takes the change on the next - // frame. See [ExplorerWindow.linkedPlaces]. - val window = windows.openHeapDump(file) - SharkLog.d { "An agent opened ${file.absolutePath} in window ${window.deepLinkId}" } - return awaitHeapDump(window) - } - override suspend fun devices(): List = onAdbThread { deviceHeapDumps.connectedDevices() } @@ -97,35 +76,9 @@ private class WindowAgentHeapDumps( deviceHeapDumps.dumpHeap(device, process) { step -> SharkLog.d { "For an agent: $step" } } } // No pixels fetched to go with it, unlike the dialog's tick box: that is a second suspension of the app, - // minutes of it, and an agent reads a bitmap's size rather than looking at it. Whoever is at the window - // can still fetch them from the panel afterwards. - val window = windows.openHeapDump(heapDumpFile) - SharkLog.d { "An agent dumped $processName into window ${window.deepLinkId}" } - return awaitHeapDump(window) - } - - /** - * Waits for [window]'s heap dump to be readable, and refuses when it never will be. - * - * Three ways it ends and only one of them is an answer — the dump opens, it fails to open, or the window is - * closed under it — because a window id handed over before its dump is open is one that refuses every call - * made with it, and either of the other two would otherwise be a call that never comes back. Which is what - * [ExplorerWindow.openProblem] exists for. - */ - private suspend fun awaitHeapDump(window: ExplorerWindow): AgentHeapDump { - snapshotFlow { - window.openHeapDump != null || window.openProblem != null || window !in windows - }.first { it } - val open = window.openHeapDump - if (open != null) { - return WindowAgentHeapDump(window, open) - } - val name = window.heapDumpFile?.name - throw AgentRefusal( - window.openProblem?.let { "$name could not be opened as a heap dump: $it" } - ?: "Window ${window.deepLinkId} was closed before $name had finished opening, so there is nothing " + - "to read. Opening it again is a call away." - ) + // minutes of it, and an agent reads a bitmap's size rather than looking at it. Whoever ends up at the + // window can still fetch them from the panel afterwards. + return open(heapDumpFile) } /** The device with this serial number, or a refusal listing the ones there are. */ @@ -145,48 +98,227 @@ private class WindowAgentHeapDumps( /** * Everything `adb` blocks on, off the connection's thread. * - * Which is not the heap dump's thread either: a dump takes minutes of shelling out, and the window it will - * open in is being read by whoever is at the machine while it does. + * Which is not the heap dump's thread either: a dump takes minutes of shelling out, and whatever heap dump + * is already open is being read while it does. */ private suspend fun onAdbThread(block: () -> T): T = withContext(Dispatchers.IO) { block() } } /** - * Whether this process was started to be a pipe between an agent and another run of the app, and what to - * exit with if it was. Null for every other command line. + * How an agent reaches the windows of this run: the app's side of `shark-explorer-agent`. + * + * The one thing worth knowing here is **why an agent is given a window and not a heap dump file**. Every read + * goes through that window's own [HeapDumpSession], so an agent's question queues on the thread the person at + * the machine is already reading with, and the verdicts and notes it writes are the ones that window is + * drawing. An agent that opened the dump itself would be answering about a heap dump nobody is looking at, and + * its conclusions would land in a file the open window would then overwrite. + * + * Which is also why opening a dump and taking one off a device end in a window here rather than in a file + * path: they are the two buttons above the map, and an agent pressing one has to end up somewhere its human + * can follow it to. [HeadlessAgentHeapDumps] is the same surface for a run that has no window at all. + */ +internal class WindowAgentHeapDumps( + private val windows: ExplorerWindows, + deviceHeapDumps: DeviceHeapDumps +) : RunAgentHeapDumps(deviceHeapDumps) { + + override fun openHeapDumps(): List = + // Asked per call rather than captured, because windows come and go while an agent is connected: a tool + // naming a window that has since closed has to be an error message and not a stale answer. + windows.mapNotNull { window -> + window.openHeapDump?.let { open -> window.agentHeapDump(open) } + } + + override suspend fun open(file: File): AgentHeapDump { + // A window already on this file rather than a second window on it, which is the opposite of what the + // button does: a person clicking `Open heap dump…` twice on one dump is comparing two readings of it, and + // an agent naming a path is naming a heap dump. Which matters most in the case this tool was written for — + // a window opened for an agent publishes this run before its dump is readable, so the agent's first move + // is to open the path it was pointed at, and a second window on it would be a second index of the same + // gigabyte and a window nobody asked for. + val already = windows.firstOrNull { it.heapDumpFile?.absoluteFile == file.absoluteFile } + // Edited from whichever thread the agent's connection is on, exactly as a link arriving from another run + // of this app edits it: a window is snapshot state, and the composition takes the change on the next + // frame. See [ExplorerWindow.linkedPlaces]. + val window = already ?: windows.openHeapDump(file) + SharkLog.d { + if (already == null) { + "An agent opened ${file.absolutePath} in window ${window.deepLinkId}" + } else { + "An agent asked for ${file.absolutePath}, which window ${window.deepLinkId} already has" + } + } + // Three ways this ends and only one of them is an answer — the dump opens, it fails to open, or the + // window is closed under it — because a window id handed over before its dump is open is one that refuses + // every call made with it, and either of the other two would otherwise be a call that never comes back. + // Which is what [ExplorerWindow.openProblem] exists for. + snapshotFlow { + window.openHeapDump != null || window.openProblem != null || window !in windows + }.first { it } + val open = window.openHeapDump + if (open != null) { + return window.agentHeapDump(open) + } + throw AgentRefusal( + window.openProblem?.let { "${file.name} could not be opened as a heap dump: $it" } + ?: "Window ${window.deepLinkId} was closed before ${file.name} had finished opening, so there is " + + "nothing to read. Opening it again is a call away." + ) + } +} + +/** + * Whether this process was started to talk MCP over stdio, and what to exit with if it was. Null for every + * other command line, which is the app opening windows. * * Answered before anything else in `main` and before any logging is installed, because the app's logger - * writes to stdout and in this mode stdout is the protocol. See [AgentStdioBridge]. + * writes to stdout and in this mode **stdout is the protocol**. Which of the two shapes it is depends only on + * whether there is a screen to investigate on: [AgentStdioBridge] pipes to a window, and [NO_UI_OPTION] + * serves the tools from this process. */ internal fun agentBridgeExitCode(args: Array): Int? { if (MCP_STDIO_OPTION !in args) { return null } + val arguments = try { + agentServerArguments(args) + } catch (invalidArguments: IllegalArgumentException) { + // On stderr, where an MCP client collects a server's log, since there is no window and no console to + // print a usage message to. + saidToTheClient(invalidArguments.message.orEmpty()) + return UNREADABLE_COMMAND_LINE + } + if (NO_UI_OPTION in args) { + return serveAgentsWithNoWindow(arguments) + } val pid = args.firstOrNull { it.startsWith(AgentStdioBridge.PID_OPTION) } ?.removePrefix(AgentStdioBridge.PID_OPTION) - return AgentStdioBridge.run(directory = AGENT_RUNS_DIRECTORY, pid = pid) + return AgentStdioBridge.run( + directory = AGENT_RUNS_DIRECTORY, + pid = pid, + // So that an agent pointed at a machine where nothing is open gets a heap dump to investigate and its + // human gets a window to watch it in, rather than being told to go and launch something. Declined when + // this run has no way of knowing what started it — see [relaunchCommand]. + openAWindow = relaunchCommand()?.let { command -> { openAnotherRun(command, arguments) } } + ) +} + +/** + * The rest of the command line, once the three options that make this a server are off it. + * + * Because what is left is an ordinary command line — heap dumps to open, a title to call their windows — and + * it means the same thing: a client's configuration says which dump to investigate the way a terminal does. + * Taken off here rather than taught to the parser, so that a window is the only thing that ever sees them. + * + * Throws [IllegalArgumentException] for a command line that doesn't read, like the parser it wraps. + */ +internal fun agentServerArguments(args: Array): ExplorerArguments = ExplorerArguments.parse( + args.filterNot { + it == MCP_STDIO_OPTION || it == NO_UI_OPTION || it.startsWith(AgentStdioBridge.PID_OPTION) + } +) + +/** + * Answers an agent's calls from this process, with no window anywhere. + * + * For a machine with no screen — a build server, or a heap dump on the far end of an ssh session — and for + * anything that drives an agent without a person watching, which is what the eval is. Everything an + * investigation leaves behind is on disk either way, so a dump worked on here opens in a window later with + * the notes and the verdicts on it. See [HeadlessAgentHeapDumps]. + */ +private fun serveAgentsWithNoWindow(arguments: ExplorerArguments): Int { + // On stderr, because stdout is the protocol and the tools run in this process: unlike the bridge, the heap + // dump's own diagnostics are in this stream too, and every one of them would be a broken JSON-RPC message. + // The log file is written as usual, which is what makes a headless session as readable as a windowed one. + return installLogging(System.err).use { + SharkLog.d { "Answering an agent over stdio, with no window" } + HeadlessAgentHeapDumps( + deviceHeapDumps = commandLineDeviceHeapDumps(), + heapDumpFiles = arguments.heapDumpFiles + ).use { heapDumps -> + AgentStdioServer.run( + heapDumps = heapDumps, + serverVersion = SharkExplorerVersion.current, + sessions = AgentServer.sessionsDirectory(AGENT_RUNS_DIRECTORY) + ) + } + } +} + +/** + * Starts another Shark Explorer, with a window, and leaves it running. + * + * **Deliberately outliving this process.** The bridge ends when the agent's client closes the pipe, and the + * window it opened is the whole point: whoever is at the machine reads the notes and the verdicts afterwards, + * on the tabs the agent left open. + * + * With the same command line this run was given, so that a client configured to investigate one heap dump + * opens a window on that dump rather than an empty one the agent then has to fill. + */ +private fun openAnotherRun( + command: List, + arguments: ExplorerArguments +) { + val titled = command + arguments.heapDumpFiles.map { it.absolutePath } + + "$TITLE_OPTION=${arguments.titlePrefix ?: AGENT_WINDOW_TITLE}" + try { + ProcessBuilder(titled) + // Its stdout is where its own diagnostics go, and they are in its log file too; its stderr is worth + // inheriting, because a window that dies before it can open a log file says why only there. + .redirectOutput(ProcessBuilder.Redirect.DISCARD) + .redirectError(ProcessBuilder.Redirect.INHERIT) + .start() + } catch (throwable: Throwable) { + saidToTheClient("Could not start a window with ${titled.joinToString(" ")}: $throwable") + } +} + +/** + * On stderr, always, which is where an MCP client collects what a server has to say. + * + * Not through `SharkLog`, and not only because stdout is the protocol: this is said before any logging has + * been installed, by the two paths that end before there is anything to install it for. + */ +private fun saidToTheClient(message: String) { + System.err.println("[shark-explorer] $message") } /** - * What a window has open, for everything that isn't drawing it. + * One heap dump open: the thread every read of it queues on, and the two things an investigation writes into. * - * The heap dump's session plus the two things an investigation writes into, gathered because they are all - * per heap dump and are all reached the same way — through the window rather than through the file. See - * [ExplorerWindow.openHeapDump]. + * Gathered because they are all per heap dump and all wanted together by everything that isn't drawing the + * window — which is an agent, whether or not there is a window. See [ExplorerWindow.openHeapDump]. */ -internal class WindowHeapDump( +internal class OpenHeapDump( val session: HeapDumpSession, val notes: HeapDumpNotes, val leakStatuses: HeapDumpLeakStatuses ) -/** One window's heap dump, as the agent surface sees it. */ -private class WindowAgentHeapDump( - private val window: ExplorerWindow, - private val open: WindowHeapDump -) : AgentHeapDump { +/** This window's heap dump as an agent sees it: shown by going to a tab, the way a link does. */ +private fun ExplorerWindow.agentHeapDump(open: OpenHeapDump): AgentHeapDump = + OpenAgentHeapDump(windowId = deepLinkId, open = open) { place -> + SharkLog.d { "An agent asked window $deepLinkId for $place" } + // The same two steps following a link takes, which is what makes an agent showing something and a + // person clicking a link land in the same place. See [ExplorerWindows.open]. + goToLinked(place) + bringToFront() + null + } - override val windowId: String get() = window.deepLinkId +/** + * One open heap dump, as the agent surface sees it, however this run came by it. + * + * One class rather than one per kind of run, because what differs between a window and a machine with no + * screen is a single call: where [show] puts a place. Everything else — the reads, the verdicts, the notes and + * every refusal about them — is about the heap dump and the files beside it, which are the same either way. + */ +internal class OpenAgentHeapDump( + override val windowId: String, + private val open: OpenHeapDump, + /** Where a place goes, answering with why it couldn't. See [AgentHeapDump.show]. */ + private val showPlace: (Place) -> String? +) : AgentHeapDump { override val heapDumpPath: String get() = open.session.heapDumpFile.absolutePath @@ -230,13 +362,15 @@ private class WindowAgentHeapDump( return open.notes.writtenAbout.mapNotNull { key -> placeOfNoteKeyOrNull(key) } } + override fun show(place: Place): String? = showPlace(place) + /** * Puts what [newText] makes of the saved note on disk, whether that is the note plus a paragraph or * something else entirely. * * **Refuses while somebody is typing in that note**, which is the one case where writing would cost * something that exists nowhere else: a draft is unsaved text, and saving over it would put half a sentence - * of theirs on disk under an answer of ours. + * of theirs on disk under an answer of ours. A run with no window has no drafts, so there it never fires. */ private suspend fun write( place: Place, @@ -277,14 +411,6 @@ private class WindowAgentHeapDump( return notepad } - override fun show(place: Place) { - SharkLog.d { "An agent asked window ${window.deepLinkId} for $place" } - // The same two steps following a link takes, which is what makes an agent showing something and a - // person clicking a link land in the same place. See [ExplorerWindows.open]. - window.goToLinked(place) - window.bringToFront() - } - /** * Refuses until the file of statuses set by hand has been read. * @@ -317,7 +443,27 @@ internal fun agentSessions(): List = AgentSessionFile.sessionsIn(AgentServer.sessionsDirectory(AGENT_RUNS_DIRECTORY)) /** Beside the runs answering links, the notes, the statuses and the logs. See [AgentServer]. */ -private val AGENT_RUNS_DIRECTORY = File(SHARK_EXPLORER_DIRECTORY, "agents") +internal val AGENT_RUNS_DIRECTORY = File(SHARK_EXPLORER_DIRECTORY, "agents") -/** What a command line says to be a pipe rather than a window. See [AgentStdioBridge]. */ +/** What a command says to talk MCP over stdio rather than open a window. See [AgentStdioBridge]. */ internal const val MCP_STDIO_OPTION = "--mcp-stdio" + +/** + * What a command says to answer an agent from this process rather than pipe it to a window. + * + * Only meaningful with [MCP_STDIO_OPTION], and deliberately not a way to run the app without a UI: the app + * *is* its windows, so a run that opened none and served nobody would sit there doing nothing. A command line + * with this and no `--mcp-stdio` is a window, which is the one thing it can't have meant. + */ +internal const val NO_UI_OPTION = "--no-ui" + +/** What a window opened for an agent that found none is called, since nobody typed a title for it. */ +private const val AGENT_WINDOW_TITLE = "Opened for an agent" + +/** + * What this process ends with when the command line it was given doesn't read. + * + * A failure rather than a message and a window, because a client that launched this has nowhere to show one: + * an MCP server that starts and lists no tools reads as a server with no tools. + */ +private const val UNREADABLE_COMMAND_LINE = 1 diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerArguments.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerArguments.kt index 956ed81761..aedfc7612b 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerArguments.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerArguments.kt @@ -73,11 +73,17 @@ internal data class ExplorerArguments( ) } - private const val TITLE_OPTION = "--title" - /** Shown with whatever was wrong, so that the message says what to type instead. */ private val USAGE = "Usage: shark-explorer [$TITLE_OPTION=\"\"] […] " + "[${DeepLink.SCHEME}:///…]" } } + +/** + * What the command line says to put in front of every window title of this run. + * + * Outside the parser because it is also written: a run that opens a window for an agent passes one, so that a + * window nobody typed a command line for still has a name to be found by in the window list. + */ +internal const val TITLE_OPTION = "--title" diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerLogging.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerLogging.kt index 21258f845b..85f89db256 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerLogging.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerLogging.kt @@ -2,31 +2,39 @@ package shark.explorer.app import java.io.Closeable import java.io.File +import java.io.PrintStream import shark.SharkLog import shark.explorer.SessionLog import shark.explorer.formatByteSize /** - * Sends everything Shark logs to stdout and to this run's own log file, and returns what closes it. + * Sends everything Shark logs to a stream and to this run's own log file, and returns what closes it. * * The file is what makes a report of "it hung" or "it showed nothing" answerable: the log says which * heap dump was open, what was read off it and how long each read took, which read failed and with * what. See [SessionLog], and [LOG_DIRECTORY] for where the files are. */ -internal fun installLogging(): Closeable { - val standardOut = StandardOutLogger() +internal fun installLogging( + /** + * Where the diagnostics go besides the file, which is stdout for a run from a terminal and **stderr for + * one talking MCP over stdio**: there, stdout is the protocol, and a log line in the middle of a JSON-RPC + * stream is a session the client reports as broken. See `shark.explorer.agent.AgentStdioServer`. + */ + diagnostics: PrintStream = System.out +): Closeable { + val streamLogger = StreamLogger(diagnostics) val sessionLog = try { SessionLog.openIn(LOG_DIRECTORY) } catch (throwable: Throwable) { // A log file is a side channel, so not being able to open one is no reason not to start: say so on - // stdout, where a run from a terminal will see it, and run with stdout alone. - standardOut.d(throwable, "Could not open a log file in $LOG_DIRECTORY, logging to stdout only") + // the stream, where a run from a terminal will see it, and run with that alone. + streamLogger.d(throwable, "Could not open a log file in $LOG_DIRECTORY, logging to the terminal only") null } SharkLog.logger = if (sessionLog == null) { - standardOut + streamLogger } else { - Loggers(listOf(standardOut, sessionLog)) + Loggers(listOf(streamLogger, sessionLog)) } logEnvironment(sessionLog) logUncaughtExceptions() @@ -76,17 +84,17 @@ private fun logUncaughtExceptions() { } } -/** Shark's diagnostics on stdout, which is where a run from a terminal expects them. */ -private class StandardOutLogger : SharkLog.Logger { +/** Shark's diagnostics on one stream, which is stdout for a run from a terminal. */ +private class StreamLogger(private val stream: PrintStream) : SharkLog.Logger { - override fun d(message: String) = println(message) + override fun d(message: String) = stream.println(message) override fun d( throwable: Throwable, message: String ) { - println(message) - throwable.printStackTrace() + stream.println(message) + throwable.printStackTrace(stream) } } diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerProcess.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerProcess.kt new file mode 100644 index 0000000000..90ece0f780 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerProcess.kt @@ -0,0 +1,66 @@ +package shark.explorer.app + +import java.io.File +import shark.SharkLog + +/** + * How this run of the app was launched, which is two quite different things and neither is visible from the + * classpath. + * + * A packaged install is a launcher `jpackage` generated — one executable that knows its own classpath and main + * class. Everything else is a JVM someone put a classpath on: `./gradlew run`, `runNamed`, an IDE run + * configuration. Anything that has to *name* this app to the OS or start another copy of it has to know which, + * and the answer is different every time somebody asks it the easy way. + */ + +/** + * The executable the OS should be told to open a `shark://` link with, and null when this run is not one. + * + * Null for a JVM on purpose: registering `java` would tell the OS to open links with a JVM and no classpath, + * which is worse than not registering at all. See [DeepLinkScheme]. + */ +internal fun launcherPathOrNull(): String? { + val command = ProcessHandle.current().info().command().orElse(null) ?: return null + return command.takeIf { File(it).name !in JVM_EXECUTABLES } +} + +/** + * What to run to start another Shark Explorer, and null when this run can't work out how it was started. + * + * Both cases, unlike [launcherPathOrNull]: a JVM *can* start another copy of itself, since this process + * already holds the classpath that would take. Which is what makes an agent able to open a window from a + * `./gradlew run` bridge as well as from an installed app — and the reason it is worth spelling the main class + * here is that the alternative is the feature only working in a packaged build, where it is slowest to try. + */ +internal fun relaunchCommand(): List? { + val command = ProcessHandle.current().info().command().orElse(null) + if (command == null) { + SharkLog.d { "This process does not say what launched it, so another run of it cannot be started" } + return null + } + if (File(command).name !in JVM_EXECUTABLES) { + return listOf(command) + } + val classPath = System.getProperty("java.class.path") + if (classPath.isNullOrEmpty()) { + SharkLog.d { "$command was launched with no classpath, so another run of it cannot be started" } + return null + } + return listOf(command, "-cp", classPath, MAIN_CLASS) +} + +/** + * A run launched as one of these is a classpath rather than an app, whatever bundle it came out of. + * + * Measured rather than assumed: a `runNamed` bundle declares its own identity in an `Info.plist` and macOS + * still records the process as `net.java.openjdk.java`, because what it launched is this. + */ +private val JVM_EXECUTABLES = setOf("java", "java.exe", "javaw.exe") + +/** + * What Kotlin calls the file `main` is in, which is the one thing here that a rename would silently break. + * + * `ExplorerProcessTest` loads it, so a rename of `Main.kt` fails a test rather than a feature nobody tries + * until an agent needs a window. + */ +internal const val MAIN_CLASS = "shark.explorer.app.MainKt" diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt index b99fd371d2..b8af60b9bf 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt @@ -63,7 +63,7 @@ internal class ExplorerWindow( * composable's state — and set by [ExplorerApp] as the session opens and closes. Null while a heap dump is * being opened, for a window that has none, and for one whose dump failed to open. */ - var openHeapDump: WindowHeapDump? by mutableStateOf(null) + var openHeapDump: OpenHeapDump? by mutableStateOf(null) /** * Why the heap dump this window was given could not be opened, and null while nothing has gone wrong. diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt new file mode 100644 index 0000000000..48557a5045 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt @@ -0,0 +1,134 @@ +package shark.explorer.app + +import java.io.Closeable +import java.io.File +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.Deferred +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.SupervisorJob +import kotlinx.coroutines.async +import kotlinx.coroutines.cancel +import shark.SharkLog +import shark.explorer.DeepLink +import shark.explorer.DeviceHeapDumps +import shark.explorer.agent.AgentHeapDump +import shark.explorer.agent.AgentRefusal + +/** + * The heap dumps of a run with no window, for an agent on a machine with no screen. + * + * Everything a window would hold, held here instead — the heap dump's own thread, the notes, the statuses set + * by hand — and **the same files on disk**, so a dump investigated over ssh today reads back with all of it in + * a window tomorrow. That is why this is in the app module rather than a program of its own: the notes and the + * verdicts are the artefact, and a headless mode writing them somewhere else would be a second app. + * + * So what is left here is the two answers that differ from a run that has windows — which dumps are open, and + * what opening one means — plus the one call a run with no window genuinely can't make: [AgentHeapDump.show] + * has nowhere to put a tab and says so rather than answering that it did. See [NO_UI_OPTION]. + */ +internal class HeadlessAgentHeapDumps( + deviceHeapDumps: DeviceHeapDumps, + /** Heap dumps named on the command line, opened as this starts. */ + heapDumpFiles: List = emptyList(), + /** The same notes a window keeps, in the same directory: a test passes its own. See [ExplorerNotes]. */ + private val notes: ExplorerNotes = ExplorerNotes(), + private val leakStatuses: ExplorerLeakStatuses = ExplorerLeakStatuses() +) : RunAgentHeapDumps(deviceHeapDumps), Closeable { + + /** + * One per heap dump being opened, keyed by file, whether it has finished or not. + * + * **What makes opening the same dump twice one open rather than two.** Which matters here in a way it + * doesn't in a window: the command line's dumps start opening while the client's first message is still on + * its way, so an agent calling `open_heap_dump` on the path it was pointed at is racing them — and losing + * that race would mean a second index of the same gigabyte, on a second thread, with the notes of the first. + */ + private val openings = mutableMapOf>() + + /** The ones that finished opening, in the order they did, which is the order a window list is in. */ + private val opened = mutableListOf() + + private val lock = Any() + + /** + * Off the connection's thread, so that opening a dump and answering about one already open can happen at + * once. Its own scope rather than the caller's: an open outlives the call that asked for it. + */ + private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO) + + init { + heapDumpFiles.forEach { file -> + // Started rather than awaited, because a client is waiting on this process to answer `initialize` and + // a large heap dump is minutes of indexing. So a quick dump is open by the time the first call comes, + // a slow one is opened by whichever call asks for it, and either way it is opened once. + opening(file) + } + } + + override fun openHeapDumps(): List = synchronized(lock) { opened.map { it.agent } } + + override suspend fun open(file: File): AgentHeapDump = opening(file).await().agent + + /** Releases the thread each open heap dump owns, and stops the ones still opening. */ + override fun close() { + scope.cancel() + synchronized(lock) { + opened.forEach { it.open.session.close() } + opened.clear() + openings.clear() + } + } + + /** This dump's open, joining one already in flight rather than starting a second. */ + private fun opening(file: File): Deferred { + val absolute = file.absoluteFile + return synchronized(lock) { + openings.getOrPut(absolute) { scope.async { openNow(absolute) } } + } + } + + private suspend fun openNow(file: File): HeadlessHeapDump { + val session = try { + HeapDumpSession.open(file) { step -> SharkLog.d { step } } + } catch (throwable: Throwable) { + SharkLog.d(throwable) { "Could not open $file" } + // Off the map, so that a path given by mistake can be given again once the file is there — and so that + // the refusal is this one rather than the same failure remembered for the rest of the session. + synchronized(lock) { openings -= file } + throw AgentRefusal("${file.absolutePath} could not be opened as a heap dump: $throwable") + } + val open = OpenHeapDump( + session = session, + notes = notes.of(file), + leakStatuses = leakStatuses.of(file) + ) + // What `HeapDumpExplorer` does as it comes up, and it has to happen somewhere: every verdict is refused + // until the file of them has been read, since saving over an unread one would delete the conclusions in + // it. A window reads it because it draws them, and a run with no window would otherwise never read it and + // refuse every verdict an agent tried to record. + open.leakStatuses.read() + // The same kind of id a window has, because it is the same question: which of the heap dumps open. Called + // `window` on the surface even here, rather than growing a second word for a run that has none — what an + // agent does with it is name a dump, and a vocabulary that changes with whether there is a screen is one + // nobody can carry between the two. + val windowId = DeepLink.newWindowId() + val dump = HeadlessHeapDump( + open = open, + agent = OpenAgentHeapDump(windowId = windowId, open = open) { place -> + SharkLog.d { "Nowhere to show $place: this run was started with $NO_UI_OPTION" } + "This Shark Explorer was started with $NO_UI_OPTION, so it has no window and nothing was shown. " + + "Say what you found in your answer instead. Whoever opens ${file.name} in a window later will " + + "find your notes and verdicts on it, since those are on disk rather than on screen." + } + ) + synchronized(lock) { opened += dump } + SharkLog.d { "${file.name} is open as $windowId, with no window" } + return dump + } +} + +/** One heap dump this run has open, with nothing drawing it. */ +private class HeadlessHeapDump( + val open: OpenHeapDump, + val agent: AgentHeapDump +) diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt index 8cb685cb3a..1e5ea4c6a1 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt @@ -257,7 +257,7 @@ internal fun ExplorerApp( * agent reaching in from outside the app. Called with the heap dump as it opens and with null as it * closes. See [ExplorerWindow.openHeapDump]. */ - onHeapDumpOpen: (WindowHeapDump?) -> Unit = {}, + onHeapDumpOpen: (OpenHeapDump?) -> Unit = {}, /** * And where a heap dump that could not be opened says so, for the same readers. * @@ -339,7 +339,7 @@ internal fun ExplorerApp( val open = currentState as? HeapDumpState.Open onHeapDumpOpen( open?.let { - WindowHeapDump( + OpenHeapDump( session = it.session, notes = notes.of(it.session.heapDumpFile), leakStatuses = leakStatuses.of(it.session.heapDumpFile) @@ -555,7 +555,7 @@ private fun cascadedPosition(cascade: Int): WindowPosition { * A function rather than a constant because it is one per run and a test's is its own — the default of a * composable that a test takes over, and what `main` hands to everything that has no window. */ -private fun commandLineDeviceHeapDumps(): DeviceHeapDumps { +internal fun commandLineDeviceHeapDumps(): DeviceHeapDumps { val adb = CommandLineAdb() // A debugger is what reaches into a process for the two things `am dumpheap` can't ask it for on an old // enough device: the pixels of a bitmap below API 35, and a collection below API 27. diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentCommandLineTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentCommandLineTest.kt new file mode 100644 index 0000000000..e5ed0335b3 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentCommandLineTest.kt @@ -0,0 +1,46 @@ +package shark.explorer.app + +import org.assertj.core.api.Assertions.assertThat +import org.junit.Test +import shark.explorer.agent.AgentStdioBridge + +/** + * Which command lines mean "be an MCP server", answered before anything else in `main`. + * + * The two cases here are the ones that end without talking to anybody, and they are the only ones a test can + * drive: everything else about this either pipes stdio to a window or serves the tools until a client closes + * its end. `HeadlessAgentHeapDumpsTest` covers what it serves. + */ +class AgentCommandLineTest { + + @Test + fun `an ordinary command line is a window`() { + assertThat(agentBridgeExitCode(arrayOf("--title=Windowed", "dump.hprof"))).isNull() + // `--no-ui` on its own is not a way to run the app with no window: there would be nothing to run. + assertThat(agentBridgeExitCode(arrayOf(NO_UI_OPTION))).isNull() + } + + @Test + fun `a command line that does not read is a failure rather than a message`() { + // A client that launched this has nowhere to show a usage message, so the exit code is what says so. + assertThat(agentBridgeExitCode(arrayOf(MCP_STDIO_OPTION, NO_UI_OPTION, "--titel=Typo"))).isEqualTo(1) + } + + @Test + fun `what is left of a server's command line is a window's`() { + val arguments = agentServerArguments( + arrayOf( + MCP_STDIO_OPTION, + NO_UI_OPTION, + "${AgentStdioBridge.PID_OPTION}12345", + "--title=For an agent", + "dump.hprof" + ) + ) + + // The heap dump and the title survive, and the three server options are not taken for heap dumps: a + // window saying `--no-ui` could not be read is what that mistake looks like. + assertThat(arguments.heapDumpFiles.map { it.name }).containsExactly("dump.hprof") + assertThat(arguments.titlePrefix).isEqualTo("For an agent") + } +} diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerProcessTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerProcessTest.kt new file mode 100644 index 0000000000..d50ac09d02 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerProcessTest.kt @@ -0,0 +1,38 @@ +package shark.explorer.app + +import org.assertj.core.api.Assertions.assertThat +import org.junit.Test + +/** + * How this run would start another one, which is what an agent that found no window needs. + * + * Only the parts that can be checked from a test JVM. Whether the command actually opens a window is a + * package away — see the deep link section of `shark/shark-explorer/AGENTS.md` — but the two ways it goes + * wrong silently are a renamed `Main.kt` and a classpath that never reaches the command, and both are here. + */ +class ExplorerProcessTest { + + @Test + fun `the main class is the one Kotlin generates`() { + // A rename of `Main.kt` fails this rather than a feature nobody tries until an agent needs a window. + assertThat(Class.forName(MAIN_CLASS)).isNotNull + } + + @Test + fun `a JVM starts another run of itself with this classpath`() { + val command = relaunchCommand() + + // The test runner is a JVM with a classpath, which is also what `./gradlew run` is. + assertThat(command).isNotNull + assertThat(command!!.last()).isEqualTo(MAIN_CLASS) + assertThat(command).contains("-cp") + assertThat(command).contains(System.getProperty("java.class.path")) + } + + @Test + fun `a JVM is not what the OS should open links with`() { + // The other half of [relaunchCommand], and deliberately the opposite answer: registering `java` would + // tell the OS to open `shark://` links with a JVM and no classpath. + assertThat(launcherPathOrNull()).isNull() + } +} diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt index 33d455faa4..444fdf54c4 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt @@ -1,11 +1,18 @@ package shark.explorer.app import java.io.File +import kotlinx.coroutines.runBlocking +import kotlinx.coroutines.withTimeoutOrNull import org.assertj.core.api.Assertions.assertThat +import org.assertj.core.api.Assertions.assertThatThrownBy import org.junit.Rule import org.junit.Test +import shark.explorer.Adb +import shark.explorer.AdbOutput import shark.explorer.DeepLink +import shark.explorer.DeviceHeapDumps import shark.explorer.Place +import shark.explorer.agent.AgentRefusal /** * How many windows the app has and which heap dump each one shows. No heap dump is read here: a window @@ -199,6 +206,43 @@ class ExplorerWindowTest { assertThat(windows.holds(CLOSED_WINDOW_ID)).isFalse() } + @Test fun `an agent asking for a heap dump a window already has gets that window`() { + val windows = explorerWindows(opening(FIRST_DUMP)) + val window = windows.single() + // What a window says when its dump turns out not to be one, which is the one outcome a test can reach + // without a real heap dump: either way it ends the wait, and what this is about is which window waited. + window.openProblem = "Not a heap dump." + + assertThatThrownBy { runBlocking { agentHeapDumps(windows).open(FIRST_DUMP.absoluteFile) } } + .isInstanceOf(AgentRefusal::class.java) + + // A second window on it would be a second index of the same gigabyte, and a window nobody asked for — + // unlike the button above the map, where a person opening one dump twice is comparing two readings of it. + assertThat(windows).hasSize(1) + assertThat(logged).anyMatch { "already has" in it && window.deepLinkId in it } + } + + @Test fun `an agent asking for a heap dump nobody has open gets a window of its own`() { + val windows = explorerWindows(opening(FIRST_DUMP)) + + // Nothing is composing these windows, so the dump never opens and the call never returns: what is being + // read here is the window it made on the way, which is the half of it that isn't the heap dump. + runBlocking { + withTimeoutOrNull(WAIT_MILLIS) { agentHeapDumps(windows).open(SECOND_DUMP.absoluteFile) } + } + + assertThat(windows.map { it.heapDumpFile }).containsExactly(FIRST_DUMP, SECOND_DUMP.absoluteFile) + assertThat(logged).anyMatch { "An agent opened" in it && SECOND_DUMP.name in it } + } + + /** The agent surface over these windows, with an `adb` that isn't there: nothing here reaches a device. */ + private fun agentHeapDumps(windows: ExplorerWindows) = WindowAgentHeapDumps( + windows = windows, + deviceHeapDumps = DeviceHeapDumps(object : Adb { + override fun run(arguments: List) = AdbOutput(exitCode = 1, text = "") + }) + ) + private fun noHeapDumps(titlePrefix: String? = null) = ExplorerArguments(heapDumpFiles = emptyList(), titlePrefix = titlePrefix) @@ -215,5 +259,8 @@ class ExplorerWindowTest { /** Shaped like one this run could have handed out, and belonging to no window of it. */ private const val CLOSED_WINDOW_ID = "qrst6789" + + /** Long enough for a window to be added and short enough not to be a pause anybody notices. */ + private const val WAIT_MILLIS = 200L } } diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt new file mode 100644 index 0000000000..d4be2c4031 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt @@ -0,0 +1,132 @@ +package shark.explorer.app + +import java.io.File +import kotlinx.coroutines.runBlocking +import org.assertj.core.api.Assertions.assertThat +import org.assertj.core.api.Assertions.assertThatThrownBy +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import shark.explorer.Adb +import shark.explorer.AdbOutput +import shark.explorer.DeviceHeapDumps +import shark.explorer.LeakStatus +import shark.explorer.LeakStatusOverride +import shark.explorer.Place +import shark.explorer.agent.AgentRefusal + +/** + * An agent's heap dumps with no window anywhere, which is what `--mcp-stdio --no-ui` serves. + * + * What is worth pinning is that this is the *same* investigation a window records rather than a second one: + * the verdicts and the notes go in the files a window reads, so the last test here opens the file again the + * way another run of the app would. + */ +class HeadlessAgentHeapDumpsTest { + + @get:Rule + val temporaryFolder = TemporaryFolder() + + @get:Rule + val log = RecordedLog() + + @Test + fun `a heap dump named on the command line is open`() { + val file = temporaryFolder.leakyHeapDump().file + headless(file).use { heapDumps -> + val dump = runBlocking { heapDumps.open(file) } + + assertThat(heapDumps.openHeapDumps().map { it.windowId }).containsExactly(dump.windowId) + assertThat(dump.heapDumpPath).isEqualTo(file.absolutePath) + } + } + + @Test + fun `opening the same heap dump twice is one heap dump`() { + val file = temporaryFolder.leakyHeapDump().file + headless().use { heapDumps -> + val first = runBlocking { heapDumps.open(file) } + val second = runBlocking { heapDumps.open(file) } + + // Not merely equal ids: a second open would be a second index of the same file, on a second thread, + // writing the notes of the first. + assertThat(second.windowId).isEqualTo(first.windowId) + assertThat(heapDumps.openHeapDumps()).hasSize(1) + } + } + + @Test + fun `showing a place says there is no window rather than that it was shown`() { + val file = temporaryFolder.leakyHeapDump().file + headless().use { heapDumps -> + val dump = runBlocking { heapDumps.open(file) } + + val problem = dump.show(Place.Leaks()) + + assertThat(problem) + .contains(NO_UI_OPTION) + .contains(file.name) + } + } + + @Test + fun `a file that is no heap dump is refused, and can be opened again once it is one`() { + val notADump = temporaryFolder.newFile("not-a-heap-dump.hprof") + headless().use { heapDumps -> + assertThatThrownBy { runBlocking { heapDumps.open(notADump) } } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining(notADump.absolutePath) + + // The failure isn't remembered for the rest of the session: a path given before the file was written is + // a path worth giving again, which is most of how a dump taken by hand arrives. + val real = temporaryFolder.leakyHeapDump().file + real.copyTo(notADump, overwrite = true) + assertThat(runBlocking { heapDumps.open(notADump) }.heapDumpPath).isEqualTo(notADump.absolutePath) + } + } + + @Test + fun `a verdict recorded with no window is on disk for the next window to read`() { + val dumped = temporaryFolder.leakyHeapDump() + val statuses = temporaryFolder.newFolder("leak-statuses") + val notes = temporaryFolder.newFolder("notes") + headless(dumped.file, statuses = statuses, notes = notes).use { heapDumps -> + val dump = runBlocking { heapDumps.open(dumped.file) } + + runBlocking { + dump.setVerdict( + LeakStatusOverride(dumped.watchedObjectId, LeakStatus.STUCK, "The app said it was done with it."), + solved = emptyList() + ) + dump.appendToNote(Place.Object(dumped.watchedObjectId), "Held by the presenters map.") + } + + // Read back the way another run of the app reads it, which is the whole claim: an investigation over + // ssh today is one a window opens tomorrow. + val reread = ExplorerLeakStatuses(statuses).of(dumped.file) + runBlocking { reread.read() } + assertThat(reread.overrides[dumped.watchedObjectId]?.status).isEqualTo(LeakStatus.STUCK) + val rereadNote = ExplorerNotes(notes).of(dumped.file).of(Place.Object(dumped.watchedObjectId)) + runBlocking { rereadNote.read() } + assertThat(rereadNote.text).contains("Held by the presenters map.") + } + } + + private fun headless( + vararg heapDumpFiles: File, + statuses: File = temporaryFolder.newFolder("statuses-${heapDumpFiles.size}"), + notes: File = temporaryFolder.newFolder("notes-${heapDumpFiles.size}") + ) = HeadlessAgentHeapDumps( + // Nothing here reaches a device, and an `adb` that answers nothing is what proves it: a test that took + // the machine's would have whatever is plugged in to answer for. + deviceHeapDumps = DeviceHeapDumps(NoAdb), + heapDumpFiles = heapDumpFiles.toList(), + notes = ExplorerNotes(notes), + leakStatuses = ExplorerLeakStatuses(statuses) + ) + + /** An `adb` that isn't there, which is what a build server running this has. */ + private object NoAdb : Adb { + override fun run(arguments: List) = AdbOutput(exitCode = 1, text = "") + } +} From eae7d4dbc602f8703b1d5d68e28b92cf51401149 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 10:01:36 +0200 Subject: [PATCH 09/27] Answer an agent with the link to what it showed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `show` and `conclude` put a place on screen, which is right once and wrong five times: raising a window over what somebody was doing is a hand-over that happens now or not at all. A `shark://` link is the same hand-over they can take with them, and the answer an agent writes lands somewhere this app can't reach — a chat window, a pull request comment, a bug report — so a link in that sentence is the difference between an answer to take on trust and one to go and look at. So `AgentHeapDump.show` answers with a `ShownPlace`: the link, or why there was nowhere. One answer rather than two calls, because a link names a window and so whether there is a link and whether anything was shown are the same fact — a `--no-ui` run handing out a link would be handing out an address nothing answers to. --- docs/shark-explorer-changelog.md | 8 +++- docs/shark-explorer.md | 21 +++++++++-- .../shark/explorer/agent/AgentHeapDump.kt | 37 ++++++++++++++++--- .../java/shark/explorer/agent/AgentMethod.kt | 5 +++ .../java/shark/explorer/agent/AgentTools.kt | 20 ++++++---- .../shark/explorer/agent/AgentToolsTest.kt | 14 +++++++ .../shark/explorer/agent/FakeAgentHeapDump.kt | 8 ++-- .../java/shark/explorer/app/ExplorerAgents.kt | 12 ++++-- .../explorer/app/HeadlessAgentHeapDumps.kt | 11 ++++-- .../app/HeadlessAgentHeapDumpsTest.kt | 7 +++- 10 files changed, 113 insertions(+), 30 deletions(-) diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index 87da8b981a..610ca4725a 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -52,9 +52,15 @@ uses, without the one for a newly recognized library leak: opens one — on the heap dump its command line named, if it named one — and leaves it open for whoever comes back to it. And with `--no-ui`, the tools are served from that process with no window anywhere, for a build server or a heap dump at the end of an ssh session: everything works the same except `show`, which says it - has nowhere to put a tab rather than answering that it showed you something. Notes and verdicts were never + has nowhere to put a tab rather than answering that it showed you something, and hands back no link since a + link names a window and this run has none. Notes and verdicts were never on the screen, so a heap dump investigated with no window opens in one later with all of it on. See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). +* ✨ **An agent answers with links into the window.** `show` and `conclude` hand back the `shark://` link to + what they put on screen, and the method the tools come with tells an agent to put those links in its reply — + so a sentence in a chat window, a pull request comment or a bug report carries a way into the heap dump + rather than instructions for finding the object again by hand. + See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). * ✨ **Agent logs**: every agent that has connected to the app is a row on a screen of its own, and opening one is everything that agent did — what each call did, which object it did it to, and the sentence it gave for making it, with the refusals in red. A row leads where the call went, so reading what an agent did and diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 653ac4e427..4fa3d80fde 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -270,9 +270,10 @@ instead of piped to a window: ``` Everything works the same except `show`, which has nowhere to put a tab and says so rather than answering that -it showed you something. Nothing else changes, because **notes and verdicts were never on the screen** — they -are files beside the heap dump, so a dump investigated over ssh today opens in a window tomorrow with the -verdicts, the reasons and the conclusion already on it. +it showed you something — and hands back no `shark://` link either, since a link names a window and this run +has none. Nothing else changes, because **notes and verdicts were never on the screen** — they are files +beside the heap dump, so a dump investigated over ssh today opens in a window tomorrow with the verdicts, the +reasons and the conclusion already on it. Then ask for what you actually want. This is the whole prompt the session below was given: @@ -298,7 +299,7 @@ press, because a surface with less than that is one whose answer is "ask your hu | `dominator_tree` | The treemap, without the pixels: where the memory has gone, a level at a time. | | `set_verdict`, `clear_verdict` | The pencil, with the reason required the same way. | | `read_notes`, `take_note` | The notes: where somebody has been, what they wrote, and adding to or replacing it. | -| `show` | Opens a tab in your window and brings it to the front. The one tool a `--no-ui` run can't do. | +| `show` | Opens a tab in your window and brings it to the front, and answers with the `shark://` link to it. The one tool a `--no-ui` run can't do. | | `conclude` | The root cause, and the only way to finish. | | `open_heap_dump` | **Open heap dump…**, for a file nobody has open yet. | | `list_devices`, `dump_heap` | **Take heap dump…**: which device, which process, and the dump itself. | @@ -395,6 +396,18 @@ stuck object and opens that tab, so the answer is in the window beside the evide > class name `MainActivity$2`, the synthetic `this$0` field and Shark's inspector label, not on a line of > source. +**And the link to that note comes back with the conclusion**, because the answer usually arrives somewhere +that isn't this app. `show` and `conclude` both answer with the `shark://` link to what they put on screen, +and the method tells an agent to put those links in its reply — so a sentence in your chat window, a pull +request comment or a bug report ends up carrying a way in: + +> The leak is `MainActivity$2.this$0`, a non-static inner class holding the activity it was declared in: +> shark://zvphq4r3/0x12d368b8 + +Clicking it opens that object in that window, with the reasoning on its tabs. Once the window is closed the +link says which window it was rather than opening the wrong one, so an answer worth keeping is worth +[copying the heap dump's path](#open-a-heap-dump) beside it. + An agent's verdicts are verdicts like any other: they say `set by hand` on every chain that runs through the object, the reason is the one it gave, and the pencil takes one off if you disagree with it. Which is the last thing this surface is for — the disagreement is about a reason you can read, not about who said it. diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt index cc682ed04b..c62c83b0d7 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt @@ -95,12 +95,39 @@ interface AgentHeapDump { * * Not suspending: this is the same hand-over a `shark://` link makes — a place put where the tabs take it * on the next frame — so there is nothing to wait for. - * - * @return null when it was shown, and why it wasn't otherwise. Which is one case, a run started with no - * window at all, and it is worth answering rather than logging: an agent told its human to look at - * something they cannot see has said the one thing worse than nothing. */ - fun show(place: Place): String? + fun show(place: Place): ShownPlace +} + +/** + * What came of putting a place in front of the person watching: the link to it, or why there was nowhere. + * + * **One answer rather than two calls**, because the two questions have one answer. A link names a window, so + * whether there is a link and whether anything was shown are the same fact — and a run with no window that + * handed out a `shark://` link anyway would be handing out an address nothing answers to. + * + * The link matters as much as the showing does: it is what an agent puts in its *reply* so that whoever asked + * can open the place themselves, later, from wherever the conversation is. Showing raises a window over + * whatever they were doing, which is right once and wrong five times; a link in a sentence is right every + * time. See [AgentTools] `show`. + */ +class ShownPlace private constructor( + /** The `shark://` link a person can click to open it, and null when nothing was shown. */ + val link: String?, + /** Why it wasn't shown, and null when it was. */ + val problem: String? +) { + + companion object { + + fun at(link: String) = ShownPlace(link = link, problem = null) + + /** + * Nothing was shown, and [problem] says why — which is worth answering rather than logging: an agent that + * told its human to look at something they cannot see has said the one thing worse than nothing. + */ + fun nowhere(problem: String) = ShownPlace(link = null, problem = problem) + } } /** diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt index c4ec2f8880..b0bf63d579 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt @@ -100,5 +100,10 @@ internal object AgentMethod { something a person can follow afterwards rather than a conclusion they have to trust. - **`show` puts what you are looking at on screen.** Use it when you reach something that matters. The window is how the person watching follows the work, and it costs you one call. + - **Put the `shark://` links you are answered with in your reply.** `show` and `conclude` hand one back: + it opens that exact object, in that window, with your notes on it. Whoever asked you can click it + while reading your answer, and again next week. So write "the leak is + `Holder.activity`(shark://…)" rather than describing which screen to open and what to click — a link + is the difference between an answer they have to take your word for and one they can go and look at. """.trimIndent() } diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index 1282a50e4c..7894d1686b 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -411,17 +411,20 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { private fun show() = AgentTool( name = "show", description = "Opens a place in a tab of this window and brings the window to the front, so that what " + - "you are looking at is what the person at the machine is looking at. One call, no answer to wait " + - "for. Use it when you reach something that matters rather than for every step.", + "you are looking at is what the person at the machine is looking at. Use it when you reach something " + + "that matters rather than for every step. It answers with a `shark://` link to that place: put that " + + "link in your reply to whoever asked you, because clicking it opens the place again, later, without " + + "you.", schema = schema(WINDOW to window(), PLACE to place()) ) { arguments -> val dump = arguments.heapDump() val place = arguments.place() - val problem = dump.show(place) + val shown = dump.show(place) buildJsonObject { - put("shown", problem == null) + put("shown", shown.problem == null) + put("link", shown.link) // So that an agent about to tell its human where to look finds out that there is nowhere. - put("problem", problem) + put("problem", shown.problem) } } @@ -469,7 +472,7 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { reason = arguments.reason ) dump.appendToNote(Place.Object(objectId), note) - val showProblem = dump.show(Place.Object(objectId)) + val shown = dump.show(Place.Object(objectId)) buildJsonObject { put("concluded", true) putJsonArray("faultyReference") { @@ -490,8 +493,11 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { put( "writtenTo", "the notes of ${exactHexObjectId(objectId)}" + - if (showProblem == null) ", and shown in window ${dump.windowId}" else ". $showProblem" + if (shown.problem == null) ", and shown in window ${dump.windowId}" else ". ${shown.problem}" ) + // The one link most worth handing back: it opens the object this conclusion is about, with the + // conclusion in its notes. Say it in your answer rather than describing where to click. + put("link", shown.link) } } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index bf64f090c2..9159e285d6 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -263,6 +263,10 @@ class AgentToolsTest { assertThat(faulty.text("field")).isEqualTo(ACTIVITY_FIELD_NAME) assertThat(faulty.text("heldObject")).isEqualTo(hex(heapDump.activityObjectId)) assertThat(faulty.text("heldClassName")).isEqualTo(ACTIVITY_CLASS_NAME) + // The one link most worth handing back, so it comes with the conclusion rather than needing a show call + // after it: it opens the object this conclusion is about, with the conclusion in its notes. + assertThat(answer.text("link")) + .isEqualTo("shark://${window.windowId}/${hex(heapDump.activityObjectId)}") } @Test @@ -444,6 +448,16 @@ class AgentToolsTest { .hasMessageContaining("is no place of a heap dump") } + @Test + fun `showing a place answers with the link to it`() { + val answer = call("show", "place" to hex(heapDump.activityObjectId)) + + // The half of showing that outlives the call: an agent writing its answer somewhere else has this to + // point at, where "open the window and click the activity" is a set of instructions. + assertThat(answer.text("link")) + .isEqualTo("shark://${window.windowId}/${hex(heapDump.activityObjectId)}") + } + @Test fun `an address written as a decimal number is refused as one`() { assertThatThrownBy { diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt index 45c4bad06d..eb1e7bab91 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt @@ -78,11 +78,11 @@ internal class FakeAgentHeapDump( override suspend fun notedPlaces(): List = notes.keys.toList() - override fun show(place: Place): String? { + override fun show(place: Place): ShownPlace { shown += place - // Null is "it was shown", which is what a window answers. The refusal a run with no window makes is - // `HeadlessAgentHeapDumpsTest`'s, since it is that run's one difference from this one. - return null + // A window's answer, which is a link. What a run with no window answers is `HeadlessAgentHeapDumpsTest`'s, + // since it is that run's one difference from this one. + return ShownPlace.at("shark://$windowId/${placeText(place)}") } override fun close() { diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt index c37719e926..34d59b5ff1 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt @@ -7,6 +7,7 @@ import kotlinx.coroutines.flow.first import kotlinx.coroutines.withContext import shark.SharkLog import shark.explorer.AndroidDevice +import shark.explorer.DeepLink import shark.explorer.DeviceHeapDumps import shark.explorer.DeviceProcess import shark.explorer.HeapExplorer @@ -21,6 +22,7 @@ import shark.explorer.agent.AgentSession import shark.explorer.agent.AgentSessionFile import shark.explorer.agent.AgentStdioBridge import shark.explorer.agent.AgentStdioServer +import shark.explorer.agent.ShownPlace import shark.explorer.placeOfNoteKeyOrNull /** @@ -303,7 +305,9 @@ private fun ExplorerWindow.agentHeapDump(open: OpenHeapDump): AgentHeapDump = // person clicking a link land in the same place. See [ExplorerWindows.open]. goToLinked(place) bringToFront() - null + // And the link itself, which is the same one the right click menu copies: an agent's answer can then + // point at this place rather than describe how to get to it. + ShownPlace.at(DeepLink(deepLinkId, place).toUri()) } /** @@ -316,8 +320,8 @@ private fun ExplorerWindow.agentHeapDump(open: OpenHeapDump): AgentHeapDump = internal class OpenAgentHeapDump( override val windowId: String, private val open: OpenHeapDump, - /** Where a place goes, answering with why it couldn't. See [AgentHeapDump.show]. */ - private val showPlace: (Place) -> String? + /** Where a place goes, and the link to it. See [AgentHeapDump.show]. */ + private val showPlace: (Place) -> ShownPlace ) : AgentHeapDump { override val heapDumpPath: String get() = open.session.heapDumpFile.absolutePath @@ -362,7 +366,7 @@ internal class OpenAgentHeapDump( return open.notes.writtenAbout.mapNotNull { key -> placeOfNoteKeyOrNull(key) } } - override fun show(place: Place): String? = showPlace(place) + override fun show(place: Place): ShownPlace = showPlace(place) /** * Puts what [newText] makes of the saved note on disk, whether that is the note plus a paragraph or diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt index 48557a5045..e8d05f145c 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt @@ -13,6 +13,7 @@ import shark.explorer.DeepLink import shark.explorer.DeviceHeapDumps import shark.explorer.agent.AgentHeapDump import shark.explorer.agent.AgentRefusal +import shark.explorer.agent.ShownPlace /** * The heap dumps of a run with no window, for an agent on a machine with no screen. @@ -116,9 +117,13 @@ internal class HeadlessAgentHeapDumps( open = open, agent = OpenAgentHeapDump(windowId = windowId, open = open) { place -> SharkLog.d { "Nowhere to show $place: this run was started with $NO_UI_OPTION" } - "This Shark Explorer was started with $NO_UI_OPTION, so it has no window and nothing was shown. " + - "Say what you found in your answer instead. Whoever opens ${file.name} in a window later will " + - "find your notes and verdicts on it, since those are on disk rather than on screen." + // And no link either, deliberately: a link names a window, so one from here would be an address + // nothing answers to, handed to somebody who would click it. + ShownPlace.nowhere( + "This Shark Explorer was started with $NO_UI_OPTION, so it has no window and nothing was shown. " + + "Say what you found in your answer instead. Whoever opens ${file.name} in a window later will " + + "find your notes and verdicts on it, since those are on disk rather than on screen." + ) } ) synchronized(lock) { opened += dump } diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt index d4be2c4031..1124ab1ba3 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt @@ -61,11 +61,14 @@ class HeadlessAgentHeapDumpsTest { headless().use { heapDumps -> val dump = runBlocking { heapDumps.open(file) } - val problem = dump.show(Place.Leaks()) + val shown = dump.show(Place.Leaks()) - assertThat(problem) + assertThat(shown.problem) .contains(NO_UI_OPTION) .contains(file.name) + // And no link, which is the half of it an agent would otherwise pass on: a `shark://` link names a + // window, so one from a run that has none is an address nothing answers to. + assertThat(shown.link).isNull() } } From 9a7cc962dc5ad06c02757730737dcf3ea43c211f Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 10:11:16 +0200 Subject: [PATCH 10/27] Drag the note's bottom edge to make it taller MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit How much of the window a note is worth is the reading of it against the reading of the heap dump, and that changes with the note: a paragraph about which of two caches this is wants four lines, and an argument about a root cause with the chain quoted in it wants the half of the window it takes to be read while looking at the chain. A number in the code can only be right for one of them, so it is an edge, dragged the way the edges between the panes already are. The height is per window, beside the pane widths and for the same reason: it is how somebody has set their desk up for the job in hand, and a note that shrank as they clicked through tabs would be the window rearranging itself. And it is never more than a share of the room the tab has, because the edge is the only way back — a note dragged tall on a big screen and then a window made small would otherwise put its own edge past the bottom of the screen, and nothing but resizing the window would bring it back. --- docs/shark-explorer-changelog.md | 3 + docs/shark-explorer.md | 6 ++ .../shark/explorer/app/HeapDumpExplorer.kt | 8 ++- .../java/shark/explorer/app/NoteSection.kt | 59 ++++++++++++--- .../src/main/java/shark/explorer/app/Panes.kt | 71 +++++++++++++++---- .../java/shark/explorer/app/ExplorerUiTest.kt | 11 ++- .../shark/explorer/app/NoteSectionTest.kt | 65 +++++++++++++++++ 7 files changed, 197 insertions(+), 26 deletions(-) diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index 610ca4725a..81f0ada3e1 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -28,6 +28,9 @@ uses, without the one for a newly recognized library leak: location are one note. Class names, addresses and `shark://` links written in a note become links back into the window, shortened to read as prose, and GitHub URLs are shortened the way GitHub shortens them. See [Take notes](shark-explorer.md#take-notes). +* ✨ Drag the line along the bottom of a note to give it more of the window or less, the same way the edges + between the panes are dragged sideways. Per window rather than per tab, and never more than its share of + the window however far it is dragged. See [Take notes](shark-explorer.md#take-notes). * ✨ Right click ← or → for the list of everywhere that arrow leads, so going back four moves is one click rather than four. * ✨ **Verdict**: whether the object a tab is on is stuck in memory — `✗ Stuck`, `✓ Expected` or `? Unknown` diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 4fa3d80fde..7c72b58be2 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -121,6 +121,12 @@ The note appears under that row and above the panes, because it is about the who Where nobody has written anything there is nothing there at all — only the button, which goes away once there is a note, since the note carries its own **✎ Edit**. +**Drag the line along its bottom edge** to give the note more of the window or less, the same way the edges +between the panes are dragged sideways. A long note scrolls rather than pushing the heap dump off the screen, +and how tall it has been dragged to is per window rather than per tab, so it stays where you put it as you +move around. It never takes more than its share of the window, however far it is dragged: the edge has to stay +somewhere you can reach it. + The notes live in `~/.shark-explorer/notes`, one directory per heap dump and one `.md` file per tab, so a note can be opened in an editor, pasted into an issue, or read by an agent without going through this app. diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt index c9540a75b3..ea6a0cda56 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt @@ -741,7 +741,9 @@ internal fun HeapDumpExplorer( if (tabNote != null) { NoteSection( notes = tabNote.notes, - onLink = followNoteLink + onLink = followNoteLink, + height = panes.noteHeight, + onResize = { delta -> panes.resizeNote(delta) } ) } Box(Modifier.weight(1f)) { @@ -936,7 +938,7 @@ private fun RowScope.ChainPane( ) } if (panes.filling != Pane.CHAIN) { - PaneDivider { delta -> panes.resize(Pane.CHAIN, delta) } + PaneDivider(resizeHint(Pane.CHAIN)) { delta -> panes.resize(Pane.CHAIN, delta) } } } @@ -1041,7 +1043,7 @@ private fun RowScope.DetailsPane( return } if (panes.filling != Pane.DETAILS) { - PaneDivider { delta -> panes.resize(Pane.DETAILS, -delta) } + PaneDivider(resizeHint(Pane.DETAILS)) { delta -> panes.resize(Pane.DETAILS, -delta) } } Column(paneWidth(panes, Pane.DETAILS).fillMaxHeight()) { PaneHeader(Pane.DETAILS) { panes.toggleFold(Pane.DETAILS) } diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/NoteSection.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/NoteSection.kt index 9389fedfb5..1acaace18b 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/NoteSection.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/NoteSection.kt @@ -1,12 +1,12 @@ package shark.explorer.app +import androidx.compose.foundation.gestures.Orientation import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.PaddingValues import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.height -import androidx.compose.foundation.layout.heightIn import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.width import androidx.compose.foundation.rememberScrollState @@ -22,6 +22,7 @@ import androidx.compose.runtime.rememberCoroutineScope import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.graphics.Color +import androidx.compose.ui.layout.layout import androidx.compose.ui.semantics.contentDescription import androidx.compose.ui.semantics.semantics import androidx.compose.ui.text.AnnotatedString @@ -35,7 +36,9 @@ import androidx.compose.ui.text.font.FontStyle import androidx.compose.ui.text.font.FontWeight import androidx.compose.ui.text.withLink import androidx.compose.ui.text.withStyle +import androidx.compose.ui.unit.Dp import androidx.compose.ui.unit.dp +import kotlin.math.roundToInt import kotlinx.coroutines.launch import shark.explorer.NoteBlock import shark.explorer.NoteLink @@ -64,6 +67,10 @@ import shark.explorer.hexObjectId * - **Written**: the note as it means, which is where the names in it lead somewhere — a class shortened to * a link, an address replaced by what is at it, a `shark://` link back to the tab it was copied from. * + * **The line along the bottom is the same divider the panes are resized by**, dragged up and down instead: + * how much of the window a note is worth is the reading of it against the reading of the heap dump, and that + * changes with the note. How tall it has been dragged to is [PanesState.noteHeight], per window. + * * One markdown file per place, kept between runs, read by the window rather than here. See [PlaceNotes]. */ @Composable @@ -71,6 +78,9 @@ internal fun NoteSection( notes: PlaceNotes, /** Where clicking a link in the written note goes. See [HeapDumpExplorer]. */ onLink: (NoteLink) -> Unit, + /** How tall it has been dragged to, and where a drag of its bottom edge goes. See [PanesState]. */ + height: Dp, + onResize: (Dp) -> Unit, modifier: Modifier = Modifier ) { val draft = notes.draft @@ -86,11 +96,13 @@ internal fun NoteSection( draft != null -> NoteEditor( notes = notes, draft = draft, + height = height, onSave = { saving.launch { notes.save() } }, onCancel = { notes.cancel() } ) notes.text.isNotEmpty() -> WrittenNote( notes = notes, + height = height, onLink = onLink, onEdit = { notes.edit() } ) @@ -105,7 +117,9 @@ internal fun NoteSection( color = MaterialTheme.colorScheme.error ) } - HorizontalDivider() + // Where a `HorizontalDivider` would be, and the same line to look at: the note's bottom edge is the + // one place a drag of it can be, since it is the only edge of the section that isn't the title above. + PaneDivider(RESIZE_NOTE_HINT, Orientation.Vertical, onResize) } } } @@ -147,6 +161,7 @@ internal fun AddNoteButton( @Composable private fun WrittenNote( notes: PlaceNotes, + height: Dp, onLink: (NoteLink) -> Unit, onEdit: () -> Unit ) { @@ -156,7 +171,7 @@ private fun WrittenNote( verticalAlignment = Alignment.Top ) { Column( - Modifier.weight(1f).heightIn(max = MAX_NOTE_HEIGHT) + Modifier.weight(1f).noteHeight(height, fill = false) .verticalScroll(rememberScrollState()) .padding(vertical = 6.dp), verticalArrangement = Arrangement.spacedBy(BLOCK_SPACING) @@ -184,6 +199,7 @@ private fun WrittenNote( private fun NoteEditor( notes: PlaceNotes, draft: String, + height: Dp, onSave: () -> Unit, onCancel: () -> Unit ) { @@ -193,7 +209,7 @@ private fun NoteEditor( onValueChange = { notes.edited(it) }, placeholder = { Text(NOTE_PLACEHOLDER, style = MaterialTheme.typography.bodySmall) }, textStyle = MaterialTheme.typography.bodyMedium, - modifier = Modifier.fillMaxWidth().height(EDITOR_HEIGHT) + modifier = Modifier.fillMaxWidth().noteHeight(height, fill = true) .semantics { contentDescription = NOTE_EDITOR_DESCRIPTION } ) Row( @@ -220,6 +236,31 @@ private fun NoteEditor( } } +/** + * As tall as the divider has been dragged to, and never more than [NOTE_SHARE] of the room the tab has. + * + * The share is what keeps the divider reachable: a note dragged tall and then a window made short would + * otherwise place its own bottom edge past the bottom of the screen, and the only way back would be to make + * the window bigger again. + * + * [fill] is the difference between the box being typed in, which is that tall whether or not anything has + * been typed yet, and the note as it reads, which takes what it needs up to that and then scrolls. + */ +private fun Modifier.noteHeight( + height: Dp, + fill: Boolean +) = layout { measurable, constraints -> + val most = if (constraints.hasBoundedHeight) { + minOf(height.roundToPx(), (constraints.maxHeight * NOTE_SHARE).roundToInt()) + } else { + height.roundToPx() + } + val placeable = measurable.measure( + constraints.copy(minHeight = if (fill) most else 0, maxHeight = most) + ) + layout(placeable.width, placeable.height) { placeable.place(0, 0) } +} + @Composable private fun NoteBlockView( block: NoteBlock, @@ -348,6 +389,9 @@ private const val NOTE_PLACEHOLDER = private const val SAVED_IN = "Saved in" +/** What dragging the note's bottom edge does, said where a bar of pixels can't say it. See [PaneDivider]. */ +internal const val RESIZE_NOTE_HINT = "Drag to make the note taller or shorter." + /** In front of a quoted line, since a quote here is one line rather than a paragraph to draw a bar beside. */ private const val QUOTE_BAR = "▎" @@ -357,11 +401,8 @@ private val BLOCK_SPACING = 4.dp private val INDENT_WIDTH = 16.dp private val MARKER_WIDTH = 24.dp -/** Enough for a few lines while writing, so that the panes under it keep the window. */ -private val EDITOR_HEIGHT = 120.dp - /** A line under the title rather than a button beside it. See [AddNoteButton]. */ private val ADD_NOTE_HEIGHT = 20.dp -/** And how much of it a long note gets before it scrolls instead of pushing the panes down. */ -private val MAX_NOTE_HEIGHT = 160.dp +/** Whatever the divider says, this much of the tab's height is the most a note gets. See [noteHeight]. */ +private const val NOTE_SHARE = 0.6f diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Panes.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Panes.kt index 501822e18f..367a31f0ca 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Panes.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Panes.kt @@ -11,6 +11,7 @@ import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.fillMaxHeight import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.height import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.width import androidx.compose.material3.MaterialTheme @@ -25,6 +26,8 @@ import androidx.compose.ui.Modifier import androidx.compose.ui.input.pointer.PointerIcon import androidx.compose.ui.input.pointer.pointerHoverIcon import androidx.compose.ui.platform.LocalDensity +import androidx.compose.ui.semantics.contentDescription +import androidx.compose.ui.semantics.semantics import androidx.compose.ui.unit.Dp import androidx.compose.ui.unit.dp import java.awt.Cursor @@ -49,17 +52,21 @@ internal enum class Pane(val paneName: String) { } /** - * How wide the three columns are and which of them are folded away. + * How the window under the tab strip is laid out: how wide the three columns are, which of them are folded + * away, and how much height the note takes above them. * * Held per window rather than per tab: the shape of the window is how someone has set their desk up for * the job at hand, and having it change under them as they switch tabs would be the window rearranging - * itself for reasons of its own. + * itself for reasons of its own. Which is also why the note's height is here and not in [NoteSection] — + * a tab with nothing written about it draws no note at all, and a height remembered inside a section that + * comes and goes is a height reset by visiting a tab nobody has written about. */ @Stable internal class PanesState { var chainWidth by mutableStateOf(ROOT_PATH_WIDTH) var detailsWidth by mutableStateOf(DETAILS_WIDTH) + var noteHeight by mutableStateOf(NOTE_HEIGHT) private var foldedChain by mutableStateOf(false) private var foldedView by mutableStateOf(false) @@ -106,8 +113,16 @@ internal class PanesState { Pane.VIEW -> Unit } } + + /** Makes the note taller or shorter by [delta], within what leaves anything under it worth reading. */ + fun resizeNote(delta: Dp) { + noteHeight = (noteHeight + delta).coerceIn(MIN_NOTE_HEIGHT, MAX_NOTE_HEIGHT) + } } +/** A pane's name as it reads mid-sentence, since the name itself starts a heading above the pane. */ +private val Pane.said: String get() = paneName.replaceFirstChar { it.lowercase() } + /** The name of a pane and the control that folds it away, along the top of it. */ @Composable internal fun PaneHeader( @@ -135,7 +150,7 @@ internal fun FoldButton( pane: Pane, onFold: () -> Unit ) { - Hint("Fold ${pane.paneName.replaceFirstChar { it.lowercase() }} away.") { + Hint("Fold ${pane.said} away.") { Text( FOLD, Modifier.clickable(onClick = onFold).padding(4.dp), @@ -160,7 +175,7 @@ internal fun FoldedPane( .background(MaterialTheme.colorScheme.surfaceVariant), horizontalAlignment = Alignment.CenterHorizontally ) { - Hint("Show ${pane.paneName.replaceFirstChar { it.lowercase() }} again.") { + Hint("Show ${pane.said} again.") { Text( UNFOLD, Modifier.clickable(onClick = onUnfold).padding(vertical = 6.dp, horizontal = 4.dp), @@ -171,31 +186,51 @@ internal fun FoldedPane( } /** - * The edge between two panes, dragged to move it. + * The edge between two things, dragged to move it: the side of a pane, or the bottom of the note. * * Wider than the line it draws, because a 1 px line is not something a pointer can be expected to hit — * the same reason the map's own containers have an [EDGE_GRAB]. + * + * [description] says what dragging it does, since a bar of pixels says nothing to a screen reader — and it + * is also the only handle a test has on an edge that draws no text. */ @Composable -internal fun PaneDivider(onDrag: (Dp) -> Unit) { +internal fun PaneDivider( + description: String, + orientation: Orientation = Orientation.Horizontal, + onDrag: (Dp) -> Unit +) { val density = LocalDensity.current Box( Modifier - .width(DIVIDER_GRAB) - .fillMaxHeight() - .pointerHoverIcon(RESIZE_CURSOR) + .alongTheEdge(orientation, DIVIDER_GRAB) + .pointerHoverIcon(if (orientation == Orientation.Horizontal) SIDEWAYS_CURSOR else UPRIGHT_CURSOR) + .semantics { contentDescription = description } .draggable( - orientation = Orientation.Horizontal, + orientation = orientation, state = rememberDraggableState { delta -> onDrag(with(density) { delta.toDp() }) } ) ) { Box( - Modifier.width(DIVIDER_LINE).fillMaxHeight().align(Alignment.Center) + Modifier.alongTheEdge(orientation, DIVIDER_LINE).align(Alignment.Center) .background(MaterialTheme.colorScheme.outlineVariant) ) } } +/** What dragging the edge of a pane does, in the words the pane is named by. */ +internal fun resizeHint(pane: Pane) = "Drag to resize ${pane.said}." + +/** As long as the edge it is on and [thickness] across it, whichever way round that is. */ +private fun Modifier.alongTheEdge( + orientation: Orientation, + thickness: Dp +) = if (orientation == Orientation.Horizontal) { + width(thickness).fillMaxHeight() +} else { + height(thickness).fillMaxWidth() +} + /** What the window shows in the middle when the last tab has been closed. */ @Composable internal fun NoTabOpen(modifier: Modifier = Modifier) { @@ -204,7 +239,8 @@ internal fun NoTabOpen(modifier: Modifier = Modifier) { } } -private val RESIZE_CURSOR = PointerIcon(Cursor(Cursor.E_RESIZE_CURSOR)) +private val SIDEWAYS_CURSOR = PointerIcon(Cursor(Cursor.E_RESIZE_CURSOR)) +private val UPRIGHT_CURSOR = PointerIcon(Cursor(Cursor.N_RESIZE_CURSOR)) private const val FOLD = "⊟" private const val UNFOLD = "⊞" @@ -218,3 +254,14 @@ private val DIVIDER_LINE = 1.dp /** Below this a pane is too narrow to read a class name in, and above it the view is the one squeezed. */ private val MIN_PANE_WIDTH = 160.dp private val MAX_PANE_WIDTH = 720.dp + +/** + * Enough of a note to read a paragraph of it, which is what most of them are, with the rest scrolling. + * + * A note is worth more room than that while it is being written or argued with, and less on a laptop + * screen — hence the divider. Whatever it is dragged to, the note never takes more than its share of the + * window; see [NoteSection]. + */ +private val NOTE_HEIGHT = 160.dp +private val MIN_NOTE_HEIGHT = 56.dp +private val MAX_NOTE_HEIGHT = 640.dp diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerUiTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerUiTest.kt index b494457cf2..0ca0133c4e 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerUiTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerUiTest.kt @@ -10,6 +10,7 @@ import androidx.compose.ui.test.hasProgressBarRangeInfo import androidx.compose.ui.test.onAllNodesWithContentDescription import androidx.compose.ui.test.onNodeWithContentDescription import androidx.compose.ui.test.v2.runSkikoComposeUiTest +import androidx.compose.ui.unit.Dp /** * Runs [block] against a window the size one opens at — [WINDOW_WIDTH] by [WINDOW_HEIGHT]. @@ -18,10 +19,16 @@ import androidx.compose.ui.test.v2.runSkikoComposeUiTest * the chain of objects holding what it's pointing at, and the details panel. At the default size the two * panes leave the view narrow enough that the controls above it are squeezed to nothing, so a test would be * pressing a window no user has. Density is 1 in a UI test, so a dp here is a pixel. + * + * [height] is there for the tests about running out of it — a window dragged short is a window with room for + * one of the things stacked up it — and every other test wants the one a window opens at. */ @OptIn(ExperimentalTestApi::class) -internal fun explorerUiTest(block: ComposeUiTest.() -> Unit) { - runSkikoComposeUiTest(size = Size(width = WINDOW_WIDTH.value, height = WINDOW_HEIGHT.value)) { +internal fun explorerUiTest( + height: Dp = WINDOW_HEIGHT, + block: ComposeUiTest.() -> Unit +) { + runSkikoComposeUiTest(size = Size(width = WINDOW_WIDTH.value, height = height.value)) { block() } } diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/NoteSectionTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/NoteSectionTest.kt index 9a784a8ea7..1506d159fd 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/NoteSectionTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/NoteSectionTest.kt @@ -1,6 +1,7 @@ package shark.explorer.app import androidx.compose.material3.MaterialTheme +import androidx.compose.ui.geometry.Offset import androidx.compose.ui.semantics.Role import androidx.compose.ui.semantics.SemanticsProperties import androidx.compose.ui.test.ComposeUiTest @@ -9,17 +10,22 @@ import androidx.compose.ui.test.SemanticsMatcher import androidx.compose.ui.test.SemanticsNodeInteraction import androidx.compose.ui.test.assertIsDisplayed import androidx.compose.ui.test.assertTextContains +import androidx.compose.ui.test.getBoundsInRoot import androidx.compose.ui.test.hasText import androidx.compose.ui.test.isEnabled import androidx.compose.ui.test.onNodeWithContentDescription import androidx.compose.ui.test.onNodeWithText import androidx.compose.ui.test.performClick +import androidx.compose.ui.test.performMouseInput import androidx.compose.ui.test.performTextInput import androidx.compose.ui.test.waitUntilAtLeastOneExists import androidx.compose.ui.test.waitUntilDoesNotExist import androidx.compose.ui.test.waitUntilExactlyOneExists +import androidx.compose.ui.unit.dp +import androidx.compose.ui.unit.height import java.io.File import org.assertj.core.api.Assertions.assertThat +import org.assertj.core.api.Assertions.within import org.junit.Rule import org.junit.Test import org.junit.rules.TemporaryFolder @@ -223,6 +229,36 @@ class NoteSectionTest { } } + @Test fun `dragging the bottom edge makes the note taller`() { + explorerUiTest { + openHeapDump() + startNote() + val before = noteEditorHeight() + + dragTheNoteEdge(by = DRAG_PIXELS) + + // How much of the window a note is worth is the reading of it against the reading of the heap dump, + // which is a judgement that changes with the note — hence an edge rather than a number in the code. + assertThat(noteEditorHeight()).isCloseTo(before + DRAG_PIXELS, within(SLOP_PIXELS)) + } + } + + @Test fun `a note is never dragged so tall that its own edge leaves the window`() { + // A short window, because that is the shape this is about: a note is dragged tall on a big screen and + // then the window is made small, which is one laptop lid away. + explorerUiTest(height = SHORT_WINDOW) { + openHeapDump() + startNote() + + dragTheNoteEdge(by = SHORT_WINDOW.value) + + // The edge is the only way back, so a drag that put it past the bottom of the window would leave the + // note as tall as it was dragged until the window itself is made bigger. + onNodeWithContentDescription(RESIZE_NOTE_HINT).assertIsDisplayed() + assertThat(noteEditorHeight()).isLessThan(SHORT_WINDOW.value) + } + } + /** Which is what a note being about a place means: a tab somewhere else is another note. */ @Test fun `a note is only about the place the tab it was written on is at`() { explorerUiTest { @@ -369,6 +405,26 @@ class NoteSectionTest { private fun ComposeUiTest.noteEditor(): SemanticsNodeInteraction = onNodeWithContentDescription(NOTE_EDITOR_DESCRIPTION) + /** + * How tall the note is, measured on the box being typed in. + * + * The section itself is no node — it is a `Surface` around whichever of the two states it is in — and the + * box is the state whose height is the whole of what the edge sets, since a written note takes what it + * needs up to that. + */ + private fun ComposeUiTest.noteEditorHeight(): Float = noteEditor().getBoundsInRoot().height.value + + /** Drags the note's bottom edge down by [by] pixels, which is what makes it taller. */ + private fun ComposeUiTest.dragTheNoteEdge(by: Float) { + onNodeWithContentDescription(RESIZE_NOTE_HINT).performMouseInput { + moveTo(center) + press() + moveBy(Offset(x = 0f, y = by)) + release() + } + waitForIdle() + } + /** Where the note about the tab a window opens on is kept. */ private fun noteFile( notesRoot: File, @@ -396,6 +452,15 @@ class NoteSectionTest { } companion object { + /** Far enough to be a drag rather than a click, and short of what the window has room for. */ + private const val DRAG_PIXELS = 120f + + /** What a drag loses to the slop that tells it from a click, which is a pixel or two of the first move. */ + private const val SLOP_PIXELS = 8f + + /** Shorter than a note can be dragged to, which is what makes the share of the window the only limit. */ + private val SHORT_WINDOW = 420.dp + private const val PAYLOAD_LENGTH = 100 private const val HOLDER_ID = 0x82182c00L From add3fb86f5054696f64823d9598d87b43f145ab3 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 10:14:41 +0200 Subject: [PATCH 11/27] Refuse an argument a tool does not take MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by asking a real dump the question this surface's own docs suggest: `find_objects` with `query`, which is what the window calls its search box, matched nothing in particular and answered with the thirty biggest objects out of 86,056 — a list of `ResourcesImpl` and bitmaps in answer to a question about `BuildConfig`. Nothing said no, because nothing read the argument. Which is the failure this surface is built to make impossible: a wrong answer that reads like an answer. So an argument no tool property matches is refused, naming it and naming the ones the tool takes, and the schema says `additionalProperties: false` so a client that validates can catch it before the call rather than after. --- docs/shark-explorer.md | 4 +++ .../java/shark/explorer/agent/AgentTool.kt | 29 ++++++++++++++++++- .../shark/explorer/agent/AgentToolsTest.kt | 16 ++++++++++ 3 files changed, 48 insertions(+), 1 deletion(-) diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 7c72b58be2..e9ea7f5560 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -320,6 +320,10 @@ cannot argue with: * **Every call has to say why it was made.** A call with none is refused — *describe_object needs `reason`, and it was not given* — and so is one whose reason is blank. What that buys is the log below. +* **An argument a tool doesn't take is refused**, naming both it and the ones the tool does take. Which + matters more than it sounds: `find_objects` given `query`, the name of the window's own search box, would + otherwise match nothing in particular and answer with the biggest objects in the heap dump, and a list of + the wrong objects reads exactly like an answer. * **A verdict needs a reason another reader can check**, exactly like one you typed, and it is kept with the verdict in the same file as yours. A verdict that contradicts one already recorded is refused with the list of what it disagrees with, the same way the window asks you. diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTool.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTool.kt index fb5ff2c8f6..d3475a566b 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTool.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTool.kt @@ -26,12 +26,16 @@ internal class AgentTool( private val handler: suspend (AgentArguments) -> JsonObject ) { + /** Every argument this tool takes, which is the same list the schema publishes. See [onlyTakes]. */ + private val takes: Set = (schema[PROPERTIES] as? JsonObject)?.keys.orEmpty() + suspend fun call(arguments: JsonObject): JsonObject { val read = AgentArguments(name, arguments) // Read before the handler and on every tool, so that a call with no reason is refused rather than // logged as a call whose reason was left blank. The schema asks for it and a client is free to ignore a // schema, so this is where "every call says why it was made" is a rule instead of a hope. read.reason + read.onlyTakes(takes) return handler(read) } } @@ -103,6 +107,24 @@ internal class AgentArguments( return text.toIntOrNull() ?: throw wrongType(name, "a whole number", text) } + /** + * Refuses an argument this tool doesn't take, naming the ones it does. + * + * Enforced here for the same reason `reason` is, and it matters more: an argument nobody reads is a call + * that answered about something else. Measured on a real dump — `find_objects` given `query`, which is + * what the window's own search box is called, matched nothing in particular and answered with the 30 + * biggest objects out of 86,056, which reads exactly like an answer to the question asked. + */ + fun onlyTakes(names: Set) { + val unknown = (arguments.keys - names).sorted() + if (unknown.isNotEmpty()) { + throw AgentRefusal( + "$toolName does not take ${unknown.joinToString { "`$it`" }}. It takes " + + "${names.sorted().joinToString { "`$it`" }}, and nothing else." + ) + } + } + fun objectId(name: String): Long = objectIdOf(name, string(name)) fun optionalObjectId(name: String): Long? = optionalString(name)?.let { objectIdOf(name, it) } @@ -202,15 +224,20 @@ internal fun schema(vararg properties: Pair): JsonObject val all = properties.toList() + (REASON to REASON_PROPERTY) return buildJsonObject { put("type", "object") - putJsonObject("properties") { + putJsonObject(PROPERTIES) { all.forEach { (name, property) -> put(name, property.schema) } } putJsonArray("required") { all.filter { it.second.isRequired }.forEach { add(it.first) } } + // Said to the client as well as enforced in [AgentTool.call], since a client that validates is one that + // can catch this before the call rather than after it. + put("additionalProperties", false) } } +private const val PROPERTIES = "properties" + private const val REASON = "reason" private val REASON_PROPERTY = string( diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index 9159e285d6..af6757470c 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -458,6 +458,22 @@ class AgentToolsTest { .isEqualTo("shark://${window.windowId}/${hex(heapDump.activityObjectId)}") } + @Test + fun `an argument the tool does not take is refused rather than ignored`() { + assertThatThrownBy { call("find_objects", "query" to HOLDER_CLASS_NAME) } + .isInstanceOf(AgentRefusal::class.java) + // Named both ways round, because the mistake is a name from somewhere else — `query` is what the + // window's own search box is called — and the fix is the name this tool uses. + .hasMessageContaining("`query`") + .hasMessageContaining("`className`") + + // Which is worth refusing rather than ignoring because ignoring it answers: a filter nothing was read + // into matches the whole heap dump, and the largest objects in it read like a list of matches. + val matched = call("find_objects", "className" to HOLDER_CLASS_NAME) + assertThat(matched.text("matchCount")).isEqualTo("2") + assertThat(matched.text("totalCount").toInt()).isGreaterThan(2) + } + @Test fun `an address written as a decimal number is refused as one`() { assertThatThrownBy { From d13390b2b5592b02b4ed2c23f21f7e7f9bd28cb1 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 10:18:18 +0200 Subject: [PATCH 12/27] Send an agent to the code, at the version the dump is of MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The method stopped where the tools stop: it says isolating the reference is not the root cause, and then leaves the reading of the code — the only place a root cause can come from — as one clause in one bullet. An agent that reads the framework on `main` and the app not at all writes "nothing clears this in onDestroy" about a class the app doesn't ship, which is confident, checkable-looking and wrong. So the method now says which copy to read and how the dump itself says which that is, and each claim in it was checked against a real dump first: `android.os.Build$VERSION` carries `SDK_INT`, `RELEASE`, `CODENAME` and `SECURITY_PATCH`; the app's `ApplicationInfo` carries its process, its `dataDir`, the APK it was installed from, `minSdkVersion`, a `targetSdkVersion=` in `seInfo` and `FLAG_DEBUGGABLE` in `flags`. And `BuildConfig` is not in a dump at all — its constants compile into their call sites, so the class never loads — which is why the app's own version is something to ask for rather than look up. --- docs/shark-explorer-changelog.md | 5 +++ docs/shark-explorer.md | 8 +++++ shark/shark-explorer/notes/agent-surface.md | 14 ++++---- .../java/shark/explorer/agent/AgentMethod.kt | 33 +++++++++++++++++-- 4 files changed, 52 insertions(+), 8 deletions(-) diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index 81f0ada3e1..5ee88cf193 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -59,6 +59,11 @@ uses, without the one for a newly recognized library leak: link names a window and this run has none. Notes and verdicts were never on the screen, so a heap dump investigated with no window opens in one later with all of it on. See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). +* ✨ **The method sends an agent to the code, at the version the heap dump is of.** Isolating the reference + says where the problem is and not how it happened, so the method that comes with the tools also says how to + work out which framework, app and library versions this dump is of — and what to ask for rather than guess, + since an app's own version number never reaches the heap. + See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). * ✨ **An agent answers with links into the window.** `show` and `conclude` hand back the `shark://` link to what they put on screen, and the method the tools come with tells an agent to put those links in its reply — so a sentence in a chat window, a pull request comment or a bug report carries a way into the heap dump diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index e9ea7f5560..b52376c41a 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -291,6 +291,14 @@ leak is — one bad reference, the three zones of a chain, the rules that spread the order that finds it, which is [the LeakCanary method](https://engineering.block.xyz/blog/the-leakcanary-method) as the tools enforce it. +**Including the part that isn't in the heap dump at all.** Isolating the reference says *where* the problem +is, not how it happened, and stopping there is the most common way an investigation fails — so the method +sends an agent to the code, at the version this dump is of, and tells it how to work out which version that +is: `android.os.Build$VERSION.SDK_INT` for the framework, the app's `ApplicationInfo` for its package, its +APK path and its target SDK, the build file or the APK for a library's version, and a decompiler when there +is no source to read. What it can't work out — the app's own version number is usually absent, since +`BuildConfig` constants never reach the heap — it is told to ask you for rather than guess. + **Everything the window can do, it can do** — there is no screen an agent can't reach and no button it can't press, because a surface with less than that is one whose answer is "ask your human to click something": diff --git a/shark/shark-explorer/notes/agent-surface.md b/shark/shark-explorer/notes/agent-surface.md index b8ff23bc55..73525c4d48 100644 --- a/shark/shark-explorer/notes/agent-surface.md +++ b/shark/shark-explorer/notes/agent-surface.md @@ -9,13 +9,15 @@ Measured off `AgentTools.all` and `AgentMethod.INSTRUCTIONS`, one `tools/list` e | | Characters | ≈ tokens | Paid | | --- | --- | --- | --- | -| Sixteen tool definitions | 18,880 | 4,720 | Every turn, while the server is connected | -| The method | 5,065 | 1,270 | Handshake, and again with `open_heap_dumps` | +| Sixteen tool definitions | 18,779 | 4,695 | Every turn, while the server is connected | +| The method | 7,845 | 1,960 | Handshake, and again with `open_heap_dumps` | -So the standing cost of this surface is **5 to 6 k tokens**, around 3% of a 200 k window. Parity took the -tool count from eleven to sixteen and the definitions from 13,116 characters to 18,880 — **a fifth of the +So the standing cost of this surface is **6 to 7 k tokens**, around 3% of a 200 k window. Parity took the +tool count from eleven to sixteen and the definitions from 13,116 characters to 18,779 — **a fifth of the window's budget for the five tools that mean an agent never has to ask its human to click something**, which -is the trade this surface exists to make. The published horror stories are still an order of magnitude worse: +is the trade this surface exists to make. The method then grew by half again for the section on reading the +code at the version the dump is of, which is the one part of the method the tools cannot enforce at all and +the part that decides whether an answer is a root cause or a reference. The published horror stories are still an order of magnitude worse: GitHub's server is ~17.6 k tokens of definitions, and three servers together have been measured at 143 k. The mitigations that shipped in 2026 (Anthropic's tool search, code execution over MCP) are aimed at that scale. **This surface is not where a context window goes to die**, and a per-tool cost of ~300 tokens is what buys @@ -33,7 +35,7 @@ descriptions that say when to reach for a tool. Re-measure it if the count doubl - **A skill** ([the open standard](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview), now read by Claude, Codex, Gemini CLI, Cursor and others) is the right home for *the method*, because progressive disclosure is exactly what the method wants: ~80 tokens of name and description at rest, the - 1,240-token body loaded only for a session that is actually investigating a heap dump. Today every session + ~2 k-token body loaded only for a session that is actually investigating a heap dump. Today every session pays for it at the handshake whether it is investigating anything or not. ## So: one core, several adapters diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt index b0bf63d579..06b1934c7b 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt @@ -73,8 +73,8 @@ internal object AgentMethod { - `find_objects` on a class you have assumed something about. Two instances of a class you took for a singleton is the answer to a surprising number of leaks: the object on the chain is not the instance you think it is. - - Read the app's source for the field that holds the next step. A verdict you can point at a line of - code for is a verdict that survives review. + - Read the code that declares the field holding the next step, at the version this dump is of — see + below. A verdict you can point at a line of code for is a verdict that survives review. 5. **Isolating the reference is not the root cause.** When one reference is left, you know *where* the problem is. You still do not know *how* it happened, and stopping here is the most common way an investigation fails. Keep going: what code assigns that field, what should have cleared it, and why @@ -82,6 +82,35 @@ internal object AgentMethod { 6. **Say how to reproduce it**, or say that you couldn't work that out. A root cause nobody can trigger is a hypothesis. + ## Read the code, at the version the dump is of + + The heap dump says what is held. Only the code says why, so an investigation that stays inside the heap + dump stops at the reference and calls that the root cause. Read what assigns the faulty field, and what + should have cleared it. + + **Which copy of the code matters as much as reading it.** A class that changed between two versions is a + root cause nobody can reproduce and a fix that doesn't apply. The dump itself says which versions: + + - **The OS.** `describe_object` on the `android.os.Build${'$'}VERSION` class: `SDK_INT` is the API level, with + `RELEASE`, `CODENAME` and `SECURITY_PATCH` beside it, and `android.os.Build` has the device and the + build fingerprint. Read AOSP at the tag for that release — an installed SDK has the framework sources + under `sources/android-` — and not `main`, which is years ahead of any device. + - **The app.** Its `android.content.pm.ApplicationInfo` is in most dumps: `processName` and `dataDir` + name the app, `sourceDir` is the APK it was installed from, `minSdkVersion` is a field of its own, + `seInfo` often carries `targetSdkVersion=`, and bit `0x2` of `flags` is `FLAG_DEBUGGABLE`. The app's + own version number usually is **not** there: `BuildConfig` constants are compiled into their call sites, + so the class is never loaded and never appears in a dump. Ask for it rather than guessing it. + - **The libraries.** A dependency's version isn't in the dump either. Ask for the build file or the + lockfile, or read the versions out of the APK at `sourceDir`, and then read that library at that tag. A + leak fixed two releases ago is worth finding out about before writing anything else. + - **Nothing to read?** Decompile. The APK is at `sourceDir` on the device the dump came from, the + dependencies are jars, and a decompiler answers most of what a verdict needs. Compiler-generated names + are evidence in themselves: `this${'$'}0` is an inner class holding what it was declared in, `val${'$'}x` is a + captured local, and neither can be cleared by any code anybody could write. + + Then **say which version of what you read**. "Nothing clears this in onDestroy" about a class the app + doesn't ship is the confident wrong answer this section exists to stop. + ## Rules you will be held to - **Every verdict needs a reason another reader can check.** A field value, an inspector label, the From c61f06340909d4ba2a97c58c0ac23b64d3356ab9 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 10:27:43 +0200 Subject: [PATCH 13/27] Write down what an agent concluded, not only that it did MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A session recorded what was asked and never what came back, which is fine for every tool that answers with data and wrong for the one that answers with a conclusion. So the row that ends a session read "Concluded about MainActivity" and the reference — the whole point of the call — was only in the note it wrote. `outcomeOfTool` reads it off `conclude`'s answer, the screen draws it after an arrow, and the eval now has something to mark against an answer key. Co-Authored-By: Claude Opus 5 --- .../shark-explorer-agent/AGENTS.md | 5 +++ .../shark/explorer/agent/AgentSessionFile.kt | 34 +++++++++++++++++ .../java/shark/explorer/agent/McpSession.kt | 27 ++++++++++++-- .../explorer/agent/AgentSessionFileTest.kt | 37 ++++++++++++++++++- .../shark/explorer/agent/McpSessionTest.kt | 18 +++++++++ .../shark/explorer/app/AgentLogsScreen.kt | 8 +++- .../shark/explorer/app/AgentLogsScreenTest.kt | 19 +++++++++- 7 files changed, 141 insertions(+), 7 deletions(-) diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index fc8ce9ce33..21e94598be 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -69,6 +69,11 @@ a refusal nobody can follow up on is a dead end on the screen. `target` derives *names* rather than from a second list of tool names — one exception, `list_leaks`, which takes no argument saying where it is. +**One field comes off the answer instead: `outcome`.** What an agent asked is what it typed, and what it +concluded is what the heap dump *agreed to* — so `outcomeOfTool` reads the reference out of `conclude`'s +answer, and nothing else records an answer. Both readers need it and neither can work it out: the screen's +last row is what a session came to, and the eval has nothing to mark against its answer key without it. + **The verbs are here rather than in the app.** `verbOfTool` is beside the tool names, so that a screen never spells them itself and drift is one list rather than two. `AgentSessionFileTest` asserts every tool in the registry has one; a tool added without a verb reads as its own name, which is the protocol showing through on diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt index 9d6c82887a..36cff74c80 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt @@ -6,6 +6,7 @@ import java.time.Instant import java.time.ZoneId import java.time.format.DateTimeFormatter import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonArray import kotlinx.serialization.json.JsonElement import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.JsonPrimitive @@ -205,6 +206,7 @@ class AgentSessionFile private constructor( // clickable: the place to go to, and a line the agent's human can paste anywhere. See [DeepLink]. link()?.let { put(LINK_KEY, it) } refusal?.let { put(REFUSAL_KEY, it) } + outcome?.let { put(OUTCOME_KEY, it) } put(MILLIS_KEY, millis) if (arguments.isNotEmpty()) { putJsonObject(ARGUMENTS_KEY) { @@ -233,6 +235,7 @@ class AgentSessionFile private constructor( place = link?.let { placeOfLinkOrNull(it, file, lineNumber) }, arguments = this[ARGUMENTS_KEY]?.asStringMap().orEmpty(), refusal = text(REFUSAL_KEY), + outcome = text(OUTCOME_KEY), millis = text(MILLIS_KEY)?.toLongOrNull() ?: 0L ) } @@ -318,6 +321,7 @@ class AgentSessionFile private constructor( private const val HEAP_DUMP_KEY = "heapDump" private const val LINK_KEY = "link" private const val REFUSAL_KEY = "refused" + private const val OUTCOME_KEY = "outcome" private const val MILLIS_KEY = "millis" private const val ARGUMENTS_KEY = "arguments" } @@ -360,6 +364,14 @@ class AgentSessionCall( val arguments: Map, /** Why the call was refused, and null for one that was answered. See [AgentRefusal]. */ val refusal: String?, + /** + * What the call came to, for the calls whose answer is worth a word. See [outcomeOfTool]. + * + * The other half of a refusal: `conclude` refused says why, and `conclude` answered says which reference + * the heap dump agreed was at fault — which is the one line of a session anybody reads it for, and the one + * the eval scores against the answer key. Null for a call whose answer is data rather than a conclusion. + */ + val outcome: String?, /** How long the app took to answer, which is mostly how long the heap dump read took. */ val millis: Long ) { @@ -390,6 +402,24 @@ val AgentSessionCall.verb: String get() = verbOfTool(tool, arguments) ?: tool.re val AgentSessionCall.subject: String? get() = arguments[SUBJECT_OBJECT] ?: arguments[SUBJECT_PLACE] ?: arguments[SUBJECT_CLASS_NAME] +/** + * What the answer to a call came to, as a couple of words, and null when the answer is data rather than a + * conclusion. + * + * Here beside [verbOfTool] and for the same reason: the tool names live in this file. Only `conclude` has one + * today, which is the point of it — a session is read to find out what somebody concluded, and every other + * call is how they got there. + */ +internal fun outcomeOfTool( + tool: String, + answer: JsonObject +): String? = when (tool) { + "conclude" -> ((answer[ANSWER_FAULTY_REFERENCE] as? JsonArray)?.firstOrNull() as? JsonObject) + ?.let { it[ANSWER_REFERENCE] as? JsonPrimitive } + ?.content + else -> null +} + /** Null for a tool this build has no verb for, which is what a test asserts never happens. */ internal fun verbOfTool( tool: String, @@ -423,6 +453,10 @@ internal fun verbOfTool( else -> null } +/** What `conclude` answers with the reference under, which is the one answer this file records. */ +private const val ANSWER_FAULTY_REFERENCE = "faultyReference" +private const val ANSWER_REFERENCE = "reference" + private const val SUBJECT_OBJECT = "object" private const val SUBJECT_PLACE = "place" private const val SUBJECT_CLASS_NAME = "className" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt index 235859869c..7fa5f7920e 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt @@ -156,14 +156,31 @@ internal class McpSession( val at = Instant.now() val startedAt = System.nanoTime() return try { - val result = toolResult(tool.call(arguments)) - record(name, arguments, target, refusal = null, at = at, startedAt = startedAt) - result + val answer = tool.call(arguments) + // The answer as well as the arguments, because one of them is a conclusion: see [outcomeOfTool]. + record( + name, + arguments, + target, + refusal = null, + outcome = outcomeOfTool(name, answer), + at = at, + startedAt = startedAt + ) + toolResult(answer) } catch (refused: AgentRefusal) { // A refusal is an answer to the agent and not a failure of the server, so it comes back as a tool // result the model reads rather than as a JSON-RPC error the client may swallow. SharkLog.d { "Refused $name: ${refused.message}" } - record(name, arguments, target, refusal = refused.message, at = at, startedAt = startedAt) + record( + name, + arguments, + target, + refusal = refused.message, + outcome = null, + at = at, + startedAt = startedAt + ) toolError(refused.message) } } @@ -181,6 +198,7 @@ internal class McpSession( arguments: JsonObject, target: AgentTarget, refusal: String?, + outcome: String?, at: Instant, startedAt: Long ) { @@ -194,6 +212,7 @@ internal class McpSession( place = target.place, arguments = arguments.recorded(), refusal = refusal, + outcome = outcome, millis = (System.nanoTime() - startedAt) / NANOS_PER_MILLI ) ) diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt index 6515df0d33..5070eeb365 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt @@ -2,6 +2,10 @@ package shark.explorer.agent import java.io.File import java.time.Instant +import kotlinx.serialization.json.addJsonObject +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put +import kotlinx.serialization.json.putJsonArray import org.assertj.core.api.Assertions.assertThat import org.junit.Rule import org.junit.Test @@ -63,6 +67,35 @@ class AgentSessionFileTest { assertThat(call.place).isEqualTo(Place.Object(OBJECT_ID)) } + @Test + fun `a call that concluded says which reference it concluded on`() { + val file = AgentSessionFile.starting(directory, SERVER_VERSION) + file.called(call(tool = "conclude", place = Place.Object(OBJECT_ID), outcome = FAULTY_REFERENCE)) + + // The one line a session is read for, and the one the eval scores against the answer key: a conclusion + // whose reference wasn't written down is a run nobody can mark. See `notes/agent-eval.md`. + assertThat(AgentSessionFile.sessionsIn(directory).single().calls.single().outcome) + .isEqualTo(FAULTY_REFERENCE) + } + + @Test + fun `what a conclusion came to is read off the answer, and nothing else is`() { + val concluded = buildJsonObject { + put("concluded", true) + putJsonArray("faultyReference") { + addJsonObject { put("reference", FAULTY_REFERENCE) } + } + } + + assertThat(outcomeOfTool("conclude", concluded)).isEqualTo(FAULTY_REFERENCE) + // Every other tool answers with data rather than a conclusion, and a row saying what a read came back + // with would be the answer printed twice. + assertThat(outcomeOfTool("describe_object", concluded)).isNull() + // A build whose conclude answers something else is a build whose sessions can't be scored, and null is + // how that shows up rather than as a crash mid-session. + assertThat(outcomeOfTool("conclude", buildJsonObject { put("concluded", true) })).isNull() + } + @Test fun `a session whose last line was cut off keeps the calls before it`() { val file = AgentSessionFile.starting(directory, SERVER_VERSION) @@ -126,7 +159,8 @@ class AgentSessionFileTest { reason: String? = "Because.", place: Place? = null, arguments: Map = emptyMap(), - refusal: String? = null + refusal: String? = null, + outcome: String? = null ) = AgentSessionCall( at = STARTED_AT, tool = tool, @@ -136,6 +170,7 @@ class AgentSessionFileTest { place = place, arguments = arguments, refusal = refusal, + outcome = outcome, millis = 12L ) diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index acada49d8c..32645eb691 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -232,6 +232,24 @@ class McpSessionTest { assertThat(call.place).isEqualTo(Place.Object(heapDump.activityObjectId)) } + @Test + fun `a call that concluded is written down with the reference it concluded on`() { + callTool( + """{"name":"set_verdict","arguments":{"object":"${hex(heapDump.holderObjectId)}",""" + + """"verdict":"EXPECTED","reason":"Holder.INSTANCE is a static singleton."}}""" + ) + callTool( + """{"name":"conclude","arguments":{"object":"${hex(heapDump.activityObjectId)}",""" + + """"rootCause":"Nothing clears Holder.activity in onDestroy.",""" + + """"reason":"The chain names one reference."}}""" + ) + + // The answer rather than the arguments, which is the only line of a session that isn't: what an agent + // asked is what it typed, and what it concluded is what the heap dump agreed to. That is the line the + // *Agent logs* screen ends a session with, and the one the eval marks against an answer key. + assertThat(sessions().single().calls.last().outcome).isEqualTo(FAULTY_REFERENCE) + } + private fun sessions(): List = AgentSessionFile.sessionsIn(sessionsDirectory) private fun callTool(params: String): JsonObject = diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt index c7c1234b44..4973a6e32e 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt @@ -187,12 +187,15 @@ private fun AgentSessionCall.isAbout(heapDumpFile: File): Boolean = /** * What the call did and what it was about, as one line: "Described MainActivity 0x12d368b8". * + * With what it came to on the end where there is one — "Concluded about MainActivity → MainActivity$2.this$0" + * — since the row that says what was concluded is the row anybody scrolling a session is looking for. + * * [title] is what this window calls that object, which is what a tab on it is called too — the row and the * tab it opens have to read the same. Without one, the address the agent wrote: a call about another heap * dump, or one this window hasn't read yet. */ private fun AgentSessionCall.line(title: String?): String = - listOfNotNull(verb, title ?: subject).joinToString(" ") + listOfNotNull(verb, title ?: subject, outcome?.let { "$LEADS_TO $it" }).joinToString(" ") /** What a session is called: who connected, and when. */ private fun AgentSession.title(): String = listOfNotNull( @@ -228,6 +231,9 @@ private val TIME_WIDTH = 60.dp private const val BECAUSE = "because:" +/** In front of what a call came to, which reads as the row's own arrow rather than as a word. */ +private const val LEADS_TO = "→" + private const val REFUSED = "Refused:" private const val A_CLIENT_THAT_DID_NOT_SAY = "An agent" diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt index 8deb01c54c..a7b5076d15 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt @@ -77,6 +77,20 @@ class AgentLogsScreenTest { } } + @Test fun `the row that concluded says which reference it came to`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(call(tool = "conclude", outcome = FAULTY_REFERENCE))))) + onNodeWithText(CLIENT, substring = true).performClick() + + // The row anybody scrolling a session is looking for: what the agent asked, and what it came to, on + // one line — so that finding the answer isn't reading every reason down the screen. + waitUntilAtLeastOneExists( + hasText("Concluded about ${activityName()} → $FAULTY_REFERENCE"), + OPEN_TIMEOUT_MILLIS + ) + } + } + @Test fun `a row leads to the object the call was about`() { explorerUiTest { openAgentLogs(listOf(session(calls = listOf(call())))) @@ -143,7 +157,8 @@ class AgentLogsScreenTest { private fun call( tool: String = "describe_object", heapDumpPath: String = heapDump.file.absolutePath, - refusal: String? = null + refusal: String? = null, + outcome: String? = null ) = AgentSessionCall( at = STARTED_AT, tool = tool, @@ -153,6 +168,7 @@ class AgentLogsScreenTest { place = Place.Object(activityObjectId()), arguments = mapOf("object" to hex(activityObjectId())), refusal = refusal, + outcome = outcome, millis = 12L ) @@ -181,6 +197,7 @@ class AgentLogsScreenTest { const val CLIENT = "claude-code 9.9.9" const val REASON = "Checking whether this activity is really destroyed." const val REFUSAL = "3 step(s) have no verdict" + const val FAULTY_REFERENCE = "Holder.activity" const val NO_AGENT_YET = "No agent has connected" val STARTED_AT: Instant = Instant.parse("2026-08-25T18:19:48.035Z") From 375ed39bb2d4d68b39cf80149d4406fe1cf00425 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 12:00:58 +0200 Subject: [PATCH 14/27] Name the heap dump a run was pointed at while it is still indexing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An agent whose first call asked what was open, 2.6 seconds into a run started on a heap dump, was told nothing was open and was not told the path it had been started on. So it guessed one, opened a different heap dump, and investigated that instead — confidently, and right through to a conclusion about a dump nobody had asked it about. A dump that is still indexing cannot be in `openHeapDumps`: a window id is a promise that every tool given it answers, and one that isn't readable can't keep it. But leaving the path out of the answer altogether is worse than either, so `open_heap_dumps` now names what this run was pointed at, and says to call `open_heap_dump` with it rather than to go looking. Both the run with a window and the run without one had the hole, since a window exists for as long as its dump takes to index. Co-Authored-By: Claude Opus 5 --- .../shark/explorer/agent/AgentHeapDump.kt | 13 +++++++ .../java/shark/explorer/agent/AgentTools.kt | 38 ++++++++++++++----- .../shark/explorer/agent/AgentToolsTest.kt | 19 ++++++++++ .../shark/explorer/agent/FakeAgentHeapDump.kt | 4 ++ .../java/shark/explorer/app/ExplorerAgents.kt | 6 +++ .../explorer/app/HeadlessAgentHeapDumps.kt | 5 +++ .../app/HeadlessAgentHeapDumpsTest.kt | 17 +++++++++ 7 files changed, 93 insertions(+), 9 deletions(-) diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt index c62c83b0d7..6ca76605fb 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt @@ -146,6 +146,19 @@ interface AgentHeapDumps { /** Every window with a heap dump open, in the order they were opened. */ fun openHeapDumps(): List + /** + * The heap dumps this run was pointed at and cannot read yet, absolute, because indexing one takes as long + * as it takes. + * + * Beside [openHeapDumps] rather than in it, since a window id is a promise that every tool given it answers + * and a dump nothing can read yet cannot keep that promise. But leaving the path out of the answer + * altogether is worse than either: an agent that asks what is open, is told nothing is, and is not told the + * path this run was started on has one move left, which is to guess a path. One did — it guessed a heap dump + * belonging to another run of the same eval, investigated that instead, and answered confidently about a + * dump nobody had asked it about. See `notes/agent-eval.md`. + */ + fun openingHeapDumpPaths(): List + /** * Opens [file] in a window of this app and answers once it can be read. * diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index 7894d1686b..f9c2bf51f5 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -74,6 +74,7 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { schema = schema() ) { _ -> val dumps = heapDumps.openHeapDumps() + val indexing = heapDumps.openingHeapDumpPaths() // Read before the JSON is built rather than inside it: a heap dump read suspends, and the JSON builders // don't take a suspending block. val described = dumps.map { dump -> @@ -89,13 +90,13 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { // instructions is a client whose model never saw them. See [AgentMethod]. put("method", AgentMethod.INSTRUCTIONS) putJsonArray("heapDumps") { described.forEach { add(it) } } + // The paths this run was started on, whether or not anything is open: a second dump still indexing while + // the first one is readable is a dump an agent would otherwise never hear about. + if (indexing.isNotEmpty()) { + putJsonArray("indexing") { indexing.forEach { add(it) } } + } if (dumps.isEmpty()) { - put( - "problem", - "No heap dump is open yet. Call $OPEN_HEAP_DUMP with the path of an `.hprof` file, or dump_heap " + - "to take one off a device. If you were pointed at a heap dump, that path is the one to open: it " + - "may be being indexed right now, and opening it again waits for that rather than starting over." - ) + put("problem", nothingToRead(indexing)) } } } @@ -714,9 +715,6 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { /** Named because [placeOrNull] is the one description of a call that has to know which tool it is. */ const val LIST_LEAKS = "list_leaks" - /** And because the refusal for a path that isn't a heap dump points at it. */ - const val OPEN_HEAP_DUMP = "open_heap_dump" - const val WINDOW = "window" const val OBJECT = "object" const val FROM = "from" @@ -902,3 +900,25 @@ private fun Long.requireOneObjectOf(tree: HeapDominatorTreemap) { } throw AgentRefusal(refusal) } + +/** + * Named out here because the refusal for a path that isn't a heap dump points at it, and because + * [nothingToRead] is not a method of [AgentTools]. + */ +private const val OPEN_HEAP_DUMP = "open_heap_dump" + +/** + * What `open_heap_dumps` answers when there is nothing to read, which depends on whether there is about to be. + * + * The indexing case names the path rather than alluding to it, because an agent that is told nothing is open + * and is not told where its heap dump is has one move left, which is to guess a path — and one guessed another + * heap dump on the same machine and investigated that instead. See [AgentHeapDumps.openingHeapDumpPaths]. + */ +private fun nothingToRead(indexing: List): String = if (indexing.isEmpty()) { + "No heap dump is open yet. Call $OPEN_HEAP_DUMP with the path of an `.hprof` file, or dump_heap to take " + + "one off a device." +} else { + "No heap dump can be read yet: this run was pointed at ${indexing.joinToString(", ")} and is indexing it. " + + "Call $OPEN_HEAP_DUMP with that path — it waits for the indexing rather than starting over — and " + + "investigate that dump. It is the one you were asked about, so don't go looking for another file." +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index af6757470c..211a2e1bfa 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -86,6 +86,24 @@ class AgentToolsTest { .hasMessageContaining(OPEN_HEAP_DUMPS) } + @Test + fun `a heap dump still being indexed is named rather than left to be guessed`() { + tools = AgentTools(FakeAgentHeapDumps(indexing = listOf("/eval/runs/4/heap-dump.hprof"))) + + val answer = call(OPEN_HEAP_DUMPS) + + assertThat(answer.array("indexing").map { it.jsonPrimitive.content }) + .containsExactly("/eval/runs/4/heap-dump.hprof") + assertThat(answer.text("problem")) + .contains("/eval/runs/4/heap-dump.hprof") + .contains(OPEN_HEAP_DUMP) + } + + @Test + fun `nothing is said to be indexing when nothing is`() { + assertThat(call(OPEN_HEAP_DUMPS).jsonObject.keys).doesNotContain("indexing") + } + @Test fun `two heap dumps open have to be named`() { val other = FakeAgentHeapDump(heapDump.explorer, windowId = "otherwindow") @@ -698,6 +716,7 @@ class AgentToolsTest { private companion object { const val OPEN_HEAP_DUMPS = "open_heap_dumps" + const val OPEN_HEAP_DUMP = "open_heap_dump" const val SET_VERDICT = "set_verdict" const val CONCLUDE = "conclude" const val OBJECT = "object" diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt index eb1e7bab91..bc58d18cd3 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt @@ -100,6 +100,8 @@ internal class FakeAgentHeapDump( */ internal class FakeAgentHeapDumps( private val open: List = emptyList(), + /** Paths this run was pointed at that aren't readable yet, which is a dump still being indexed. */ + private val indexing: List = emptyList(), /** Keyed by serial number, each with the processes that device is running. */ private val devices: Map> = emptyMap(), /** What a file, or a dump pulled off a device, opens as. Refuses by default, since most tests open none. */ @@ -114,6 +116,8 @@ internal class FakeAgentHeapDumps( override fun openHeapDumps(): List = open + override fun openingHeapDumpPaths(): List = indexing + override suspend fun open(file: File): AgentHeapDump { opened += file return opens(file) diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt index 34d59b5ff1..61ccc9e88b 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt @@ -131,6 +131,12 @@ internal class WindowAgentHeapDumps( window.openHeapDump?.let { open -> window.agentHeapDump(open) } } + override fun openingHeapDumpPaths(): List = windows.mapNotNull { window -> + // A window opened on a file it is still indexing, which is what a run started on a path looks like for as + // long as the indexing takes — and what an agent connecting in that window is otherwise told nothing about. + window.heapDumpFile?.takeIf { window.openHeapDump == null }?.absolutePath + } + override suspend fun open(file: File): AgentHeapDump { // A window already on this file rather than a second window on it, which is the opposite of what the // button does: a person clicking `Open heap dump…` twice on one dump is comparing two readings of it, and diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt index e8d05f145c..221938dc46 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt @@ -68,6 +68,11 @@ internal class HeadlessAgentHeapDumps( override fun openHeapDumps(): List = synchronized(lock) { opened.map { it.agent } } + override fun openingHeapDumpPaths(): List = synchronized(lock) { + val readable = opened.map { it.agent.heapDumpPath }.toSet() + openings.keys.map { it.absolutePath }.filter { it !in readable } + } + override suspend fun open(file: File): AgentHeapDump = opening(file).await().agent /** Releases the thread each open heap dump owns, and stops the ones still opening. */ diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt index 1124ab1ba3..2f1d1eb428 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt @@ -41,6 +41,23 @@ class HeadlessAgentHeapDumpsTest { } } + @Test + fun `a heap dump named on the command line is named before it can be read`() { + val file = temporaryFolder.leakyHeapDump().file + headless(file).use { heapDumps -> + // Either open or indexing, and the assertion is the union because which one it is at this line is a race + // with the open this started: the invariant that matters is that a dump this run was pointed at is never + // in neither list. An agent that asks what is open and is told nothing, with no path, guesses a path. + val named = heapDumps.openingHeapDumpPaths() + heapDumps.openHeapDumps().map { it.heapDumpPath } + assertThat(named).containsExactly(file.absolutePath) + + runBlocking { heapDumps.open(file) } + + assertThat(heapDumps.openHeapDumps().map { it.heapDumpPath }).containsExactly(file.absolutePath) + assertThat(heapDumps.openingHeapDumpPaths()).isEmpty() + } + } + @Test fun `opening the same heap dump twice is one heap dump`() { val file = temporaryFolder.leakyHeapDump().file From fb7aa3e18bd338e4ae87f5782854a05c1abc01bc Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 12:01:07 +0200 Subject: [PATCH 15/27] Measure whether an agent can solve a leak MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every change to a tool description, a refusal or the method is a prompt change, and a prompt change is not something anyone can review by reading it. JProfiler measured theirs and found one model going from 38/55 scenarios to 55/55 on the same tools with better descriptions — a change nobody would have predicted from the diff. So: heap dumps whose answer key is known before the tools are asked anything, a client per run, and a score that is a string comparison and a count over the session file the server wrote while the agent worked. No model marks anything, because a model judging an answer is a second unverified opinion. `shark-explorer-eval` is the dumps and the scoring — its own module because writing a scenario needs `shark-hprof-test` in a main source set, and that is not a dependency anything shipped should have. `harness/eval/run-eval.sh` is the process handling between them. Four things about it are only there because a run was handed its own answer, and every one of those was found by running it: the dump's file name, the client's working directory, the notes of the run before, and the notes of the eval before that one. The fourth cost a day and is written up in notes/agent-eval.md, because it read exactly like a model getting a leak wrong. Co-Authored-By: Claude Opus 5 --- build.gradle.kts | 1 + settings.gradle | 1 + shark/shark-explorer/AGENTS.md | 5 +- shark/shark-explorer/notes/agent-eval.md | 188 ++++++++---- .../shark-explorer-agent/AGENTS.md | 11 + .../harness/eval/run-eval.sh | 286 ++++++++++++++++++ .../shark-explorer-eval/build.gradle.kts | 23 ++ .../main/java/shark/explorer/eval/EvalMain.kt | 145 +++++++++ .../java/shark/explorer/eval/EvalScenarios.kt | 223 ++++++++++++++ .../java/shark/explorer/eval/EvalScore.kt | 186 ++++++++++++ .../shark/explorer/eval/EvalScenariosTest.kt | 113 +++++++ .../java/shark/explorer/eval/EvalScoreTest.kt | 156 ++++++++++ 12 files changed, 1281 insertions(+), 57 deletions(-) create mode 100755 shark/shark-explorer/shark-explorer-agent/harness/eval/run-eval.sh create mode 100644 shark/shark-explorer/shark-explorer-eval/build.gradle.kts create mode 100644 shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalMain.kt create mode 100644 shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalScenarios.kt create mode 100644 shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalScore.kt create mode 100644 shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScenariosTest.kt create mode 100644 shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt diff --git a/build.gradle.kts b/build.gradle.kts index 4d61c99b9f..92913643fc 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -200,6 +200,7 @@ val modulesWithoutPublicApi = listOf( "shark-explorer-agent", "shark-explorer-app", "shark-explorer-core", + "shark-explorer-eval", "shark-explorer-jdwp", "shark-hprof-test", "shark-test", diff --git a/settings.gradle b/settings.gradle index 407800bb6d..8c8a01ec26 100644 --- a/settings.gradle +++ b/settings.gradle @@ -32,6 +32,7 @@ include ':shark:shark-cli' include ':shark:shark-explorer:shark-explorer-agent' include ':shark:shark-explorer:shark-explorer-app' include ':shark:shark-explorer:shark-explorer-core' +include ':shark:shark-explorer:shark-explorer-eval' include ':shark:shark-explorer:shark-explorer-jdwp' include ':shark:shark-graph' include ':shark:shark-hprof' diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index ae8f7481b2..2f058e5314 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -15,6 +15,7 @@ reading the source alone — everything else is in the code. Keep it that way. | `shark-explorer-jdwp` | Attaches to a live app as a debugger to read the pixels of its bitmaps. | **Imports `com.sun.jdi`, so it needs a JDK and can't be loaded on Android.** That's the whole reason it isn't in `core`. | | `shark-explorer-agent` | The MCP server a window answers agents through, the `--mcp-stdio` pipe that reaches it, and `--no-ui` for a run with no window at all. | **No Compose, Java 8 target, and desktop only** — it calls `ProcessHandle`. Has its own `AGENTS.md`. | | `shark-explorer-app` | Compose Desktop UI: window, the canvas each shape draws into, details panel. | **Java 17 target** — see below. | +| `shark-explorer-eval` | The heap dumps an agent is measured on, and the scoring of what it did with them. Driven by `shark-explorer-agent/harness/eval/run-eval.sh`. | **The only module with `shark-hprof-test` in its main source set**, which is why it is a module: writing the scenarios is what it does, and a dump-building DSL can be no dependency of anything the app ships. Runs no model. | `shark/shark-explorer/` itself holds no code, matching how `shark/` and `leakcanary/` are grouping directories in this repo. @@ -585,8 +586,8 @@ Design decisions and findings, kept current as the work proceeds: about the provider rather than the component, and how to dump really generated code - `notes/agent-surface.md` — what the MCP surface costs a client in tokens, measured, and why a CLI and a skill are adapters over the same registry rather than second implementations -- `notes/agent-eval.md` — the plan for scoring how well an agent solves a leak, with no model doing the - scoring +- `notes/agent-eval.md` — how well an agent solves a leak, scored with no model doing the scoring: the answer + keys, the three ways a run gets handed its own answer, and the baseline to beat Update these in the same change that makes them stale. They're for agents, so keep them short and skip anything derivable from the code. diff --git a/shark/shark-explorer/notes/agent-eval.md b/shark/shark-explorer/notes/agent-eval.md index 96ad4bf1c7..f26609a3e0 100644 --- a/shark/shark-explorer/notes/agent-eval.md +++ b/shark/shark-explorer/notes/agent-eval.md @@ -1,6 +1,11 @@ # Measuring whether an agent can solve a leak -The plan for an eval of the agent surface. Not built yet; this is what to build and why it is shaped this way. +An eval of the agent surface. `shark-explorer-eval` is the heap dumps and the scoring; +`shark-explorer-agent/harness/eval/run-eval.sh` is the process handling between them. + +```bash +shark/shark-explorer/shark-explorer-agent/harness/eval/run-eval.sh --models opus,sonnet --repetitions 5 +``` ## What it is for @@ -14,82 +19,152 @@ weak models are where a surface is measured**, since a strong one papers over a ## The rule: no model scores this -An LLM judging an answer is a second unverified opinion. Everything below is decided by string comparison or -by counting, off artefacts the app already writes. +An LLM judging an answer is a second unverified opinion. Every number is a string comparison or a count over +the session file the server wrote while the agent worked — `EvalResult`, and nothing in it is a judgement. + +**The answer key is the faulty reference**, `OwnerClass.field`, per heap dump, and it is known before the tools +are asked anything: -**The answer key is the faulty reference**, `OwnerClass.field`, per heap dump. Two sources for it, both -independent of what the tools would answer: +- **Synthetic dumps built with the `dump { }` DSL**, where the fixture *writes* the leak, so the key is true + by construction. +- **This repository's real Android dumps**, whose key is what LeakCanary's own analysis names. + `leak_asynctask_o.hprof` is `MainActivity$2.this$0`, and `LegacyHprofTest` already pins the same dump's + leaking object and its 211,038 retained bytes, so a key that drifts from the library's reading fails a test. -- **Synthetic dumps built with the `dump { }` DSL**, where the fixture *writes* the leak, so the key is - known by construction. `AgentHeapDumps.applicationHoldsActivityThroughHolder` is the first one: - `Holder.activity`, by construction. This is where the interesting variants live — see the families below. -- **The repository's real Android dumps**, whose key is written down once by hand and checked against - LeakCanary's own leak trace for the same dump. `leak_asynctask_o.hprof` is `MainActivity$2.this$0`, and - `LegacyHprofTest` already pins the same dump's leaking object and its 211,038 retained bytes, so a key that - drifts from the library's reading is a key that fails a test. +`EvalScenariosTest` is what keeps a scenario honest, and it checks three things about every one of them: the +key is on the chain and **one verdict on the owner of it solves the dump**, the chain names **nothing** before +a verdict has been set, and the leak is one `list_leaks` finds on its own. Which is not a check that the +answer is right — it is right by construction — but that the dump can be *investigated* to it. A scenario an +agent can't finish, or one that hands over the answer with no work, is a scenario whose score is a fact about +nothing. ## What one run is scored on | Signal | How it is read | | --- | --- | -| Concluded at all | A `conclude` that was not refused | -| **Right reference** | Exact match of the concluded `OwnerClass.field` against the key | -| Wrong reference | Concluded, but on another step of the chain — the failure that matters most, since it is a confident wrong answer | -| Stopped short | Text answer produced with no `conclude` — the failure mode of [the shark-cli draft](https://github.com/square/leakcanary/pull/2796) | -| Verdicts against the key | An `EXPECTED` on the object the key says is stuck, or the reverse | -| Rounds | Tool calls, and refusals among them | -| Cost | Wall clock, and the client's own token and dollar report where it has one | +| `RIGHT` | The concluded `OwnerClass.field` equals the key | +| `WRONG` | Concluded another reference — the failure that matters most, since it is a confident wrong answer somebody would have acted on | +| `REFUSED` | Tried to conclude and was refused every time, so it never claimed a root cause | +| `NOT_CONCLUDED` | Never tried, which is a surface an agent answered around rather than through | +| `WANDERED` | Concluded about a heap dump this run was not given, so the run measured nothing — see below | +| Calls, refusals | Counted off the session, median over the repetitions | +| Conclude attempts | More than one is the refused-then-verdict-then-concluded story, working | +| Cost | The client's own report, in `/client.json` | Rounds and refusals are the interesting secondary numbers rather than pass/fail: a change that keeps the pass rate and halves the calls is a better surface, and a rise in refusals with the same pass rate says a refusal -message is not telling an agent what to do next. - -## Where the numbers come from - -**A machine-readable session record, one file per agent session**, written beside the human log: the client -that connected, and per call the tool, its arguments, the reason, whether it was refused, and how long the -read took. The eval reads that rather than scraping prose, and the same file is what the window's *Agent -logs* screen draws. One artefact, two readers — build it once. - -**That part exists**: `AgentSessionFile` writes `~/.shark-explorer/agents/sessions/*.jsonl` and reads it back, -so a scorer is a walk over `AgentSessionFile.sessionsIn(…)`. Every signal in the table above is on it except -the two the client reports — an answer written with no `conclude` at all, and the cost — which come from the -adapter's own output. Which session belongs to which scenario run is the file the connection was given: -`AgentServer` logs it as the connection opens, and one run of the eval is one connection. +message is not telling an agent what to do next. **A surface that turns wrong answers into refusals has got +better even if its pass rate hasn't moved**, which is why those are two columns and not one. + +Not scored, deliberately: whether a verdict contradicts the key. It would take resolving the addresses in the +arguments against an open dump, and a verdict that was wrong and then corrected is not a worse run. + +## Four ways a run gets handed its own answer + +All four of these were runs that scored well or failed for the wrong reason, and every one was found by running +the script rather than by reading it. They live in `set_up_run`, and they are the part of this worth knowing +before changing anything: + +- **The heap dump's file name.** An agent is answered with the path of what it is reading, so a dump called + `cache-never-evicts.hprof` names the answer before it has read a byte. Every run's dump is + `heap-dump.hprof`, and the scenario's own copy sits in a numbered directory rather than a named one — the + fourth item below is why the name has to be off the filesystem entirely and not merely off this run's copy. +- **The client's working directory.** Its own environment lists that directory in what the model is told. With + the three dumps in it, the first run of this script opened all three and solved all three — one session, + three conclusions, and a score that meant nothing. A run's working directory now holds one file: its MCP + config. +- **The notes and the verdicts of the run before, and of the eval before that one.** They are kept per heap + dump, keyed by file name and directory, so five repetitions over one path are one investigation and four + agents reading the first one's conclusion — which the very first run demonstrated by calling `read_notes` + third. Each run gets a directory of its own with a symlink in it, since the key doesn't resolve symlinks, and + every invocation puts its runs under a directory named for when it started. That second half was missing for + a day, and the next item is what it cost. +- **An agent with nothing left to investigate goes and finds something.** Worth reading in full: it is the one + that would have been written up as a model failing. + +### The two runs that wandered + +`runs/3/heap-dump.hprof` was the third run of *every* eval, so the second eval's third agent opened a heap dump +the first eval's third agent had already solved — same path, same notes, same verdicts, four of them, with the +faulty reference already named. Its own words for what it did next, in the reason it gave for the call: + +> The dump open in the window is already concluded (CacheEntry.activity). Opening the real 8 MB dump for this +> run, which has no verdicts on it yet, to investigate it. + +The path it opened was a guess — `runs/3/heap-dump.hprof` with `runs` swapped for `dumps` — and it landed on +another scenario's dump, which it then investigated properly and concluded correctly about. Scored against the +scenario it had been given, that is a confidently wrong answer. It is nothing of the kind, and the day before, +the same thing had been written down as sonnet getting a leak wrong. + +Three things came out of it: + +- **A directory per invocation**, which is the actual fix and is one line of the script. +- **`WANDERED`.** Scoring compares the heap dump each conclusion was recorded against with the one the run was + given, and a mismatch is its own outcome rather than a wrong answer. It is not being removed now that the + cause is gone: an eval whose failures look like model failures is worse than no eval. +- **`AgentHeapDumps.openingHeapDumpPaths`.** Not the cause, but the reason the first of the two had nothing + better to do: its first call asked what was open 2.6 seconds in, the dump it had been started on was still + indexing, and the answer said nothing was open without naming the path the run had been pointed at. An agent + told that has one move left, which is to guess a path. That hole is in the *product* rather than in the eval — + an agent connecting to a window that is still indexing falls into exactly the same one — and it is the first + thing this eval found that was worth fixing in the app. + +## Baseline, 2026-08-25 + +Shark Explorer 1.0.0, `claude` 2.1.223, one repetition each, $3.33 and 13 minutes for the six. One repetition +is a smoke test and not a measurement — five is what a result worth arguing from takes — but it is the number +this table is honest about. + +| Scenario | Model | Right | Wrong | Refused | No conclusion | Wandered | Calls | Refusals | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | +| two-apart | opus | 1/1 | 0/1 | 0/1 | 0/1 | 0/1 | 15 | 0 | +| two-apart | sonnet | 1/1 | 0/1 | 0/1 | 0/1 | 0/1 | 9 | 1 | +| cache-never-evicts | opus | 1/1 | 0/1 | 0/1 | 0/1 | 0/1 | 19 | 0 | +| cache-never-evicts | sonnet | 1/1 | 0/1 | 0/1 | 0/1 | 0/1 | 13 | 0 | +| real-asynctask | opus | 1/1 | 0/1 | 0/1 | 0/1 | 0/1 | 28 | 0 | +| real-asynctask | sonnet | 1/1 | 0/1 | 0/1 | 0/1 | 0/1 | 16 | 0 | + +Six for six, which is a ceiling and therefore not much of a baseline: **these three scenarios cannot show a +change to the method or a refusal making anything better**, only worse. What the numbers to beat are is the +call counts, and the one row worth pointing at is sonnet on `two-apart` — refused once, set a verdict, then +concluded, in 9 calls against opus's 15. That is the surface working as designed on the weaker model, which is +the model a surface is measured on. Harder scenarios are what the families below are for, and the cost per run +($0.23 to $1.07) is what says how many repetitions of them are affordable. ## The scenario families -Start with two dumps to get the harness working, then grow the synthetic side, because the whole point is -cases a real dump doesn't happen to contain: +Three exist. The rest are what the synthetic side is *for* — shapes a real dump doesn't happen to contain: -- **Two apart** — one unexplained step between the verdicts, which is `conclude`'s refusal made real. -- **A long unknown zone** — five or six steps with nothing known, so the agent has to work inwards. -- **A decoy** — an object that reads like a leak (destroyed activity in a cache that is meant to hold it) - above the real one, where the key is the reference below. -- **Two candidates** — two references that both cross into stuck, so the answer depends on a verdict the - agent has to defend rather than on the shape of the chain. +- ✅ **Two apart** (`two-apart`) — one unexplained step between the verdicts, which is `conclude`'s refusal + made real. +- ✅ **A long unknown zone** (`cache-never-evicts`) — four steps of infrastructure with no verdict, rooted at + a static singleton so that "this belongs in memory" is a fact of the dump rather than an assumption. +- ✅ **A real dump** (`real-asynctask`) — 8 MB, real framework classes, and a chain nobody wrote for this eval. +- **A decoy** — an object that reads like a leak above the real one, where the key is the reference below. +- **Two candidates** — two references that both cross into stuck, so the answer depends on a verdict the agent + has to defend rather than on the shape of the chain. - **A loop** — objects holding each other, where the chain's order is arbitrary and the conflict machinery reports nothing (see `LeakStatusOverrides.isAbove`). - **A library leak** — the fault is in the framework, and the right answer says so rather than naming app code. +- **Source to read** — the method sends an agent to the code at the version the dump is of, and no scenario + here has any code to read. Measuring that means shipping a source tree with the dump and letting the client + keep its file tools, which the runs above turn off on purpose so that the surface is the only variable. -## The runner +## What the runs leave behind -``` -harness/eval/run-eval.sh --scenarios all --model --repetitions 5 -``` - -Per scenario × model × repetition: open the dump (a window, or headless once that exists), run the client -non-interactively with the same one-line prompt the harness uses today, then score from the session record. -Five repetitions because a model is not deterministic, reported as `x/5` rather than averaged. +Every run is a directory under `$TMPDIR/shark-explorer-eval//runs`: the heap dump as that run +saw it, what the client reported, and which scenario it was. The session goes where every other session goes, +so **a run is readable in a window afterwards** — open that run's `heap-dump.hprof` and the *Agent logs* screen +has the whole investigation, call by call, with the verdicts and the note the agent wrote on the tabs it left. +That is the artefact to look at when a scenario fails: a score says which runs to read, and the log says why. -**One adapter per client**, each a few lines: `claude -p --output-format json` reports turns and usage, -`codex exec` and `opencode run` have their own. The prompt stays identical across clients — what is being -measured is the surface, and a prompt tuned per client measures the prompt. +Until the next eval, which deletes the ones before it: an 8 MB dump per run adds up, and the run that has to be +read is the one that just failed. So read a failure before rerunning. -**Not in CI.** It costs money and needs the network. Run it before and after a change to the method or a -refusal, and commit the table to this file with the date and the versions, so the next change has a baseline -to beat. +An eval also leaves one `~/.shark-explorer/notes` directory and one `leak-statuses` file per run, which is what +makes the above work. They can go once the runs have been read, and nothing depends on them going: the paths +they are keyed to belong to an eval that has already been deleted. ## What to do with a result @@ -97,3 +172,6 @@ A scenario that fails the same way across models is a bug in this surface, not i one of the four things that JProfiler's numbers moved: a more prescriptive description, a refusal that says what to do next, a tool that cannot be called out of order, or a piece of the method that has to be in the tool's own description because the method was skipped. + +**Not in CI.** It costs money and needs the network. Run it before and after a change to the method or a +refusal, and commit the table with the date and the versions, so the next change has a baseline to beat. diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index 21e94598be..9ad398d814 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -24,6 +24,7 @@ being talked to by a program that is not this app. | `AgentStdioBridge.kt` | `--mcp-stdio`: the pipe an MCP client launches. | | `AgentStdioServer.kt` | And `--no-ui`: the same tools over this process's own stdio, for a run with no window. | | `harness/start-harness.sh` | Opens a window and prints the command that throws an agent at it. | +| `harness/eval/run-eval.sh` | Throws an agent at a heap dump whose answer is known, and scores what it did. The dumps and the scoring are `shark-explorer-eval`. | Nothing here is public API — the module is in `modulesWithoutPublicApi`, like the rest of the explorer — with two deliberate exceptions, `AgentServer`/`AgentStdioBridge`/`AgentHeapDump*` because the app calls them, and @@ -187,6 +188,9 @@ the reads happen on the heap dump's thread and the tests run headless. # The whole surface end to end, in a real window, with an agent that has never seen this repository. shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh [heap-dump.hprof] + +# And the same surface scored: heap dumps whose faulty reference is known, and a number per run. +shark/shark-explorer/shark-explorer-agent/harness/eval/run-eval.sh --models opus,sonnet --repetitions 5 ``` Every test here runs against a heap dump built with the `dump { }` DSL and no window, which is what @@ -199,3 +203,10 @@ one heap dump in it, and writes an MCP config pinned to that run plus a prompt t root cause" — so what the agent follows is the method the server handed it. Then read `~/.shark-explorer/logs`: a run that went well and a run that guessed look completely different there, and neither of them looks like anything in a unit test. + +**And `harness/eval` is the measured half of the same idea.** The harness shows how one investigation goes; +the eval runs an agent against a dump whose faulty reference is already known and scores whether it found it, +by string comparison and counting, with no model marking anything. So it is what says whether a change to a +description or a refusal made things better rather than only different. +`shark/shark-explorer/notes/agent-eval.md` has the answer keys — and the three ways a run gets handed its own +answer, each of which was a score that meant nothing. diff --git a/shark/shark-explorer/shark-explorer-agent/harness/eval/run-eval.sh b/shark/shark-explorer/shark-explorer-agent/harness/eval/run-eval.sh new file mode 100755 index 0000000000..74c242080b --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/harness/eval/run-eval.sh @@ -0,0 +1,286 @@ +#!/bin/bash +# +# Measures whether an agent can solve a leak through Shark Explorer's MCP surface, and scores it by string +# comparison and counting. No model marks anything: see shark/shark-explorer/notes/agent-eval.md. +# +# One run is one scenario, one model, one repetition, and one session file. The heap dumps and the scoring +# come from `shark-explorer-eval`; everything here is process handling — launching a client per run, finding +# the session it produced, and writing down which run that session belongs to. +# +# ./run-eval.sh every scenario, the default model, once each +# ./run-eval.sh --scenarios two-apart --repetitions 5 one scenario, five times +# ./run-eval.sh --models opus,sonnet two models over the same dumps, in one table +# +# Costs money and needs the network, so it is not in CI. Run it before and after a change to the method or a +# refusal and commit the table it prints, or the change is a prompt change nobody reviewed. +# +# **Four things here are about keeping a run from being told the answer**, and all four were found by running +# it rather than by thinking about it — see `set_up_run`. A run that leaks its own answer scores well and +# measures nothing, which is the one failure of an eval that doesn't announce itself. + +set -euo pipefail + +readonly REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../../.." && pwd)" +readonly APP_PATH="shark/shark-explorer/shark-explorer-app/build/compose/binaries/main/app/Shark Explorer.app" +readonly SESSIONS_DIRECTORY="$HOME/.shark-explorer/agents/sessions" +readonly TEMPORARY_DIRECTORY="${TMPDIR:-/tmp}" +readonly EVAL_DIRECTORY="${SHARK_EVAL_DIR:-${TEMPORARY_DIRECTORY%/}/shark-explorer-eval}" +# Every invocation gets a directory of its own, named for when it started, because notes and verdicts are kept +# per heap dump path: `runs/3/heap-dump.hprof` reused a week later is the same heap dump as far as they are +# concerned, and the second agent to be given it opens a dump somebody else already solved. Which is not a +# hypothetical — see `set_up_run`. +readonly RUN_SET="$EVAL_DIRECTORY/$(date +%Y-%m-%d_%H-%M-%S)" +readonly EVAL_MODULE=":shark:shark-explorer:shark-explorer-eval" +# Long enough for a real dump to be indexed and an investigation to run, short enough that a client which +# hung on a refusal doesn't hold the whole eval. A run that hits it is scored for what it did before it. +readonly RUN_TIMEOUT_SECONDS="${SHARK_EVAL_TIMEOUT:-900}" + +# What every run of every scenario is asked, and it says nothing about how to investigate. What the agent +# follows has to come from the server, or the eval is measuring this file. +readonly PROMPT="A heap dump is open in Shark Explorer, which you can reach through its MCP tools. Something \ +in it is leaking. Find the root cause." + +main() { + local scenarios="all" models="opus" repetitions=1 + while (($#)); do + case "$1" in + --scenarios) scenarios="$2"; shift 2 ;; + --models | --model) models="$2"; shift 2 ;; + --repetitions) repetitions="$2"; shift 2 ;; + --help | -h) usage; exit 0 ;; + *) echo "Unknown option $1" >&2; usage >&2; exit 1 ;; + esac + done + + require_client + + # The sets before this one, because each is a directory of heap dumps and one of them is 8 MB. Which means a + # run is readable in a window until the next eval starts and not after it, so read a failure before rerunning. + rm -rf "$EVAL_DIRECTORY" + mkdir -p "$RUN_SET/dumps" "$RUN_SET/runs" + local app + app="$(built_app)" + + echo "Writing the scenario heap dumps into $RUN_SET/dumps." + local scenario_lines + scenario_lines="$(eval_module scenarios "$RUN_SET/dumps" "$REPO_ROOT")" + + local runs="$RUN_SET/runs.tsv" + : >"$runs" + local run_number=0 + local name dump key about + while IFS=$'\t' read -r name dump key about; do + if [[ "$scenarios" != "all" && ",$scenarios," != *",$name,"* ]]; then + # Said rather than skipped silently: a table of one scenario looks exactly like a table of all of them + # that only one of them passed. + echo "Skipping $name." + continue + fi + echo + echo "$name — $about" + echo " the answer is $key, and nothing the agent is told mentions it" + local model repetition + for model in ${models//,/ }; do + for ((repetition = 1; repetition <= repetitions; repetition++)); do + run_number=$((run_number + 1)) + run_once "$app" "$name" "$dump" "$model" "$repetition" "$run_number" "$runs" + done + done + done <<<"$scenario_lines" + + echo + echo "Scoring." + echo + eval_module score "$runs" "$REPO_ROOT" "$SESSIONS_DIRECTORY" + cat <>"$runs" + echo " $((ended - started))s, session $session" +} + +# One run's own directory, and it prints where it is. +# +# Four things about the shape of it are what keep a run from being handed its own answer. Each was a run that +# scored well and measured nothing: +# +# **The heap dump is called `heap-dump.hprof`, whatever the scenario is**, and the scenario's own dump sits in a +# numbered directory rather than a named one. An agent is answered with the path of the dump it is reading, so a +# file called `cache-never-evicts.hprof` tells it where to look before it has read anything — and one run +# reached a *sibling* scenario's dump by path, so the name has to be off the filesystem and not merely off this +# run's copy of it. +# +# **A run's dump is in a directory of its own, and every invocation's runs are under [RUN_SET].** Notes and +# verdicts are kept per heap dump, keyed by the file name and the directory it is in — so five runs over one +# path would be one run and four agents reading the conclusion of the first, and `runs/3` reused by the next +# eval a week later is that same agent again. A symlink rather than a copy: the key is not the resolved path, so +# the identity is fresh and the 8 MB is not copied five times. +# +# **The client's working directory holds nothing but its MCP config.** Its own environment lists that +# directory in what it is told, so the other scenarios' dumps being in it is the eval naming every answer at +# once. Which is exactly what the first run of this script did: it opened all three and solved all three. +# +# **And a run that wandered anyway is scored as having wandered**, not as having answered wrongly. Two did, +# before [RUN_SET] existed: given a dump the previous eval had already solved — same path, so the same notes and +# verdicts — an agent that has nothing left to investigate goes looking for a dump that does, and both of them +# guessed a path in this eval's own directory and investigated that instead. The check stays now that the cause +# is gone, because that is how this eval reports its own failure rather than blaming a model for one. +set_up_run() { + local scenario="$1" dump="$2" model="$3" repetition="$4" run_number="$5" + local directory="$RUN_SET/runs/$run_number" + mkdir -p "$directory/cwd" + ln -sf "$dump" "$directory/heap-dump.hprof" + # Beside the run rather than in it, so that a directory of numbers is still readable afterwards. + printf '%s\t%s\t%s\n' "$scenario" "$model" "$repetition" >"$directory/what.txt" + echo "$directory" +} + +# The client, with nothing of this machine to work with but the heap dump. +# +# `--tools ""` turns off every built-in tool, which is the control that makes the number mean something: the +# heap dump and the tool descriptions are then the whole of what the model has, so a change in the score is a +# change in this surface rather than in what it managed to read off the disk. The method tells an agent to go +# and read the code, and it is right to — but a scenario that measures *that* has to ship the code to read, +# which none of these do yet. +# +# `--strict-mcp-config` for the same reason the interactive harness uses it: no other MCP server, and no +# memory of this project. Your own ~/.claude/CLAUDE.md still loads, which is the one thing this cannot keep +# out. +run_client() { + local directory="$1" model="$2" + ( + cd "$directory/cwd" + timeout_command "$RUN_TIMEOUT_SECONDS" claude \ + --print "$PROMPT" \ + --model "$model" \ + --mcp-config mcp.json \ + --strict-mcp-config \ + --allowedTools "mcp__shark-explorer" \ + --tools "" \ + --output-format json \ + --no-session-persistence \ + >"$directory/client.json" 2>"$directory/client.stderr" + ) +} + +# `timeout` is GNU, and macOS has it only if coreutils is installed. Without one, the run is unbounded and +# says so rather than silently having no limit. +timeout_command() { + local seconds="$1" + shift + if command -v timeout >/dev/null; then + timeout "$seconds" "$@" + elif command -v gtimeout >/dev/null; then + gtimeout "$seconds" "$@" + else + echo " no timeout command, so this run is unbounded (brew install coreutils)" >&2 + "$@" + fi +} + +# The server the client launches: this app, answering from its own process with no window. +# +# `--no-ui` rather than a window per run, because thirty windows is thirty indexed heap dumps and nobody is +# watching any of them. What an investigation leaves behind is on disk either way, so the run is still +# readable in a window afterwards — which is the last thing this script prints. +write_mcp_config() { + local app="$1" directory="$2" + cat >"$directory/cwd/mcp.json" <&2 + (cd "$REPO_ROOT" && ./gradlew --quiet :shark:shark-explorer:shark-explorer-app:createDistributable) + if [[ ! -d "$built" ]]; then + echo "The app was not built at $built" >&2 + exit 1 + fi + echo "$built" +} + +# The eval module, on stdout, with Gradle's own noise on stderr where it belongs. +eval_module() { + (cd "$REPO_ROOT" && ./gradlew --quiet "$EVAL_MODULE:run" --args="$*" 2>/dev/null) +} + +session_files() { + ls "$SESSIONS_DIRECTORY" 2>/dev/null | sort || true +} + +require_client() { + if ! command -v claude >/dev/null; then + cat >&2 <,] [--models ,] [--repetitions ] + + --scenarios Which to run, comma separated. Default: all. + --models What to pass the client as its model, comma separated. Default: opus. A weak model is + where a surface is measured: a strong one papers over a bad description. + --repetitions Runs per scenario, reported as x/n rather than averaged, because a model is not + deterministic. Default: 1, and 5 is what a result worth committing takes. +END +} + +main "$@" diff --git a/shark/shark-explorer/shark-explorer-eval/build.gradle.kts b/shark/shark-explorer/shark-explorer-eval/build.gradle.kts new file mode 100644 index 0000000000..f41f309ba5 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-eval/build.gradle.kts @@ -0,0 +1,23 @@ +plugins { + id("org.jetbrains.kotlin.jvm") + id("application") +} + +dependencies { + // The session record an agent leaves behind, which is the whole of what a run is scored from. Nothing + // here talks to a model or to the tools: the agent's client does that, over this module's heap dumps. + implementation(projects.shark.sharkExplorer.sharkExplorerAgent) + + implementation(libs.kotlin.stdlib) + // In the main source set, unlike every other module here, because writing the scenarios *is* what this + // module does. Which is the reason this is a module of its own: a dump built by the DSL cannot be a + // dependency of anything the app ships. + implementation(projects.shark.sharkHprofTest) + + testImplementation(libs.junit) + testImplementation(libs.assertjCore) +} + +application { + mainClass.set("shark.explorer.eval.EvalMainKt") +} diff --git a/shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalMain.kt b/shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalMain.kt new file mode 100644 index 0000000000..2f9a800dc5 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalMain.kt @@ -0,0 +1,145 @@ +package shark.explorer.eval + +import java.io.File +import kotlin.system.exitProcess +import shark.explorer.agent.AgentServer +import shark.explorer.agent.AgentSession +import shark.explorer.agent.AgentSessionFile + +/** + * The two halves of an eval run that are code rather than shell: writing the heap dumps, and scoring the + * sessions the agents left behind. + * + * Everything between them — launching a client, per model, per repetition — is + * `shark-explorer-agent/harness/eval/run-eval.sh`, because it is process handling and nothing else, and + * because the adapter for each client is a line of its own arguments. This module never runs a model. + * + * Two commands rather than one that does everything, since they happen at different times: the dumps are + * written once and read by every run, and the scoring happens after the last one has finished. A run that + * crashed halfway is still scored for what it did. + */ +fun main(args: Array) { + val command = args.firstOrNull() + val rest = args.drop(1) + when (command) { + "scenarios" -> writeScenarios(rest) + "score" -> scoreRuns(rest) + else -> { + System.err.println(USAGE) + exitProcess(1) + } + } +} + +/** + * Writes every scenario's heap dump into a directory, and says on stdout what it wrote. + * + * One line per scenario, tab separated, because the caller is a shell script: the name, the file, the key and + * what the scenario is about. The key is printed for whoever is reading the run rather than for the script — + * scoring reads it back out of this module, so a key can never drift between the two halves. + * + * A numbered directory each, so that **no path an agent could reach spells the scenario's name**: the name says + * what the leak is, and this is the eval's answer key. The number is this listing's order and nothing else, and + * the mapping from it to a name is on stdout, where only the script reads it. + */ +private fun writeScenarios(args: List) { + val directory = File(args.firstOrNull() ?: fail("`scenarios` needs a directory to write the dumps into.")) + val repositoryRoot = File(args.getOrNull(1) ?: ".") + EvalScenarios.all(repositoryRoot).forEachIndexed { index, scenario -> + val file = scenario.writeHeapDumpIn(File(directory, "${index + 1}")) + println(listOf(scenario.name, file.absolutePath, scenario.key, scenario.about).joinToString("\t")) + } +} + +/** + * Scores the runs listed in a file, and prints the table to commit and the line per run under it. + * + * The runs file is written by the script as it goes, a line per finished run: the scenario, the model, the name + * of the session file that run's server wrote, and the heap dump it was pointed at. Written as it goes rather + * than at the end so that an eval + * somebody stopped halfway is still an eval — thirty runs is an hour, and the reason to stop one is usually + * that the first few already answered the question. + */ +private fun scoreRuns(args: List) { + val runsFile = File(args.firstOrNull() ?: fail("`score` needs the file the runs were recorded in.")) + if (!runsFile.isFile) { + fail("There is no runs file at ${runsFile.absolutePath}.") + } + val repositoryRoot = File(args.getOrNull(1) ?: ".") + val sessionsDirectory = args.getOrNull(2)?.let { File(it) } ?: DEFAULT_SESSIONS_DIRECTORY + // Read once and indexed, rather than once per run: the whole point of a session being a small file is that + // a hundred of them is one cheap directory read, and a run names one of them. + val sessions = AgentSessionFile.sessionsIn(sessionsDirectory).associateBy { it.file.name } + val results = runsFile.readLines() + .filter { it.isNotBlank() } + .mapNotNull { line -> scoreRun(line, repositoryRoot, sessions) } + if (results.isEmpty()) { + fail("None of the runs in ${runsFile.absolutePath} could be scored.") + } + println(results.asMarkdownTable()) + println() + println(results.asRunLines()) +} + +/** + * One line of the runs file as a result, or null with a line on stderr saying why not. + * + * Skipped rather than fatal, because the runs a script has already paid for are worth scoring even when one + * of them names a session that isn't there — which is what a client that failed to start the server looks + * like, and it is a finding of its own. + */ +private fun scoreRun( + line: String, + repositoryRoot: File, + sessions: Map +): EvalResult? { + val fields = line.split("\t") + if (fields.size < 4) { + System.err.println( + "Not a run: \"$line\". Each line is scenario, model, session file and heap dump, tab separated." + ) + return null + } + val (scenarioName, model, sessionName, heapDumpPath) = fields + val scenario = EvalScenarios.byName(scenarioName, repositoryRoot) + if (scenario == null) { + System.err.println("There is no scenario called \"$scenarioName\".") + return null + } + val session = sessions[sessionName] + if (session == null) { + System.err.println( + "No session called \"$sessionName\" was written. ${sessions.size} session(s) are there, so this " + + "run is one whose server never got as far as a handshake." + ) + return null + } + // The path as the run's server was pointed at it, which is what its session calls the dump: a run that + // investigated a different file is a run this eval measured nothing with. See [EvalOutcome.WANDERED]. + return EvalResult.of(scenario, model, session, heapDumpPath) +} + +private fun fail(message: String): Nothing { + System.err.println(message) + exitProcess(1) +} + +/** Where a run with no window writes its sessions, which is where every other one writes them too. */ +private val DEFAULT_SESSIONS_DIRECTORY: File + get() = AgentServer.sessionsDirectory( + File(File(System.getProperty("user.home")), ".shark-explorer/agents") + ) + +private val USAGE = """ + Shark Explorer's agent eval, the half of it that isn't process handling. + + scenarios [repository root] + Writes every scenario's heap dump into . Prints one tab separated line per scenario: + name, heap dump, answer key, what it is about. + + score [repository root] [sessions directory] + Scores the runs recorded in , one tab separated line each: scenario, model, session file + name, heap dump. Prints the markdown table to commit, and a line per run under it. + + Run it through shark-explorer-agent/harness/eval/run-eval.sh rather than by hand. +""".trimIndent() diff --git a/shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalScenarios.kt b/shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalScenarios.kt new file mode 100644 index 0000000000..06bd64e9d4 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalScenarios.kt @@ -0,0 +1,223 @@ +package shark.explorer.eval + +import java.io.File +import shark.GcRoot.JniGlobal +import shark.HprofWriterHelper +import shark.ValueHolder.BooleanHolder +import shark.ValueHolder.IntHolder +import shark.ValueHolder.ReferenceHolder +import shark.dump + +/** + * One heap dump to hand an agent, and the reference an investigation of it has to end on. + * + * **The key is known before the tools are asked anything**, which is the whole rule of this eval: either the + * fixture below writes the leak, so the answer is true by construction, or the dump is one of this + * repository's real ones and the key is what LeakCanary's own analysis names. Nothing here asks the surface + * under test what the answer is, and no model decides whether a run got it. + * + * A scenario is [about] one thing that is hard, so that a column of results says *which* part of an + * investigation a change to the method or a refusal moved. See `notes/agent-eval.md`. + */ +class EvalScenario internal constructor( + val name: String, + /** The faulty reference, spelled the way `PathReference.leakLabel` spells one: `Holder.activity`. */ + val key: String, + /** What this dump makes an agent do that the others don't, for the table a run prints. */ + val about: String, + private val writeHeapDump: (File) -> Unit +) { + + /** + * Writes this scenario's heap dump into [directory] and answers with the file. + * + * Written per run rather than committed, like every other test heap dump in this repository — and rewritten + * even when it is already there, since a dump left over from an older build of the DSL would be scored as + * this scenario while being a different one. + * + * **Called `heap-dump.hprof` and not [name]**, so the caller gives each scenario a directory of its own: a + * file called `cache-never-evicts.hprof` hands its answer to whatever opens it, and an agent is answered with + * the path of what it is reading. See `notes/agent-eval.md`. + */ + fun writeHeapDumpIn(directory: File): File { + directory.mkdirs() + val file = File(directory, HEAP_DUMP_FILE_NAME) + file.delete() + writeHeapDump(file) + return file + } + + override fun toString(): String = "$name → $key" +} + +/** + * Every scenario an eval run works through. + * + * Deliberately few and deliberately different from each other. The families still to add are in + * `notes/agent-eval.md`, and each of them is a shape a real dump doesn't happen to contain — two candidate + * references, a loop, a fault in the framework — which is what the synthetic side is for. + */ +object EvalScenarios { + + /** + * [repositoryRoot] is where the real dumps are read from, since the ones under + * `shark/shark-android/src/test/resources` are part of this eval and are not ours to rewrite. + */ + fun all(repositoryRoot: File): List = listOf( + twoApart(), + aCacheThatNeverEvicts(), + aRealAsyncTaskLeak(repositoryRoot) + ) + + fun byName( + name: String, + repositoryRoot: File + ): EvalScenario? = all(repositoryRoot).firstOrNull { it.name == name } + + /** + * The smallest dump that takes an investigation: one unexplained object between the two the heap dump can + * read for itself. + * + * Which is `conclude`'s refusal made real. The application belongs in memory and the activity is destroyed, + * so a surface that named a faulty reference off the dump alone would answer `App.holder` — and the answer + * is one step further down, reachable only by someone deciding what the holder is for. A run that fails + * here fails at the first thing the method asks for. + */ + private fun twoApart() = EvalScenario( + name = "two-apart", + key = "Holder.activity", + about = "One unexplained step between what belongs in memory and what shouldn't be there" + ) { file -> + file.dump { + androidBuild() + val activity = destroyedActivity() + val holder = HOLDER_CLASS_NAME instance { field["activity"] = activity } + val application = instance( + clazz( + className = "com.example.ExampleApplication", + superclassId = clazz(className = "android.app.Application"), + fields = listOf("holder" to ReferenceHolder::class) + ), + fields = listOf(holder) + ) + gcRoot(JniGlobal(id = application.value, jniGlobalRefId = 0)) + } + } + + /** + * A singleton cache holding a destroyed activity through four steps of infrastructure. + * + * The unknown zone is the point: the loader, the cache, its array and the entry all have to be given a + * verdict before one reference is left, and none of them is anything an inspector knows about. What makes + * the answer checkable rather than a matter of taste is the static field — `ImageLoader.INSTANCE` holds the + * loader, so "this is meant to be in memory" is a fact of the dump and not an assumption, and it spreads + * down to the entry. The activity is watched, so the other end is the app's own word for it. + * + * Written bottom up because the loader's class holds the loader: [reserveObjectId] is how an object points + * at something written after it. + */ + private fun aCacheThatNeverEvicts() = EvalScenario( + name = "cache-never-evicts", + key = "CacheEntry.activity", + about = "Four steps of infrastructure with no verdict, rooted at a static singleton" + ) { file -> + file.dump { + androidBuild() + val loader = reserveObjectId() + val activity = destroyedActivity() + // The app's own record that it is done with this activity, which the method says to start from. + keyedWeakReference(activity) + val entry = CACHE_ENTRY_CLASS_NAME instance { + field["key"] = string("screen:main") + field["activity"] = activity + } + val entries = objectArray(entry) + val cache = "com.example.image.MemoryCache" instance { + field["entries"] = entries + field["size"] = IntHolder(1) + } + instance( + clazz( + className = "com.example.image.ImageLoader", + // A class is a GC root of its own, so this static field is what roots the whole chain — and it is + // what an agent can point at to defend a verdict on everything below it. + staticFields = listOf("INSTANCE" to loader), + fields = listOf("cache" to ReferenceHolder::class) + ), + fields = listOf(cache), + objectId = loader + ) + } + } + + /** + * A real Android heap dump of a real leak, which is the one scenario nothing about this repository invented. + * + * `leak_asynctask_o.hprof` is the dump `LegacyHprofTest` pins the leaking object and the retained size of, + * so the key is checked against the library's own reading rather than against ours: an anonymous + * `AsyncTask` subclass holding the activity that declared it, through the field the compiler generates for + * exactly that. Copied into the run's directory rather than opened where it lies, so that an eval run + * writes nothing into the repository — an agent sets verdicts and notes as it works, and those land beside + * the dump. + */ + private fun aRealAsyncTaskLeak(repositoryRoot: File) = EvalScenario( + name = "real-asynctask", + key = "MainActivity\$2.this\$0", + about = "A real dump: 8 MB, an inner class, and a chain nobody wrote for this eval" + ) { file -> + val real = File(repositoryRoot, REAL_ASYNC_TASK_DUMP) + require(real.isFile) { + "There is no heap dump at ${real.absolutePath}. The real-dump scenarios are read out of this " + + "repository, so an eval run has to say where it is: pass the repository root." + } + real.copyTo(file, overwrite = true) + } +} + +/** + * An instance of the app's own `Activity` subclass whose inherited `mDestroyed` is true, which is what the + * object inspectors read to say an object shouldn't be in memory. + * + * Field values are written most derived class first, and the subclass declares none, so the instance is + * written with the one field its superclass has. + */ +private fun HprofWriterHelper.destroyedActivity(): ReferenceHolder = instance( + clazz( + className = "com.example.MainActivity", + superclassId = clazz( + className = "android.app.Activity", + fields = listOf("mDestroyed" to BooleanHolder::class) + ) + ), + fields = listOf(BooleanHolder(true)) +) + +/** + * What `android.os.Build` looks like in a dump, which is what Shark matches its library leak patterns + * against — and a dump with the class but not these three fields makes it throw a bare NPE from under + * everything. See `shark/shark-explorer/AGENTS.md`. + * + * Duplicated from the other modules' tests rather than shared, since a test helper is not worth a module's + * public API. + */ +private fun HprofWriterHelper.androidBuild() { + "android.os.Build" clazz { + staticField["MANUFACTURER"] = string("Google") + staticField["ID"] = string("BP31.250610.004") + } + "android.os.Build\$VERSION" clazz { + // Recent enough that none of Shark's known library leaks is in these dumps, so the references a chain + // names are the app's own — a library leak is a scenario of its own, not a surprise in another one. + staticField["SDK_INT"] = IntHolder(34) + } +} + +/** What every scenario's dump is called, whichever scenario it is. See [EvalScenario.writeHeapDumpIn]. */ +const val HEAP_DUMP_FILE_NAME = "heap-dump.hprof" + +private const val HOLDER_CLASS_NAME = "com.example.Holder" + +private const val CACHE_ENTRY_CLASS_NAME = "com.example.image.CacheEntry" + +/** Where the real dump lives, which is a test resource of `shark-android` and stays one. */ +private const val REAL_ASYNC_TASK_DUMP = "shark/shark-android/src/test/resources/leak_asynctask_o.hprof" diff --git a/shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalScore.kt b/shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalScore.kt new file mode 100644 index 0000000000..c5d78e425e --- /dev/null +++ b/shark/shark-explorer/shark-explorer-eval/src/main/java/shark/explorer/eval/EvalScore.kt @@ -0,0 +1,186 @@ +package shark.explorer.eval + +import shark.explorer.agent.AgentSession +import shark.explorer.agent.AgentSessionCall + +/** + * What one agent's session came to, measured against the scenario's answer key. + * + * **Nothing here is a judgement.** Every field is a string comparison or a count over the session file the + * server wrote while the agent worked, which is the rule this eval exists under: a model scoring another + * model's answer is a second unverified opinion, and the reason to have numbers at all is to be able to + * review a change to a tool description. See `notes/agent-eval.md`. + */ +class EvalResult( + val scenario: String, + val model: String, + val outcome: EvalOutcome, + /** The reference the agent concluded on, and null for a run that concluded nothing. */ + val concluded: String?, + /** What it should have been, repeated here so that a result is readable without the scenario beside it. */ + val key: String, + /** The heap dump a wandering run concluded about instead, and null for every run that stayed. */ + val wanderedTo: String?, + /** How many tools calls it took, refusals included: the number a better surface lowers. */ + val callCount: Int, + /** + * How many of those were refused. + * + * The secondary number worth the most: refusals rising while the pass rate holds still says a refusal + * message is not telling an agent what to do next, and refusals falling to zero says the refusals stopped + * biting. Neither shows up in pass or fail. + */ + val refusalCount: Int, + /** How many times it tried to finish, which is how a run that was refused into giving up reads. */ + val concludeCount: Int, + /** Time the heap dump spent being read for it, summed over every call. */ + val readMillis: Long, + /** Which session file this was read from, so that a row of a table leads back to what the agent did. */ + val sessionId: String +) { + + companion object { + + /** + * Scores [session] against [scenario], which is a walk over the calls and no more than that. + * + * [model] is what ran it, which the session file has no idea about: an MCP server is told the name of the + * client and never the name of the model behind it. + */ + fun of( + scenario: EvalScenario, + model: String, + session: AgentSession, + heapDumpPath: String + ): EvalResult { + val concludes = session.calls.filter { it.tool == CONCLUDE } + val concluded = concludes.firstNotNullOfOrNull { it.outcome } + return EvalResult( + scenario = scenario.name, + model = model, + outcome = outcomeOf(concludes, concluded, scenario.key, heapDumpPath), + concluded = concluded, + key = scenario.key, + wanderedTo = concludes.mapNotNull { it.heapDumpPath }.firstOrNull { it != heapDumpPath }, + callCount = session.calls.size, + refusalCount = session.refusedCount, + concludeCount = concludes.size, + readMillis = session.calls.sumOf { it.millis }, + sessionId = session.sessionId + ) + } + + private fun outcomeOf( + concludes: List, + concluded: String?, + key: String, + heapDumpPath: String + ): EvalOutcome = when { + // Before the answer is compared to anything, because a conclusion about another heap dump is not an + // answer to this scenario however right it reads. + concludes.any { it.heapDumpPath != null && it.heapDumpPath != heapDumpPath } -> EvalOutcome.WANDERED + // The reference and nothing else, because that is what the answer key is: a run that named it and + // explained it badly still found it, and a run that explained the wrong reference beautifully didn't. + concluded == key -> EvalOutcome.RIGHT + concluded != null -> EvalOutcome.WRONG + concludes.isNotEmpty() -> EvalOutcome.REFUSED + else -> EvalOutcome.NOT_CONCLUDED + } + + private const val CONCLUDE = "conclude" + } +} + +/** + * The five ways a run ends, which are five different things to do about it. + * + * [WRONG] is the one that matters most, and the reason a pass rate alone is not enough: an agent that + * concluded the wrong reference produced a confident answer somebody would have acted on, while [REFUSED] and + * [NOT_CONCLUDED] left the question open. A surface that turns wrong answers into refusals has got better + * even if its pass rate hasn't moved. + * + * [WANDERED] is the one that is not about the model at all. It is this eval failing to measure anything, and it + * is here because it happened: a run whose first call found nothing open, and which was not told the path it + * had been started on, guessed one — and the path it guessed was another run's heap dump. Scoring that as a + * wrong answer would have blamed a model for a hole in the surface. See `notes/agent-eval.md`. + */ +enum class EvalOutcome( + /** One word for a table, since a column of enum constants is a column nobody reads. */ + val label: String +) { + /** Concluded, and on the reference the key names. */ + RIGHT("right"), + + /** Concluded on another reference: the confident wrong answer. */ + WRONG("wrong"), + + /** Tried to conclude and was refused every time, so it never claimed a root cause. */ + REFUSED("refused"), + + /** Never tried, which is the failure mode of a surface an agent answers around rather than through. */ + NOT_CONCLUDED("no conclusion"), + + /** Concluded about a heap dump this run was not given, so the run measured nothing and is not the model's. */ + WANDERED("wandered") +} + +/** + * Every result as a markdown table, ready to be committed to `notes/agent-eval.md`. + * + * A table rather than a number, because the number a change is judged by depends on which change it is: a + * pass rate for a new refusal, the call count for a description that was meant to save a round, the wrong + * column for anything that touches the method. Grouped by scenario and model, `x/n` rather than averaged, + * since five runs of a model are five samples and not a measurement of one. + */ +fun List.asMarkdownTable(): String { + val header = + "| Scenario | Model | Right | Wrong | Refused | No conclusion | Wandered | Calls | Refusals |" + val rule = "| --- | --- | --- | --- | --- | --- | --- | --- | --- |" + val rows = groupBy { it.scenario to it.model }.map { (key, results) -> + val (scenario, model) = key + val count = results.size + "| $scenario | $model " + + "| ${results.count { it.outcome == EvalOutcome.RIGHT }}/$count " + + "| ${results.count { it.outcome == EvalOutcome.WRONG }}/$count " + + "| ${results.count { it.outcome == EvalOutcome.REFUSED }}/$count " + + "| ${results.count { it.outcome == EvalOutcome.NOT_CONCLUDED }}/$count " + + // In the table rather than only in the run lines, because a column of zeroes is the claim that these + // numbers are about the models — and a column that isn't zero says to fix the harness before reading + // the rest of the row. + "| ${results.count { it.outcome == EvalOutcome.WANDERED }}/$count " + + "| ${results.map { it.callCount }.median()} " + + "| ${results.map { it.refusalCount }.median()} |" + } + return (listOf(header, rule) + rows).joinToString("\n") +} + +/** + * Every run, one line each, in the order they were scored. + * + * Under the table because the table is what a change is argued from and this is what an argument about one + * row goes to: which reference was concluded, and which session file to open to see how. + */ +fun List.asRunLines(): String = joinToString("\n") { result -> + listOfNotNull( + result.scenario, + result.model, + result.outcome.label, + result.wanderedTo?.let { "concluded about $it, not the dump it was given" }, + result.concluded?.takeIf { it != result.key }?.let { "concluded $it, key ${result.key}" }, + "${result.callCount} call(s)", + "${result.refusalCount} refused".takeIf { result.refusalCount > 0 }, + "${result.concludeCount} conclude attempt(s)".takeIf { result.concludeCount > 1 }, + "${result.readMillis}ms reading", + result.sessionId + ).joinToString(" · ") +} + +/** + * The middle call count rather than the mean, because a run that went in circles is worth ten that didn't and + * would drag an average with it. Averaged over two for an even count, which is what a median is. + */ +private fun List.median(): Int { + val sorted = sorted() + val middle = sorted.size / 2 + return if (sorted.size % 2 == 1) sorted[middle] else (sorted[middle - 1] + sorted[middle]) / 2 +} diff --git a/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScenariosTest.kt b/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScenariosTest.kt new file mode 100644 index 0000000000..48bbd6e237 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScenariosTest.kt @@ -0,0 +1,113 @@ +package shark.explorer.eval + +import java.io.File +import org.assertj.core.api.Assertions.assertThat +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import shark.explorer.HeapExplorer +import shark.explorer.LeakStatus +import shark.explorer.LeakStatusOverride +import shark.explorer.LeakStatusOverrides +import shark.explorer.RootPath +import shark.explorer.faultyReference +import shark.explorer.leakLabel + +/** + * That every scenario is one an agent could actually solve, and that the key names the reference it would end + * on. + * + * **This is not a check that the answer is right** — the answer is right by construction, since the fixture + * writes the leak — it is a check that the dump can be *investigated* to it. A scenario whose key is nowhere + * on the chain, or one whose chain names it before anybody has read anything, is a scenario every model fails + * or passes for the wrong reason, and the eval would report that as a fact about the models. + * + * So each case here does the one thing the method asks an agent to do and no more: find the leak, get the + * chain, set the one verdict that closes the unknown zone, and read off what the chain then names. Which is + * also why it is the test to run after touching the tools — it is the shortest thing in this repository that + * says the surface can be finished. + */ +class EvalScenariosTest { + + @get:Rule + val temporaryFolder = TemporaryFolder() + + @Test + fun `every scenario names its key once the unknown zone is closed`() { + EvalScenarios.all(repositoryRoot).forEach { scenario -> + val file = scenario.writeHeapDumpIn(temporaryFolder.newFolder(scenario.name)) + HeapExplorer.open(file).use { explorer -> + val chain = explorer.chainToTheFirstLeak() + val keyIndex = chain.steps.indexOfFirst { it.step.reference?.leakLabel() == scenario.key } + assertThat(keyIndex) + .describedAs( + "${scenario.name} has no reference called ${scenario.key} on the chain to its first leak. " + + "What it does have: ${chain.references()}" + ) + .isGreaterThan(0) + // Everything above the object that owns the key is meant to be in memory, which one verdict says: + // an EXPECTED spreads upwards. What is below is stuck already, because the dump itself says so. + val owner = chain.steps[keyIndex - 1].step.objectId + val solved = explorer.tree.rootPathTo( + objectId = chain.steps.last().step.objectId, + overrides = LeakStatusOverrides.of( + listOf( + LeakStatusOverride(owner, LeakStatus.EXPECTED, "The scenario says this belongs in memory.") + ) + ) + ) + assertThat(solved.faultyReference()?.leakLabel()) + .describedAs("${scenario.name} was not solved by one verdict on the owner of ${scenario.key}") + .isEqualTo(scenario.key) + } + } + } + + @Test + fun `no scenario names its key before anybody has read anything`() { + EvalScenarios.all(repositoryRoot).forEach { scenario -> + val file = scenario.writeHeapDumpIn(temporaryFolder.newFolder("unread-${scenario.name}")) + HeapExplorer.open(file).use { explorer -> + // Because a chain that names the faulty reference with no verdict set is a scenario an agent finishes + // by reading one answer, and a run of it measures nothing about the method. `conclude` would allow it. + assertThat(explorer.chainToTheFirstLeak().faultyReference()) + .describedAs("${scenario.name} names a faulty reference before anybody has set a verdict") + .isNull() + } + } + } + + @Test + fun `every scenario is a leak the heap dump finds on its own`() { + EvalScenarios.all(repositoryRoot).forEach { scenario -> + val file = scenario.writeHeapDumpIn(temporaryFolder.newFolder("leaks-${scenario.name}")) + HeapExplorer.open(file).use { explorer -> + // The first step of the method is `list_leaks`, so a scenario whose leak isn't in it is one an agent + // has to go looking for by other means — a different investigation from the one being measured. + assertThat(explorer.tree.findLeaks().leakingObjectCount) + .describedAs("${scenario.name} has no leak of the app's own for list_leaks to answer with") + .isGreaterThan(0) + } + } + } + + /** The chain to the first leaking object the dump reports, which is where the method starts. */ + private fun HeapExplorer.chainToTheFirstLeak(): RootPath { + val leaks = tree.findLeaks() + val leaking = leaks.leakSections.flatMap { it.groups }.flatMap { it.objects } + assertThat(leaking).describedAs("nothing is leaking in this dump").isNotEmpty + return tree.rootPathTo(leaking.first().objectId) + } + + /** Every reference the chain names, for a failure message that says what the key should have been. */ + private fun RootPath.references(): String = + steps.mapNotNull { it.step.reference?.leakLabel() }.joinToString(", ") + + /** + * Where the real dumps are read from. + * + * Four directories up from this module, since a JVM test runs with the module as its working directory and + * the real dumps are test resources of another one. + */ + private val repositoryRoot: File get() = File("../../..").canonicalFile +} diff --git a/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt b/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt new file mode 100644 index 0000000000..834a29e601 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt @@ -0,0 +1,156 @@ +package shark.explorer.eval + +import java.io.File +import java.time.Instant +import org.assertj.core.api.Assertions.assertThat +import org.junit.Test +import shark.explorer.agent.AgentSession +import shark.explorer.agent.AgentSessionCall + +/** + * What a session is scored as, which is a handful of counts and one string comparison. + * + * Worth testing rather than reading because the outcomes are the whole of what this eval reports, and most of + * them are failures that mean different things: a wrong answer somebody would have acted on, a run the refusals + * stopped, a run that never tried, and a run that investigated the wrong heap dump — which is this eval's own + * failure and not a model's. A scorer that folded any two of those together would report a change as neutral + * that had turned confident wrong answers into refusals, which is the change most worth making. + */ +class EvalScoreTest { + + @Test + fun `a conclusion on the key is right`() { + val result = score(calls = listOf(call("list_leaks"), concluded(KEY))) + + assertThat(result.outcome).isEqualTo(EvalOutcome.RIGHT) + assertThat(result.concluded).isEqualTo(KEY) + assertThat(result.callCount).isEqualTo(2) + } + + @Test + fun `a conclusion on another reference is wrong, not a near miss`() { + val result = score(calls = listOf(concluded("ExampleApplication.holder"))) + + // The failure the whole eval exists to count: the run produced an answer, and somebody would have gone + // and changed the wrong line. Nothing here scales it by how close the reference was. + assertThat(result.outcome).isEqualTo(EvalOutcome.WRONG) + assertThat(result.concluded).isEqualTo("ExampleApplication.holder") + } + + @Test + fun `a run the refusals stopped is told from one that never tried`() { + val refused = score(calls = listOf(call("conclude", refusal = "Not concluded. 1 step(s) have no verdict"))) + val neverTried = score(calls = listOf(call("list_leaks"), call("chain_from_gc_root"))) + + assertThat(refused.outcome).isEqualTo(EvalOutcome.REFUSED) + assertThat(refused.concludeCount).isEqualTo(1) + assertThat(neverTried.outcome).isEqualTo(EvalOutcome.NOT_CONCLUDED) + assertThat(neverTried.concludeCount).isZero + } + + @Test + fun `a run refused and then right is right, and says how many attempts it took`() { + val result = score( + calls = listOf( + call("conclude", refusal = "Not concluded. 1 step(s) have no verdict"), + call("set_verdict"), + concluded(KEY) + ) + ) + + // Which is the story the surface is built for — refused, a verdict, then concluded — so a scorer that + // called this a refusal would mark the method working as the method failing. + assertThat(result.outcome).isEqualTo(EvalOutcome.RIGHT) + assertThat(result.refusalCount).isEqualTo(1) + assertThat(result.concludeCount).isEqualTo(2) + } + + @Test + fun `a conclusion about another heap dump is not an answer to this scenario`() { + val result = score( + calls = listOf(call("open_heap_dump"), concluded(KEY, heapDumpPath = "/dumps/2/heap-dump.hprof")) + ) + + // Even though it concluded the key: this run was given another dump, so what it found was a leak in + // somebody else's scenario. Scored as the harness failing rather than as the model answering. + assertThat(result.outcome).isEqualTo(EvalOutcome.WANDERED) + assertThat(result.wanderedTo).isEqualTo("/dumps/2/heap-dump.hprof") + assertThat(result.asRunLine()).contains("not the dump it was given") + } + + @Test + fun `the table is one row per scenario and model, counted out of the repetitions`() { + val results = listOf( + score(calls = listOf(concluded(KEY))), + score(calls = listOf(concluded("Other.field"))), + score(calls = listOf(call("conclude", refusal = "Not concluded"))) + ) + + val table = results.asMarkdownTable() + + // `x/n` rather than a rate, because three runs of a model are three samples: a rate of 33% reads as a + // measurement and hides that the answer to "does this work" was yes once. + assertThat(table).contains("| two-apart | opus | 1/3 | 1/3 | 1/3 | 0/3 | 0/3 |") + } + + @Test + fun `a run leads back to the session it was read from`() { + val result = score(calls = listOf(concluded(KEY))) + + // Because every number above is an argument about a session somebody then has to go and read, and the + // *Agent logs* screen finds one by this id. + assertThat(result.asRunLine()).contains(SESSION_ID) + } + + private fun score(calls: List) = EvalResult.of( + scenario = EvalScenario( + name = "two-apart", + key = KEY, + about = "A scenario of this test's own, so that nothing here depends on which dumps exist" + ) { error("This test scores sessions and opens no heap dump") }, + model = "opus", + session = AgentSession( + sessionId = SESSION_ID, + startedAt = AT, + client = "claude-code 9.9.9", + serverVersion = "1.2.3", + file = File("/sessions/agent-$SESSION_ID.jsonl"), + calls = calls + ), + heapDumpPath = HEAP_DUMP_PATH + ) + + private fun concluded( + reference: String, + heapDumpPath: String = HEAP_DUMP_PATH + ) = call("conclude", outcome = reference, heapDumpPath = heapDumpPath) + + private fun call( + tool: String, + refusal: String? = null, + outcome: String? = null, + heapDumpPath: String = HEAP_DUMP_PATH + ) = AgentSessionCall( + at = AT, + tool = tool, + reason = "Because.", + windowId = null, + heapDumpPath = heapDumpPath, + place = null, + arguments = emptyMap(), + refusal = refusal, + outcome = outcome, + millis = 12L + ) + + private fun EvalResult.asRunLine() = listOf(this).asRunLines() + + private companion object { + const val KEY = "Holder.activity" + const val SESSION_ID = "1a2b3c4d" + + /** The dump this run was given, named the way every run's is: the scenario is not in the path. */ + const val HEAP_DUMP_PATH = "/runs/1/heap-dump.hprof" + val AT: Instant = Instant.parse("2026-08-25T18:19:48.035Z") + } +} From 1826618d68ae99bece7aafe7149143441da28ff9 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 19:04:59 +0200 Subject: [PATCH 16/27] Name the object an agent asked about where the heap dump is open MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The *Agent logs* screen drew a bare `0x12d368b8` for most rows, and the rows it did name were the only ones that led anywhere. Both came from the same mistake: the window resolved the addresses itself, so it could only speak for calls about the dump it happened to have open, and a session spans every dump that was open while it ran. So the name is recorded as the call is made, by the run that has that dump open — `AgentTarget.about`, one small read per call — and the screen shows what is written down. A row reads the same in any window now, and a row about another heap dump opens that dump instead of doing nothing. It only leads nowhere when the file has been deleted, which it says by naming it. Sessions written before this have no name against their calls, so `subject` falls back to the address the agent typed. Co-Authored-By: Claude Opus 5 --- .../shark-explorer-agent/AGENTS.md | 8 ++ .../shark/explorer/agent/AgentSessionFile.kt | 28 ++++- .../java/shark/explorer/agent/AgentTools.kt | 40 +++++- .../java/shark/explorer/agent/McpSession.kt | 4 +- .../explorer/agent/AgentSessionFileTest.kt | 20 +++ .../shark/explorer/agent/McpSessionTest.kt | 21 +++- .../shark/explorer/app/AgentLogsScreen.kt | 114 ++++++++++++------ .../java/shark/explorer/app/ExplorerWindow.kt | 28 +++++ .../shark/explorer/app/HeapDumpExplorer.kt | 62 ++-------- .../src/main/java/shark/explorer/app/Main.kt | 11 ++ .../shark/explorer/app/AgentLogsScreenTest.kt | 61 ++++++++-- .../shark/explorer/app/ExplorerWindowTest.kt | 27 +++++ .../java/shark/explorer/eval/EvalScoreTest.kt | 1 + 13 files changed, 318 insertions(+), 107 deletions(-) diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index 9ad398d814..8551ed0551 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -70,6 +70,14 @@ a refusal nobody can follow up on is a dead end on the screen. `target` derives *names* rather than from a second list of tool names — one exception, `list_leaks`, which takes no argument saying where it is. +**And it is named here, not by whoever reads it.** `target` costs one extra read per call, `agentPlaceTitle`, +because what an agent typed is an address and what the screen shows is `MainActivity 0x12d368b8` — and +resolving an address means having *that* heap dump open. A session spans every dump that was open while it +ran, and it is read afterwards in whichever window happens to be open, so a screen that resolved these itself +could only name the calls about its own dump and every other row would stay a bare address. Which is exactly +what it did, until this was recorded. `about` is null for a session written before that, and `subject` falls +back to the address, so an old session still reads. + **One field comes off the answer instead: `outcome`.** What an agent asked is what it typed, and what it concluded is what the heap dump *agreed to* — so `outcomeOfTool` reads the reference out of `conclude`'s answer, and nothing else records an answer. Both readers need it and neither can work it out: the screen's diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt index 36cff74c80..29e0400797 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt @@ -202,6 +202,9 @@ class AgentSessionFile private constructor( reason?.let { put(REASON_KEY, it) } windowId?.let { put(WINDOW_KEY, it) } heapDumpPath?.let { put(HEAP_DUMP_KEY, it) } + // What the object is called, and not only the address the agent wrote: naming it takes the heap dump + // it is in, and this is written while that dump is open. See [AgentSessionCall.about]. + about?.let { put(ABOUT_KEY, it) } // As the link the window hands out for that place, which is the whole of what a row has to be // clickable: the place to go to, and a line the agent's human can paste anywhere. See [DeepLink]. link()?.let { put(LINK_KEY, it) } @@ -233,6 +236,7 @@ class AgentSessionFile private constructor( windowId = text(WINDOW_KEY), heapDumpPath = text(HEAP_DUMP_KEY), place = link?.let { placeOfLinkOrNull(it, file, lineNumber) }, + about = text(ABOUT_KEY), arguments = this[ARGUMENTS_KEY]?.asStringMap().orEmpty(), refusal = text(REFUSAL_KEY), outcome = text(OUTCOME_KEY), @@ -319,6 +323,7 @@ class AgentSessionFile private constructor( private const val REASON_KEY = "reason" private const val WINDOW_KEY = "window" private const val HEAP_DUMP_KEY = "heapDump" + private const val ABOUT_KEY = "about" private const val LINK_KEY = "link" private const val REFUSAL_KEY = "refused" private const val OUTCOME_KEY = "outcome" @@ -360,6 +365,16 @@ class AgentSessionCall( val windowId: String?, val heapDumpPath: String?, val place: Place?, + /** + * What [place] is called — `MainActivity 0x12d368b8` — as the window naming a tab on it would. + * + * Recorded rather than worked out on the way in, because working it out is a read of *that* heap dump: a + * session spans the dumps that were open while it ran, and it is read afterwards in whichever window + * happens to be open. So a screen that resolved these itself could only name the calls about its own dump, + * and every other row would stay the bare address an agent wrote — which is the one thing this screen + * exists to not show. Null for a call about no place, and for a session written before this was recorded. + */ + val about: String?, /** The rest of the arguments, by name, with `reason` and `window` left out: they have fields of their own. */ val arguments: Map, /** Why the call was refused, and null for one that was answered. See [AgentRefusal]. */ @@ -395,12 +410,21 @@ class AgentSessionCall( val AgentSessionCall.verb: String get() = verbOfTool(tool, arguments) ?: tool.replace('_', ' ') /** - * What the call was about, in the words the window uses for it: an address, a class name, a place. + * What the call was about, in the words the window uses for it: an object with its class name, a class name, + * a place. + * + * Which is [AgentSessionCall.about] wherever there is one, so that a row and the tab clicking it opens are + * recognisably one object. What the agent typed is the fallback, and it is what a call about a place of no + * heap dump reads as — an address on its own, which is the whole of what a session held before the app + * started writing the name beside it. * * Null for a call whose subject is the whole heap dump or the app itself, where the verb says all of it. */ val AgentSessionCall.subject: String? - get() = arguments[SUBJECT_OBJECT] ?: arguments[SUBJECT_PLACE] ?: arguments[SUBJECT_CLASS_NAME] + get() = about + ?: arguments[SUBJECT_OBJECT] + ?: arguments[SUBJECT_PLACE] + ?: arguments[SUBJECT_CLASS_NAME] /** * What the answer to a call came to, as a couple of words, and null when the answer is data rather than a diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index f9c2bf51f5..6adb04e60f 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -23,6 +23,7 @@ import shark.explorer.leakLabel import shark.explorer.leakStatusConflictsWith import shark.explorer.nodeIdText import shark.explorer.outlineOf +import shark.explorer.titleOf /** * Everything an agent can do to an open heap dump, as MCP tools. @@ -592,14 +593,14 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { } /** - * Which window and which place a call was about, for the log of the session it was made in. + * Which window, which place and what to call it, for the log of the session the call was made in. * * Read off the arguments rather than out of the handler, so that a call that was refused is recorded * pointing at whatever it was asking about — which is most of what makes a refusal worth reading * afterwards. Nothing here refuses: this is a description of a call, and a call with an argument this * can't make sense of is one the handler is about to refuse with a message of its own. */ - fun target( + suspend fun target( name: String, arguments: JsonObject ): AgentTarget { @@ -609,7 +610,17 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { return AgentTarget( windowId = dump?.windowId, heapDumpPath = dump?.heapDumpPath, - place = place + place = place, + // Named here, while the dump it is a place of is open, rather than by whoever reads the log + // afterwards: a session is read in whichever window happens to be open, and a window that has + // another heap dump cannot say what an address in this one stands for. See [agentPlaceTitle]. + about = if (dump == null || place == null) { + null + } else { + dump.read("what to call ${placeText(place) ?: place} for the log") { + it.tree.agentPlaceTitle(place) + } + } ) } @@ -763,7 +774,7 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { } /** - * What a call was about: which window, which heap dump, and which place of it. + * What a call was about: which window, which heap dump, which place of it, and what that place is called. * * Only for the session log, which is the one reader that needs this without needing the answer: a row of the * *Agent logs* screen is a verb, a subject and somewhere to go when it is clicked. See [AgentSessionCall]. @@ -771,9 +782,28 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { internal class AgentTarget( val windowId: String?, val heapDumpPath: String?, - val place: Place? + val place: Place?, + /** What the window calls [place] — `MainActivity 0x12d368b8` — read while the dump was open. */ + val about: String? ) +/** + * What the window calls a place an agent asked about: the title a tab on it would have. + * + * The same [titleOf] the tabs are named by, so that a row of a session and the tab clicking it opens read + * the same — an agent and the person watching it are looking at one object, and two spellings of it would + * be two objects to them. + * + * With the one difference that makes this a function of its own: an agent can name an address the heap dump + * has no object at, which is a call it was refused and still a row worth reading. [titleOf] would throw on + * it, so the address is asked about first and stands for itself when it is nothing here. + */ +private fun HeapDominatorTreemap.agentPlaceTitle(place: Place): String = when (place) { + is Place.Object -> + if (objectNameOrNull(place.objectId) == null) exactHexObjectId(place.objectId) else titleOf(place) + else -> titleOf(place) +} + /** * What the verdicts on a chain add up to: whether one reference is at fault, and what to say when none is. * diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt index 7fa5f7920e..45f7d8ad71 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt @@ -151,7 +151,8 @@ internal class McpSession( // trying to learn and then what that cost. See [AgentTools]. SharkLog.d { "An agent called $name${arguments.logLine()}" } // What the call is about, read before it is made rather than after: a refused call is recorded pointing - // at whatever it was asking about, which is most of what makes a refusal worth reading afterwards. + // at whatever it was asking about, which is most of what makes a refusal worth reading afterwards. And + // named here, while that heap dump is open, because nothing reading the session later can. val target = tools.target(name, arguments) val at = Instant.now() val startedAt = System.nanoTime() @@ -210,6 +211,7 @@ internal class McpSession( windowId = target.windowId, heapDumpPath = target.heapDumpPath, place = target.place, + about = target.about, arguments = arguments.recorded(), refusal = refusal, outcome = outcome, diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt index 5070eeb365..40708ce258 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt @@ -38,6 +38,7 @@ class AgentSessionFileTest { tool = "describe_object", reason = "Reading the holder's fields.", place = Place.Object(OBJECT_ID), + about = "Holder 0x12d368b8", arguments = mapOf("object" to "0x12d368b8") ) ) @@ -55,6 +56,23 @@ class AgentSessionFileTest { assertThat(call.heapDumpPath).isEqualTo("/dumps/leak.hprof") assertThat(call.arguments).containsEntry("object", "0x12d368b8") assertThat(call.millis).isEqualTo(12L) + // What the object is called, which the window that answered the agent wrote down: the screen reading + // this is in whichever window is open, and naming an address means having that heap dump. + assertThat(call.about).isEqualTo("Holder 0x12d368b8") + assertThat(call.subject).isEqualTo("Holder 0x12d368b8") + } + + @Test + fun `a call with no name recorded is read as the address the agent wrote`() { + val file = AgentSessionFile.starting(directory, SERVER_VERSION) + file.called( + call(tool = "describe_object", place = Place.Object(OBJECT_ID), arguments = mapOf("object" to "0x12d368b8")) + ) + + // Which is every session written before the name was recorded beside the address, and the reason the + // screen asks for a subject rather than for a name: an old session still has rows worth reading. + assertThat(AgentSessionFile.sessionsIn(directory).single().calls.single().subject) + .isEqualTo("0x12d368b8") } @Test @@ -158,6 +176,7 @@ class AgentSessionFileTest { tool: String, reason: String? = "Because.", place: Place? = null, + about: String? = null, arguments: Map = emptyMap(), refusal: String? = null, outcome: String? = null @@ -168,6 +187,7 @@ class AgentSessionFileTest { windowId = WINDOW_ID, heapDumpPath = "/dumps/leak.hprof", place = place, + about = about, arguments = arguments, refusal = refusal, outcome = outcome, diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index 32645eb691..2ef09d078f 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -15,6 +15,7 @@ import org.junit.Test import org.junit.rules.TemporaryFolder import shark.explorer.Place import shark.explorer.exactHexObjectId +import shark.explorer.hexObjectId /** * What a client of this server gets back, as JSON-RPC rather than as Kotlin. @@ -207,7 +208,9 @@ class McpSessionTest { assertThat(session.serverVersion).isEqualTo(SERVER_VERSION) val call = session.calls.single() assertThat(call.verb).isEqualTo("Described") - assertThat(call.subject).isEqualTo(hex(heapDump.holderObjectId)) + // The object as the window names a tab on it, not the address the agent typed: the screen reading this + // is in whichever window is open later, and naming an address takes the heap dump it is in. + assertThat(call.subject).isEqualTo("Holder ${hexObjectId(heapDump.holderObjectId)}") assertThat(call.reason).isEqualTo("Checking whether the holder is the singleton it looks like.") assertThat(call.refusal).isNull() // Which is what makes the row clickable: the place, in the window the call was made against. @@ -227,9 +230,21 @@ class McpSessionTest { assertThat(call.verb).isEqualTo("Concluded about") assertThat(call.refusal).contains("Not concluded") assertThat(call.reason).isEqualTo("I know what this is.") - // Refused, and still pointing at the object it was refused about: a refusal nobody can follow up on is - // the half of a session that is worth reading afterwards. + // Refused, and still pointing at the object it was refused about, named: a refusal nobody can follow up + // on is the half of a session that is worth reading afterwards. assertThat(call.place).isEqualTo(Place.Object(heapDump.activityObjectId)) + assertThat(call.subject).isEqualTo("MainActivity ${hexObjectId(heapDump.activityObjectId)}") + } + + @Test + fun `an address of no object of the heap dump is written down as the address`() { + callTool("""{"name":"describe_object","arguments":{"object":"0xdeadbeef","reason":"Guessing."}}""") + + // A refusal, and a row of it still says what was asked about. There is nothing to name it after, so it + // stands for itself rather than making the call unrecordable. + val call = sessions().single().calls.single() + assertThat(call.refusal).contains("0xdeadbeef") + assertThat(call.subject).isEqualTo("0xdeadbeef") } @Test diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt index 4973a6e32e..45e708e390 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt @@ -19,6 +19,7 @@ import java.io.File import java.time.Instant import java.time.ZoneId import java.time.format.DateTimeFormatter +import shark.SharkLog import shark.explorer.Place import shark.explorer.agent.AgentSession import shark.explorer.agent.AgentSessionCall @@ -74,26 +75,31 @@ internal fun AgentLogsScreen( * from each other, and that is a question about what was asked and why — a screen of JSON is the same * information in the one form nobody reads. So a row is what the call did, what it was about, and the * sentence the agent gave for making it, which is its own words and not a paraphrase. An agent names objects - * by address, and this names them the way the rest of the window does, so that a row and the tab it opens are - * recognisably the same object. + * by address, and a row names them the way the rest of the window does, so that a row and the tab it opens + * are recognisably the same object. Which is [AgentSessionCall.subject], written down as the call was made: + * naming an address means reading the dump it is in, and this screen is read from whichever window is open. * - * A row about an object of the heap dump this window has open leads to it, like every other way to an - * object here. One about another dump says which, and leads nowhere: a session can span windows, and - * silently landing on the wrong dump's object at the same address would be worse than not moving. + * **And every row that names a place leads to it.** One about the heap dump this window has open goes there + * the way every other way to an object here does; one about another dump opens that dump. A session is one + * agent's connection and can read as many dumps as were open, so a row leading nowhere would be the app + * showing somebody what an agent looked at and then declining to show them the thing. */ @Composable internal fun AgentLogScreen( session: AgentSession?, - /** Which heap dump this window has open, which is what decides whether a row leads anywhere. */ + /** Which heap dump this window has open, which is what decides whether a row moves this window. */ heapDumpFile: File, - /** - * What this window calls each place a call was about — `MainActivity 0x12d368b8` — for the places it has - * been asked about yet. A place that isn't in here is drawn as the address the agent wrote, which is what - * a call about another heap dump stays as: naming it would mean reading a dump this window doesn't have. - */ - placeTitles: Map, onOpen: (Place, OpenIn) -> Unit, onCopyLink: (Place) -> Unit, + /** + * Where a row about another heap dump goes: that dump, in the window that has it or one of its own. + * + * Nothing by default, because routing this is a question about every window of the run and a screen + * composed without an answer must not silently look like a screen whose rows lead somewhere. + */ + onOpenHeapDump: (File, Place) -> Unit = { file, place -> + SharkLog.d { "Nothing here to open $place of $file with" } + }, modifier: Modifier = Modifier ) { Surface(modifier, color = MaterialTheme.colorScheme.surface) { @@ -121,9 +127,9 @@ internal fun AgentLogScreen( AgentCallRow( call = call, heapDumpFile = heapDumpFile, - title = call.place?.let { placeTitles[it] }, onOpen = onOpen, - onCopyLink = onCopyLink + onCopyLink = onCopyLink, + onOpenHeapDump = onOpenHeapDump ) } } @@ -135,14 +141,19 @@ internal fun AgentLogScreen( private fun AgentCallRow( call: AgentSessionCall, heapDumpFile: File, - /** What this window calls what the call was about, and null while it hasn't been read or can't be. */ - title: String?, onOpen: (Place, OpenIn) -> Unit, - onCopyLink: (Place) -> Unit + onCopyLink: (Place) -> Unit, + onOpenHeapDump: (File, Place) -> Unit ) { - // A row leads somewhere only when the place it names is a place of the dump this window has open. An - // address is an address of one heap dump, so the same one in another dump is a different object. - val place = call.place?.takeIf { call.isAbout(heapDumpFile) } + val place = call.place + // Which heap dump the row is about when it isn't this window's, and null when it is. An address is an + // address of one dump, so the same number in another one is another object: this window cannot go there, + // and the dump that can has to be opened first. + val elsewhere = call.otherHeapDumpOrNull(heapDumpFile) + // And whether that is still possible. A session outlives the heap dumps it was about, so a row naming one + // that has been deleted says which and leads nowhere. + val opens = elsewhere?.takeIf { it.isFile } + val line = call.line(elsewhere) Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) { Text( call.at.clockTime(), @@ -151,17 +162,29 @@ private fun AgentCallRow( color = MUTED_TEXT ) Column { - if (place == null) { - Text(call.line(title), style = MaterialTheme.typography.bodyMedium) - } else { - val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } - OpenTarget(open, { onCopyLink(place) }) { - Text( - call.line(title), - Modifier.openable(open), - style = MaterialTheme.typography.bodyMedium, - color = LINK_COLOR - ) + when { + // Asking which heap dumps are open is about the app rather than about one of them. + place == null -> Text(line, style = MaterialTheme.typography.bodyMedium) + opens != null -> Text( + line, + // No tab to choose and no link to copy: what a link names is a window, and the window this call + // was made against belongs to a run that has usually ended. The heap dump is what outlived it. + Modifier.openable { onOpenHeapDump(opens, place) }, + style = MaterialTheme.typography.bodyMedium, + color = LINK_COLOR + ) + // The heap dump it names is gone, so there is nothing left to open it on. + elsewhere != null -> Text(line, style = MaterialTheme.typography.bodyMedium) + else -> { + val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } + OpenTarget(open, { onCopyLink(place) }) { + Text( + line, + Modifier.openable(open), + style = MaterialTheme.typography.bodyMedium, + color = LINK_COLOR + ) + } } } call.reason?.let { reason -> @@ -180,9 +203,16 @@ private fun AgentCallRow( } } -/** Whether the call was about the heap dump this window has open. See [AgentLogScreen]. */ -private fun AgentSessionCall.isAbout(heapDumpFile: File): Boolean = - heapDumpPath == null || heapDumpPath == heapDumpFile.absolutePath +/** + * The heap dump the call was about when it is one this window hasn't got open, and null when it has. + * + * Null as well for a call about no heap dump at all, which is asking the app which dumps are open. Whether + * the file is still there is a separate question, and the one that decides whether the row leads anywhere: + * a deleted dump is worth naming and impossible to open. See [AgentLogScreen]. + */ +private fun AgentSessionCall.otherHeapDumpOrNull(heapDumpFile: File): File? = heapDumpPath + ?.takeIf { it != heapDumpFile.absolutePath } + ?.let { File(it) } /** * What the call did and what it was about, as one line: "Described MainActivity 0x12d368b8". @@ -190,12 +220,15 @@ private fun AgentSessionCall.isAbout(heapDumpFile: File): Boolean = * With what it came to on the end where there is one — "Concluded about MainActivity → MainActivity$2.this$0" * — since the row that says what was concluded is the row anybody scrolling a session is looking for. * - * [title] is what this window calls that object, which is what a tab on it is called too — the row and the - * tab it opens have to read the same. Without one, the address the agent wrote: a call about another heap - * dump, or one this window hasn't read yet. + * And with [otherHeapDump] named at the end of a row about a dump this window hasn't got open, because + * clicking that row opens a heap dump: which one is a thing to know before rather than after. */ -private fun AgentSessionCall.line(title: String?): String = - listOfNotNull(verb, title ?: subject, outcome?.let { "$LEADS_TO $it" }).joinToString(" ") +private fun AgentSessionCall.line(otherHeapDump: File?): String = listOfNotNull( + verb, + subject, + outcome?.let { "$LEADS_TO $it" }, + otherHeapDump?.let { "$IN ${it.name}" } +).joinToString(" ") /** What a session is called: who connected, and when. */ private fun AgentSession.title(): String = listOfNotNull( @@ -234,6 +267,9 @@ private const val BECAUSE = "because:" /** In front of what a call came to, which reads as the row's own arrow rather than as a word. */ private const val LEADS_TO = "→" +/** And in front of the heap dump a row is about, for the rows that are about another one. */ +private const val IN = "in" + private const val REFUSED = "Refused:" private const val A_CLIENT_THAT_DID_NOT_SAY = "An agent" diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt index b8af60b9bf..c8fa2ce62e 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt @@ -269,5 +269,33 @@ internal fun ExplorerWindows.openHeapDump( return window } +/** + * Goes to [place] of [heapDumpFile], opening that dump if no window of this run has it. + * + * Which is what a row of the *Agent logs* screen about another heap dump leads to: a session is one agent's + * connection and can read whichever dumps were open, so the object a call was about is often not in the + * window the log is being read in. A window already showing that dump is the one to raise rather than a + * second one on the same file — the same rule [openHeapDump] follows, one window per heap dump — and the + * window that has just been opened for it is in front already, so only an existing one is brought forward. + * + * Not [DeepLink]: a link names the window it was copied from, and the window a session recorded is usually + * one from a run that has since ended. The heap dump outlives it, which is why this goes by the file. + */ +internal fun ExplorerWindows.goToHeapDump( + heapDumpFile: File, + place: Place +) { + // By absolute path, because a window opened from a command line holds the relative path it was given while + // a session recorded the absolute one, and those are the same heap dump. + val showing = firstOrNull { it.heapDumpFile?.absoluteFile == heapDumpFile.absoluteFile } + SharkLog.d { + val where = if (showing == null) "a window it is not open in yet" else "window ${showing.deepLinkId}" + "A row of an agent's session asked $where for $place of ${heapDumpFile.name}" + } + val window = showing ?: openHeapDump(heapDumpFile) + window.goToLinked(place) + showing?.bringToFront() +} + /** Between what a run is called and which heap dump a window shows, as elsewhere in this window. */ private const val TITLE_SEPARATOR = " · " diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt index ea6a0cda56..f47edb25f4 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt @@ -79,7 +79,6 @@ import shark.explorer.TreemapPresentation import shark.explorer.TreemapRect import shark.explorer.agent.AgentSession import shark.explorer.detours -import shark.explorer.exactHexObjectId import shark.explorer.formatObjectCount import shark.explorer.hexObjectId import shark.explorer.leakStatusConflictsWith @@ -132,6 +131,15 @@ internal fun HeapDumpExplorer( * window of this run, another run of the app, or nowhere. See [DeepLinkPeers.follow]. */ followDeepLink: (DeepLink) -> Unit = { link -> SharkLog.d { "Nothing here to follow $link with" } }, + /** + * And where a row of an agent's session about another heap dump goes, which is that heap dump. + * + * Only the application knows, for the reason [followDeepLink] is its too: which window a heap dump opens + * in is a question about every window of the run. See [ExplorerWindows.goToHeapDump]. + */ + onOpenHeapDump: (File, Place) -> Unit = { file, place -> + SharkLog.d { "Nothing here to open $place of $file with" } + }, /** * What every agent that has connected to this app did, read whenever the screen showing them is open. * @@ -193,12 +201,6 @@ internal fun HeapDumpExplorer( var sessions by remember { mutableStateOf(emptyList()) } /** What each tab is called, by the place it is on. Only grows: a place is named once and stays named. */ var placeTitles by remember { mutableStateOf(emptyMap()) } - /** - * And what to call the places an agent asked about, which is the same question with one difference: an - * agent can name an address this heap dump has no object at, so this map answers for a place a tab could - * not be opened on. See [agentPlaceTitle]. - */ - var agentPlaceTitles by remember { mutableStateOf(emptyMap()) } /** * The note about the tab on screen, and null once the last tab has been closed — which is the one state * with no tab to write about. @@ -526,27 +528,6 @@ internal fun HeapDumpExplorer( } } - // And what to call the objects those agents asked about, so that a row of a session names an object the - // way the tab it opens does — `MainActivity 0x12d368b8` — rather than as the bare address the agent wrote. - // The session file holds addresses on purpose: an address is what an agent said, and what it stands for is - // a read of the heap dump this window has open, which is the same read that names a tab. - val unnamedAgentPlaces = (place as? Place.AgentLog) - ?.let { open -> sessions.firstOrNull { it.sessionId == open.sessionId } } - ?.calls.orEmpty() - .filter { it.heapDumpPath == null || it.heapDumpPath == session.heapDumpFile.absolutePath } - .mapNotNull { it.place } - .filter { it !in agentPlaceTitles } - .distinct() - LaunchedEffect(session, unnamedAgentPlaces) { - if (unnamedAgentPlaces.isEmpty()) { - return@LaunchedEffect - } - val named = session.read("what to call ${unnamedAgentPlaces.size} places an agent asked about") { explorer -> - unnamedAgentPlaces.associateWith { explorer.tree.agentPlaceTitle(it) } - } - agentPlaceTitles = agentPlaceTitles + named - } - // And what has been decided about this heap dump's objects by hand, also once per run: one small file, // read before anything is drawn from it, because a chain read without it would be the heap dump's own // answer where someone has already recorded another. See [HeapDumpLeakStatuses]. @@ -758,7 +739,7 @@ internal fun HeapDumpExplorer( favourites = favourites, sessions = sessions, heapDumpFile = session.heapDumpFile, - agentPlaceTitles = agentPlaceTitles, + onOpenHeapDump = onOpenHeapDump, sizes = sizes, onOpen = openObject, onCopyLink = copyObjectLink, @@ -1100,8 +1081,8 @@ private fun ListPlace( sessions: List, /** Which heap dump this window has open, which is what decides where an agent's row leads. */ heapDumpFile: File, - /** What this window calls the places those agents asked about. See [agentPlaceTitle]. */ - agentPlaceTitles: Map, + /** And where one about another heap dump leads: that dump. See [AgentLogScreen]. */ + onOpenHeapDump: (File, Place) -> Unit, sizes: HeapSizes, onOpen: (Long, OpenIn) -> Unit, onCopyLink: (Long) -> Unit, @@ -1160,9 +1141,9 @@ private fun ListPlace( // Null for a session that has been pushed out by newer ones, or one from another machine's link. session = sessions.firstOrNull { it.sessionId == place.sessionId }, heapDumpFile = heapDumpFile, - placeTitles = agentPlaceTitles, onOpen = onOpenPlace, onCopyLink = onCopyPlaceLink, + onOpenHeapDump = onOpenHeapDump, modifier = modifier ) // The places with a view of their own are drawn by the panes, not here. @@ -1641,23 +1622,6 @@ private suspend fun HeapDumpSession.describing( ) } -/** - * What this window calls a place an agent asked about: the title a tab on it would have. - * - * The same [titleOf] the tabs are named by, so that a row of a session and the tab clicking it opens read the - * same — an agent and the person watching it are looking at one object, and two spellings of it would be two - * objects to them. - * - * With the one difference that makes this a function of its own: an agent can name an address this heap dump - * has no object at, which is a call it was refused and still a row worth reading. [titleOf] would throw on - * it, so the address is asked about first and stands for itself when it is nothing here. - */ -private fun HeapDominatorTreemap.agentPlaceTitle(place: Place): String = when (place) { - is Place.Object -> - if (objectNameOrNull(place.objectId) == null) exactHexObjectId(place.objectId) else titleOf(place) - else -> titleOf(place) -} - /** What the panes are being filled in for, for the log. See [HeapDumpSession.read]. */ private fun Place.description(): String = when (this) { is Place.Object -> "what ${nodeIdText(objectId)} is" diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt index 1e5ea4c6a1..095d294f34 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt @@ -211,6 +211,9 @@ private fun explorerApplication( // The same way a link arriving from the OS is followed, which is what makes a `shark://` link // written in a note work wherever it is read from. followDeepLink = { link -> DeepLinkPeers.follow(link, windows) }, + // A row of an agent's session about another heap dump, which is a file rather than a window: the + // run that answered that agent has usually ended, and its window ids with it. + onOpenHeapDump = { file, place -> windows.goToHeapDump(file, place) }, linkedPlaces = window.linkedPlaces, onLinkedPlaceOpened = { place -> window.linkedPlaceOpened(place) }, deepLinkProblem = window.deepLinkProblem, @@ -283,6 +286,13 @@ internal fun ExplorerApp( * question about every window of the run, so a window composed without it says so in the log. */ followDeepLink: (DeepLink) -> Unit = { link -> SharkLog.d { "Nothing here to follow $link with" } }, + /** + * And where a row of an agent's session about another heap dump goes, which is that heap dump: the window + * showing it, or a new one. Only the application knows, for the same reason. See [ExplorerWindows]. + */ + onOpenHeapDump: (File, Place) -> Unit = { file, place -> + SharkLog.d { "Nothing here to open $place of $file with" } + }, /** * Why this window has no heap dump, for one a link opened because the window it named had gone. * @@ -400,6 +410,7 @@ internal fun ExplorerApp( linkedPlaces = linkedPlaces, onLinkedPlaceOpened = onLinkedPlaceOpened, followDeepLink = followDeepLink, + onOpenHeapDump = onOpenHeapDump, openUrl = openUrl, copyToClipboard = copyToClipboard, modifier = Modifier.weight(1f) diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt index a7b5076d15..dddc3dbe40 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt @@ -14,6 +14,7 @@ import androidx.compose.ui.test.performClick import androidx.compose.ui.test.waitUntilAtLeastOneExists import java.io.File import java.time.Instant +import org.assertj.core.api.Assertions.assertThat import org.junit.Before import org.junit.Rule import org.junit.Test @@ -32,8 +33,8 @@ import shark.explorer.hexObjectId * * An investigation an agent ran and one a person ran are the same investigation: it reads this heap dump, * sets the verdicts they see and writes the same notes. So what it did is read here in words, and **a row - * leads where the call went** — which is what these tests are about, along with the one case where it must - * not: a call about another heap dump, whose addresses mean nothing here. + * leads where the call went** — which is what these tests are about, including the row about a heap dump + * this window hasn't got open, since a session can span every dump that was open while it ran. */ @OptIn(ExperimentalTestApi::class) class AgentLogsScreenTest { @@ -106,15 +107,49 @@ class AgentLogsScreenTest { } } - @Test fun `a call about another heap dump is read here and leads nowhere`() { + @Test fun `a call about another heap dump names its object, and leads to that dump`() { + val otherHeapDump = testFolder.newFile("another.hprof") + var opened: Pair? = null explorerUiTest { - openAgentLogs(listOf(session(calls = listOf(call(heapDumpPath = "/dumps/another.hprof"))))) + openAgentLogs( + sessions = listOf(session(calls = listOf(call(heapDumpPath = otherHeapDump.absolutePath)))), + onOpenHeapDump = { file, place -> opened = file to place } + ) + onNodeWithText(CLIENT, substring = true).performClick() + waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) + + // Named the same way as a row about this window's own dump, because the name was written down when + // the call was made — this window has never read that file and could not work it out. + val row = "Described ${activityName()} in ${otherHeapDump.name}" + waitUntilAtLeastOneExists(hasText(row), OPEN_TIMEOUT_MILLIS) + onNodeWithText(row).performClick() + } + + // Not this window: an address is an address of one heap dump, so going there means opening that dump. + assertThat(opened).isEqualTo(otherHeapDump to Place.Object(activityObjectId())) + } + + @Test fun `a call about a heap dump that has gone leads nowhere`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(call(heapDumpPath = "/dumps/deleted.hprof"))))) onNodeWithText(CLIENT, substring = true).performClick() waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) - // An address is an address of one heap dump, so the same one here is a different object — or no - // object at all. Read it as the agent wrote it, and don't follow it. - onNodeWithText("Described ${hex(activityObjectId())}").assertHasNoClickAction() + // Still worth reading, and there is nothing to open: a session outlives the heap dumps it was about. + onNodeWithText("Described ${activityName()} in deleted.hprof").assertHasNoClickAction() + } + } + + @Test fun `a session written before the app recorded names reads as the address the agent wrote`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(call(about = null))))) + onNodeWithText(CLIENT, substring = true).performClick() + waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) + + // Which is what every session on disk holds from before there was a name beside the address, and a + // row of one still leads to the object: this window has that heap dump open. + onNodeWithText("Described ${hex(activityObjectId())}").performClick() + waitUntilAtLeastOneExists(hasText("mDestroyed", substring = true), OPEN_TIMEOUT_MILLIS) } } @@ -127,7 +162,10 @@ class AgentLogsScreenTest { } /** Opens the window on [leakyHeapDump] with [sessions] as the agents that have worked through it. */ - private fun ComposeUiTest.openAgentLogs(sessions: List) { + private fun ComposeUiTest.openAgentLogs( + sessions: List, + onOpenHeapDump: (File, Place) -> Unit = { _, _ -> } + ) { setContent { MaterialTheme { ExplorerApp( @@ -136,6 +174,9 @@ class AgentLogsScreenTest { // Given rather than read off this machine: the sessions under whoever is running the tests are // their investigations, and none of this window's business. agentSessions = { sessions }, + // Where a row about another heap dump goes, which is a question about every window of the run and + // so answered outside this one. See [ExplorerWindowTest]. + onOpenHeapDump = onOpenHeapDump, deviceHeapDumps = DeviceHeapDumps(NO_DEVICE_ADB) ) } @@ -157,6 +198,9 @@ class AgentLogsScreenTest { private fun call( tool: String = "describe_object", heapDumpPath: String = heapDump.file.absolutePath, + // What the app wrote down for the object as the call was made, which is what a row reads as. Null for a + // session from a build that recorded only the address. + about: String? = activityName(), refusal: String? = null, outcome: String? = null ) = AgentSessionCall( @@ -166,6 +210,7 @@ class AgentLogsScreenTest { windowId = "zvphq4r3", heapDumpPath = heapDumpPath, place = Place.Object(activityObjectId()), + about = about, arguments = mapOf("object" to hex(activityObjectId())), refusal = refusal, outcome = outcome, diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt index 444fdf54c4..f0666a4129 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt @@ -198,6 +198,33 @@ class ExplorerWindowTest { assertThat(empty.deepLinkProblem).isNull() } + @Test fun `a row of an agent's session goes to the window that has that heap dump`() { + val windows = explorerWindows(opening(FIRST_DUMP, SECOND_DUMP)) + val (first, second) = windows + + // The absolute path, which is what a session recorded, against a window holding the relative one it was + // given on the command line: the same heap dump, and one window of it. + windows.goToHeapDump(SECOND_DUMP.absoluteFile, Place.Leaks()) + + // Not a second window on the same dump: a session names the heap dump it read rather than a window, + // since the run that answered that agent has usually ended and its window ids with it. + assertThat(second.linkedPlaces).containsExactly(Place.Leaks()) + assertThat(first.linkedPlaces).isEmpty() + assertThat(windows).hasSize(2) + } + + @Test fun `a row about a heap dump no window has open opens it`() { + val windows = explorerWindows(opening(FIRST_DUMP)) + + windows.goToHeapDump(SECOND_DUMP.absoluteFile, Place.Starred) + + // Because the alternative is the app showing somebody what an agent looked at and then declining to + // show them the thing. + val opened = windows.last() + assertThat(opened.heapDumpFile).isEqualTo(SECOND_DUMP.absoluteFile) + assertThat(opened.linkedPlaces).containsExactly(Place.Starred) + } + @Test fun `a run knows which windows are its own`() { val windows = explorerWindows(opening(FIRST_DUMP)) diff --git a/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt b/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt index 834a29e601..f61024de10 100644 --- a/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt +++ b/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt @@ -137,6 +137,7 @@ class EvalScoreTest { windowId = null, heapDumpPath = heapDumpPath, place = null, + about = null, arguments = emptyMap(), refusal = refusal, outcome = outcome, From fef41993f258c239c3b000de39ab12c26569cfb5 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 19:26:05 +0200 Subject: [PATCH 17/27] Leave the one place no argument named unnamed in a session MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `list_leaks` is the only tool whose place comes from which tool it is rather than from an argument, so recording a name for it made its row read "Listed the leaks Leaks" — the verb already says the whole of it. --- .../java/shark/explorer/agent/AgentTools.kt | 23 +++++++++++-------- .../shark/explorer/agent/McpSessionTest.kt | 12 ++++++++++ 2 files changed, 26 insertions(+), 9 deletions(-) diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index 6adb04e60f..151b9e07b2 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -606,7 +606,8 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { ): AgentTarget { val read = AgentArguments(name, arguments) val dump = read.orNull { resolvedDump(optionalString(WINDOW)) } - val place = read.orNull { placeOrNull(name) } + val named = read.orNull { namedPlaceOrNull() } + val place = named ?: if (name == LIST_LEAKS) Place.Leaks() else null return AgentTarget( windowId = dump?.windowId, heapDumpPath = dump?.heapDumpPath, @@ -614,28 +615,32 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { // Named here, while the dump it is a place of is open, rather than by whoever reads the log // afterwards: a session is read in whichever window happens to be open, and a window that has // another heap dump cannot say what an address in this one stands for. See [agentPlaceTitle]. - about = if (dump == null || place == null) { + // + // And only a place the call itself pointed at, so that the one place derived from the tool rather + // than from an argument goes unnamed: a row already reading "Listed the leaks" would otherwise say + // "Listed the leaks Leaks". + about = if (dump == null || named == null) { null } else { - dump.read("what to call ${placeText(place) ?: place} for the log") { - it.tree.agentPlaceTitle(place) + dump.read("what to call ${placeText(named) ?: named} for the log") { + it.tree.agentPlaceTitle(named) } } ) } /** - * Which place of the heap dump a call is about, from what it was given rather than from which tool it is. + * Which place of the heap dump a call named, from what it was given rather than from which tool it is. * * By argument name, so that a tool added here is described by this without being listed in it: everything * about an object takes `object`, everything about a place takes `place`, and the search takes a class - * name. The one tool whose subject is in neither is the list of leaks, which takes nothing at all. + * name. The one tool whose subject is in none of them is the list of leaks, which takes nothing at all — + * see [target], which is where that one is filled in. */ - private fun AgentArguments.placeOrNull(name: String): Place? = when { + private fun AgentArguments.namedPlaceOrNull(): Place? = when { optionalString(PLACE) != null -> place() optionalString(OBJECT) != null -> Place.Object(objectId(OBJECT)) optionalString(CLASS_NAME) != null -> Place.Objects(ObjectListFilter(query = string(CLASS_NAME))) - name == LIST_LEAKS -> Place.Leaks() else -> null } @@ -723,7 +728,7 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { const val SET_VERDICT = "set_verdict" const val CONCLUDE = "conclude" - /** Named because [placeOrNull] is the one description of a call that has to know which tool it is. */ + /** Named because [target] is the one description of a call that has to know which tool it is. */ const val LIST_LEAKS = "list_leaks" const val WINDOW = "window" diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index 2ef09d078f..c5e75e0609 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -236,6 +236,18 @@ class McpSessionTest { assertThat(call.subject).isEqualTo("MainActivity ${hexObjectId(heapDump.activityObjectId)}") } + @Test + fun `a call that named no place is written down with somewhere to go and nothing to call it`() { + callTool("""{"name":"list_leaks","arguments":{"reason":"Starting with what the dump says."}}""") + + val call = sessions().single().calls.single() + // The leaks screen to go to, and no name for it: the verb already says the whole of what this call did, + // so naming the place as well would read "Listed the leaks Leaks". + assertThat(call.verb).isEqualTo("Listed the leaks") + assertThat(call.place).isEqualTo(Place.Leaks()) + assertThat(call.subject).isNull() + } + @Test fun `an address of no object of the heap dump is written down as the address`() { callTool("""{"name":"describe_object","arguments":{"object":"0xdeadbeef","reason":"Guessing."}}""") From ffeb51bfb6e97c1cb32b31cb805e1f227f62f42a Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 23:10:25 +0200 Subject: [PATCH 18/27] List the agents that worked on this heap dump, and no others A window is a heap dump, so the *Agent logs* screen of one is the agents that read *that* dump: `AgentSession.heapDumpPaths` is what decides, and the sessions about another dump are listed under `Other heap dumps` and opened in a window of theirs. There is no window that isn't a heap dump for them to be read in, and read against the wrong one they are rows of addresses that mean other objects. Which is also why naming an object goes back to the window, undoing the `about` recorded on every call: the reader has the heap dump the session is about, so it can resolve an address itself, and a session written before the recording was added no longer reads differently from one written after. The one row left as an address is a call that went on to another dump, which says which file and opens it. Co-Authored-By: Claude Opus 5 --- .../shark-explorer-agent/AGENTS.md | 17 ++- .../shark/explorer/agent/AgentSessionFile.kt | 37 ++--- .../java/shark/explorer/agent/AgentTools.kt | 57 ++------ .../java/shark/explorer/agent/McpSession.kt | 4 +- .../explorer/agent/AgentSessionFileTest.kt | 20 --- .../shark/explorer/agent/McpSessionTest.kt | 27 +--- .../shark/explorer/app/AgentLogsScreen.kt | 126 +++++++++++++----- .../shark/explorer/app/HeapDumpExplorer.kt | 54 +++++++- .../shark/explorer/app/AgentLogsScreenTest.kt | 65 +++++---- .../java/shark/explorer/eval/EvalScoreTest.kt | 1 - 10 files changed, 223 insertions(+), 185 deletions(-) diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index 8551ed0551..13d42d9013 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -70,13 +70,16 @@ a refusal nobody can follow up on is a dead end on the screen. `target` derives *names* rather than from a second list of tool names — one exception, `list_leaks`, which takes no argument saying where it is. -**And it is named here, not by whoever reads it.** `target` costs one extra read per call, `agentPlaceTitle`, -because what an agent typed is an address and what the screen shows is `MainActivity 0x12d368b8` — and -resolving an address means having *that* heap dump open. A session spans every dump that was open while it -ran, and it is read afterwards in whichever window happens to be open, so a screen that resolved these itself -could only name the calls about its own dump and every other row would stay a bare address. Which is exactly -what it did, until this was recorded. `about` is null for a session written before that, and `subject` falls -back to the address, so an old session still reads. +**A session records addresses, and is read in the window of its heap dump.** What an agent types is +`0x12d368b8` and what the screen shows is `MainActivity 0x12d368b8`, so somebody has to resolve it — and +resolving an address means having *that* dump open. Which the reader does: the *Agent logs* screen of a window +lists the sessions that read the dump it has open, `AgentSession.heapDumpPaths`, and the rest are opened in a +window of theirs. So nothing here writes the name down. Recording it was tried and reverted: it put one extra +heap dump read on every call to answer a question the reader already has the dump for. + +`heapDumpPath` per call rather than per session is what makes that work, and it is not redundant — an agent +can open a second dump, and a call about one this window hasn't got is a row it leaves as the address, saying +which file, and opens that dump when clicked. **One field comes off the answer instead: `outcome`.** What an agent asked is what it typed, and what it concluded is what the heap dump *agreed to* — so `outcomeOfTool` reads the reference out of `conclude`'s diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt index 29e0400797..752246155d 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt @@ -202,9 +202,6 @@ class AgentSessionFile private constructor( reason?.let { put(REASON_KEY, it) } windowId?.let { put(WINDOW_KEY, it) } heapDumpPath?.let { put(HEAP_DUMP_KEY, it) } - // What the object is called, and not only the address the agent wrote: naming it takes the heap dump - // it is in, and this is written while that dump is open. See [AgentSessionCall.about]. - about?.let { put(ABOUT_KEY, it) } // As the link the window hands out for that place, which is the whole of what a row has to be // clickable: the place to go to, and a line the agent's human can paste anywhere. See [DeepLink]. link()?.let { put(LINK_KEY, it) } @@ -236,7 +233,6 @@ class AgentSessionFile private constructor( windowId = text(WINDOW_KEY), heapDumpPath = text(HEAP_DUMP_KEY), place = link?.let { placeOfLinkOrNull(it, file, lineNumber) }, - about = text(ABOUT_KEY), arguments = this[ARGUMENTS_KEY]?.asStringMap().orEmpty(), refusal = text(REFUSAL_KEY), outcome = text(OUTCOME_KEY), @@ -323,7 +319,6 @@ class AgentSessionFile private constructor( private const val REASON_KEY = "reason" private const val WINDOW_KEY = "window" private const val HEAP_DUMP_KEY = "heapDump" - private const val ABOUT_KEY = "about" private const val LINK_KEY = "link" private const val REFUSAL_KEY = "refused" private const val OUTCOME_KEY = "outcome" @@ -347,6 +342,15 @@ class AgentSession( /** How many of the calls were refused, which is the one number a list of sessions is worth showing. */ val refusedCount: Int get() = calls.count { it.refusal != null } + + /** + * Which heap dumps it read, in the order it first read each of them. + * + * Usually one, and a session is not *bound* to one: an agent can open a second dump, and comparing two is + * a thing to do. Which is what the window listing these needs — a window is one heap dump, so the sessions + * it shows are the ones that read the dump it has open, and the rest are read in the window of theirs. + */ + val heapDumpPaths: List get() = calls.mapNotNull { it.heapDumpPath }.distinct() } /** @@ -365,16 +369,6 @@ class AgentSessionCall( val windowId: String?, val heapDumpPath: String?, val place: Place?, - /** - * What [place] is called — `MainActivity 0x12d368b8` — as the window naming a tab on it would. - * - * Recorded rather than worked out on the way in, because working it out is a read of *that* heap dump: a - * session spans the dumps that were open while it ran, and it is read afterwards in whichever window - * happens to be open. So a screen that resolved these itself could only name the calls about its own dump, - * and every other row would stay the bare address an agent wrote — which is the one thing this screen - * exists to not show. Null for a call about no place, and for a session written before this was recorded. - */ - val about: String?, /** The rest of the arguments, by name, with `reason` and `window` left out: they have fields of their own. */ val arguments: Map, /** Why the call was refused, and null for one that was answered. See [AgentRefusal]. */ @@ -410,21 +404,12 @@ class AgentSessionCall( val AgentSessionCall.verb: String get() = verbOfTool(tool, arguments) ?: tool.replace('_', ' ') /** - * What the call was about, in the words the window uses for it: an object with its class name, a class name, - * a place. - * - * Which is [AgentSessionCall.about] wherever there is one, so that a row and the tab clicking it opens are - * recognisably one object. What the agent typed is the fallback, and it is what a call about a place of no - * heap dump reads as — an address on its own, which is the whole of what a session held before the app - * started writing the name beside it. + * What the call was about, in the words the window uses for it: an address, a class name, a place. * * Null for a call whose subject is the whole heap dump or the app itself, where the verb says all of it. */ val AgentSessionCall.subject: String? - get() = about - ?: arguments[SUBJECT_OBJECT] - ?: arguments[SUBJECT_PLACE] - ?: arguments[SUBJECT_CLASS_NAME] + get() = arguments[SUBJECT_OBJECT] ?: arguments[SUBJECT_PLACE] ?: arguments[SUBJECT_CLASS_NAME] /** * What the answer to a call came to, as a couple of words, and null when the answer is data rather than a diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index 151b9e07b2..f9c2bf51f5 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -23,7 +23,6 @@ import shark.explorer.leakLabel import shark.explorer.leakStatusConflictsWith import shark.explorer.nodeIdText import shark.explorer.outlineOf -import shark.explorer.titleOf /** * Everything an agent can do to an open heap dump, as MCP tools. @@ -593,54 +592,39 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { } /** - * Which window, which place and what to call it, for the log of the session the call was made in. + * Which window and which place a call was about, for the log of the session it was made in. * * Read off the arguments rather than out of the handler, so that a call that was refused is recorded * pointing at whatever it was asking about — which is most of what makes a refusal worth reading * afterwards. Nothing here refuses: this is a description of a call, and a call with an argument this * can't make sense of is one the handler is about to refuse with a message of its own. */ - suspend fun target( + fun target( name: String, arguments: JsonObject ): AgentTarget { val read = AgentArguments(name, arguments) val dump = read.orNull { resolvedDump(optionalString(WINDOW)) } - val named = read.orNull { namedPlaceOrNull() } - val place = named ?: if (name == LIST_LEAKS) Place.Leaks() else null + val place = read.orNull { placeOrNull(name) } return AgentTarget( windowId = dump?.windowId, heapDumpPath = dump?.heapDumpPath, - place = place, - // Named here, while the dump it is a place of is open, rather than by whoever reads the log - // afterwards: a session is read in whichever window happens to be open, and a window that has - // another heap dump cannot say what an address in this one stands for. See [agentPlaceTitle]. - // - // And only a place the call itself pointed at, so that the one place derived from the tool rather - // than from an argument goes unnamed: a row already reading "Listed the leaks" would otherwise say - // "Listed the leaks Leaks". - about = if (dump == null || named == null) { - null - } else { - dump.read("what to call ${placeText(named) ?: named} for the log") { - it.tree.agentPlaceTitle(named) - } - } + place = place ) } /** - * Which place of the heap dump a call named, from what it was given rather than from which tool it is. + * Which place of the heap dump a call is about, from what it was given rather than from which tool it is. * * By argument name, so that a tool added here is described by this without being listed in it: everything * about an object takes `object`, everything about a place takes `place`, and the search takes a class - * name. The one tool whose subject is in none of them is the list of leaks, which takes nothing at all — - * see [target], which is where that one is filled in. + * name. The one tool whose subject is in neither is the list of leaks, which takes nothing at all. */ - private fun AgentArguments.namedPlaceOrNull(): Place? = when { + private fun AgentArguments.placeOrNull(name: String): Place? = when { optionalString(PLACE) != null -> place() optionalString(OBJECT) != null -> Place.Object(objectId(OBJECT)) optionalString(CLASS_NAME) != null -> Place.Objects(ObjectListFilter(query = string(CLASS_NAME))) + name == LIST_LEAKS -> Place.Leaks() else -> null } @@ -728,7 +712,7 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { const val SET_VERDICT = "set_verdict" const val CONCLUDE = "conclude" - /** Named because [target] is the one description of a call that has to know which tool it is. */ + /** Named because [placeOrNull] is the one description of a call that has to know which tool it is. */ const val LIST_LEAKS = "list_leaks" const val WINDOW = "window" @@ -779,7 +763,7 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { } /** - * What a call was about: which window, which heap dump, which place of it, and what that place is called. + * What a call was about: which window, which heap dump, and which place of it. * * Only for the session log, which is the one reader that needs this without needing the answer: a row of the * *Agent logs* screen is a verb, a subject and somewhere to go when it is clicked. See [AgentSessionCall]. @@ -787,28 +771,9 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { internal class AgentTarget( val windowId: String?, val heapDumpPath: String?, - val place: Place?, - /** What the window calls [place] — `MainActivity 0x12d368b8` — read while the dump was open. */ - val about: String? + val place: Place? ) -/** - * What the window calls a place an agent asked about: the title a tab on it would have. - * - * The same [titleOf] the tabs are named by, so that a row of a session and the tab clicking it opens read - * the same — an agent and the person watching it are looking at one object, and two spellings of it would - * be two objects to them. - * - * With the one difference that makes this a function of its own: an agent can name an address the heap dump - * has no object at, which is a call it was refused and still a row worth reading. [titleOf] would throw on - * it, so the address is asked about first and stands for itself when it is nothing here. - */ -private fun HeapDominatorTreemap.agentPlaceTitle(place: Place): String = when (place) { - is Place.Object -> - if (objectNameOrNull(place.objectId) == null) exactHexObjectId(place.objectId) else titleOf(place) - else -> titleOf(place) -} - /** * What the verdicts on a chain add up to: whether one reference is at fault, and what to say when none is. * diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt index 45f7d8ad71..7fa5f7920e 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt @@ -151,8 +151,7 @@ internal class McpSession( // trying to learn and then what that cost. See [AgentTools]. SharkLog.d { "An agent called $name${arguments.logLine()}" } // What the call is about, read before it is made rather than after: a refused call is recorded pointing - // at whatever it was asking about, which is most of what makes a refusal worth reading afterwards. And - // named here, while that heap dump is open, because nothing reading the session later can. + // at whatever it was asking about, which is most of what makes a refusal worth reading afterwards. val target = tools.target(name, arguments) val at = Instant.now() val startedAt = System.nanoTime() @@ -211,7 +210,6 @@ internal class McpSession( windowId = target.windowId, heapDumpPath = target.heapDumpPath, place = target.place, - about = target.about, arguments = arguments.recorded(), refusal = refusal, outcome = outcome, diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt index 40708ce258..5070eeb365 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt @@ -38,7 +38,6 @@ class AgentSessionFileTest { tool = "describe_object", reason = "Reading the holder's fields.", place = Place.Object(OBJECT_ID), - about = "Holder 0x12d368b8", arguments = mapOf("object" to "0x12d368b8") ) ) @@ -56,23 +55,6 @@ class AgentSessionFileTest { assertThat(call.heapDumpPath).isEqualTo("/dumps/leak.hprof") assertThat(call.arguments).containsEntry("object", "0x12d368b8") assertThat(call.millis).isEqualTo(12L) - // What the object is called, which the window that answered the agent wrote down: the screen reading - // this is in whichever window is open, and naming an address means having that heap dump. - assertThat(call.about).isEqualTo("Holder 0x12d368b8") - assertThat(call.subject).isEqualTo("Holder 0x12d368b8") - } - - @Test - fun `a call with no name recorded is read as the address the agent wrote`() { - val file = AgentSessionFile.starting(directory, SERVER_VERSION) - file.called( - call(tool = "describe_object", place = Place.Object(OBJECT_ID), arguments = mapOf("object" to "0x12d368b8")) - ) - - // Which is every session written before the name was recorded beside the address, and the reason the - // screen asks for a subject rather than for a name: an old session still has rows worth reading. - assertThat(AgentSessionFile.sessionsIn(directory).single().calls.single().subject) - .isEqualTo("0x12d368b8") } @Test @@ -176,7 +158,6 @@ class AgentSessionFileTest { tool: String, reason: String? = "Because.", place: Place? = null, - about: String? = null, arguments: Map = emptyMap(), refusal: String? = null, outcome: String? = null @@ -187,7 +168,6 @@ class AgentSessionFileTest { windowId = WINDOW_ID, heapDumpPath = "/dumps/leak.hprof", place = place, - about = about, arguments = arguments, refusal = refusal, outcome = outcome, diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index c5e75e0609..845a6e4c4a 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -15,7 +15,6 @@ import org.junit.Test import org.junit.rules.TemporaryFolder import shark.explorer.Place import shark.explorer.exactHexObjectId -import shark.explorer.hexObjectId /** * What a client of this server gets back, as JSON-RPC rather than as Kotlin. @@ -208,9 +207,7 @@ class McpSessionTest { assertThat(session.serverVersion).isEqualTo(SERVER_VERSION) val call = session.calls.single() assertThat(call.verb).isEqualTo("Described") - // The object as the window names a tab on it, not the address the agent typed: the screen reading this - // is in whichever window is open later, and naming an address takes the heap dump it is in. - assertThat(call.subject).isEqualTo("Holder ${hexObjectId(heapDump.holderObjectId)}") + assertThat(call.subject).isEqualTo(hex(heapDump.holderObjectId)) assertThat(call.reason).isEqualTo("Checking whether the holder is the singleton it looks like.") assertThat(call.refusal).isNull() // Which is what makes the row clickable: the place, in the window the call was made against. @@ -230,35 +227,23 @@ class McpSessionTest { assertThat(call.verb).isEqualTo("Concluded about") assertThat(call.refusal).contains("Not concluded") assertThat(call.reason).isEqualTo("I know what this is.") - // Refused, and still pointing at the object it was refused about, named: a refusal nobody can follow up - // on is the half of a session that is worth reading afterwards. + // Refused, and still pointing at the object it was refused about: a refusal nobody can follow up on is + // the half of a session that is worth reading afterwards. assertThat(call.place).isEqualTo(Place.Object(heapDump.activityObjectId)) - assertThat(call.subject).isEqualTo("MainActivity ${hexObjectId(heapDump.activityObjectId)}") } @Test - fun `a call that named no place is written down with somewhere to go and nothing to call it`() { + fun `a call that named no place is written down with somewhere to go all the same`() { callTool("""{"name":"list_leaks","arguments":{"reason":"Starting with what the dump says."}}""") + // The leaks screen to go to, and nothing after the verb, which already says the whole of what this call + // did. The one place a call is about without naming it in an argument. See [AgentTools.target]. val call = sessions().single().calls.single() - // The leaks screen to go to, and no name for it: the verb already says the whole of what this call did, - // so naming the place as well would read "Listed the leaks Leaks". assertThat(call.verb).isEqualTo("Listed the leaks") assertThat(call.place).isEqualTo(Place.Leaks()) assertThat(call.subject).isNull() } - @Test - fun `an address of no object of the heap dump is written down as the address`() { - callTool("""{"name":"describe_object","arguments":{"object":"0xdeadbeef","reason":"Guessing."}}""") - - // A refusal, and a row of it still says what was asked about. There is nothing to name it after, so it - // stands for itself rather than making the call unrecordable. - val call = sessions().single().calls.single() - assertThat(call.refusal).contains("0xdeadbeef") - assertThat(call.subject).isEqualTo("0xdeadbeef") - } - @Test fun `a call that concluded is written down with the reference it concluded on`() { callTool( diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt index 45e708e390..5d7fbf18b1 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt @@ -27,34 +27,52 @@ import shark.explorer.agent.subject import shark.explorer.agent.verb /** - * Every agent that has worked on a heap dump through this app, one row each. + * Every agent that has worked on **this** heap dump, one row each, and a way to the ones that worked on + * another. * * Because an agent works in this window: it reads the dump the person at the machine is reading, sets the * verdicts they see and writes into the same notes. So what it did has to be here, in words, rather than in * a JSON stream a client happens to have kept — and a row of it has to lead where it went, which is what * makes the two of them one investigation instead of two. * - * Not per heap dump, unlike the notes and the verdicts: a session is one agent's connection to this app and - * can read whichever dumps were open. Whether a row is about *this* window's dump is what decides whether - * clicking it goes anywhere. See [AgentLogScreen]. + * Per heap dump, like the notes and the verdicts, because a window is a heap dump: a session listed in the + * wrong window is one whose addresses mean nothing here. The sessions that read other dumps are still worth + * reaching from here — an agent is usually handed a dump nobody has open yet — and each of those is opened + * in a window of *its* dump rather than read in this one. There is no window that is not a heap dump for + * them to be listed in on their own. */ @Composable internal fun AgentLogsScreen( sessions: List, + /** Which heap dump this window has open, which is what decides which sessions are this window's. */ + heapDumpFile: File, onOpen: (Place, OpenIn) -> Unit, onCopyLink: (Place) -> Unit, + /** + * Where a session about another heap dump goes: that dump, in the window that has it or one of its own. + * + * Nothing by default, because routing this is a question about every window of the run and a screen + * composed without an answer must not silently look like a screen whose rows lead somewhere. + */ + onOpenHeapDump: (File, Place) -> Unit = { file, place -> + SharkLog.d { "Nothing here to open $place of $file with" } + }, modifier: Modifier = Modifier ) { + val here = sessions.filter { heapDumpFile.absolutePath in it.heapDumpPaths } + // Which leaves a session that read no heap dump at all — a client that connected and asked nothing — with + // the ones about other dumps, since it is not about this one either. + val elsewhere = sessions - here.toSet() Surface(modifier, color = MaterialTheme.colorScheme.surface) { Column( Modifier.verticalScroll(rememberScrollState()).padding(16.dp), verticalArrangement = Arrangement.spacedBy(8.dp) ) { Text(Place.AGENT_LOGS_LABEL, style = MaterialTheme.typography.titleMedium) - if (sessions.isEmpty()) { + if (here.isEmpty()) { Text(NO_SESSIONS, style = MaterialTheme.typography.bodyMedium) } - sessions.forEach { session -> + here.forEach { session -> val place = Place.AgentLog(session.sessionId) val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } OpenTarget(open, { onCopyLink(place) }) { @@ -64,39 +82,75 @@ internal fun AgentLogsScreen( } } } + if (elsewhere.isNotEmpty()) { + HorizontalDivider() + Text(OTHER_HEAP_DUMPS, style = MaterialTheme.typography.titleMedium) + elsewhere.forEach { session -> OtherHeapDumpSessionRow(session, onOpenHeapDump) } + } } } } +/** + * One agent that worked on another heap dump: what it did, and that dump to open it in. + * + * Not opened here. An address is an address of one heap dump, so a session read against the wrong one is a + * screen of rows that name other objects than the ones the agent saw — which is the whole reason this list is + * per dump. A session that read no dump at all, or one whose dump has been deleted, has nowhere to be opened + * and says which file it wanted. + */ +@Composable +private fun OtherHeapDumpSessionRow( + session: AgentSession, + onOpenHeapDump: (File, Place) -> Unit +) { + val opens = session.heapDumpPaths.firstOrNull()?.let { File(it) }?.takeIf { it.isFile } + val title = session.title() + val summary = session.summary() + if (opens == null) { + Column { + Text(title, style = MaterialTheme.typography.bodyMedium) + Text(summary, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) + } + return + } + // No tab to choose and no link to copy: what a link names is a window, and the window this session was + // read in belongs to a run that has usually ended. The heap dump is what outlived it. + val open = { onOpenHeapDump(opens, Place.AgentLog(session.sessionId)) } + Column(Modifier.openable { open() }) { + Text(title, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) + Text(summary, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) + } +} + /** * What one agent did, call by call, in the order it made them. * * **Verbs and object names rather than the protocol.** What is worth reading here is whether the steps follow * from each other, and that is a question about what was asked and why — a screen of JSON is the same * information in the one form nobody reads. So a row is what the call did, what it was about, and the - * sentence the agent gave for making it, which is its own words and not a paraphrase. An agent names objects - * by address, and a row names them the way the rest of the window does, so that a row and the tab it opens - * are recognisably the same object. Which is [AgentSessionCall.subject], written down as the call was made: - * naming an address means reading the dump it is in, and this screen is read from whichever window is open. + * sentence the agent gave for making it, which is its own words and not a paraphrase. * - * **And every row that names a place leads to it.** One about the heap dump this window has open goes there - * the way every other way to an object here does; one about another dump opens that dump. A session is one - * agent's connection and can read as many dumps as were open, so a row leading nowhere would be the app - * showing somebody what an agent looked at and then declining to show them the thing. + * **An agent names objects by address, and a row names them the way the rest of the window does**, so that a + * row and the tab it opens are recognisably the same object. Which is a read of the heap dump this window has + * open — the same read that names a tab — and it is why this screen is reached from the sessions about *this* + * dump: a window can only speak for the dump it has. See [AgentLogsScreen] and [placeTitles]. + * + * **And every row that names a place leads to it.** The exception is the call of a session that went on to + * another heap dump, which names that dump and opens it: a session is one agent's connection and can read as + * many dumps as were open, so a row leading nowhere would be the app showing somebody what an agent looked at + * and then declining to show them the thing. */ @Composable internal fun AgentLogScreen( session: AgentSession?, /** Which heap dump this window has open, which is what decides whether a row moves this window. */ heapDumpFile: File, + /** What this window calls the places the agent asked about, for the calls about its own heap dump. */ + placeTitles: Map, onOpen: (Place, OpenIn) -> Unit, onCopyLink: (Place) -> Unit, - /** - * Where a row about another heap dump goes: that dump, in the window that has it or one of its own. - * - * Nothing by default, because routing this is a question about every window of the run and a screen - * composed without an answer must not silently look like a screen whose rows lead somewhere. - */ + /** Where a row about another heap dump goes. See [AgentLogsScreen]. */ onOpenHeapDump: (File, Place) -> Unit = { file, place -> SharkLog.d { "Nothing here to open $place of $file with" } }, @@ -127,6 +181,7 @@ internal fun AgentLogScreen( AgentCallRow( call = call, heapDumpFile = heapDumpFile, + placeTitles = placeTitles, onOpen = onOpen, onCopyLink = onCopyLink, onOpenHeapDump = onOpenHeapDump @@ -141,19 +196,23 @@ internal fun AgentLogScreen( private fun AgentCallRow( call: AgentSessionCall, heapDumpFile: File, + placeTitles: Map, onOpen: (Place, OpenIn) -> Unit, onCopyLink: (Place) -> Unit, onOpenHeapDump: (File, Place) -> Unit ) { val place = call.place // Which heap dump the row is about when it isn't this window's, and null when it is. An address is an - // address of one dump, so the same number in another one is another object: this window cannot go there, - // and the dump that can has to be opened first. + // address of one dump, so the same number in another one is another object: this window cannot name it or + // go there, and the dump that can has to be opened first. val elsewhere = call.otherHeapDumpOrNull(heapDumpFile) // And whether that is still possible. A session outlives the heap dumps it was about, so a row naming one // that has been deleted says which and leads nowhere. val opens = elsewhere?.takeIf { it.isFile } - val line = call.line(elsewhere) + // Named for a call about this window's own heap dump, and not for one about another: this window has never + // read that file, so what a number in it stands for is not a question it can answer. + val named = if (elsewhere == null) place?.let { placeTitles[it] } else null + val line = call.line(named, elsewhere) Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) { Text( call.at.clockTime(), @@ -167,8 +226,6 @@ private fun AgentCallRow( place == null -> Text(line, style = MaterialTheme.typography.bodyMedium) opens != null -> Text( line, - // No tab to choose and no link to copy: what a link names is a window, and the window this call - // was made against belongs to a run that has usually ended. The heap dump is what outlived it. Modifier.openable { onOpenHeapDump(opens, place) }, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR @@ -221,11 +278,15 @@ private fun AgentSessionCall.otherHeapDumpOrNull(heapDumpFile: File): File? = he * — since the row that says what was concluded is the row anybody scrolling a session is looking for. * * And with [otherHeapDump] named at the end of a row about a dump this window hasn't got open, because - * clicking that row opens a heap dump: which one is a thing to know before rather than after. + * clicking that row opens a heap dump: which one is a thing to know before rather than after. Those are the + * rows with no [named] to show, where the address the agent wrote stands for itself. */ -private fun AgentSessionCall.line(otherHeapDump: File?): String = listOfNotNull( +private fun AgentSessionCall.line( + named: String?, + otherHeapDump: File? +): String = listOfNotNull( verb, - subject, + named ?: subject, outcome?.let { "$LEADS_TO $it" }, otherHeapDump?.let { "$IN ${it.name}" } ).joinToString(" ") @@ -244,7 +305,7 @@ private fun AgentSession.title(): String = listOfNotNull( * which is either an agent that was made to go back and look, or a refusal message that isn't landing. */ private fun AgentSession.summary(): String { - val dumps = calls.mapNotNull { it.heapDumpPath }.distinct().map { File(it).name } + val dumps = heapDumpPaths.map { File(it).name } return listOfNotNull( "${calls.size} call(s)", "$refusedCount refused".takeIf { refusedCount > 0 }, @@ -274,9 +335,12 @@ private const val REFUSED = "Refused:" private const val A_CLIENT_THAT_DID_NOT_SAY = "An agent" +/** The sessions that read another dump, which open in a window of that dump. See [AgentLogsScreen]. */ +private const val OTHER_HEAP_DUMPS = "Other heap dumps" + private const val NO_SESSIONS = - "No agent has connected to this app yet. Hand a heap dump to one by pointing its MCP client at Shark " + - "Explorer, and everything it does lands here." + "No agent has worked on this heap dump. Hand it to one by pointing its MCP client at Shark Explorer, and " + + "everything it does lands here." private const val NOTHING_ASKED = "This agent connected and asked nothing before it went away." diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt index f47edb25f4..d55b5fd213 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt @@ -79,6 +79,7 @@ import shark.explorer.TreemapPresentation import shark.explorer.TreemapRect import shark.explorer.agent.AgentSession import shark.explorer.detours +import shark.explorer.exactHexObjectId import shark.explorer.formatObjectCount import shark.explorer.hexObjectId import shark.explorer.leakStatusConflictsWith @@ -201,6 +202,12 @@ internal fun HeapDumpExplorer( var sessions by remember { mutableStateOf(emptyList()) } /** What each tab is called, by the place it is on. Only grows: a place is named once and stays named. */ var placeTitles by remember { mutableStateOf(emptyMap()) } + /** + * And what to call the places an agent asked about, which is the same question with one difference: an + * agent can name an address this heap dump has no object at, so this map answers for a place a tab could + * not be opened on. See [agentPlaceTitle]. + */ + var agentPlaceTitles by remember { mutableStateOf(emptyMap()) } /** * The note about the tab on screen, and null once the last tab has been closed — which is the one state * with no tab to write about. @@ -528,6 +535,28 @@ internal fun HeapDumpExplorer( } } + // And what to call the objects those agents asked about, so that a row of a session names an object the + // way the tab it opens does — `MainActivity 0x12d368b8` — rather than as the bare address the agent wrote. + // The session file holds addresses on purpose: an address is what an agent said, and what it stands for is + // a read of the heap dump this window has open, which is the same read that names a tab. Which is why the + // sessions listed here are the ones that read this dump — the calls about another are left as written. + val unnamedAgentPlaces = (place as? Place.AgentLog) + ?.let { open -> sessions.firstOrNull { it.sessionId == open.sessionId } } + ?.calls.orEmpty() + .filter { it.heapDumpPath == null || it.heapDumpPath == session.heapDumpFile.absolutePath } + .mapNotNull { it.place } + .filter { it !in agentPlaceTitles } + .distinct() + LaunchedEffect(session, unnamedAgentPlaces) { + if (unnamedAgentPlaces.isEmpty()) { + return@LaunchedEffect + } + val named = session.read("what to call ${unnamedAgentPlaces.size} places an agent asked about") { explorer -> + unnamedAgentPlaces.associateWith { explorer.tree.agentPlaceTitle(it) } + } + agentPlaceTitles = agentPlaceTitles + named + } + // And what has been decided about this heap dump's objects by hand, also once per run: one small file, // read before anything is drawn from it, because a chain read without it would be the heap dump's own // answer where someone has already recorded another. See [HeapDumpLeakStatuses]. @@ -739,6 +768,7 @@ internal fun HeapDumpExplorer( favourites = favourites, sessions = sessions, heapDumpFile = session.heapDumpFile, + agentPlaceTitles = agentPlaceTitles, onOpenHeapDump = onOpenHeapDump, sizes = sizes, onOpen = openObject, @@ -1081,7 +1111,9 @@ private fun ListPlace( sessions: List, /** Which heap dump this window has open, which is what decides where an agent's row leads. */ heapDumpFile: File, - /** And where one about another heap dump leads: that dump. See [AgentLogScreen]. */ + /** What this window calls the places those agents asked about. See [agentPlaceTitle]. */ + agentPlaceTitles: Map, + /** And where a session or a row about another heap dump leads: that dump. See [AgentLogsScreen]. */ onOpenHeapDump: (File, Place) -> Unit, sizes: HeapSizes, onOpen: (Long, OpenIn) -> Unit, @@ -1133,14 +1165,17 @@ private fun ListPlace( ) is Place.AgentLogs -> AgentLogsScreen( sessions = sessions, + heapDumpFile = heapDumpFile, onOpen = onOpenPlace, onCopyLink = onCopyPlaceLink, + onOpenHeapDump = onOpenHeapDump, modifier = modifier ) is Place.AgentLog -> AgentLogScreen( // Null for a session that has been pushed out by newer ones, or one from another machine's link. session = sessions.firstOrNull { it.sessionId == place.sessionId }, heapDumpFile = heapDumpFile, + placeTitles = agentPlaceTitles, onOpen = onOpenPlace, onCopyLink = onCopyPlaceLink, onOpenHeapDump = onOpenHeapDump, @@ -1622,6 +1657,23 @@ private suspend fun HeapDumpSession.describing( ) } +/** + * What this window calls a place an agent asked about: the title a tab on it would have. + * + * The same [titleOf] the tabs are named by, so that a row of a session and the tab clicking it opens read the + * same — an agent and the person watching it are looking at one object, and two spellings of it would be two + * objects to them. + * + * With the one difference that makes this a function of its own: an agent can name an address this heap dump + * has no object at, which is a call it was refused and still a row worth reading. [titleOf] would throw on + * it, so the address is asked about first and stands for itself when it is nothing here. + */ +private fun HeapDominatorTreemap.agentPlaceTitle(place: Place): String = when (place) { + is Place.Object -> + if (objectNameOrNull(place.objectId) == null) exactHexObjectId(place.objectId) else titleOf(place) + else -> titleOf(place) +} + /** What the panes are being filled in for, for the log. See [HeapDumpSession.read]. */ private fun Place.description(): String = when (this) { is Place.Object -> "what ${nodeIdText(objectId)} is" diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt index dddc3dbe40..aa548621d6 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt @@ -33,8 +33,9 @@ import shark.explorer.hexObjectId * * An investigation an agent ran and one a person ran are the same investigation: it reads this heap dump, * sets the verdicts they see and writes the same notes. So what it did is read here in words, and **a row - * leads where the call went** — which is what these tests are about, including the row about a heap dump - * this window hasn't got open, since a session can span every dump that was open while it ran. + * leads where the call went** — which is what these tests are about, along with the boundary of that: a + * window is one heap dump, so the agents listed in it are the ones that read that dump, and the rest are + * reached by opening theirs. */ @OptIn(ExperimentalTestApi::class) class AgentLogsScreenTest { @@ -107,7 +108,7 @@ class AgentLogsScreenTest { } } - @Test fun `a call about another heap dump names its object, and leads to that dump`() { + @Test fun `an agent that worked on another heap dump is listed apart, and opens in a window of that dump`() { val otherHeapDump = testFolder.newFile("another.hprof") var opened: Pair? = null explorerUiTest { @@ -115,41 +116,50 @@ class AgentLogsScreenTest { sessions = listOf(session(calls = listOf(call(heapDumpPath = otherHeapDump.absolutePath)))), onOpenHeapDump = { file, place -> opened = file to place } ) - onNodeWithText(CLIENT, substring = true).performClick() - waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) - // Named the same way as a row about this window's own dump, because the name was written down when - // the call was made — this window has never read that file and could not work it out. - val row = "Described ${activityName()} in ${otherHeapDump.name}" - waitUntilAtLeastOneExists(hasText(row), OPEN_TIMEOUT_MILLIS) - onNodeWithText(row).performClick() + // Not among this window's agents, because a window is a heap dump and this one read another: its + // addresses are addresses of that file. Still reachable, since a dump handed to an agent is usually + // one nobody has open. + onNodeWithText(NO_AGENT_YET, substring = true).assertIsDisplayed() + onNodeWithText(OTHER_HEAP_DUMPS).assertIsDisplayed() + onNodeWithText(CLIENT, substring = true).performClick() } - // Not this window: an address is an address of one heap dump, so going there means opening that dump. - assertThat(opened).isEqualTo(otherHeapDump to Place.Object(activityObjectId())) + // Its own log, in a window of its own heap dump, rather than read here against the wrong one. + assertThat(opened).isEqualTo(otherHeapDump to Place.AgentLog(SESSION_ID)) } - @Test fun `a call about a heap dump that has gone leads nowhere`() { + @Test fun `a call that went on to another heap dump reads as the address, and leads to that dump`() { + val otherHeapDump = testFolder.newFile("another.hprof") + var opened: Pair? = null explorerUiTest { - openAgentLogs(listOf(session(calls = listOf(call(heapDumpPath = "/dumps/deleted.hprof"))))) + openAgentLogs( + sessions = listOf( + session(calls = listOf(call(), call(heapDumpPath = otherHeapDump.absolutePath))) + ), + onOpenHeapDump = { file, place -> opened = file to place } + ) onNodeWithText(CLIENT, substring = true).performClick() - waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) + waitUntilAtLeastOneExists(hasText(describedRow()), OPEN_TIMEOUT_MILLIS) - // Still worth reading, and there is nothing to open: a session outlives the heap dumps it was about. - onNodeWithText("Described ${activityName()} in deleted.hprof").assertHasNoClickAction() + // The address as the agent wrote it, and the file it means something in: this window has never read + // that dump, so what the number stands for there is not a question it can answer. + val row = "Described ${hex(activityObjectId())} in ${otherHeapDump.name}" + onNodeWithText(row).performClick() } + + // Going there means opening that dump, where the same address is that dump's object. + assertThat(opened).isEqualTo(otherHeapDump to Place.Object(activityObjectId())) } - @Test fun `a session written before the app recorded names reads as the address the agent wrote`() { + @Test fun `a call about a heap dump that has gone leads nowhere`() { explorerUiTest { - openAgentLogs(listOf(session(calls = listOf(call(about = null))))) + openAgentLogs(listOf(session(calls = listOf(call(), call(heapDumpPath = "/dumps/deleted.hprof"))))) onNodeWithText(CLIENT, substring = true).performClick() - waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) + waitUntilAtLeastOneExists(hasText(describedRow()), OPEN_TIMEOUT_MILLIS) - // Which is what every session on disk holds from before there was a name beside the address, and a - // row of one still leads to the object: this window has that heap dump open. - onNodeWithText("Described ${hex(activityObjectId())}").performClick() - waitUntilAtLeastOneExists(hasText("mDestroyed", substring = true), OPEN_TIMEOUT_MILLIS) + // Still worth reading, and there is nothing to open: a session outlives the heap dumps it was about. + onNodeWithText("Described ${hex(activityObjectId())} in deleted.hprof").assertHasNoClickAction() } } @@ -198,9 +208,6 @@ class AgentLogsScreenTest { private fun call( tool: String = "describe_object", heapDumpPath: String = heapDump.file.absolutePath, - // What the app wrote down for the object as the call was made, which is what a row reads as. Null for a - // session from a build that recorded only the address. - about: String? = activityName(), refusal: String? = null, outcome: String? = null ) = AgentSessionCall( @@ -210,7 +217,6 @@ class AgentLogsScreenTest { windowId = "zvphq4r3", heapDumpPath = heapDumpPath, place = Place.Object(activityObjectId()), - about = about, arguments = mapOf("object" to hex(activityObjectId())), refusal = refusal, outcome = outcome, @@ -243,7 +249,8 @@ class AgentLogsScreenTest { const val REASON = "Checking whether this activity is really destroyed." const val REFUSAL = "3 step(s) have no verdict" const val FAULTY_REFERENCE = "Holder.activity" - const val NO_AGENT_YET = "No agent has connected" + const val NO_AGENT_YET = "No agent has worked on this heap dump" + const val OTHER_HEAP_DUMPS = "Other heap dumps" val STARTED_AT: Instant = Instant.parse("2026-08-25T18:19:48.035Z") diff --git a/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt b/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt index f61024de10..834a29e601 100644 --- a/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt +++ b/shark/shark-explorer/shark-explorer-eval/src/test/java/shark/explorer/eval/EvalScoreTest.kt @@ -137,7 +137,6 @@ class EvalScoreTest { windowId = null, heapDumpPath = heapDumpPath, place = null, - about = null, arguments = emptyMap(), refusal = refusal, outcome = outcome, From e4504137a9044638e9360c8d51b547d81db44155 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Tue, 25 Aug 2026 23:40:24 +0200 Subject: [PATCH 19/27] Call a tool from a shell, at the window that is already open MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `--agent name=value …` makes one call and prints the answer, and `--agent-help` says what the calls are with no window, no heap dump and no Gradle. For an agent whose client speaks no MCP, and for one that has a shell and hasn't been configured with anything. It is argument translation and nothing else: it builds a `tools/call` on the loopback socket the run already publishes, so a refusal it prints was thrown by the handler that would have refused an MCP client, and the help is generated from the same registry `tools/list` answers from. Which also means a call from a shell is not a slower call — measured at 160 to 180 ms, a JVM starting and a socket, against a heap dump that was parsed and indexed once in the window somebody is watching. What a process per call would otherwise lose is the session: a connection is what gathers an MCP investigation, and a command per question would be a row per call on the *Agent logs* screen. So the handshake carries an optional session name after the token, `AgentSessionFile.continuing` appends to the file that already has it, and a shell's calls default to `cli`. Checked at both ends, because it becomes part of a file name — refused before anything is called at this end, served with a session of its own and a line in the log at the other. Exit codes carry the rest: 0 with JSON on stdout, 2 with the refusal on stderr, 1 when there was nothing to answer it. --- shark/shark-explorer/AGENTS.md | 2 +- shark/shark-explorer/notes/agent-surface.md | 59 ++- .../shark-explorer-agent/AGENTS.md | 29 ++ .../shark/explorer/agent/AgentCommandLine.kt | 488 ++++++++++++++++++ .../java/shark/explorer/agent/AgentServer.kt | 42 +- .../shark/explorer/agent/AgentSessionFile.kt | 65 ++- .../shark/explorer/agent/AgentStdioBridge.kt | 183 ++++--- .../java/shark/explorer/agent/McpSession.kt | 11 +- .../explorer/agent/AgentCommandLineTest.kt | 235 +++++++++ .../shark/explorer/agent/AgentServerTest.kt | 51 +- .../java/shark/explorer/app/ExplorerAgents.kt | 85 ++- .../src/main/java/shark/explorer/app/Main.kt | 3 + .../explorer/app/AgentCommandLineTest.kt | 46 -- .../shark/explorer/app/AgentOptionsTest.kt | 118 +++++ 14 files changed, 1245 insertions(+), 172 deletions(-) create mode 100644 shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt create mode 100644 shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentCommandLineTest.kt delete mode 100644 shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentCommandLineTest.kt create mode 100644 shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentOptionsTest.kt diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index 2f058e5314..3b91c59a56 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -13,7 +13,7 @@ reading the source alone — everything else is in the code. Keep it that way. | --- | --- | --- | | `shark-explorer-core` | Heap dump → dominator tree → layout model. Layout, hit testing, navigation state. | **No Compose dependency, Java 8 target.** Must stay reusable from the Android `leakcanary-app`. | | `shark-explorer-jdwp` | Attaches to a live app as a debugger to read the pixels of its bitmaps. | **Imports `com.sun.jdi`, so it needs a JDK and can't be loaded on Android.** That's the whole reason it isn't in `core`. | -| `shark-explorer-agent` | The MCP server a window answers agents through, the `--mcp-stdio` pipe that reaches it, and `--no-ui` for a run with no window at all. | **No Compose, Java 8 target, and desktop only** — it calls `ProcessHandle`. Has its own `AGENTS.md`. | +| `shark-explorer-agent` | The MCP server a window answers agents through, the `--mcp-stdio` pipe and the `--agent` command line that reach it, and `--no-ui` for a run with no window at all. | **No Compose, Java 8 target, and desktop only** — it calls `ProcessHandle`. Has its own `AGENTS.md`. | | `shark-explorer-app` | Compose Desktop UI: window, the canvas each shape draws into, details panel. | **Java 17 target** — see below. | | `shark-explorer-eval` | The heap dumps an agent is measured on, and the scoring of what it did with them. Driven by `shark-explorer-agent/harness/eval/run-eval.sh`. | **The only module with `shark-hprof-test` in its main source set**, which is why it is a module: writing the scenarios is what it does, and a dump-building DSL can be no dependency of anything the app ships. Runs no model. | diff --git a/shark/shark-explorer/notes/agent-surface.md b/shark/shark-explorer/notes/agent-surface.md index 73525c4d48..c08463d276 100644 --- a/shark/shark-explorer/notes/agent-surface.md +++ b/shark/shark-explorer/notes/agent-surface.md @@ -23,15 +23,37 @@ mitigations that shipped in 2026 (Anthropic's tool search, code execution over M **This surface is not where a context window goes to die**, and a per-tool cost of ~300 tokens is what buys descriptions that say when to reach for a tool. Re-measure it if the count doubles again. +## What the command line costs, now that there is one + +`--agent name=value …` is a process per call, and the thing to know is what that *doesn't* cost. +Measured against a packaged build with one window open on `leak_asynctask_o.hprof`: + +| | Measured | Paid | +| --- | --- | --- | +| One call, JVM start to JSON on stdout | 160–180 ms | Per call | +| `--agent-help`, all sixteen tools | 13,391 characters, ≈3,350 tokens | Only when read | +| `--agent-help `, one of them | ~1,200 characters, ≈300 tokens | Only when read | + +So the standing cost is nothing, and the whole surface as text is *smaller* than the `tools/list` definitions +of it (13,391 against 18,779) because `reason` is explained once rather than sixteen times. + +**A call from a shell is not a slower call.** It reaches the same window over the loopback socket the run +already publishes, so the heap dump is the one that was parsed and indexed once and the read queues on that +window's own thread — the 170 ms is a JVM starting and a socket, not a heap dump being reopened. The +process-per-call shape costs exactly one thing, and it isn't speed: **a connection can no longer be what +gathers an investigation**, which is what `--agent-session=` and `AgentSessionFile.continuing` exist for. A +call says which session it is one of, defaulting to `cli`, so a conversation's calls are one +row of the *Agent logs* screen the way one held-open MCP connection is. + ## What each shape is actually good at -- **MCP** is the only one of the three that gets a *session*: a process already holding a parsed heap dump, - its indexes, the window a person is watching, and the verdicts set so far. Reopening `large-dump.hprof` - costs seconds and hundreds of megabytes, so a stateless call per question is not a smaller version of - this, it is a different and much slower tool. It is also the only shape a client discovers on its own. -- **A CLI** is what an agent reaches for without being told, costs nothing until it is run, and pipes into - `grep`. Two things it would buy that MCP can't: the **no window open** case, and clients that speak no - MCP. What it must not be is a second implementation — see below. +- **MCP** is the shape a client *discovers*: the tools, their schemas and every refusal arrive in band, so + nothing has to teach a model what this surface is. And a connection is a session for free. What it is not + is the only way to reach a live window — that was the assumption this note was written under, and it was + wrong. +- **The command line** is what an agent reaches for without being configured, costs nothing until it is run, + pipes into `grep`, and is the only one of the two an agent whose client speaks no MCP can use. It pays for + the discovery MCP gets free: something has to tell it `--agent-help` exists, which is the skill's job. - **A skill** ([the open standard](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview), now read by Claude, Codex, Gemini CLI, Cursor and others) is the right home for *the method*, because progressive disclosure is exactly what the method wants: ~80 tokens of name and description at rest, the @@ -45,21 +67,26 @@ The thing worth protecting is that **the enforcement is not in the transport**. arguments in and JSON out, not a second copy of the rules: - `McpSession` — JSON-RPC over the socket. Exists. -- A CLI adapter — one subcommand that names a tool and its arguments, printing the answer or the refusal, and - a `--agent-help` that prints the same descriptions the schema carries so nothing has to be written twice. - Talks to a published run when there is one, and opens a heap dump itself when there isn't. +- `AgentCommandLine` — `--agent name=value …`, which turns a command line into one `tools/call` on that + same socket and prints what came back. Exists. It refuses nothing itself: every refusal it reports was + thrown by the handler that would have refused an MCP client. `--agent-help` is generated from the registry, + so a tool cannot be on one and missing from the other, and it is described through `NoHeapDumpToDescribe` — + a heap dump whose every method throws — which makes "printed, never called" hold rather than be a habit. - The skill — the method as `SKILL.md`, plus how to reach either adapter. Prose, not generated, and it points at `--agent-help` rather than listing tools that would go stale. -What that leaves duplicated is argument parsing per adapter, which is tens of lines. What it must never -become is two places that decide whether an investigation may conclude. +What that leaves duplicated is argument parsing per adapter, which is tens of lines — and less than that +here, because `AgentArguments` reads a number and a boolean out of text (the tools were written for a model, +which sends `limit=30` as a string as often as not). So a command line sends every value as it was typed and +the only shape needing a spelling of its own is a list, which is comma separated because a shell has no +brackets. What it must never become is two places that decide whether an investigation may conclude. ## The judgement, in one line -Keep MCP for the window somebody is watching, add the CLI for the window that isn't open yet, and move the -method into a skill so it costs nothing until it is needed. The criticism of MCP is about surfaces ten times -this size and about servers whose tools are one HTTP call each; ours is a session against a live process, -which is the case that criticism still concedes. +MCP for a client that can be configured, the command line for everything else, and the method in a skill so +it costs nothing until it is needed — all three over one registry. The criticism of MCP is about surfaces ten +times this size and about servers whose tools are one HTTP call each; ours is a session against a live +process, which is the case that criticism still concedes. Sources worth reading before changing this: the [Milvus comparison of the three shapes](https://milvus.io/blog/is-mcp-dead-cli-and-skills-for-ai-agents.md), Anthropic's diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index 13d42d9013..88dc6a05a3 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -23,6 +23,7 @@ being talked to by a program that is not this app. | `AgentServer.kt` | The loopback socket a run publishes, and the file that says where. | | `AgentStdioBridge.kt` | `--mcp-stdio`: the pipe an MCP client launches. | | `AgentStdioServer.kt` | And `--no-ui`: the same tools over this process's own stdio, for a run with no window. | +| `AgentCommandLine.kt` | `--agent name=value …`: one call typed at a window, over the same socket. And `--agent-help`, generated from the registry. | | `harness/start-harness.sh` | Opens a window and prints the command that throws an agent at it. | | `harness/eval/run-eval.sh` | Throws an agent at a heap dump whose answer is known, and scores what it did. The dumps and the scoring are `shark-explorer-eval`. | @@ -110,6 +111,30 @@ The app's own side of it — a window answering an agent — logs through `Shark reads as the reason for each call followed by the reads it caused. That is the artefact to ask for when somebody reports that an agent got it wrong. +## Two adapters, and the handshake line that lets a shell have a session + +`--mcp-stdio` is a client holding a session open. `--agent name=value …` is one call typed at a window +that is already up, and it is **argument translation and nothing else**: it builds a `tools/call` on the same +socket, so a refusal it prints was thrown by the handler that would have refused an MCP client. Adding a rule +to one adapter and not the other is the mistake this shape exists to make impossible — see +`notes/agent-surface.md`, which also has what a call costs. + +**A process per call would otherwise be a session per call**, and a session is what somebody reads afterwards. +So the handshake is `token[ sessionName]` on one line, `AgentSessionFile.continuing` appends to the newest file +whose name carries that id, and a command line defaults to `cli` — an agent's calls come out +of one shell the way its MCP calls come out of one connection. A client that says nothing gets a session of +its own, which is what every MCP client does. + +**The name is checked at both ends**, because it becomes part of a file name: the command line refuses one +that isn't letters and digits before calling anything, and `AgentServer` serves the connection anyway with a +session of its own and a line in the log. Refusing the connection would lose the investigation to protect a +file name; the calls are none the worse for it. + +**Exit codes are the second half of the answer.** 0 with JSON on stdout, 2 with the refusal on stderr, 1 when +there was nothing to answer it. A refusal is not a failure of the command — it is what the surface said, and +the message is the next thing to do — so a script can tell "it said no" from "nothing was there", and a shell +keeping stdout for the JSON still shows the sentence. + ## The transport, and why it is three things **A run publishes a loopback port and a token** to `~/.shark-explorer/agents/.agent`, and `--mcp-stdio` @@ -197,6 +222,10 @@ the reads happen on the heap dump's thread and the tests run headless. ```bash ./gradlew :shark:shark-explorer:shark-explorer-agent:check # test + detekt +# What the surface is, from a shell, with nothing open and no Gradle. Then one call at a window. +"Shark Explorer.app/Contents/MacOS/Shark Explorer" --agent-help +"Shark Explorer.app/Contents/MacOS/Shark Explorer" --agent list_leaks window= reason="Trying it" + # The whole surface end to end, in a real window, with an agent that has never seen this repository. shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh [heap-dump.hprof] diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt new file mode 100644 index 0000000000..74d757635b --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt @@ -0,0 +1,488 @@ +package shark.explorer.agent + +import java.io.BufferedReader +import java.io.File +import java.io.InputStreamReader +import java.io.OutputStreamWriter +import java.io.PrintWriter +import java.net.InetAddress +import java.net.InetSocketAddress +import java.net.Socket +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.add +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put +import kotlinx.serialization.json.putJsonArray +import kotlinx.serialization.json.putJsonObject +import shark.explorer.AndroidDevice +import shark.explorer.DeviceProcess + +/** + * One tool call typed rather than spoken over a session: `--agent name=value …`. + * + * The second adapter over the registry in [AgentTools], and deliberately not a second surface: all it does is + * turn a command line into a `tools/call` and print what came back, so a refusal met here is the same refusal + * thrown by the same handler. See `notes/agent-surface.md`. + * + * **It talks to the window that is already open**, over the loopback socket every run publishes — the same + * one [AgentStdioBridge] pipes to. So a call from here is as cheap as one made over MCP: the heap dump was + * parsed and indexed once, in the window somebody is watching, and this queues on that window's own reading + * thread. Being a process per call costs exactly one thing, which is that a connection can no longer be what + * gathers an investigation. See [SESSION_OPTION]. + * + * Why have it at all, given the pipe: it is what an agent reaches for without being configured, it costs + * nothing until it is run, it pipes into `grep`, and it is the only one of the two that an agent whose client + * speaks no MCP can use. + */ +object AgentCommandLine { + + /** + * Makes one call, prints the answer, and returns the exit code the process should end with. + * + * [words] is what was typed after `--agent`: the name of a tool, then its arguments as `name=value`. + */ + fun run( + /** Where the runs of the app publish themselves. See [AgentServer]. */ + directory: File, + words: List, + /** Which run, by process id, or null for the one that started most recently. */ + pid: String? = null, + /** + * Which session this call belongs to on the *Agent logs* screen. See [defaultSessionName]. + * + * Refused rather than sanitised when it is no name: it becomes part of a file name, and a caller that + * got it wrong wants to hear so on the call it got wrong. + */ + sessionName: String = defaultSessionName(), + /** How long to wait for a run to appear, for a call made while the app is still starting. */ + waitMillis: Long = DEFAULT_RUN_WAIT_MILLIS, + /** How to open a window when no run of the app is open, and null to only look for one. */ + openAWindow: (() -> Unit)? = null + ): Int { + val toolName = words.firstOrNull() + if (toolName == null || isCallArgument(toolName)) { + say("$AGENT_OPTION needs the name of a tool. $HELP_OPTION prints the ones there are.") + return NOTHING_ANSWERED + } + if (!AgentSessionFile.isSessionName(sessionName)) { + say( + "\"$sessionName\" is no session name: it becomes part of a file name, so it is letters and digits, " + + "up to ${AgentSessionFile.MAX_SESSION_NAME_LENGTH} of them." + ) + return NOTHING_ANSWERED + } + val arguments = try { + argumentsOf(toolName, words.drop(1)) + } catch (unreadable: IllegalArgumentException) { + say(unreadable.message.orEmpty()) + return NOTHING_ANSWERED + } + val run = waitForRun(directory, pid, waitMillis, openAWindow) ?: return NOTHING_ANSWERED + val socket = try { + Socket().apply { + connect(InetSocketAddress(InetAddress.getLoopbackAddress(), run.port), CONNECT_TIMEOUT_MILLIS) + } + } catch (throwable: Throwable) { + // Which is a run that was killed: the file is still there and nothing is on the port. + say("Shark Explorer run ${run.pid} does not answer on port ${run.port}: $throwable") + run.file.delete() + return NOTHING_ANSWERED + } + return socket.use { call(it, run, toolName, arguments, sessionName) } + } + + /** + * Every tool of this build as text: what each is for, and the arguments it takes. + * + * Generated from the same registry `tools/list` answers from, so a tool cannot be on one and missing from + * the other — which is the rule this adapter is under. [toolName] narrows it to one, because a surface of + * sixteen tools is worth reading a piece at a time. + * + * Answered with no run of the app and no heap dump anywhere, since it describes a build rather than + * anything open: an agent reads this *before* there is something to read. [NoHeapDumpToDescribe] is what + * makes that literal. + */ + fun help( + /** What to type to run this app, which is what the examples are written with. */ + command: String, + toolName: String? = null + ): String { + val tools = described() + val asked = toolName?.let { name -> tools.filter { it.name == name } } + if (asked != null && asked.isEmpty()) { + return "There is no tool called \"$toolName\". This build has " + + tools.joinToString(", ") { it.name } + "." + } + return buildString { + if (asked == null) { + appendLine(preamble(command)) + } + (asked ?: tools).forEach { appendLine(it.helpText()) } + } + } + + /** + * Whether a word of a command line is one argument of a call, `name=value`. + * + * The one rule both ends of `--agent` read a command line by: whatever this says is an argument is sent to + * the tool, and whatever it doesn't is the command line of the window. Two definitions of that would be a + * heap dump path quietly sent as an argument, or an argument quietly opened as a heap dump. + * + * A name of letters and digits, which is what every argument on this surface is called, so that a path is + * still a path — `/tmp/a=b.hprof` has a slash in its name and is therefore no argument. The one it gets + * wrong is a relative path with an `=` in it and no directory, which is a file nobody has. + */ + fun isCallArgument(word: String): Boolean { + val name = word.substringBefore('=', missingDelimiterValue = "") + return name.isNotEmpty() && name.first().isAsciiLetter() && name.all { it.isAsciiLetterOrDigit() } + } + + /** + * What a call joins when nothing said: the process that ran it, which for an agent is its shell. + * + * A shell lives as long as the conversation does and an agent's calls are commands in it, so its process + * id gathers an investigation the way one held-open connection gathers an MCP one. Falls back to this + * process, which is a session per call — a shell that cannot be named is one whose calls cannot be + * gathered, and a row each is better than landing in somebody else's session. + */ + fun defaultSessionName(): String { + val current = ProcessHandle.current() + val pid = current.parent().map { it.pid() }.orElse(current.pid()) + return "$SESSION_NAME_PREFIX$pid" + } + + private fun call( + socket: Socket, + run: AgentServer.PublishedRun, + toolName: String, + arguments: JsonObject, + sessionName: String + ): Int { + val toApp = PrintWriter(OutputStreamWriter(socket.getOutputStream(), Charsets.UTF_8), true) + val fromApp = BufferedReader(InputStreamReader(socket.getInputStream(), Charsets.UTF_8)) + // The token, and then which session this call is one of: one line, because the alternative is a + // handshake that has to be answered before the protocol can start. See [AgentServer]. + toApp.println("${run.token} $sessionName") + if (fromApp.readLine() != AgentServer.ACCEPTED) { + say("Shark Explorer run ${run.pid} refused the token in ${run.file}, so it is not the run that wrote it") + return NOTHING_ANSWERED + } + // Says who is calling, which is what puts a client name in a session's first line. Nothing else here + // needs it: the tools are the same whether or not anybody introduced themselves. + if (ask(toApp, fromApp, INITIALIZE_ID, "initialize", initializeParameters()) == null) { + return NOTHING_ANSWERED + } + val result = ask( + toApp, + fromApp, + CALL_ID, + "tools/call", + buildJsonObject { + put("name", toolName) + put("arguments", arguments) + } + ) ?: return NOTHING_ANSWERED + return printed(result) + } + + /** One JSON-RPC call and its answer, or null having said on stderr why there wasn't one. */ + private fun ask( + toApp: PrintWriter, + fromApp: BufferedReader, + id: Int, + method: String, + parameters: JsonObject + ): JsonObject? { + val message = buildJsonObject { + put("jsonrpc", JSONRPC_VERSION) + put("id", id) + put("method", method) + put("params", parameters) + } + toApp.println(JSON.encodeToString(JsonElement.serializer(), message)) + val line = fromApp.readLine() + if (line == null) { + say("Shark Explorer stopped answering during $method, so the window it was in has gone") + return null + } + val answer = try { + JSON.parseToJsonElement(line) as? JsonObject + } catch (notJson: Exception) { + say("Shark Explorer answered $method with something that is no JSON-RPC message: $notJson") + return null + } + val error = answer?.get("error") as? JsonObject + if (error != null) { + // Not a refusal: a refusal is an answer a tool gave. This is the app failing to answer at all. + say("Shark Explorer could not answer $method: ${(error["message"] as? JsonPrimitive)?.content}") + return null + } + return answer?.get("result") as? JsonObject + } + + /** + * Puts the answer on stdout, or the refusal on stderr, and says which happened in the exit code. + * + * Two streams and two codes because **a refusal is not a failure of the command**: it is what the surface + * answered, and its message is the next thing to do. So a shell keeping stdout for the JSON still shows + * the sentence, and a script can tell "it said no" from "there was nothing to ask". + */ + private fun printed(result: JsonObject): Int { + val text = ((result["content"] as? JsonArray)?.firstOrNull() as? JsonObject) + ?.let { (it["text"] as? JsonPrimitive)?.content } + if ((result["isError"] as? JsonPrimitive)?.content == "true") { + say(text ?: "The call was refused, and the refusal said nothing.") + return REFUSED + } + val structured = result["structuredContent"] as? JsonObject + val out = PrintWriter(OutputStreamWriter(System.out, Charsets.UTF_8), true) + out.println( + structured?.let { PRETTY_JSON.encodeToString(JsonElement.serializer(), it) } ?: text.orEmpty() + ) + out.flush() + return ANSWERED + } + + /** + * The arguments as JSON: every value as it was typed, except the ones the schema says are lists. + * + * Which works because [AgentArguments] reads a number and a boolean out of text — the tools were written + * for a model, and a model sends `limit=30` as a string as often as not. So a command line spells + * everything the way a person types it, and the one shape with no spelling of its own is a list: those are + * comma separated, a shell having no brackets. + * + * The schema is this build's own, so an argument of a tool this build has never heard of goes as text and + * is refused at the other end, by a message that lists the tools there are. + */ + private fun argumentsOf( + toolName: String, + words: List + ): JsonObject { + val lists = described().firstOrNull { it.name == toolName }?.listArguments().orEmpty() + return buildJsonObject { + words.forEach { word -> + require(isCallArgument(word)) { + "\"$word\" is no argument of $toolName. An argument is `name=value`, and a value with spaces in " + + "it is quoted: reason=\"why I am asking\"." + } + val name = word.substringBefore('=') + val value = word.substringAfter('=') + if (name in lists) { + putJsonArray(name) { value.split(LIST_SEPARATOR).forEach { add(it.trim()) } } + } else { + put(name, value) + } + } + } + } + + /** The tools of this build, described. Built per call, so nothing here is shared between threads. */ + private fun described(): List = AgentTools(NoHeapDumpToDescribe).all + + private fun initializeParameters(): JsonObject = buildJsonObject { + put("protocolVersion", PROTOCOL_VERSION) + putJsonObject("clientInfo") { + put("name", CLIENT_NAME) + } + } + + private fun preamble(command: String): String = """ + |Shark Explorer's heap dump tools, from a shell. One call per command, answered by the window that has + |the heap dump open — or by a window this opens when none is. + | + | $command $AGENT_OPTION name=value … + | $command $AGENT_OPTION open_heap_dumps reason="Finding out which heap dump is open" + | $command $AGENT_OPTION describe_object object=0x7205 reason="Reading the holder's fields" + | + |Start with open_heap_dumps: its answer carries the method to follow, the window ids every other tool + |names a heap dump by, and whatever verdicts somebody has already recorded about that dump. + | + |Every tool takes `reason`, which is why you are making the call. It is logged beside the reads it causes + |and read afterwards on the *Agent logs* screen of the window, so write the sentence you would say to the + |person watching. Addresses are `0x…`, exactly as this surface writes them, and never decimal. + | + |${options()} + | + |Exit code $ANSWERED when the answer is on stdout, $REFUSED when the call was refused and the refusal is + |on stderr, $NOTHING_ANSWERED when there was nothing to answer it. + | + |TOOLS + """.trimMargin() + + private fun options(): String = listOf( + "$PID_OPTION" to "Which run of the app to call, when more than one is open.", + "$SESSION_OPTION" to + "Which session these calls are one of, letters and digits. One per shell by default, so that an " + + "investigation is one row of the *Agent logs* screen rather than a row per call.", + "$HELP_OPTION " to "Just that tool." + ).joinToString("\n") { (option, what) -> " ${option.padEnd(OPTION_WIDTH)}$what" } + + /** Answered: the tool's own JSON is on stdout. */ + const val ANSWERED = 0 + + /** + * Nothing answered: no run to talk to, one that has gone, or a command line this could not read. + * + * The same code [AgentStdioBridge] ends with for the same case and for the same reason: a command that did + * nothing has to fail, or whatever ran it carries on as though it had an answer. + */ + const val NOTHING_ANSWERED = 1 + + /** Refused: the tool said no, and stderr says what to do about it. */ + const val REFUSED = 2 + + /** What a command line says to make one call. See `shark.explorer.app.ExplorerArguments`. */ + const val AGENT_OPTION = "--agent" + + /** And to read what the calls are, which needs no window and no heap dump. See [help]. */ + const val HELP_OPTION = "--agent-help" + + /** + * What a command line says to put its calls in one session, rather than one session per call. + * + * For an agent whose calls do not all come out of one shell — a harness that starts one per command — and + * for a person following an investigation of their own. See [defaultSessionName]. + */ + const val SESSION_OPTION = "--agent-session=" + + /** Which run to call, spelled the way the pipe spells it. See [AgentStdioBridge.PID_OPTION]. */ + const val PID_OPTION = AgentStdioBridge.PID_OPTION + + /** How the session of a call from here is named, so that a file says what made it. */ + private const val SESSION_NAME_PREFIX = "cli" + + /** What the window's *Agent logs* screen says connected, for a session started from a shell. */ + private const val CLIENT_NAME = "shark-explorer-cli" + + /** Wide enough for the longest option above, since the descriptions read as a column or as nothing. */ + private const val OPTION_WIDTH = 24 + + private const val LIST_SEPARATOR = ',' + + private const val JSONRPC_VERSION = "2.0" + + /** The revision this was written against, which the app echoes back. See [McpSession]. */ + private const val PROTOCOL_VERSION = "2025-06-18" + + private const val INITIALIZE_ID = 1 + private const val CALL_ID = 2 + + private const val CONNECT_TIMEOUT_MILLIS = 1_000 + + private val JSON = Json { ignoreUnknownKeys = true } + + /** + * Indented, because the reader is either a person or a model reading a chain of twenty steps. + * + * The same shape the text of an MCP answer is in, so that what an agent reads is the same either way. + */ + private val PRETTY_JSON = Json(JSON) { prettyPrint = true } +} + +internal fun Char.isAsciiLetter(): Boolean = this in 'a'..'z' || this in 'A'..'Z' + +internal fun Char.isAsciiLetterOrDigit(): Boolean = isAsciiLetter() || this in '0'..'9' + +/** One argument of a tool, as the help prints it. */ +private class HelpArgument( + val name: String, + val schema: JsonObject, + val isRequired: Boolean +) + +private fun AgentTool.helpText(): String = buildString { + appendLine(name) + appendLine(" $description") + // `reason` is on every tool and is said once, in the preamble: sixteen copies of the same paragraph is a + // sixth of the help, and the one argument nobody needs reminding of per tool is the mandatory one. + arguments().filter { it.name != REASON_ARGUMENT }.forEach { appendLine(" ${it.helpLine()}") } +} + +/** Which of this tool's arguments are lists, which is the one thing a command line has to spell specially. */ +private fun AgentTool.listArguments(): Set = + arguments().filter { it.schema.type() == ARRAY_TYPE }.map { it.name }.toSet() + +private fun AgentTool.arguments(): List { + val properties = schema[PROPERTIES_KEY] as? JsonObject ?: return emptyList() + val required = (schema[REQUIRED_KEY] as? JsonArray).orEmpty() + .mapNotNull { (it as? JsonPrimitive)?.content } + return properties.mapNotNull { (name, element) -> + (element as? JsonObject)?.let { HelpArgument(name, it, name in required) } + } +} + +private fun HelpArgument.helpLine(): String { + val kind = listOfNotNull(schema.kind(), "optional".takeIf { !isRequired }).joinToString(", ") + return "$name ($kind) — ${schema.description()}" +} + +/** + * How a value of this argument is written on a command line, which is not its JSON type. + * + * Every value is typed as text and read as whatever the tool asks for — see `AgentCommandLine.argumentsOf` — + * so what a reader needs here is what to type, rather than that JSON has numbers in it. + */ +private fun JsonObject.kind(): String { + val values = (this[ENUM_KEY] as? JsonArray)?.contents() + val itemValues = ((this[ITEMS_KEY] as? JsonObject)?.get(ENUM_KEY) as? JsonArray)?.contents() + return when { + values != null -> values.joinToString(" or ") + type() == ARRAY_TYPE -> + "comma separated" + itemValues?.let { ", from ${it.joinToString(", ")}" }.orEmpty() + type() == INTEGER_TYPE -> "a whole number" + type() == BOOLEAN_TYPE -> "true or false" + else -> "text" + } +} + +private fun JsonArray.contents(): List = mapNotNull { (it as? JsonPrimitive)?.content } + +private fun JsonObject.type(): String? = (this["type"] as? JsonPrimitive)?.content + +private fun JsonObject.description(): String = (this["description"] as? JsonPrimitive)?.content.orEmpty() + +/** + * The heap dumps of a run that is answering nobody, which is every method throwing. + * + * [AgentTools] holds these so that its handlers can read a heap dump, and **describing a tool never calls + * its handler** — so a registry built on this can be printed and cannot be used. Throwing rather than + * answering with nothing, because "no heap dump is open" is an answer an agent would act on and this is not + * that: it is a description of a build, with nowhere for a call to go. + */ +private object NoHeapDumpToDescribe : AgentHeapDumps { + + override fun openHeapDumps(): List = nothing() + + override fun openingHeapDumpPaths(): List = nothing() + + override suspend fun open(file: File): AgentHeapDump = nothing() + + override suspend fun devices(): List = nothing() + + override suspend fun processesOf(serialNumber: String): List = nothing() + + override suspend fun dumpHeap( + serialNumber: String, + processName: String + ): AgentHeapDump = nothing() + + private fun nothing(): Nothing = throw IllegalStateException( + "These tools are only being described, so there is no heap dump here and nothing to call: a call goes " + + "to the run of the app that has one open. See AgentCommandLine." + ) +} + +/** Said once in the preamble rather than under every tool. See [AgentTool]. */ +private const val REASON_ARGUMENT = "reason" + +private const val PROPERTIES_KEY = "properties" +private const val REQUIRED_KEY = "required" +private const val ENUM_KEY = "enum" +private const val ITEMS_KEY = "items" +private const val ARRAY_TYPE = "array" +private const val INTEGER_TYPE = "integer" +private const val BOOLEAN_TYPE = "boolean" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt index 8f673ce06c..694d105c80 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt @@ -158,7 +158,8 @@ object AgentServer { } /** - * One connection: the token, then a JSON-RPC message per line until the agent goes away. + * One connection: the token and optionally a session to join, then a JSON-RPC message per line until the + * agent goes away. * * **No read timeout**, unlike the link socket. An agent thinking, or waiting for the person at the * machine, is a connection with nothing on it for minutes at a time, and a session dropped for being @@ -174,8 +175,8 @@ object AgentServer { socket.use { val reader = BufferedReader(InputStreamReader(socket.getInputStream(), Charsets.UTF_8)) val writer = PrintWriter(OutputStreamWriter(socket.getOutputStream(), Charsets.UTF_8), true) - val sentToken = reader.readLine() - if (sentToken != token) { + val handshake = reader.readLine().orEmpty().split(HANDSHAKE_SEPARATOR) + if (handshake.firstOrNull() != token) { // Loopback only, so this is a stale file being read far more often than it is anything to worry // about. SharkLog.d { "An agent connected quoting the wrong token, so it was not listened to" } @@ -183,10 +184,10 @@ object AgentServer { return } writer.println(ACCEPTED) - // A file per accepted connection, so that two agents at one heap dump are two sessions to read rather - // than one file with both of their reasoning in it. Named before the handshake, since a client that - // connects and says nothing is itself worth a line on that screen. - val sessionFile = AgentSessionFile.starting(sessions, serverVersion) + // A file per accepted connection unless it asked to join one, so that two agents at one heap dump are + // two sessions to read rather than one file with both of their reasoning in it. Named before the + // handshake, since a client that connects and says nothing is itself worth a line on that screen. + val sessionFile = sessionFile(sessions, serverVersion, handshake.getOrNull(1)) SharkLog.d { "An agent's session is being written to ${sessionFile.file}" } val session = McpSession(AgentTools(heapDumps), serverVersion, sessionFile) while (true) { @@ -206,6 +207,30 @@ object AgentServer { } } + /** + * Where this connection's calls are written down: a session of its own, or the one it asked to join. + * + * A connection is a session for a client that holds one open, which is what MCP over the pipe is. A + * command line is a process per call, so it names the session its calls belong to instead — see + * [AgentCommandLine]. The name is checked here as well as there, because it becomes part of a file name and + * it arrived from another process; a name this cannot use is a session of its own and a line saying so, + * rather than a connection refused, since the calls themselves are none the worse for it. + */ + private fun sessionFile( + sessions: File, + serverVersion: String, + name: String? + ): AgentSessionFile { + if (name == null) { + return AgentSessionFile.starting(sessions, serverVersion) + } + if (!AgentSessionFile.isSessionName(name)) { + SharkLog.d { "\"$name\" is no session name, so this connection was given a session of its own" } + return AgentSessionFile.starting(sessions, serverVersion) + } + return AgentSessionFile.continuing(sessions, serverVersion, name) + } + private fun newToken(): String { val bytes = ByteArray(TOKEN_BYTES) SecureRandom().nextBytes(bytes) @@ -232,6 +257,9 @@ object AgentServer { private const val SESSIONS_DIRECTORY = "sessions" internal const val ACCEPTED = "OK" internal const val DECLINED = "NO" + + /** Between the token and the session a connection is joining, which is why a name has no spaces in it. */ + private const val HANDSHAKE_SEPARATOR = ' ' private const val PORT_PROPERTY = "port" private const val TOKEN_PROPERTY = "token" private const val THREAD_NAME = "shark-explorer-agents" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt index 752246155d..dbca9e72b7 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt @@ -41,11 +41,11 @@ class AgentSessionFile private constructor( /** What this session is called, in the window and in the file. See [newSessionId]. */ val sessionId: String, private val startedAt: Instant, - private val serverVersion: String + private val serverVersion: String, + /** Whether the file already says whose session it is, which a call joining one finds true. */ + private var isHeaderWritten: Boolean ) { - private var isHeaderWritten = false - /** * Says who connected, which is the handshake and therefore the first thing to land in the file. * @@ -124,9 +124,58 @@ class AgentSessionFile private constructor( directory.mkdirs() val name = FILE_NAME_PREFIX + FILE_NAME_TIME.format(startedAt) + "-$sessionId$FILE_NAME_SUFFIX" deleteOlderSessions(directory, keepSessionCount - 1) - return AgentSessionFile(File(directory, name), sessionId, startedAt, serverVersion) + return AgentSessionFile( + file = File(directory, name), + sessionId = sessionId, + startedAt = startedAt, + serverVersion = serverVersion, + isHeaderWritten = false + ) + } + + /** + * The session called [sessionId] to add to, which is the newest file of that name or a new one. + * + * What a command line needs and a connection doesn't. An MCP client holds one connection open for a + * whole investigation, so a connection is a session; `--agent` is a process per call, so without this a + * morning's work would be thirty files and the *Agent logs* screen would list thirty agents where there + * was one. See [AgentCommandLine]. + * + * The header is not written again, since a file with two of them is two sessions to whoever reads it — + * so the client, the protocol and the build in it are the ones from the call that started the session. + */ + fun continuing( + directory: File, + serverVersion: String, + sessionId: String, + startedAt: Instant = Instant.now(), + keepSessionCount: Int = KEEP_SESSION_COUNT + ): AgentSessionFile { + val existing = directory.listFiles { file: File -> file.name.isSessionFile() }.orEmpty() + .filter { it.name.sessionIdOfName() == sessionId } + // Named after when it started, so the newest of them is a sort by name — the same one the window + // lists first. Several only happen for a session named again after the older one aged out. + .maxByOrNull { it.name } + ?: return starting(directory, serverVersion, startedAt, sessionId, keepSessionCount) + return AgentSessionFile( + file = existing, + sessionId = sessionId, + startedAt = startedAt, + serverVersion = serverVersion, + isHeaderWritten = true + ) } + /** + * Whether [name] can name a session, which is strict because it becomes part of a file name. + * + * Letters and digits, and no '-' in particular: a file is `agent--.jsonl` and the id is read + * back out of it by splitting on the last one. Checked on both sides of the socket — the command line + * refuses a name it cannot use, and this end never names a file after something it was told. + */ + fun isSessionName(name: String): Boolean = name.length in 1..MAX_SESSION_NAME_LENGTH && + name.all { it.isAsciiLetterOrDigit() } + /** * Every session written in [directory], newest first, with the calls of each in the order they were * made. @@ -299,6 +348,14 @@ class AgentSessionFile private constructor( */ const val KEEP_SESSION_COUNT = 100 + /** + * How long a name a caller can give a session, which is enough for a word and a process id. + * + * A bound at all because it is part of a file name: the ids this hands out are eight characters, and a + * name nobody can read on the *Agent logs* screen is no better than one of those. + */ + const val MAX_SESSION_NAME_LENGTH = 16 + private const val SESSION_ID_BYTES = 4 private const val FILE_NAME_PREFIX = "agent-" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt index 9fb915ee49..2025249911 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioBridge.kt @@ -40,15 +40,8 @@ object AgentStdioBridge { /** Which run, by process id, or null for the one that started most recently. */ pid: String? = null, /** How long to wait for a run to appear, for a client that launched this before the app was open. */ - waitMillis: Long = DEFAULT_WAIT_MILLIS, - /** - * How to open a window to investigate in when no run of the app is open, and null to wait for one. - * - * Because the alternative is an agent whose only answer is "ask somebody to launch Shark Explorer", and - * a window opened here is a window the person at the machine can then watch — which is the whole reason - * this surface is a window rather than a library. Not called when a run was asked for by [pid]: that - * names a window, and opening a different one would be answering about the wrong heap dump. - */ + waitMillis: Long = DEFAULT_RUN_WAIT_MILLIS, + /** How to open a window when no run of the app is open, and null to wait for one. See [waitForRun]. */ openAWindow: (() -> Unit)? = null ): Int { val run = waitForRun(directory, pid, waitMillis, openAWindow) ?: return NOTHING_TO_TALK_TO @@ -118,85 +111,9 @@ object AgentStdioBridge { return 0 } - private fun waitForRun( - directory: File, - pid: String?, - waitMillis: Long, - openAWindow: (() -> Unit)? - ): AgentServer.PublishedRun? { - var waited = 0L - var deadline = waitMillis - var opened = false - // Naming a run names a window and therefore a heap dump, so opening a different one would be answering - // about the wrong dump: for that command line there is nothing to open, only something to wait for. - val opensAWindow = openAWindow != null && pid == null - while (true) { - val runs = AgentServer.publishedRuns(directory) - val run = if (pid == null) runs.firstOrNull() else runs.firstOrNull { it.pid == pid } - if (run == null && !opened && opensAWindow) { - say("No Shark Explorer is running, so one is being opened to investigate in.") - requireNotNull(openAWindow).invoke() - opened = true - // From here rather than from the start, because what is being waited for changed: a JVM starting, - // Compose coming up and a window appearing, rather than a file that may already be there. - deadline = waited + OPENING_WAIT_MILLIS - } - if (run != null) { - if (pid == null && runs.size > 1) { - // Which run an agent ends up in is worth saying rather than leaving to be worked out from what - // heap dump it finds open: several explorers at once is the normal way this app is used. - say( - "${runs.size} Shark Explorer runs are open; talking to ${run.pid}, the one that started most " + - "recently. Pass $PID_OPTION to pick another: " + runs.joinToString(", ") { it.pid } - ) - } - return run - } - if (waited >= deadline) { - say( - if (pid == null && opened) { - "A Shark Explorer was started and has not published itself in " + - "${OPENING_WAIT_MILLIS / 1000} seconds, so something went wrong opening it. Its log is in " + - "the newest file under ~/.shark-explorer/logs." - } else if (pid == null) { - "No Shark Explorer is running, so there is no heap dump to investigate. Open one — every run " + - "of the app publishes itself in $directory — and start this again." - } else { - "No Shark Explorer run is $pid. Open runs: " + - AgentServer.publishedRuns(directory).joinToString(", ") { it.pid }.ifEmpty { "none" } - } - ) - return null - } - Thread.sleep(POLL_MILLIS) - waited += POLL_MILLIS - } - } - - /** - * On stderr, always, which is where an MCP client collects what a server has to say. - * - * Not through `SharkLog`: this process deliberately never installs the app's logging, since that writes - * to stdout and stdout is the protocol. - */ - private fun say(message: String) { - System.err.println("[shark-explorer] $message") - } - /** What the command line says to pick a run by process id. See `shark.explorer.app.ExplorerArguments`. */ const val PID_OPTION = "--agent-run=" - private const val DEFAULT_WAIT_MILLIS = 10_000L - - /** - * How long a window opened from here is given to publish itself. - * - * Longer than [DEFAULT_WAIT_MILLIS] by a lot, because it covers a cold JVM, Compose starting and jlink's - * runtime being paged in — and because the alternative to waiting is telling an agent there is no window - * while one is in the middle of appearing. - */ - private const val OPENING_WAIT_MILLIS = 60_000L - private const val POLL_MILLIS = 250L private const val CONNECT_TIMEOUT_MILLIS = 1_000 private const val SHUTDOWN_MILLIS = 500L @@ -206,3 +123,99 @@ object AgentStdioBridge { */ private const val NOTHING_TO_TALK_TO = 1 } + +/** + * The run of the app to talk to, opening one if there is none and nothing is open, and null for a machine + * where there was never going to be one. + * + * Out here rather than in [AgentStdioBridge] because both adapters ask this and the answer must not differ: + * picking a run when several are open, opening a window when none is, and giving up are one question, and a + * command line that chose a different run from the pipe would be a second surface. See [AgentCommandLine]. + */ +internal fun waitForRun( + directory: File, + pid: String?, + waitMillis: Long, + /** + * How to open a window to investigate in when no run of the app is open, and null to wait for one. + * + * Because the alternative is an agent whose only answer is "ask somebody to launch Shark Explorer", and a + * window opened here is a window the person at the machine can then watch — which is the whole reason this + * surface is a window rather than a library. Not called when a run was asked for by [pid]: that names a + * window, and opening a different one would be answering about the wrong heap dump. + */ + openAWindow: (() -> Unit)? +): AgentServer.PublishedRun? { + var waited = 0L + var deadline = waitMillis + var opened = false + // Naming a run names a window and therefore a heap dump, so opening a different one would be answering + // about the wrong dump: for that command line there is nothing to open, only something to wait for. + val opensAWindow = openAWindow != null && pid == null + while (true) { + val runs = AgentServer.publishedRuns(directory) + val run = if (pid == null) runs.firstOrNull() else runs.firstOrNull { it.pid == pid } + if (run == null && !opened && opensAWindow) { + say("No Shark Explorer is running, so one is being opened to investigate in.") + requireNotNull(openAWindow).invoke() + opened = true + // From here rather than from the start, because what is being waited for changed: a JVM starting, + // Compose coming up and a window appearing, rather than a file that may already be there. + deadline = waited + OPENING_WAIT_MILLIS + } + if (run != null) { + if (pid == null && runs.size > 1) { + // Which run an agent ends up in is worth saying rather than leaving to be worked out from what heap + // dump it finds open: several explorers at once is the normal way this app is used. + say( + "${runs.size} Shark Explorer runs are open; talking to ${run.pid}, the one that started most " + + "recently. Pass ${AgentStdioBridge.PID_OPTION} to pick another: " + + runs.joinToString(", ") { it.pid } + ) + } + return run + } + if (waited >= deadline) { + say( + if (pid == null && opened) { + "A Shark Explorer was started and has not published itself in " + + "${OPENING_WAIT_MILLIS / 1000} seconds, so something went wrong opening it. Its log is in " + + "the newest file under ~/.shark-explorer/logs." + } else if (pid == null) { + "No Shark Explorer is running, so there is no heap dump to investigate. Open one — every run " + + "of the app publishes itself in $directory — and start this again." + } else { + "No Shark Explorer run is $pid. Open runs: " + + AgentServer.publishedRuns(directory).joinToString(", ") { it.pid }.ifEmpty { "none" } + } + ) + return null + } + Thread.sleep(POLL_MILLIS) + waited += POLL_MILLIS + } +} + +/** + * On stderr, always: where an MCP client collects what a server has to say, and where a shell shows what a + * command is doing. + * + * Not through `SharkLog`: neither of these processes installs the app's logging, since that writes to stdout + * and stdout carries the protocol in one and the answer in the other. + */ +internal fun say(message: String) { + System.err.println("[shark-explorer] $message") +} + +/** + * How long a window opened for an agent is given to publish itself. + * + * A lot longer than a wait for one that should already be there, because it covers a cold JVM, Compose + * starting and jlink's runtime being paged in — and because the alternative to waiting is telling an agent + * there is no window while one is in the middle of appearing. + */ +private const val OPENING_WAIT_MILLIS = 60_000L +private const val POLL_MILLIS = 250L + +/** How long either adapter waits for a run that ought to be there already, before saying there is none. */ +internal const val DEFAULT_RUN_WAIT_MILLIS = 10_000L diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt index 7fa5f7920e..f1e7c70409 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt @@ -17,10 +17,13 @@ import shark.SharkLog /** * One agent's connection to this app, spoken as [MCP](https://modelcontextprotocol.io). * - * MCP rather than a command line or a protocol of ours, for one reason: **the app is already running and an - * agent has to reach into it**. A CLI would have to open the heap dump again — seconds and hundreds of - * megabytes per question, and answers about a dump nobody is looking at — while MCP is the one interface - * every agent already has, so a client is configured once and nothing here ever calls a model. + * MCP rather than a protocol of ours because it is the one interface every agent already has: a client is + * configured once, discovers the tools and their schemas itself, and nothing here ever calls a model. + * + * What it has over [AgentCommandLine], which reaches the same run over the same socket, is that **a + * connection is a session**. One handshake for an investigation, the tools and the method arriving in band, + * and every call of it in one file for the *Agent logs* screen to draw. A command line has to be told which + * session it is joining to get the last of those, and nothing at all to get the first two. * * JSON-RPC 2.0, one message per line. That framing is stdio MCP's own, which is what lets the bridge in * [AgentStdioBridge] be a pipe and nothing more. diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentCommandLineTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentCommandLineTest.kt new file mode 100644 index 0000000000..f7098b4273 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentCommandLineTest.kt @@ -0,0 +1,235 @@ +package shark.explorer.agent + +import java.io.ByteArrayOutputStream +import java.io.Closeable +import java.io.File +import java.io.PrintStream +import org.assertj.core.api.Assertions.assertThat +import org.junit.After +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import shark.explorer.exactHexObjectId + +/** + * A tool call typed at a window that is already open, which is the other adapter over the same tools. + * + * Two things here are worth a test and the rest is translation. **A refusal has to come back as a refusal** — + * the message on stderr and an exit code of its own, since the whole method rests on an agent being told no + * in words it can act on. And **a shell's worth of calls has to be one session**: a connection is what + * gathers an MCP investigation, and a process per call has nothing to gather it with unless it says which + * session it is joining. See [AgentServerTest] for the socket under this. + */ +class AgentCommandLineTest { + + @get:Rule + val temporaryFolder = TemporaryFolder() + + @get:Rule + val log = RecordedLog() + + private lateinit var directory: File + private lateinit var heapDump: InvestigationHeapDump + private lateinit var window: FakeAgentHeapDump + private val closeables = mutableListOf() + + /** What a shell would show, which is the answer on stdout and everything else on stderr. */ + private val printed = ByteArrayOutputStream() + private val said = ByteArrayOutputStream() + + @Before + fun setUp() { + directory = temporaryFolder.newFolder("agents") + heapDump = temporaryFolder.applicationHoldsActivityThroughHolder() + window = FakeAgentHeapDump(heapDump.explorer) + } + + @After + fun tearDown() { + closeables.forEach { it.close() } + heapDump.close() + } + + @Test + fun `a call is answered with the tool's own JSON`() { + listen() + + val exitCode = agent( + "describe_object", + "object=${exactHexObjectId(heapDump.holderObjectId)}", + "reason=Reading the holder's fields." + ) + + assertThat(exitCode).isEqualTo(AgentCommandLine.ANSWERED) + // Indented, and the same JSON an MCP client is answered with: a person reads this one and a model reads + // the other, and neither of them is reading a different surface. + assertThat(printed()).contains(HOLDER_CLASS_NAME).contains("\n ") + assertThat(window.reads).isNotEmpty + } + + @Test + fun `a refusal is what the shell gets back, and it says what to do instead`() { + listen() + + val exitCode = agent("describe_object", "object=${exactHexObjectId(heapDump.holderObjectId)}") + + // Its own exit code, because a refusal is not a failure of the command: the tool answered, and what it + // answered is the next thing to do. Nothing on stdout, so a shell keeping that for the JSON gets none. + assertThat(exitCode).isEqualTo(AgentCommandLine.REFUSED) + assertThat(printed()).isEmpty() + assertThat(said()).contains("needs `reason`") + } + + @Test + fun `the calls of one shell are one session`() { + listen() + + agent("open_heap_dumps", "reason=Finding out what is open.") + agent("list_leaks", "reason=Reading what the dump says about itself.") + + // One row of the *Agent logs* screen rather than two, which is the whole of what naming a session buys: + // an investigation is what somebody reads afterwards, and a process per call would have cut it up. + val session = sessions().single() + assertThat(session.sessionId).isEqualTo(SESSION_NAME) + assertThat(session.calls.map { it.tool }).containsExactly("open_heap_dumps", "list_leaks") + // Said once, by the call that started the session, since a file with two headers is two sessions. + assertThat(session.client).isEqualTo("shark-explorer-cli") + } + + @Test + fun `the calls of another shell are another session`() { + listen() + + agent("open_heap_dumps", "reason=Finding out what is open.") + agent("open_heap_dumps", "reason=Finding out what is open.", sessionName = "cli99") + + // Two agents at one heap dump are two investigations to read, exactly as two connections are. + assertThat(sessions().map { it.sessionId }).containsExactlyInAnyOrder(SESSION_NAME, "cli99") + } + + @Test + fun `a list argument is spelled with commas`() { + listen() + + val exitCode = agent( + "find_objects", + "className=Holder", + "kinds=INSTANCE,CLASS", + "reason=Checking there is only one holder." + ) + + // A shell has no brackets, so the one argument shape with no spelling of its own gets one here. Refused + // rather than misread if it arrived as text, which is what makes this assertion about the commas. + assertThat(exitCode).isEqualTo(AgentCommandLine.ANSWERED) + assertThat(printed()).contains(HOLDER_CLASS_NAME) + } + + @Test + fun `a call with no run to talk to says so rather than making something up`() { + val exitCode = agent("open_heap_dumps", "reason=Finding out what is open.") + + assertThat(exitCode).isEqualTo(AgentCommandLine.NOTHING_ANSWERED) + assertThat(printed()).isEmpty() + assertThat(said()).contains("No Shark Explorer is running") + } + + @Test + fun `a session name that could be a path is refused before anything is called`() { + listen() + + val exitCode = agent("open_heap_dumps", "reason=Finding out what is open.", sessionName = "../../evil") + + // It becomes part of a file name, so the caller hears about it on the call it got wrong rather than + // finding a session file somewhere else. The app end checks it too — see [AgentServerTest]. + assertThat(exitCode).isEqualTo(AgentCommandLine.NOTHING_ANSWERED) + assertThat(said()).contains("is no session name") + assertThat(sessions()).isEmpty() + } + + @Test + fun `a word that is no argument is a message about arguments`() { + listen() + + val exitCode = agent("describe_object", "0x7205") + + assertThat(exitCode).isEqualTo(AgentCommandLine.NOTHING_ANSWERED) + assertThat(said()).contains("An argument is `name=value`") + } + + @Test + fun `the help is every tool of this build, with how to type its arguments`() { + val help = AgentCommandLine.help(command = "shark-explorer") + + // Generated from the registry, so a tool added without being described here is a test failure rather + // than a tool an agent using the command line never hears about. + AgentTools(FakeAgentHeapDumps()).all.forEach { tool -> + assertThat(help).contains(tool.name).contains(tool.description) + } + // The one thing the schema doesn't say, because JSON has brackets and a command line hasn't. + assertThat(help).contains("comma separated") + // And `reason` is said in the preamble rather than under each of sixteen tools, which would be a sixth + // of the help spent on the one argument every tool takes. + assertThat(help).contains("Every tool takes `reason`") + assertThat(help.lines().filter { it.trim().startsWith("reason (") }).isEmpty() + } + + @Test + fun `the help of one tool is that tool, and of no tool says which there are`() { + val one = AgentCommandLine.help(command = "shark-explorer", toolName = "conclude") + + assertThat(one).contains("conclude").doesNotContain("list_leaks") + + val none = AgentCommandLine.help(command = "shark-explorer", toolName = "chain_from_a_gc_root") + + assertThat(none).contains("There is no tool").contains("chain_from_gc_root") + } + + private fun listen(): Closeable = AgentServer.listen( + heapDumps = FakeAgentHeapDumps(listOf(window)), + serverVersion = "1.2.3", + directory = directory + ).also { closeables += it } + + /** + * Runs one command and hands back its exit code, with stdout and stderr collected. + * + * Nothing waited for: the run these tests are about is either already published or never will be, and a + * window is not something a test can open. + */ + private fun agent( + vararg words: String, + sessionName: String = SESSION_NAME + ): Int { + val previousOut = System.out + val previousErr = System.err + System.setOut(PrintStream(printed, true, Charsets.UTF_8.name())) + System.setErr(PrintStream(said, true, Charsets.UTF_8.name())) + return try { + AgentCommandLine.run( + directory = directory, + words = words.toList(), + pid = null, + sessionName = sessionName, + waitMillis = 0L, + openAWindow = null + ) + } finally { + System.setOut(previousOut) + System.setErr(previousErr) + } + } + + private fun sessions(): List = + AgentSessionFile.sessionsIn(AgentServer.sessionsDirectory(directory)) + + private fun printed(): String = printed.toString(Charsets.UTF_8.name()) + + private fun said(): String = said.toString(Charsets.UTF_8.name()) + + private companion object { + + /** What one shell's calls are gathered under, which is `cli` for a real one. */ + const val SESSION_NAME = "cli1234" + } +} diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt index fa0cf46bc0..57b0cd6864 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentServerTest.kt @@ -94,6 +94,39 @@ class AgentServerTest { } } + @Test + fun `two connections that name one session are one session`() { + listen() + val run = AgentServer.publishedRuns(directory).single() + + connect(run, sessionName = "cli7").use { it.ask(PING) } + connect(run, sessionName = "cli7").use { it.ask(CALL_OPEN_HEAP_DUMPS) } + + // What a command line needs of this end: a connection per call, and one file to read them in. A client + // that holds a connection open says nothing and gets a session of its own. See [AgentCommandLineTest]. + val session = sessions().single() + assertThat(session.sessionId).isEqualTo("cli7") + assertThat(session.calls.map { it.tool }).containsExactly("open_heap_dumps") + } + + @Test + fun `a session name that could be a path is not made into one`() { + listen() + val run = AgentServer.publishedRuns(directory).single() + + connect(run, sessionName = "../../evil").use { client -> + // Served, because the calls are none the worse for the name: what it loses is being gathered with the + // others, and refusing the connection would lose the investigation instead. + assertThat(client.accepted).isTrue() + client.ask(CALL_OPEN_HEAP_DUMPS) + } + + assertThat(sessions().single().sessionId).isNotEqualTo("../../evil") + assertThat(log).anyMatch { it.contains("is no session name") } + assertThat(temporaryFolder.root.walkTopDown().filter { it.name.endsWith(".jsonl") }.toList()) + .hasSize(1) + } + @Test fun `closing a run takes it off the list`() { val listening = listen() @@ -121,13 +154,19 @@ class AgentServerTest { private fun connect( run: AgentServer.PublishedRun, - token: String = run.token - ): TestClient = TestClient(run.port, token) + token: String = run.token, + sessionName: String? = null + ): TestClient = TestClient(run.port, token, sessionName) + + private fun sessions(): List = + AgentSessionFile.sessionsIn(AgentServer.sessionsDirectory(directory)) /** An agent's end of the connection, as far as this test needs one: a token, then a line at a time. */ private class TestClient( port: Int, - token: String + token: String, + /** The session this connection joins, which a command line names and a client holding one open doesn't. */ + sessionName: String? ) : Closeable { private val socket = Socket(InetAddress.getLoopbackAddress(), port) @@ -137,7 +176,7 @@ class AgentServerTest { val accepted: Boolean init { - toApp.println(token) + toApp.println(listOfNotNull(token, sessionName).joinToString(" ")) accepted = fromApp.readLine() == AgentServer.ACCEPTED } @@ -154,5 +193,9 @@ class AgentServerTest { private companion object { const val PING = """{"jsonrpc":"2.0","id":1,"method":"ping"}""" + + const val CALL_OPEN_HEAP_DUMPS = + """{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"open_heap_dumps",""" + + """"arguments":{"reason":"Finding out what is open."}}}""" } } diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt index 61ccc9e88b..fd32d2f7de 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt @@ -14,6 +14,7 @@ import shark.explorer.HeapExplorer import shark.explorer.LeakStatusOverride import shark.explorer.LeakStatusOverrides import shark.explorer.Place +import shark.explorer.agent.AgentCommandLine import shark.explorer.agent.AgentHeapDump import shark.explorer.agent.AgentHeapDumps import shark.explorer.agent.AgentRefusal @@ -189,7 +190,7 @@ internal fun agentBridgeExitCode(args: Array): Int? { return null } val arguments = try { - agentServerArguments(args) + windowArguments(args) } catch (invalidArguments: IllegalArgumentException) { // On stderr, where an MCP client collects a server's log, since there is no window and no console to // print a usage message to. @@ -212,7 +213,49 @@ internal fun agentBridgeExitCode(args: Array): Int? { } /** - * The rest of the command line, once the three options that make this a server are off it. + * Whether this process was started to make one tool call from a shell, and what to exit with if it was. + * + * The other adapter over the same tools: `--mcp-stdio` is a client holding a session open, and this is an + * agent — or a person — typing one command at a window that is already up. See [AgentCommandLine]. + * + * Answered before any logging is installed for the reason the pipe is: **stdout carries the answer**, and a + * log line in the middle of it is JSON that whatever ran this cannot parse. + */ +internal fun agentCommandExitCode(args: Array): Int? { + val helpIndex = args.indexOf(AgentCommandLine.HELP_OPTION) + val callIndex = args.indexOf(AgentCommandLine.AGENT_OPTION) + if (helpIndex < 0 && callIndex < 0) { + return null + } + if (helpIndex >= 0) { + // On stdout, since this is the whole of what the command was run for. No window, no heap dump and no + // waiting: the tools are text this build carries. + println(AgentCommandLine.help(command = commandToRunThis(), toolName = args.toolNameAt(helpIndex))) + return 0 + } + val toolName = args.toolNameAt(callIndex) + val arguments = try { + // Everything that isn't the call is the command line of the window this may have to open. + windowArguments(args, toolName) + } catch (invalidArguments: IllegalArgumentException) { + saidToTheClient(invalidArguments.message.orEmpty()) + return UNREADABLE_COMMAND_LINE + } + return AgentCommandLine.run( + directory = AGENT_RUNS_DIRECTORY, + words = listOfNotNull(toolName) + args.filter { AgentCommandLine.isCallArgument(it) }, + pid = args.optionValue(AgentStdioBridge.PID_OPTION), + sessionName = args.optionValue(AgentCommandLine.SESSION_OPTION) + ?: AgentCommandLine.defaultSessionName(), + // The same window a client that found nothing open gets, and here it is worth more: the next call from + // this shell finds that run published and talks to it, so one command line opening a window is what + // makes every command after it cheap. + openAWindow = relaunchCommand()?.let { command -> { openAnotherRun(command, arguments) } } + ) +} + +/** + * The rest of the command line, once the options that make this a server or a call are off it. * * Because what is left is an ordinary command line — heap dumps to open, a title to call their windows — and * it means the same thing: a client's configuration says which dump to investigate the way a terminal does. @@ -220,12 +263,44 @@ internal fun agentBridgeExitCode(args: Array): Int? { * * Throws [IllegalArgumentException] for a command line that doesn't read, like the parser it wraps. */ -internal fun agentServerArguments(args: Array): ExplorerArguments = ExplorerArguments.parse( - args.filterNot { - it == MCP_STDIO_OPTION || it == NO_UI_OPTION || it.startsWith(AgentStdioBridge.PID_OPTION) +internal fun windowArguments( + args: Array, + /** The one word of a call that isn't `name=value`, and null for a command line that is no call. */ + toolName: String? = null +): ExplorerArguments = ExplorerArguments.parse( + args.filterNot { word -> + word.isAgentOption() || AgentCommandLine.isCallArgument(word) || (toolName != null && word == toolName) } ) +/** + * The word naming a tool at [index] of the command line, which is the one after the option. + * + * Positional because a call reads as a command — `--agent describe_object object=0x7205` — and null for an + * option that was given nothing, which is `--agent-help` on its own. + */ +private fun Array.toolNameAt(index: Int): String? = + getOrNull(index + 1)?.takeIf { !it.startsWith("-") && !AgentCommandLine.isCallArgument(it) } + +private fun Array.optionValue(option: String): String? = + firstOrNull { it.startsWith(option) }?.removePrefix(option) + +/** What a word of the command line has to be to reach an agent rather than a window. */ +private fun String.isAgentOption(): Boolean = this == MCP_STDIO_OPTION || this == NO_UI_OPTION || + this == AgentCommandLine.AGENT_OPTION || this == AgentCommandLine.HELP_OPTION || + startsWith(AgentStdioBridge.PID_OPTION) || startsWith(AgentCommandLine.SESSION_OPTION) + +/** + * What to type to run this app, for the examples in the help. + * + * The launcher of a packaged install, which is a path somebody can copy — and the generic name for a run + * from source, where the real command line is a JVM and a classpath nobody wants printed at them. + */ +private fun commandToRunThis(): String { + val launcher = launcherPathOrNull() ?: return "shark-explorer" + return if (' ' in launcher) "\"$launcher\"" else launcher +} + /** * Answers an agent's calls from this process, with no window anywhere. * diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt index 095d294f34..b80169bbce 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt @@ -63,6 +63,9 @@ fun main(args: Array) { // installs no logging: stdout is the protocol in that mode, and one log line in the middle of it is a // session the agent's client reports as broken. See [AgentStdioBridge]. agentBridgeExitCode(args)?.let { exitProcess(it) } + // And a run asked to make one call from a shell prints the answer and ends, for the same reason: what is + // on stdout is what whoever typed it is reading. See [AgentCommandLine]. + agentCommandExitCode(args)?.let { exitProcess(it) } // Launched from a terminal, so Shark's own diagnostics and any failure to open a heap dump belong on // stdout as well as in the window — and in a file, so that a session someone reports on can be read // back after it. See [installLogging]. diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentCommandLineTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentCommandLineTest.kt deleted file mode 100644 index e5ed0335b3..0000000000 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentCommandLineTest.kt +++ /dev/null @@ -1,46 +0,0 @@ -package shark.explorer.app - -import org.assertj.core.api.Assertions.assertThat -import org.junit.Test -import shark.explorer.agent.AgentStdioBridge - -/** - * Which command lines mean "be an MCP server", answered before anything else in `main`. - * - * The two cases here are the ones that end without talking to anybody, and they are the only ones a test can - * drive: everything else about this either pipes stdio to a window or serves the tools until a client closes - * its end. `HeadlessAgentHeapDumpsTest` covers what it serves. - */ -class AgentCommandLineTest { - - @Test - fun `an ordinary command line is a window`() { - assertThat(agentBridgeExitCode(arrayOf("--title=Windowed", "dump.hprof"))).isNull() - // `--no-ui` on its own is not a way to run the app with no window: there would be nothing to run. - assertThat(agentBridgeExitCode(arrayOf(NO_UI_OPTION))).isNull() - } - - @Test - fun `a command line that does not read is a failure rather than a message`() { - // A client that launched this has nowhere to show a usage message, so the exit code is what says so. - assertThat(agentBridgeExitCode(arrayOf(MCP_STDIO_OPTION, NO_UI_OPTION, "--titel=Typo"))).isEqualTo(1) - } - - @Test - fun `what is left of a server's command line is a window's`() { - val arguments = agentServerArguments( - arrayOf( - MCP_STDIO_OPTION, - NO_UI_OPTION, - "${AgentStdioBridge.PID_OPTION}12345", - "--title=For an agent", - "dump.hprof" - ) - ) - - // The heap dump and the title survive, and the three server options are not taken for heap dumps: a - // window saying `--no-ui` could not be read is what that mistake looks like. - assertThat(arguments.heapDumpFiles.map { it.name }).containsExactly("dump.hprof") - assertThat(arguments.titlePrefix).isEqualTo("For an agent") - } -} diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentOptionsTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentOptionsTest.kt new file mode 100644 index 0000000000..6f0cdb7e5a --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentOptionsTest.kt @@ -0,0 +1,118 @@ +package shark.explorer.app + +import java.io.ByteArrayOutputStream +import java.io.PrintStream +import org.assertj.core.api.Assertions.assertThat +import org.junit.Test +import shark.explorer.agent.AgentCommandLine +import shark.explorer.agent.AgentStdioBridge + +/** + * Which command lines reach an agent rather than a window, answered before anything else in `main`. + * + * The cases here are the ones that end without talking to anybody, and they are the only ones a test can + * drive: everything else either pipes stdio to a window, serves the tools until a client closes its end, or + * calls a run of the app that whoever is running the tests may have open. `HeadlessAgentHeapDumpsTest` covers + * what is served, and `AgentCommandLineTest` in `shark-explorer-agent` covers a call over a real socket. + * + * What is worth pinning here is the **split**: a command line carries both a call and the window that may + * have to be opened to answer it, and mistaking one for the other means an argument opened as a heap dump. + */ +class AgentOptionsTest { + + @Test + fun `an ordinary command line is a window`() { + assertThat(agentBridgeExitCode(arrayOf("--title=Windowed", "dump.hprof"))).isNull() + // `--no-ui` on its own is not a way to run the app with no window: there would be nothing to run. + assertThat(agentBridgeExitCode(arrayOf(NO_UI_OPTION))).isNull() + assertThat(agentCommandExitCode(arrayOf("--title=Windowed", "dump.hprof"))).isNull() + } + + @Test + fun `a command line that does not read is a failure rather than a message`() { + // A client that launched this has nowhere to show a usage message, so the exit code is what says so. + assertThat(agentBridgeExitCode(arrayOf(MCP_STDIO_OPTION, NO_UI_OPTION, "--titel=Typo"))).isEqualTo(1) + } + + @Test + fun `what is left of a server's command line is a window's`() { + val arguments = windowArguments( + arrayOf( + MCP_STDIO_OPTION, + NO_UI_OPTION, + "${AgentStdioBridge.PID_OPTION}12345", + "--title=For an agent", + "dump.hprof" + ) + ) + + // The heap dump and the title survive, and the three server options are not taken for heap dumps: a + // window saying `--no-ui` could not be read is what that mistake looks like. + assertThat(arguments.heapDumpFiles.map { it.name }).containsExactly("dump.hprof") + assertThat(arguments.titlePrefix).isEqualTo("For an agent") + } + + @Test + fun `what is left of a call's command line is a window's too`() { + val arguments = windowArguments( + arrayOf( + AgentCommandLine.AGENT_OPTION, + "describe_object", + "object=0x7205", + "reason=Reading the holder's fields.", + "${AgentCommandLine.SESSION_OPTION}cli99", + "--title=For an agent", + "dump.hprof" + ), + toolName = "describe_object" + ) + + // The tool and its arguments are the call, and what remains is the window this would open to answer it — + // which is the same window a command line with no call in it would have opened. + assertThat(arguments.heapDumpFiles.map { it.name }).containsExactly("dump.hprof") + assertThat(arguments.titlePrefix).isEqualTo("For an agent") + } + + @Test + fun `a call with no tool named says so rather than calling something`() { + val exitCode = onItsOwnStreams { agentCommandExitCode(arrayOf(AgentCommandLine.AGENT_OPTION)) } + + assertThat(exitCode).isEqualTo(AgentCommandLine.NOTHING_ANSWERED) + } + + @Test + fun `the help is printed, and needs nothing open`() { + val printed = ByteArrayOutputStream() + + val exitCode = onItsOwnStreams(printed) { + agentCommandExitCode(arrayOf(AgentCommandLine.HELP_OPTION, "conclude")) + } + + assertThat(exitCode).isZero + // The tool asked about, and not the fifteen others: reading a surface a piece at a time is what naming + // one is for. + assertThat(printed.toString(Charsets.UTF_8.name())).contains("conclude").doesNotContain("list_leaks") + } + + /** + * Runs [block] with stdout and stderr taken over, since these two paths write to both. + * + * A test that let them through would put the help of sixteen tools in the middle of the test report, and + * the messages beside it read as failures of whatever ran next. + */ + private fun onItsOwnStreams( + printed: ByteArrayOutputStream = ByteArrayOutputStream(), + block: () -> Int? + ): Int? { + val previousOut = System.out + val previousErr = System.err + System.setOut(PrintStream(printed, true, Charsets.UTF_8.name())) + System.setErr(PrintStream(ByteArrayOutputStream(), true, Charsets.UTF_8.name())) + return try { + block() + } finally { + System.setOut(previousOut) + System.setErr(previousErr) + } + } +} From f18da6c496e0288604f96349bf35ae51a4f801a5 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Wed, 26 Aug 2026 08:59:06 +0200 Subject: [PATCH 20/27] Answer an agent with what has already been tried on this heap dump MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The *Agent logs* screen had no tool behind it, which broke the rule that every screen has an agent equivalent — and it is one of the more useful screens to have: an investigation somebody already ran is either the answer or the half of the dump not worth doing again. `agent_log` lists the sessions that read this dump, newest first, with what each concluded and how many calls were refused; with `session=`, every call of one in order with the reason the agent gave. Refused by name for a session that read another dump, because an address is an address of one dump: reading it here would be rows meaning other objects. Which is the rule the screen groups by, so both readers of a session file agree. `dump_heap` said the garbage was collected "where the device is new enough", which was vague and wrong: it is collected either way — `am dumpheap -g` from API 27, and `JdwpGc` running the same collection in the process below that. Detekt's LargeClass now excludes tests, the way TooManyFunctions already does, and allows 700: a test class is as long as the story it walks, and AgentTools is a registry whose length is tool descriptions rather than behaviour. Co-Authored-By: Claude Opus 5 --- config/detekt-config.yml | 6 +- shark/shark-explorer/notes/agent-surface.md | 20 +-- .../shark/explorer/agent/AgentCommandLine.kt | 13 +- .../java/shark/explorer/agent/AgentJson.kt | 45 ++++++ .../java/shark/explorer/agent/AgentServer.kt | 8 +- .../shark/explorer/agent/AgentSessionFile.kt | 4 + .../shark/explorer/agent/AgentStdioServer.kt | 6 +- .../java/shark/explorer/agent/AgentTools.kt | 66 ++++++++- .../explorer/agent/AgentCommandLineTest.kt | 2 +- .../explorer/agent/AgentSessionFileTest.kt | 2 +- .../shark/explorer/agent/AgentToolsTest.kt | 129 ++++++++++++++++-- .../shark/explorer/agent/FakeAgentHeapDump.kt | 11 ++ .../shark/explorer/agent/McpSessionTest.kt | 3 +- 13 files changed, 282 insertions(+), 33 deletions(-) diff --git a/config/detekt-config.yml b/config/detekt-config.yml index f62e45f261..98e78437c2 100644 --- a/config/detekt-config.yml +++ b/config/detekt-config.yml @@ -62,8 +62,12 @@ complexity: active: false ignoredLabels: "" LargeClass: + #LeakCanary - excluded tests, and increased from 600 to 700: a test class is as long as the story it + # walks, which is why TooManyFunctions already excludes them, and shark-explorer-agent's AgentTools is a + # registry whose length is tool descriptions rather than behaviour. active: true - threshold: 600 + excludes: "**/test/**,**/androidTest/**,**/*.Test.kt,**/*.Spec.kt,**/*.Spek.kt" + threshold: 700 # Leave me alone! # LongMethod: # #LeakCanary - increased from 60 to 90 diff --git a/shark/shark-explorer/notes/agent-surface.md b/shark/shark-explorer/notes/agent-surface.md index c08463d276..10bd1fa65f 100644 --- a/shark/shark-explorer/notes/agent-surface.md +++ b/shark/shark-explorer/notes/agent-surface.md @@ -9,13 +9,15 @@ Measured off `AgentTools.all` and `AgentMethod.INSTRUCTIONS`, one `tools/list` e | | Characters | ≈ tokens | Paid | | --- | --- | --- | --- | -| Sixteen tool definitions | 18,779 | 4,695 | Every turn, while the server is connected | +| Seventeen tool definitions | 20,938 | 5,235 | Every turn, while the server is connected | | The method | 7,845 | 1,960 | Handshake, and again with `open_heap_dumps` | -So the standing cost of this surface is **6 to 7 k tokens**, around 3% of a 200 k window. Parity took the -tool count from eleven to sixteen and the definitions from 13,116 characters to 18,779 — **a fifth of the -window's budget for the five tools that mean an agent never has to ask its human to click something**, which -is the trade this surface exists to make. The method then grew by half again for the section on reading the +So the standing cost of this surface is **7 to 8 k tokens**, around 3.5% of a 200 k window. Parity took the +tool count from eleven to seventeen and the definitions from 13,116 characters to 20,938 — **a fifth of the +window's budget for the six tools that mean an agent never has to ask its human to click something**, which +is the trade this surface exists to make. The sixth is `agent_log`, 1,237 characters of the total, and the +900 the other sixteen grew by are the two agent-log places added to the sentence naming every place, which +`show`, `read_notes` and `take_note` all repeat. The method then grew by half again for the section on reading the code at the version the dump is of, which is the one part of the method the tools cannot enforce at all and the part that decides whether an answer is a root cause or a reference. The published horror stories are still an order of magnitude worse: GitHub's server is ~17.6 k tokens of definitions, and three servers together have been measured at 143 k. The @@ -31,11 +33,13 @@ Measured against a packaged build with one window open on `leak_asynctask_o.hpro | | Measured | Paid | | --- | --- | --- | | One call, JVM start to JSON on stdout | 160–180 ms | Per call | -| `--agent-help`, all sixteen tools | 13,391 characters, ≈3,350 tokens | Only when read | -| `--agent-help `, one of them | ~1,200 characters, ≈300 tokens | Only when read | +| `--agent-help`, all seventeen tools | 14,414 characters, ≈3,600 tokens | Only when read | +| `--agent-help `, one of them | 500–1,250 characters, ≈125–310 tokens | Only when read | So the standing cost is nothing, and the whole surface as text is *smaller* than the `tools/list` definitions -of it (13,391 against 18,779) because `reason` is explained once rather than sixteen times. +of it (14,414 against 20,938) because `reason` is explained once rather than seventeen times. Both +`--agent-help` figures include the invocation path twice, since what it prints is the command to type on this +machine; a shorter install path is a slightly shorter help. **A call from a shell is not a slower call.** It reaches the same window over the loopback socket the run already publishes, so the heap dump is the one that was parsed and indexed once and the read queues on that diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt index 74d757635b..0c861e446c 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt @@ -281,7 +281,7 @@ object AgentCommandLine { } /** The tools of this build, described. Built per call, so nothing here is shared between threads. */ - private fun described(): List = AgentTools(NoHeapDumpToDescribe).all + private fun described(): List = AgentTools(NoHeapDumpToDescribe) { nothingToDescribeWith() }.all private fun initializeParameters(): JsonObject = buildJsonObject { put("protocolVersion", PROTOCOL_VERSION) @@ -470,12 +470,15 @@ private object NoHeapDumpToDescribe : AgentHeapDumps { processName: String ): AgentHeapDump = nothing() - private fun nothing(): Nothing = throw IllegalStateException( - "These tools are only being described, so there is no heap dump here and nothing to call: a call goes " + - "to the run of the app that has one open. See AgentCommandLine." - ) + private fun nothing(): Nothing = nothingToDescribeWith() } +/** The same for the sessions the log tool reads, which a build being described has no directory for. */ +private fun nothingToDescribeWith(): Nothing = throw IllegalStateException( + "These tools are only being described, so there is no heap dump here and nothing to call: a call goes to " + + "the run of the app that has one open. See AgentCommandLine." +) + /** Said once in the preamble rather than under every tool. See [AgentTool]. */ private const val REASON_ARGUMENT = "reason" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt index 616a32fa57..ed2fa6b1ab 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt @@ -74,6 +74,51 @@ internal object AgentJson { put("verdictsSetByHand", verdicts(verdicts)) } + /** + * One investigation somebody else already ran: who was working, how it went, and what it came to. + * + * The numbers the *Agent logs* screen shows in the same order, because they are what makes a session worth + * opening or not: how much was asked, how much of it was refused, and whether it ended in a conclusion. + */ + fun agentSession(session: AgentSession): JsonObject = buildJsonObject { + put("session", session.sessionId) + put("client", session.client) + put("startedAt", session.startedAt?.toString()) + put("calls", session.calls.size) + put("refused", session.refusedCount) + // What it concluded, which is the one thing a reader is looking for — and null for a session that + // concluded nothing, which is most of them. + put("concluded", session.calls.mapNotNull { it.outcome }.lastOrNull()) + putJsonArray("heapDumps") { session.heapDumpPaths.forEach { add(it) } } + } + + /** + * Every call of one session, in the order it made them, with the reason the agent gave for each. + * + * The reasons are the point. A session read as a list of tool names is the protocol showing through; read + * as what was asked and why, it either follows from itself or doesn't — which is the same judgement the + * person at the window makes on that screen. + */ + fun agentSessionCalls(session: AgentSession): JsonObject = buildJsonObject { + put("session", session.sessionId) + put("client", session.client) + putJsonArray("calls") { + session.calls.forEach { call -> + addJsonObject { + put("at", call.at.toString()) + put("tool", call.tool) + put("reason", call.reason) + // What the call was about, as the agent wrote it: an address is that dump's address, and this is + // read by something that can resolve it. + put("about", call.subject) + put("heapDumpPath", call.heapDumpPath) + put("refused", call.refusal) + put("outcome", call.outcome) + } + } + } + } + /** * Every verdict set by hand, so that an agent arriving at a window someone has been working in reads the * conclusions already reached rather than starting over on top of them. diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt index 694d105c80..8e9082a92e 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentServer.kt @@ -189,7 +189,13 @@ object AgentServer { // handshake, since a client that connects and says nothing is itself worth a line on that screen. val sessionFile = sessionFile(sessions, serverVersion, handshake.getOrNull(1)) SharkLog.d { "An agent's session is being written to ${sessionFile.file}" } - val session = McpSession(AgentTools(heapDumps), serverVersion, sessionFile) + val session = McpSession( + // Read off disk per call rather than captured, so that an agent asking what has been done to a + // heap dump sees what another one working on it right now has done so far. + AgentTools(heapDumps) { AgentSessionFile.sessionsIn(sessions) }, + serverVersion, + sessionFile + ) while (true) { val line = reader.readLine() ?: break if (line.isBlank()) { diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt index dbca9e72b7..262e9bbfb2 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt @@ -467,6 +467,7 @@ val AgentSessionCall.verb: String get() = verbOfTool(tool, arguments) ?: tool.re */ val AgentSessionCall.subject: String? get() = arguments[SUBJECT_OBJECT] ?: arguments[SUBJECT_PLACE] ?: arguments[SUBJECT_CLASS_NAME] + ?: arguments[SUBJECT_SESSION] /** * What the answer to a call came to, as a couple of words, and null when the answer is data rather than a @@ -507,6 +508,8 @@ internal fun verbOfTool( // Worth the difference on the screen: a note replaced is a paragraph that was there and isn't any more, // which is the one thing an agent does here that a reader can't get back. "take_note" -> if (arguments[SUBJECT_REPLACE] == "true") "Rewrote the note on" else "Wrote a note on" + // Reading what other agents did, which is the one call whose subject is another session of this screen. + "agent_log" -> if (SUBJECT_SESSION in arguments) "Read what an agent did in" else "Read the agent log" "show" -> "Showed" "conclude" -> "Concluded about" // The app rather than a heap dump, so each of these says the whole of what it did: there is no subject @@ -531,3 +534,4 @@ private const val SUBJECT_REPLACE = "replace" private const val SUBJECT_PATH = "path" private const val SUBJECT_DEVICE = "device" private const val SUBJECT_PROCESS = "process" +private const val SUBJECT_SESSION = "session" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioServer.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioServer.kt index b7e8daa0e7..098f8226b1 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioServer.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentStdioServer.kt @@ -40,7 +40,11 @@ object AgentStdioServer { // Named before the handshake, like a socket session is, so that a client which connects and says nothing // is still a row on the *Agent logs* screen of whoever reads these later. val sessionFile = AgentSessionFile.starting(sessions, serverVersion) - val session = McpSession(AgentTools(heapDumps), serverVersion, sessionFile) + val session = McpSession( + AgentTools(heapDumps) { AgentSessionFile.sessionsIn(sessions) }, + serverVersion, + sessionFile + ) val reader = BufferedReader(InputStreamReader(System.`in`, Charsets.UTF_8)) val writer = PrintWriter(OutputStreamWriter(System.out, Charsets.UTF_8), true) while (true) { diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index f9c2bf51f5..8fb189738c 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -42,12 +42,23 @@ import shark.explorer.outlineOf * not a quality gate — asking a model to explain itself does not make it right — but it is what turns a run * into something a person can follow afterwards instead of a conclusion they have to take on trust. */ -internal class AgentTools(private val heapDumps: AgentHeapDumps) { +internal class AgentTools( + private val heapDumps: AgentHeapDumps, + /** + * Every session this machine has a record of, for [AGENT_LOG] — the sessions of other runs of the app + * included, since what has been tried on a heap dump outlives the run it was tried in. + * + * A function rather than a list, because it is read off disk per call: an agent asking what has been done + * to this dump while another one is working on it has to see what that one has done so far. + */ + private val recordedSessions: () -> List +) { /** In the order an investigation uses them, which is the order a client lists them in. */ val all: List = listOf( openHeapDumps(), listLeaks(), + agentLog(), describeObject(), chainFromGcRoot(), waysHeld(), @@ -114,6 +125,45 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { AgentJson.leaks(leaks) } + private fun agentLog() = AgentTool( + name = AGENT_LOG, + description = "What has already been done to this heap dump, by you and by anybody else: one entry per " + + "session, newest first, with what it concluded and how many of its calls were refused — and with " + + "`$SESSION`, every call of one session in order, with the reason the agent gave for each. The " + + "window's *Agent logs* screen, which is where a person reads the same thing. Worth reading before " + + "starting: an investigation somebody already ran is either the answer or the half of the dump not " + + "worth doing again. Sessions of earlier runs of the app are in it, and so is this one.", + schema = schema( + WINDOW to window(), + SESSION to string("Optional: one session's id, from the list, to read every call it made.").optional() + ) + ) { arguments -> + val dump = arguments.heapDump() + val sessionId = arguments.optionalString(SESSION) + // The sessions about this heap dump, because a session is only readable against the dump it read: an + // address in another one is another object. Which is the same rule the screen groups by. + val sessions = recordedSessions().filter { dump.heapDumpPath in it.heapDumpPaths } + if (sessionId == null) { + return@AgentTool buildJsonObject { + put("heapDumpPath", dump.heapDumpPath) + putJsonArray("sessions") { sessions.forEach { add(AgentJson.agentSession(it)) } } + if (sessions.isEmpty()) { + put("problem", "Nothing has been done to this heap dump through Shark Explorer yet, so there is " + + "nothing to read. What you do now is what the next reader of this will find.") + } + } + } + val session = sessions.firstOrNull { it.sessionId == sessionId } + ?: throw AgentRefusal( + "No session called \"$sessionId\" read this heap dump. " + if (sessions.isEmpty()) { + "None has: this dump has no agent log yet." + } else { + "The ones that did are " + sessions.joinToString(", ") { it.sessionId } + "." + } + ) + AgentJson.agentSessionCalls(session) + } + private fun describeObject() = AgentTool( name = "describe_object", description = "What one object is: its class, what the inspectors made of it, its verdict and the " + @@ -565,8 +615,9 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { description = "Takes a heap dump of a running process, opens it in a window of Shark Explorer and " + "answers once it can be read — the whole of what the window's `Take heap dump` button does. " + "**Minutes, on a large app**: the device writes the dump, it is pulled over `adb`, and then opened. " + - "The garbage is collected first where the device is new enough, so what is in the dump is what is " + - "really still held. Ask list_devices first for the device and the process.", + "The garbage is collected first either way, so what is in the dump is what is really still held — " + + "with `am dumpheap -g` from API 27, and below that by running the same collection in the process " + + "over JDWP. Ask list_devices first for the device and the process.", schema = schema( DEVICE to string("The serial number of the device, from list_devices."), PROCESS to string("The name of the process to dump, from list_devices.") @@ -617,14 +668,17 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { * Which place of the heap dump a call is about, from what it was given rather than from which tool it is. * * By argument name, so that a tool added here is described by this without being listed in it: everything - * about an object takes `object`, everything about a place takes `place`, and the search takes a class - * name. The one tool whose subject is in neither is the list of leaks, which takes nothing at all. + * about an object takes `object`, everything about a place takes `place`, the search takes a class name and + * one session of the log takes its id. The two tools whose subject is in none of them name a screen and + * take nothing at all — the leaks, and the log read as a list. */ private fun AgentArguments.placeOrNull(name: String): Place? = when { optionalString(PLACE) != null -> place() optionalString(OBJECT) != null -> Place.Object(objectId(OBJECT)) optionalString(CLASS_NAME) != null -> Place.Objects(ObjectListFilter(query = string(CLASS_NAME))) + optionalString(SESSION) != null -> Place.AgentLog(string(SESSION)) name == LIST_LEAKS -> Place.Leaks() + name == AGENT_LOG -> Place.AgentLogs else -> null } @@ -714,8 +768,10 @@ internal class AgentTools(private val heapDumps: AgentHeapDumps) { /** Named because [placeOrNull] is the one description of a call that has to know which tool it is. */ const val LIST_LEAKS = "list_leaks" + const val AGENT_LOG = "agent_log" const val WINDOW = "window" + const val SESSION = "session" const val OBJECT = "object" const val FROM = "from" const val CLASS_NAME = "className" diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentCommandLineTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentCommandLineTest.kt index f7098b4273..9d039fc29d 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentCommandLineTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentCommandLineTest.kt @@ -163,7 +163,7 @@ class AgentCommandLineTest { // Generated from the registry, so a tool added without being described here is a test failure rather // than a tool an agent using the command line never hears about. - AgentTools(FakeAgentHeapDumps()).all.forEach { tool -> + agentTools(FakeAgentHeapDumps()).all.forEach { tool -> assertThat(help).contains(tool.name).contains(tool.description) } // The one thing the schema doesn't say, because JSON has brackets and a command line hasn't. diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt index 5070eeb365..e8a35ddf6b 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt @@ -142,7 +142,7 @@ class AgentSessionFileTest { @Test fun `every tool has a verb, so that no screen ends up showing the protocol`() { - val withoutAVerb = AgentTools(FakeAgentHeapDumps()).all + val withoutAVerb = agentTools(FakeAgentHeapDumps()).all .map { it.name } .filter { verbOfTool(it, emptyMap()) == null } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index 211a2e1bfa..3f035b7b88 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -1,5 +1,6 @@ package shark.explorer.agent +import java.time.Instant import kotlinx.coroutines.runBlocking import kotlinx.serialization.json.JsonArray import kotlinx.serialization.json.JsonNull @@ -47,7 +48,7 @@ class AgentToolsTest { fun setUp() { heapDump = temporaryFolder.applicationHoldsActivityThroughHolder() window = FakeAgentHeapDump(heapDump.explorer) - tools = AgentTools(FakeAgentHeapDumps(listOf(window))) + tools = agentTools(FakeAgentHeapDumps(listOf(window))) } @After @@ -78,7 +79,7 @@ class AgentToolsTest { @Test fun `a run with no heap dump open says so rather than answering`() { - tools = AgentTools(FakeAgentHeapDumps()) + tools = agentTools(FakeAgentHeapDumps()) assertThat(call(OPEN_HEAP_DUMPS).text("problem")).contains("No heap dump is open") assertThatThrownBy { call("list_leaks") } @@ -88,7 +89,7 @@ class AgentToolsTest { @Test fun `a heap dump still being indexed is named rather than left to be guessed`() { - tools = AgentTools(FakeAgentHeapDumps(indexing = listOf("/eval/runs/4/heap-dump.hprof"))) + tools = agentTools(FakeAgentHeapDumps(indexing = listOf("/eval/runs/4/heap-dump.hprof"))) val answer = call(OPEN_HEAP_DUMPS) @@ -99,6 +100,63 @@ class AgentToolsTest { .contains(OPEN_HEAP_DUMP) } + @Test + fun `the agent log is what has already been tried on this heap dump`() { + tools = agentTools( + FakeAgentHeapDumps(listOf(window)), + sessions = listOf( + recordedSession("cli7", concluded = "Holder.activity"), + // Another dump's investigation, which is another dump's addresses: not this window's to answer with. + recordedSession("cli8", heapDumpPath = "/dumps/another.hprof") + ) + ) + + val answer = call(AGENT_LOG) + + val sessions = answer.array("sessions").map { it.jsonObject } + assertThat(sessions.map { it.text("session") }).containsExactly("cli7") + // The one field a reader is looking for, and the one neither this screen nor the eval can work out for + // itself: what the investigation came to. + assertThat(sessions.single().text("concluded")).isEqualTo("Holder.activity") + assertThat(sessions.single().text("refused")).isEqualTo("1") + } + + @Test + fun `one session of the log is every call it made, with the reasons`() { + tools = agentTools( + FakeAgentHeapDumps(listOf(window)), + sessions = listOf(recordedSession("cli7", concluded = "Holder.activity")) + ) + + val calls = call(AGENT_LOG, "session" to "cli7").array("calls").map { it.jsonObject } + + // The reasons are the point: a session read as a list of tool names is the protocol showing through. + assertThat(calls.map { it.text("tool") }).containsExactly("list_leaks", "conclude") + assertThat(calls.first().text("reason")).isEqualTo("Reading what the dump says about itself.") + assertThat(calls.last().text("outcome")).isEqualTo("Holder.activity") + assertThat(calls.first().text("refused")).contains("needs `reason`") + } + + @Test + fun `a session that read another heap dump is refused by name here`() { + tools = agentTools( + FakeAgentHeapDumps(listOf(window)), + sessions = listOf(recordedSession("cli7"), recordedSession("cli8", "/dumps/another.hprof")) + ) + + assertThatThrownBy { call(AGENT_LOG, "session" to "cli8") } + .isInstanceOf(AgentRefusal::class.java) + // Named, and the ones that did read this dump listed: an address of another dump is another object, so + // reading that session here would be a screen of rows meaning something else. + .hasMessageContaining("No session called \"cli8\" read this heap dump") + .hasMessageContaining("cli7") + } + + @Test + fun `a heap dump nothing has been done to says so`() { + assertThat(call(AGENT_LOG).text("problem")).contains("Nothing has been done to this heap dump") + } + @Test fun `nothing is said to be indexing when nothing is`() { assertThat(call(OPEN_HEAP_DUMPS).jsonObject.keys).doesNotContain("indexing") @@ -107,7 +165,7 @@ class AgentToolsTest { @Test fun `two heap dumps open have to be named`() { val other = FakeAgentHeapDump(heapDump.explorer, windowId = "otherwindow") - tools = AgentTools(FakeAgentHeapDumps(listOf(window, other))) + tools = agentTools(FakeAgentHeapDumps(listOf(window, other))) assertThatThrownBy { call("list_leaks") } .isInstanceOf(AgentRefusal::class.java) @@ -592,7 +650,7 @@ class AgentToolsTest { fun `a heap dump nobody has open can be opened by its path`() { val other = FakeAgentHeapDump(heapDump.explorer, windowId = "openedwindow") val heapDumps = FakeAgentHeapDumps(listOf(window), opens = { other }) - tools = AgentTools(heapDumps) + tools = agentTools(heapDumps) val answer = call("open_heap_dump", "path" to heapDump.explorer.heapDumpFile.absolutePath) @@ -604,7 +662,7 @@ class AgentToolsTest { @Test fun `a path with no file at it is refused before anything is opened`() { val heapDumps = FakeAgentHeapDumps(listOf(window)) - tools = AgentTools(heapDumps) + tools = agentTools(heapDumps) assertThatThrownBy { call("open_heap_dump", "path" to "/no/such/dump.hprof") } .isInstanceOf(AgentRefusal::class.java) @@ -624,7 +682,7 @@ class AgentToolsTest { isDebuggableBuild = true ) val process = DeviceProcess(processId = 4231, name = "com.example.app") - tools = AgentTools(FakeAgentHeapDumps(listOf(window), devices = mapOf(device to listOf(process)))) + tools = agentTools(FakeAgentHeapDumps(listOf(window), devices = mapOf(device to listOf(process)))) val devices = call("list_devices").array("devices").map { it.jsonObject } assertThat(devices.single().text("device")).isEqualTo("emulator-5554") @@ -638,7 +696,7 @@ class AgentToolsTest { @Test fun `a machine with nothing plugged in says so rather than answering with an empty list`() { - tools = AgentTools(FakeAgentHeapDumps(listOf(window))) + tools = agentTools(FakeAgentHeapDumps(listOf(window))) assertThat(call("list_devices").text("problem")).contains("connected to no device") } @@ -660,7 +718,7 @@ class AgentToolsTest { devices = mapOf(device to listOf(process)), opens = { dumped } ) - tools = AgentTools(heapDumps) + tools = agentTools(heapDumps) val answer = call("dump_heap", "device" to "emulator-5554", "process" to "com.example.app") @@ -690,6 +748,58 @@ class AgentToolsTest { ) } + /** + * A session somebody else already ran on a heap dump: two calls, one of them refused. + * + * Written by hand rather than by running the tools, because what `agent_log` answers with is what + * [AgentSessionFile] read back off disk — and how a call becomes a line of that file is + * [AgentSessionFileTest]'s. + */ + private fun recordedSession( + sessionId: String, + heapDumpPath: String = heapDump.explorer.heapDumpFile.absolutePath, + concluded: String? = null + ) = AgentSession( + sessionId = sessionId, + startedAt = Instant.parse("2026-08-26T09:15:00Z"), + client = "claude-code 9.9.9", + serverVersion = "1.2.3", + file = temporaryFolder.newFile("agent-$sessionId.jsonl"), + calls = listOf( + recordedCall( + tool = "list_leaks", + heapDumpPath = heapDumpPath, + reason = "Reading what the dump says about itself.", + refusal = "list_leaks needs `reason`, and it was not given." + ), + recordedCall( + tool = "conclude", + heapDumpPath = heapDumpPath, + reason = "Naming the reference the chain agrees on.", + outcome = concluded + ) + ) + ) + + private fun recordedCall( + tool: String, + heapDumpPath: String, + reason: String, + refusal: String? = null, + outcome: String? = null + ) = AgentSessionCall( + at = Instant.parse("2026-08-26T09:15:01Z"), + tool = tool, + reason = reason, + windowId = window.windowId, + heapDumpPath = heapDumpPath, + place = null, + arguments = emptyMap(), + refusal = refusal, + outcome = outcome, + millis = 3L + ) + private fun call( name: String, vararg arguments: Pair @@ -717,6 +827,7 @@ class AgentToolsTest { const val OPEN_HEAP_DUMPS = "open_heap_dumps" const val OPEN_HEAP_DUMP = "open_heap_dump" + const val AGENT_LOG = "agent_log" const val SET_VERDICT = "set_verdict" const val CONCLUDE = "conclude" const val OBJECT = "object" diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt index bc58d18cd3..b2fc4da080 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt @@ -139,3 +139,14 @@ internal class FakeAgentHeapDumps( return opens(File("$processName.hprof")) } } + +/** + * The registry over [heapDumps], with [sessions] as everything agents have recorded on this machine. + * + * Sessions are what `agent_log` answers with and nothing else here reads, so a test about any other tool + * says nothing about them — which is a run that has recorded none, not a run whose log is unreadable. + */ +internal fun agentTools( + heapDumps: AgentHeapDumps, + sessions: List = emptyList() +) = AgentTools(heapDumps) { sessions } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index 845a6e4c4a..1dcbebcb8c 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -43,7 +43,7 @@ class McpSessionTest { window = FakeAgentHeapDump(heapDump.explorer) sessionsDirectory = File(temporaryFolder.root, "sessions") session = McpSession( - tools = AgentTools(FakeAgentHeapDumps(listOf(window))), + tools = agentTools(FakeAgentHeapDumps(listOf(window))), serverVersion = SERVER_VERSION, sessionFile = AgentSessionFile.starting(sessionsDirectory, SERVER_VERSION) ) @@ -83,6 +83,7 @@ class McpSessionTest { assertThat(tools.map { it.text("name") }).containsExactly( "open_heap_dumps", "list_leaks", + "agent_log", "describe_object", "chain_from_gc_root", "ways_held", From 5faeb13a35fbf6f97322a756087f5971dc7ae53a Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Wed, 26 Aug 2026 08:59:14 +0200 Subject: [PATCH 21/27] Head each heap dump's agent logs with the file, this window's first MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The screen listed this window's sessions and then "Other heap dumps", which buried the one thing a reader needs to know about a session: which dump its addresses belong to. Now it is "Agent log" over one group per heap dump, headed with the file name — numbered when two dumps share one — and this window's group first, saying so. A row of a session was clickable end to end, so clicking the word "Described" navigated. Only the object leads anywhere now, unless the call named none, where the verb is the whole sentence and is the link itself. Which also fixes "Listed the leaks Leaks": the window named a place derived from the tool rather than from an argument, so the leaks screen's title landed after a verb that already said it. It only names what the call itself said it was about, which is one fewer heap dump read as well. Co-Authored-By: Claude Opus 5 --- .../shark/explorer/app/AgentLogsScreen.kt | 234 +++++++++++------- .../shark/explorer/app/HeapDumpExplorer.kt | 4 + .../shark/explorer/app/AgentLogsScreenTest.kt | 104 ++++++-- 3 files changed, 233 insertions(+), 109 deletions(-) diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt index 5d7fbf18b1..10e4390b6d 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt @@ -2,6 +2,7 @@ package shark.explorer.app import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.FlowRow import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.width @@ -27,24 +28,23 @@ import shark.explorer.agent.subject import shark.explorer.agent.verb /** - * Every agent that has worked on **this** heap dump, one row each, and a way to the ones that worked on - * another. + * Every agent that has worked through this app, under the heap dump it worked on, this window's first. * * Because an agent works in this window: it reads the dump the person at the machine is reading, sets the * verdicts they see and writes into the same notes. So what it did has to be here, in words, rather than in * a JSON stream a client happens to have kept — and a row of it has to lead where it went, which is what * makes the two of them one investigation instead of two. * - * Per heap dump, like the notes and the verdicts, because a window is a heap dump: a session listed in the - * wrong window is one whose addresses mean nothing here. The sessions that read other dumps are still worth - * reaching from here — an agent is usually handed a dump nobody has open yet — and each of those is opened - * in a window of *its* dump rather than read in this one. There is no window that is not a heap dump for - * them to be listed in on their own. + * **Grouped by heap dump, because a session only means anything against one.** An address is an address of + * one dump, so a session read in the wrong window is a screen of rows naming other objects — which is why a + * group that isn't this window's opens in a window of *its* dump instead of being read here. This window's + * group comes first and says so; the dump an agent was handed is usually one nobody has open, so the rest are + * as much of the screen as it is. */ @Composable internal fun AgentLogsScreen( sessions: List, - /** Which heap dump this window has open, which is what decides which sessions are this window's. */ + /** Which heap dump this window has open, which is the group that is read here rather than opened. */ heapDumpFile: File, onOpen: (Place, OpenIn) -> Unit, onCopyLink: (Place) -> Unit, @@ -59,54 +59,58 @@ internal fun AgentLogsScreen( }, modifier: Modifier = Modifier ) { - val here = sessions.filter { heapDumpFile.absolutePath in it.heapDumpPaths } - // Which leaves a session that read no heap dump at all — a client that connected and asked nothing — with - // the ones about other dumps, since it is not about this one either. - val elsewhere = sessions - here.toSet() + val groups = sessions.byHeapDump(heapDumpFile) Surface(modifier, color = MaterialTheme.colorScheme.surface) { Column( Modifier.verticalScroll(rememberScrollState()).padding(16.dp), verticalArrangement = Arrangement.spacedBy(8.dp) ) { Text(Place.AGENT_LOGS_LABEL, style = MaterialTheme.typography.titleMedium) - if (here.isEmpty()) { - Text(NO_SESSIONS, style = MaterialTheme.typography.bodyMedium) - } - here.forEach { session -> - val place = Place.AgentLog(session.sessionId) - val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } - OpenTarget(open, { onCopyLink(place) }) { - Column(Modifier.openable(open)) { - Text(session.title(), style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) - Text(session.summary(), style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) - } + groups.forEachIndexed { index, group -> + if (index > 0) { + HorizontalDivider() + } + Text(group.label, style = MaterialTheme.typography.titleSmall) + if (group.sessions.isEmpty()) { + Text(NO_SESSIONS, style = MaterialTheme.typography.bodyMedium) + } + group.sessions.forEach { session -> + SessionRow(session, group, onOpen, onCopyLink, onOpenHeapDump) } - } - if (elsewhere.isNotEmpty()) { - HorizontalDivider() - Text(OTHER_HEAP_DUMPS, style = MaterialTheme.typography.titleMedium) - elsewhere.forEach { session -> OtherHeapDumpSessionRow(session, onOpenHeapDump) } } } } } /** - * One agent that worked on another heap dump: what it did, and that dump to open it in. + * One agent's session: read in this window when the heap dump it read is the one open here, and otherwise a + * way to that dump. * - * Not opened here. An address is an address of one heap dump, so a session read against the wrong one is a - * screen of rows that name other objects than the ones the agent saw — which is the whole reason this list is - * per dump. A session that read no dump at all, or one whose dump has been deleted, has nowhere to be opened - * and says which file it wanted. + * A session that read no dump at all — a client that connected and asked nothing — leads nowhere, and neither + * does one whose dump has been deleted: a session outlives the files it was about. */ @Composable -private fun OtherHeapDumpSessionRow( +private fun SessionRow( session: AgentSession, + group: HeapDumpSessions, + onOpen: (Place, OpenIn) -> Unit, + onCopyLink: (Place) -> Unit, onOpenHeapDump: (File, Place) -> Unit ) { - val opens = session.heapDumpPaths.firstOrNull()?.let { File(it) }?.takeIf { it.isFile } + val place = Place.AgentLog(session.sessionId) val title = session.title() val summary = session.summary() + if (group.isThisWindow) { + val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } + OpenTarget(open, { onCopyLink(place) }) { + Column(Modifier.openable(open)) { + Text(title, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) + Text(summary, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) + } + } + return + } + val opens = group.heapDumpFile?.takeIf { it.isFile } if (opens == null) { Column { Text(title, style = MaterialTheme.typography.bodyMedium) @@ -116,13 +120,68 @@ private fun OtherHeapDumpSessionRow( } // No tab to choose and no link to copy: what a link names is a window, and the window this session was // read in belongs to a run that has usually ended. The heap dump is what outlived it. - val open = { onOpenHeapDump(opens, Place.AgentLog(session.sessionId)) } - Column(Modifier.openable { open() }) { + Column(Modifier.openable { onOpenHeapDump(opens, place) }) { Text(title, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) Text(summary, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) } } +/** The sessions that read one heap dump, under the name this screen calls that dump. */ +private class HeapDumpSessions( + /** What the group is headed with: the file's name, and which of several dumps of that name it is. */ + val label: String, + /** The dump itself, and null for the sessions that read none. */ + val heapDumpFile: File?, + /** Whether it is the dump this window has open, which is the one group that is read here. */ + val isThisWindow: Boolean, + val sessions: List +) + +/** + * These sessions under the heap dumps they read, this window's dump first and always present. + * + * A session that read two dumps is under both, because it is one agent's work on each of them and reading it + * against either is reading what it did there. One that read none is last, under a heading of its own: it is + * not about this dump either, and there is no window that isn't a heap dump to list it in. + * + * The file's name rather than its path, which is what a reader recognises — and two dumps of the same name + * from different directories are told apart by a number, since the name on its own would read as one dump + * whose sessions disagree about what its addresses mean. + */ +private fun List.byHeapDump(heapDumpFile: File): List { + val thisDump = heapDumpFile.absolutePath + // Newest session first, which is the order these arrive in, so the dump worked on most recently is the + // group after this window's. + val paths = listOf(thisDump) + flatMap { it.heapDumpPaths }.distinct().filter { it != thisDump } + val names = mutableMapOf() + val groups = paths.map { path -> + val file = File(path) + val seen = names.merge(file.name, 1, Int::plus)!! + val isThisWindow = path == thisDump + HeapDumpSessions( + // Numbered only from the second one on, since a name that is the only one of itself needs no number. + label = file.name + (if (seen > 1) " ($seen)" else "") + + (if (isThisWindow) " ($THIS_HEAP_DUMP)" else ""), + heapDumpFile = file, + isThisWindow = isThisWindow, + sessions = filter { path in it.heapDumpPaths } + ) + } + val readNothing = filter { it.heapDumpPaths.isEmpty() } + return groups + if (readNothing.isEmpty()) { + emptyList() + } else { + listOf( + HeapDumpSessions( + label = NO_HEAP_DUMP_READ, + heapDumpFile = null, + isThisWindow = false, + sessions = readNothing + ) + ) + } +} + /** * What one agent did, call by call, in the order it made them. * @@ -136,10 +195,10 @@ private fun OtherHeapDumpSessionRow( * open — the same read that names a tab — and it is why this screen is reached from the sessions about *this* * dump: a window can only speak for the dump it has. See [AgentLogsScreen] and [placeTitles]. * - * **And every row that names a place leads to it.** The exception is the call of a session that went on to - * another heap dump, which names that dump and opens it: a session is one agent's connection and can read as - * many dumps as were open, so a row leading nowhere would be the app showing somebody what an agent looked at - * and then declining to show them the thing. + * **And what a row names leads to it** — the object, not the verb, since the object is what a reader wants to + * look at. A call that went on to another heap dump leads to that dump instead, named on the row: a session is + * one agent's connection and can read as many dumps as were open, and a row leading nowhere would be the app + * showing somebody what an agent looked at and then declining to show them the thing. */ @Composable internal fun AgentLogScreen( @@ -191,7 +250,14 @@ internal fun AgentLogScreen( } } -/** One call: when, what it did, and why the agent said it was doing it. */ +/** + * One call: when, what it did, and why the agent said it was doing it. + * + * **What leads somewhere is the object, not the verb.** A row is a sentence about a thing — "Described + * MainActivity 0x12d368b8" — and the thing is what a reader wants to go and look at, so it is the only part + * that is a link. Where the call named nothing, the verb is the whole of what it was about and is the link + * itself: "Listed the leaks" is the leaks screen. + */ @Composable private fun AgentCallRow( call: AgentSessionCall, @@ -209,10 +275,12 @@ private fun AgentCallRow( // And whether that is still possible. A session outlives the heap dumps it was about, so a row naming one // that has been deleted says which and leads nowhere. val opens = elsewhere?.takeIf { it.isFile } - // Named for a call about this window's own heap dump, and not for one about another: this window has never - // read that file, so what a number in it stands for is not a question it can answer. - val named = if (elsewhere == null) place?.let { placeTitles[it] } else null - val line = call.line(named, elsewhere) + // What the call itself said it was about, and null for the calls that named nothing — where the verb says + // the whole of it. Only those are named by this window: naming a place derived from which tool it is would + // put "Leaks" after "Listed the leaks", and a call about another dump names a file this window never read. + val target = call.subject?.let { subject -> + if (elsewhere == null) place?.let { placeTitles[it] } ?: subject else subject + } Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) { Text( call.at.clockTime(), @@ -221,28 +289,30 @@ private fun AgentCallRow( color = MUTED_TEXT ) Column { - when { - // Asking which heap dumps are open is about the app rather than about one of them. - place == null -> Text(line, style = MaterialTheme.typography.bodyMedium) - opens != null -> Text( - line, - Modifier.openable { onOpenHeapDump(opens, place) }, - style = MaterialTheme.typography.bodyMedium, - color = LINK_COLOR - ) - // The heap dump it names is gone, so there is nothing left to open it on. - elsewhere != null -> Text(line, style = MaterialTheme.typography.bodyMedium) - else -> { - val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } - OpenTarget(open, { onCopyLink(place) }) { - Text( - line, - Modifier.openable(open), - style = MaterialTheme.typography.bodyMedium, - color = LINK_COLOR - ) + // Nowhere to go for a call about the app rather than about a heap dump — which dumps are open — or + // about one that has since been deleted. + val leadsTo = place?.takeIf { elsewhere == null || opens != null } + // Wrapped rather than truncated, since a class name is as long as it is and the reason under it is a + // sentence: this row is read, not scanned past. + FlowRow(horizontalArrangement = Arrangement.spacedBy(4.dp)) { + // The link is the target, or the verb where the call named no target: what a reader clicks is the + // thing, and a verb that is the whole sentence is the thing. + val linked = target ?: call.verb + if (target != null) { + Text(call.verb, style = MaterialTheme.typography.bodyMedium) + } + when { + leadsTo == null -> Text(linked, style = MaterialTheme.typography.bodyMedium) + opens != null -> LinkText(linked, Modifier.openable { onOpenHeapDump(opens, leadsTo) }) + else -> { + val open: (OpenIn) -> Unit = { openIn -> onOpen(leadsTo, openIn) } + OpenTarget(open, { onCopyLink(leadsTo) }) { LinkText(linked, Modifier.openable(open)) } } } + // What the answer came to, and — for a row that opens another dump when clicked — which dump that + // is: worth knowing before rather than after. + call.outcome?.let { Text("$LEADS_TO $it", style = MaterialTheme.typography.bodyMedium) } + elsewhere?.let { Text("$IN ${it.name}", style = MaterialTheme.typography.bodyMedium) } } call.reason?.let { reason -> // The agent's own sentence, indented under what it did: read down the column of these and a session @@ -271,25 +341,12 @@ private fun AgentSessionCall.otherHeapDumpOrNull(heapDumpFile: File): File? = he ?.takeIf { it != heapDumpFile.absolutePath } ?.let { File(it) } -/** - * What the call did and what it was about, as one line: "Described MainActivity 0x12d368b8". - * - * With what it came to on the end where there is one — "Concluded about MainActivity → MainActivity$2.this$0" - * — since the row that says what was concluded is the row anybody scrolling a session is looking for. - * - * And with [otherHeapDump] named at the end of a row about a dump this window hasn't got open, because - * clicking that row opens a heap dump: which one is a thing to know before rather than after. Those are the - * rows with no [named] to show, where the address the agent wrote stands for itself. - */ -private fun AgentSessionCall.line( - named: String?, - otherHeapDump: File? -): String = listOfNotNull( - verb, - named ?: subject, - outcome?.let { "$LEADS_TO $it" }, - otherHeapDump?.let { "$IN ${it.name}" } -).joinToString(" ") +/** One piece of a row that leads somewhere, which is the piece a reader clicks. */ +@Composable +private fun LinkText( + text: String, + modifier: Modifier +) = Text(text, modifier, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) /** What a session is called: who connected, and when. */ private fun AgentSession.title(): String = listOfNotNull( @@ -335,8 +392,11 @@ private const val REFUSED = "Refused:" private const val A_CLIENT_THAT_DID_NOT_SAY = "An agent" -/** The sessions that read another dump, which open in a window of that dump. See [AgentLogsScreen]. */ -private const val OTHER_HEAP_DUMPS = "Other heap dumps" +/** After the heap dump this window has open, which is the one group of sessions that is read here. */ +private const val THIS_HEAP_DUMP = "this heap dump" + +/** And over the sessions of a client that connected and read nothing, which no window can be about. */ +private const val NO_HEAP_DUMP_READ = "No heap dump" private const val NO_SESSIONS = "No agent has worked on this heap dump. Hand it to one by pointing its MCP client at Shark Explorer, and " + diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt index d55b5fd213..896f011bf6 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt @@ -78,6 +78,7 @@ import shark.explorer.TreemapLayout import shark.explorer.TreemapPresentation import shark.explorer.TreemapRect import shark.explorer.agent.AgentSession +import shark.explorer.agent.subject import shark.explorer.detours import shark.explorer.exactHexObjectId import shark.explorer.formatObjectCount @@ -544,6 +545,9 @@ internal fun HeapDumpExplorer( ?.let { open -> sessions.firstOrNull { it.sessionId == open.sessionId } } ?.calls.orEmpty() .filter { it.heapDumpPath == null || it.heapDumpPath == session.heapDumpFile.absolutePath } + // Only the calls that named what they were about, since those are the only rows that show a name: the + // place of a call that named nothing comes from which tool it is, and its verb already says it. + .filter { it.subject != null } .mapNotNull { it.place } .filter { it !in agentPlaceTitles } .distinct() diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt index aa548621d6..f152cdfc66 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt @@ -6,9 +6,11 @@ import androidx.compose.ui.semantics.SemanticsProperties import androidx.compose.ui.test.ComposeUiTest import androidx.compose.ui.test.ExperimentalTestApi import androidx.compose.ui.test.SemanticsMatcher +import androidx.compose.ui.test.assertHasClickAction import androidx.compose.ui.test.assertHasNoClickAction import androidx.compose.ui.test.assertIsDisplayed import androidx.compose.ui.test.hasText +import androidx.compose.ui.test.onAllNodesWithText import androidx.compose.ui.test.onNodeWithText import androidx.compose.ui.test.performClick import androidx.compose.ui.test.waitUntilAtLeastOneExists @@ -62,7 +64,38 @@ class AgentLogsScreenTest { // The verb, the object named the way a tab on it is named, and the agent's own sentence for why it // asked: no JSON and no bare address on any of it. waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) - waitUntilAtLeastOneExists(hasText(describedRow()), OPEN_TIMEOUT_MILLIS) + waitUntilAtLeastOneExists(hasText(DESCRIBED), OPEN_TIMEOUT_MILLIS) + waitUntilAtLeastOneExists(hasText(activityName()), OPEN_TIMEOUT_MILLIS) + } + } + + @Test fun `the object a call was about is the link, and the verb is not`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(call())))) + onNodeWithText(CLIENT, substring = true).performClick() + waitUntilAtLeastOneExists(hasText(activityName()), OPEN_TIMEOUT_MILLIS) + + // What a reader wants to go and look at is the object, so that is the whole of what leads anywhere: + // a row where clicking the word "Described" navigates is a row with a hand cursor over prose. + onNodeWithText(activityName()).assertHasClickAction() + onNodeWithText(DESCRIBED).assertHasNoClickAction() + } + } + + @Test fun `a call that named nothing is the verb itself, and leads where it went`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(leaksCall())))) + onNodeWithText(CLIENT, substring = true).performClick() + waitUntilAtLeastOneExists(hasText(LISTED_THE_LEAKS), OPEN_TIMEOUT_MILLIS) + + // "Listed the leaks" is the whole sentence and the leaks screen is where it went, so the verb is the + // link — and nothing follows it. A screen name after it, from the tool rather than from an argument, + // read as "Listed the leaks Leaks". + onNodeWithText(LISTED_THE_LEAKS).assertHasClickAction() + onNodeWithText(LISTED_THE_LEAKS).performClick() + + // The leaks screen, named by the reference each leak is: the same screen the agent was reading. + waitUntilAtLeastOneExists(hasText(ACTIVITY_LEAK_NAME, substring = true), OPEN_TIMEOUT_MILLIS) } } @@ -72,10 +105,10 @@ class AgentLogsScreenTest { onNodeWithText(CLIENT, substring = true).performClick() waitUntilAtLeastOneExists(hasText(REFUSAL, substring = true), OPEN_TIMEOUT_MILLIS) - waitUntilAtLeastOneExists( - hasText("Concluded about ${activityName()}"), - OPEN_TIMEOUT_MILLIS - ) + waitUntilAtLeastOneExists(hasText(CONCLUDED_ABOUT), OPEN_TIMEOUT_MILLIS) + // Refused, and still leading to the object it was refused about: the refusals are the half of a + // session worth reading afterwards. + onNodeWithText(activityName()).assertHasClickAction() } } @@ -86,10 +119,9 @@ class AgentLogsScreenTest { // The row anybody scrolling a session is looking for: what the agent asked, and what it came to, on // one line — so that finding the answer isn't reading every reason down the screen. - waitUntilAtLeastOneExists( - hasText("Concluded about ${activityName()} → $FAULTY_REFERENCE"), - OPEN_TIMEOUT_MILLIS - ) + waitUntilAtLeastOneExists(hasText(CONCLUDED_ABOUT), OPEN_TIMEOUT_MILLIS) + onNodeWithText(activityName()).assertIsDisplayed() + onNodeWithText("→ $FAULTY_REFERENCE").assertIsDisplayed() } } @@ -99,8 +131,8 @@ class AgentLogsScreenTest { onNodeWithText(CLIENT, substring = true).performClick() waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) - waitUntilAtLeastOneExists(hasText(describedRow()), OPEN_TIMEOUT_MILLIS) - onNodeWithText(describedRow()).performClick() + waitUntilAtLeastOneExists(hasText(activityName()), OPEN_TIMEOUT_MILLIS) + onNodeWithText(activityName()).performClick() // What the inspectors made of the object the agent was reading, which is the whole promise of the // screen: reading what it did and going to look at it are one move. @@ -121,7 +153,10 @@ class AgentLogsScreenTest { // addresses are addresses of that file. Still reachable, since a dump handed to an agent is usually // one nobody has open. onNodeWithText(NO_AGENT_YET, substring = true).assertIsDisplayed() - onNodeWithText(OTHER_HEAP_DUMPS).assertIsDisplayed() + // The dump this window has open is the first group and says so; the other is a group of its own, + // headed with the file name a reader recognises rather than with a path. + onNodeWithText("${heapDump.file.name} (this heap dump)").assertIsDisplayed() + onNodeWithText(otherHeapDump.name).assertIsDisplayed() onNodeWithText(CLIENT, substring = true).performClick() } @@ -139,13 +174,15 @@ class AgentLogsScreenTest { ), onOpenHeapDump = { file, place -> opened = file to place } ) - onNodeWithText(CLIENT, substring = true).performClick() - waitUntilAtLeastOneExists(hasText(describedRow()), OPEN_TIMEOUT_MILLIS) + // Listed under both dumps, because it is one agent's work on each of them; the group read here is + // this window's, which is the first. + thisWindowsSession().performClick() + waitUntilAtLeastOneExists(hasText(activityName()), OPEN_TIMEOUT_MILLIS) // The address as the agent wrote it, and the file it means something in: this window has never read // that dump, so what the number stands for there is not a question it can answer. - val row = "Described ${hex(activityObjectId())} in ${otherHeapDump.name}" - onNodeWithText(row).performClick() + onNodeWithText("in ${otherHeapDump.name}").assertIsDisplayed() + onNodeWithText(hex(activityObjectId())).performClick() } // Going there means opening that dump, where the same address is that dump's object. @@ -155,11 +192,12 @@ class AgentLogsScreenTest { @Test fun `a call about a heap dump that has gone leads nowhere`() { explorerUiTest { openAgentLogs(listOf(session(calls = listOf(call(), call(heapDumpPath = "/dumps/deleted.hprof"))))) - onNodeWithText(CLIENT, substring = true).performClick() - waitUntilAtLeastOneExists(hasText(describedRow()), OPEN_TIMEOUT_MILLIS) + thisWindowsSession().performClick() + waitUntilAtLeastOneExists(hasText(activityName()), OPEN_TIMEOUT_MILLIS) // Still worth reading, and there is nothing to open: a session outlives the heap dumps it was about. - onNodeWithText("Described ${hex(activityObjectId())} in deleted.hprof").assertHasNoClickAction() + onNodeWithText("in deleted.hprof").assertIsDisplayed() + onNodeWithText(hex(activityObjectId())).assertHasNoClickAction() } } @@ -223,14 +261,32 @@ class AgentLogsScreenTest { millis = 12L ) + /** + * The session as this window's group of them lists it, which is the first: a session that read two dumps is + * listed under both, and only this window's group is read here. + */ + private fun ComposeUiTest.thisWindowsSession() = onAllNodesWithText(CLIENT, substring = true)[0] + + /** The one call that names nothing: the leaks screen is the whole of what it was about. */ + private fun leaksCall() = AgentSessionCall( + at = STARTED_AT, + tool = "list_leaks", + reason = REASON, + windowId = "zvphq4r3", + heapDumpPath = heapDump.file.absolutePath, + place = Place.Leaks(), + arguments = emptyMap(), + refusal = null, + outcome = null, + millis = 12L + ) + private fun activityObjectId() = heapDump.activityObjectIds.first() /** How the window names the activity: the same title the tab a row opens carries. */ private fun activityName() = "${LEAKING_ACTIVITY_CLASS_NAME.substringAfterLast('.')} ${hexObjectId(activityObjectId())}" - private fun describedRow() = "Described ${activityName()}" - private fun hex(objectId: Long) = exactHexObjectId(objectId) /** @@ -250,7 +306,11 @@ class AgentLogsScreenTest { const val REFUSAL = "3 step(s) have no verdict" const val FAULTY_REFERENCE = "Holder.activity" const val NO_AGENT_YET = "No agent has worked on this heap dump" - const val OTHER_HEAP_DUMPS = "Other heap dumps" + + /** The verbs the rows read as, which are [shark.explorer.agent.verb]'s and not this screen's. */ + const val DESCRIBED = "Described" + const val CONCLUDED_ABOUT = "Concluded about" + const val LISTED_THE_LEAKS = "Listed the leaks" val STARTED_AT: Instant = Instant.parse("2026-08-25T18:19:48.035Z") From 5a5f9bad1eaaf75b138768b3470ad5f4d3e4eb7f Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Wed, 26 Aug 2026 09:10:16 +0200 Subject: [PATCH 22/27] Tell an agent this surface exists, and that it isn't only about leaks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The tools were reachable and undiscoverable: MCP needed a client someone had configured, and `--agent` needed somebody to say it was there. So the skill — `.claude/skills/shark-explorer/SKILL.md`, which is the directory every client that reads the standard looks in, so it is the project skill here and the directory a user copies into `~/.claude/skills`. It frames the three cases an agent is actually in — something already open, a file to open, a device to dump — rather than assuming a window is up, and it points at `--agent-help` and at the method the tools hand over rather than repeating either. The description is not leak-only, and neither is the surface any more: taking a dump used to say "call list_leaks to see what the dump says about itself", which points a dump somebody took because the app was using a gigabyte away from the question. It names dominator_tree too. And `find_objects` now says that with no className it is the biggest objects in the dump, which is what "what's big in my heap" needs and which nothing said. Checked by doing it: `--agent dump_heap` took a 146 MB dump of com.squareup off an API 29 emulator in 90 seconds and opened it, and `dominator_tree` and `find_objects` on that window found the class loader's 42 MB of classes and, below it, a Coil image cache holding a 4 MB bitmap. The eval note has the plan for scoring that as a scenario, including why the class loader answer is the failure a "what's big" dump has to provoke. Co-Authored-By: Claude Opus 5 --- .claude/skills/shark-explorer/SKILL.md | 115 ++++++++++++++++++ docs/shark-explorer.md | 36 ++++++ shark/shark-explorer/notes/agent-eval.md | 46 +++++++ shark/shark-explorer/notes/agent-surface.md | 40 +++++- .../java/shark/explorer/agent/AgentTools.kt | 19 ++- 5 files changed, 247 insertions(+), 9 deletions(-) create mode 100644 .claude/skills/shark-explorer/SKILL.md diff --git a/.claude/skills/shark-explorer/SKILL.md b/.claude/skills/shark-explorer/SKILL.md new file mode 100644 index 0000000000..263c1406e9 --- /dev/null +++ b/.claude/skills/shark-explorer/SKILL.md @@ -0,0 +1,115 @@ +--- +name: shark-explorer +description: "Use when investigating an Android or JVM heap dump (.hprof): what is leaking and why, what is holding an object, what the biggest objects are, what a process is spending its memory on. Drives Shark Explorer, which reads the dump in a window a person can watch, from a shell or over MCP." +allowed-tools: + - Bash +--- + +# Investigating a heap dump with Shark Explorer + +Shark Explorer is a desktop app that reads a heap dump, and every screen and button of it is also a tool you +can call. So you read the dump **in the window somebody is looking at**: what you look at, they can look at, +and the verdicts and notes you leave are on their screen and in files that outlive the run. + +It is not only for leaks. `list_leaks` is the dump's own answer about what shouldn't be there, and +`dominator_tree` is what the memory is actually going on, which is a different question — a heap where nothing +is leaking still has a biggest object. + +## Start by working out which case you are in + +**Something is already open.** Ask, and the answer carries the method to follow, the window ids every other +tool names a dump by, and any verdicts somebody has already reached: + +```bash +"/Applications/Shark Explorer.app/Contents/MacOS/Shark Explorer" --agent open_heap_dumps \ + reason="Finding out what is already open" +``` + +If that says nothing is open, it opened a window for you, so the same command again lists it. + +**You have a file.** A dump that came with a bug report, or one you took earlier: + +```bash +… --agent open_heap_dump path=/absolute/path/bug-4821.hprof reason="The dump the report came with" +``` + +It answers once the dump is readable, which on a large one is a wait rather than a moment. + +**You need to take one.** From a device or emulator `adb` is connected to: + +```bash +… --agent list_devices reason="Finding the device" +… --agent list_devices device=emulator-5554 reason="Finding the process to dump" +… --agent dump_heap device=emulator-5554 process=com.example.app reason="Reproduced the bug, dumping now" +``` + +`dump_heap` collects the garbage first, writes the dump on the device, pulls it and opens it — minutes on a +large app, and one call that does not come back until it is readable. A process can only be dumped if the app +was built debuggable or the whole device build is; `list_devices` says which. + +**And before investigating anything, read what has already been tried on that dump:** + +```bash +… --agent agent_log reason="Finding out whether somebody has already been through this" +``` + +An investigation somebody already ran is either the answer or the half of the dump not worth doing again. + +## The command line + +```bash +"/Applications/Shark Explorer.app/Contents/MacOS/Shark Explorer" --agent name=value … +``` + +- `--agent-help` prints every tool, with its arguments and what each one is for. `--agent-help ` prints + one tool instead of all of them. **Read that rather than guessing at a tool**, and rather than trusting a + list in a file like this one, which goes stale. +- **Find the launcher first** — the path above is where a `.dmg` install puts it, and the space in it has to + stay quoted: + ```bash + ls -d /Applications/"Shark Explorer.app" ~/Applications/"Shark Explorer.app" 2>/dev/null + ``` +- **Every tool takes `reason`**, which is why you are making the call. It is logged beside the reads it causes + and read afterwards by a person on the *Agent logs* screen, so write the sentence you would say to somebody + watching over your shoulder. +- **Exit code 0** means the answer is the JSON on stdout. **2 means the call was refused**, and the refusal on + stderr is the next thing to do, not an error to retry. **1** means nothing was there to answer it. +- **Addresses are `0x…`, exactly as the surface writes them.** Never decimal: a heap dump's addresses do not + survive a JSON number. +- `--agent-run=` picks between several open runs. `--agent-session=` says which investigation these + calls are one of; by default one shell is one session, so what you did reads as one row of that screen + rather than a row per call. + +**Over MCP instead, if your client can be configured**, which gets the same tools with their schemas in band: + +```json +{ "mcpServers": { "shark-explorer": { + "command": "/Applications/Shark Explorer.app/Contents/MacOS/Shark Explorer", + "args": ["--mcp-stdio"] +} } } +``` + +Add `--no-ui` for a machine with no screen — a build server, or a dump at the far end of an ssh session. +Everything works the same except `show`, which has nowhere to put a tab. + +## What to do with it + +**The method comes with the tools.** `open_heap_dumps` hands back the whole of it — what a leak is, the three +zones of a chain, how a verdict spreads, and the order that finds the faulty reference. Follow that; it is +[the LeakCanary method](https://engineering.block.xyz/blog/the-leakcanary-method) as the tools enforce it, and +it does not need repeating here. + +Two things about it that are easy to miss: + +- **`conclude` will refuse you** until the heap dump agrees that one reference is at fault, and the refusal + says which of the three reasons it is. That is the surface working. Go and do what it says — usually + `set_verdict` on the object it named — rather than reporting a root cause it would not accept. +- **Isolating the reference is not the root cause.** It says where the problem is, not how it happened, so the + method sends you to the code at the version this dump is of, and tells you how to work out which version + that is. + +**When the question isn't a leak**, the tools are the same and the order is yours. What is big is +`dominator_tree`, top down, and `describe_object` on whatever it names; what is holding one thing is +`ways_held`; what instances of a class there are, and how much they retain between them, is `find_objects`. +`show` puts any of it on the person's screen, and `take_note` writes what you found where they and the next +reader will find it. diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index b52376c41a..5baf4dd2b6 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -281,6 +281,41 @@ has none. Nothing else changes, because **notes and verdicts were never on the s beside the heap dump, so a dump investigated over ssh today opens in a window tomorrow with the verdicts, the reasons and the conclusion already on it. +### Or from a shell, with nothing configured + +The same tools are a command away, for an agent whose client speaks no MCP and for one that has a terminal and +hasn't been set up with anything: + +```bash +"/Applications/Shark Explorer.app/Contents/MacOS/Shark Explorer" --agent-help +"/Applications/Shark Explorer.app/Contents/MacOS/Shark Explorer" \ + --agent list_leaks reason="Starting from what the dump says about itself" +``` + +`--agent-help` prints every tool with its arguments; `--agent-help ` prints one of them. A call goes to +the window that has the heap dump open — the same socket the MCP pipe uses — or opens one when nothing is +running. It exits 0 with the answer as JSON on stdout, **2 when the call was refused**, with the refusal on +stderr where a script can read it, and 1 when there was nothing to answer it. + +Calls from one shell are one session, so what an agent did reads as one row of the *Agent logs* screen rather +than a row per command. `--agent-session=` says so explicitly, and `--agent-run=` picks between +several explorers. + +### The skill + +An agent still has to be told that any of this exists. This repository carries a +[skill](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview) that does it — the three +ways in, the command line, and what to do with an answer — and every client that reads the standard (Claude +Code, Codex, Cursor, Gemini CLI) picks it up from the same directory: + +```bash +git clone git@github.com:square/leakcanary.git +cp -R leakcanary/.claude/skills/shark-explorer ~/.claude/skills/ +``` + +Then "there's a heap dump in ~/Downloads, what's using all the memory" is enough: the skill is what turns that +into opening the dump in a window you can watch. + Then ask for what you actually want. This is the whole prompt the session below was given: > A heap dump is open in Shark Explorer, which you can reach through its MCP tools. Something in it is @@ -306,6 +341,7 @@ press, because a surface with less than that is one whose answer is "ask your hu | --- | --- | | `open_heap_dumps` | Every window and what is open in it, with the method to follow. | | `list_leaks` | The **Leaks** screen: what this heap dump says shouldn't be there. | +| `agent_log` | The **Agent logs** screen: what has already been tried on this dump, and what it came to. | | `chain_from_gc_root` | One chain, every step with its labels and its verdict. | | `describe_object` | What an object is: its class, fields, labels, size. | | `ways_held` | Every way an object is held, rather than the one chain — the *X ways from here* list. | diff --git a/shark/shark-explorer/notes/agent-eval.md b/shark/shark-explorer/notes/agent-eval.md index f26609a3e0..6f6aa9fc38 100644 --- a/shark/shark-explorer/notes/agent-eval.md +++ b/shark/shark-explorer/notes/agent-eval.md @@ -166,6 +166,52 @@ An eval also leaves one `~/.shark-explorer/notes` directory and one `leak-status makes the above work. They can go once the runs have been read, and nothing depends on them going: the paths they are keyed to belong to an eval that has already been deleted. +## Planned: a question that isn't a leak + +Everything above scores one question — *which reference is at fault* — and the surface answers others. "What +is using all this memory" is the one people ask most after that, it has a checkable answer, and it exercises a +different half of the tools: `dominator_tree` and `find_objects` rather than the chain and the verdicts. What +follows is the design, not something that runs yet. + +**The prompt is the whole of the input, as it is for the leak runs**: *"A heap dump is open in Shark Explorer. +What is using most of the memory in this app?"* — no mention of a tool. + +**The answer key is an object, not a string.** A leak's key is `OwnerClass.field` because that is what +`conclude` answers with; here the scenario builder knows which object it made the biggest, so the key is that +object's identity, and the score resolves what the agent named back to a class in the dump. Which means the +same score works on a real dump with no key written by hand at all: `HeapDominatorTree` says what the biggest +retainer is, and the eval can ask it. + +**What is scored, all of it off the session file and the dump, with no model:** + +| Signal | How it is read | +| --- | --- | +| `RIGHT` | The agent `show`ed or wrote a note naming the key object, or its class | +| `WRONG` | Named another object as the answer — the confident wrong answer again | +| `TRIVIAL` | Named the class loader's class array, or anything else above the app's own objects: true, useless, and the failure this dump is shaped to provoke | +| `NO_ANSWER` | Never named an object at all | +| Calls to get there | The number worth halving, since this question is a fan-out | + +**`take_note` and `show` are what an answer is written in**, because they already are: the window's own way of +saying "this is the thing" is a note on the object and a tab on the screen. So this needs no new tool, and +that is the point — a surface that needs a `report_the_answer` tool per question is a surface that has stopped +being the window. + +**The `TRIVIAL` row is the finding that prompted this.** Measured on a 146 MB dump of a real app taken through +`dump_heap`: the top of the dominator tree is `6 × PathClassLoader` at 42 MB, 38% of a 111 MB heap, and under +it one `Object[]` of 79,655 loaded classes. The biggest ten children of that array come to 1 MB — 2% of it — +so an agent that walks down the biggest branch and reports what it finds has reported "the classes", which is +both true and no use to anybody. The interesting answer was two rows further down `find_objects`: a Coil image +cache holding a 4 MB bitmap. A scenario shaped like that is what says whether a description or a refusal can +get an agent past it. + +**Two scenarios to start with**, matching how the leak families are split: + +- **A synthetic one** where a named static cache retains a known share of the heap under a class loader made + deliberately fat, so `TRIVIAL` and `RIGHT` are both reachable and the key is exact. +- **A real one**, a dump taken off a device with `dump_heap`, scored against what `HeapDominatorTree` says. + Which also makes it the first eval scenario whose dump nobody wrote. + ## What to do with a result A scenario that fails the same way across models is a bug in this surface, not in the model, and the fix is diff --git a/shark/shark-explorer/notes/agent-surface.md b/shark/shark-explorer/notes/agent-surface.md index 10bd1fa65f..11feebd3e1 100644 --- a/shark/shark-explorer/notes/agent-surface.md +++ b/shark/shark-explorer/notes/agent-surface.md @@ -9,11 +9,11 @@ Measured off `AgentTools.all` and `AgentMethod.INSTRUCTIONS`, one `tools/list` e | | Characters | ≈ tokens | Paid | | --- | --- | --- | --- | -| Seventeen tool definitions | 20,938 | 5,235 | Every turn, while the server is connected | +| Seventeen tool definitions | 21,123 | 5,280 | Every turn, while the server is connected | | The method | 7,845 | 1,960 | Handshake, and again with `open_heap_dumps` | So the standing cost of this surface is **7 to 8 k tokens**, around 3.5% of a 200 k window. Parity took the -tool count from eleven to seventeen and the definitions from 13,116 characters to 20,938 — **a fifth of the +tool count from eleven to seventeen and the definitions from 13,116 characters to 21,123 — **a fifth of the window's budget for the six tools that mean an agent never has to ask its human to click something**, which is the trade this surface exists to make. The sixth is `agent_log`, 1,237 characters of the total, and the 900 the other sixteen grew by are the two agent-log places added to the sentence naming every place, which @@ -33,11 +33,11 @@ Measured against a packaged build with one window open on `leak_asynctask_o.hpro | | Measured | Paid | | --- | --- | --- | | One call, JVM start to JSON on stdout | 160–180 ms | Per call | -| `--agent-help`, all seventeen tools | 14,414 characters, ≈3,600 tokens | Only when read | +| `--agent-help`, all seventeen tools | 14,594 characters, ≈3,650 tokens | Only when read | | `--agent-help `, one of them | 500–1,250 characters, ≈125–310 tokens | Only when read | So the standing cost is nothing, and the whole surface as text is *smaller* than the `tools/list` definitions -of it (14,414 against 20,938) because `reason` is explained once rather than seventeen times. Both +of it (14,594 against 21,123) because `reason` is explained once rather than seventeen times. Both `--agent-help` figures include the invocation path twice, since what it prints is the command to type on this machine; a shorter install path is a slightly shorter help. @@ -76,8 +76,9 @@ arguments in and JSON out, not a second copy of the rules: thrown by the handler that would have refused an MCP client. `--agent-help` is generated from the registry, so a tool cannot be on one and missing from the other, and it is described through `NoHeapDumpToDescribe` — a heap dump whose every method throws — which makes "printed, never called" hold rather than be a habit. -- The skill — the method as `SKILL.md`, plus how to reach either adapter. Prose, not generated, and it points - at `--agent-help` rather than listing tools that would go stale. +- The skill — `.claude/skills/shark-explorer/SKILL.md`. Exists. Prose, not generated, and it points at + `--agent-help` and at the method the tools hand over rather than repeating either, since a list of tools in + a file is a list that goes stale. See the next section for why it is in `.claude/`. What that leaves duplicated is argument parsing per adapter, which is tens of lines — and less than that here, because `AgentArguments` reads a number and a boolean out of text (the tools were written for a model, @@ -85,6 +86,33 @@ which sends `limit=30` as a string as often as not). So a command line sends eve the only shape needing a spelling of its own is a list, which is comma separated because a shell has no brackets. What it must never become is two places that decide whether an investigation may conclude. +## Where the skill lives, and how an agent finds it + +A skill nobody loads is a file. The two things that decide where it goes are that **skills are discovered by +directory, not by search** — every client that reads the +[standard](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview) looks in +`.claude/skills//SKILL.md` under the project and in `~/.claude/skills/` for the machine — and that +**the people who need it don't have this repository**: they installed a `.dmg`. + +So `.claude/skills/shark-explorer/` is the one place it can be that is not arbitrary. In this repository it is +the project skill, so an agent working on the explorer has it without being told. And it is the directory a +user copies: + +```bash +cp -R .claude/skills/shark-explorer ~/.claude/skills/ +``` + +Which is what `docs/shark-explorer.md` says, and what a release should carry as an asset. **The alternative +worth knowing about and not taking** is having the app write it into `~/.claude/skills` as it starts: it would +need no install step and would always match the build, and it would also be an app that writes into another +program's configuration directory without being asked, which is not a thing to do to somebody's machine. + +**A skill is not how an agent finds the binary.** It names the `.dmg` install path and how to look for it, +because there is nothing on `PATH` — the bundle is `/Applications/Shark Explorer.app`, the space stays and +gets quoted, and the name was deliberately given that space once Block's signing service could take it (see +`packageName` in the app's build script). What would remove the quoting for good is a launcher shim on `PATH`, +which is a separate decision about writing outside the bundle. + ## The judgement, in one line MCP for a client that can be configured, the command line for everything else, and the method in a skill so diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index 8fb189738c..a6e07edfd2 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -237,7 +237,9 @@ internal class AgentTools( description = "The objects of this heap dump whose class name matches, largest retained size first, " + "with how many matched in total. Use it on a class you have assumed something about: two instances " + "of a class you took for a singleton is the answer to a surprising number of leaks, because the " + - "object on the chain then isn't the instance you thought it was.", + "object on the chain then isn't the instance you thought it was. **With no className it is every " + + "object, so it is also the answer to \"what are the biggest things in this heap\"** — one object at " + + "a time, where dominator_tree is what holds them.", schema = schema( WINDOW to window(), CLASS_NAME to string("Matched against the class name.").optional(), @@ -575,7 +577,7 @@ internal class AgentTools( put("window", dump.windowId) put("heapDumpPath", dump.heapDumpPath) put("opened", true) - put("next", "Call $LIST_LEAKS with this window to see what the dump says about itself.") + put("next", NEXT_WITH_A_NEW_DUMP) } } @@ -631,7 +633,7 @@ internal class AgentTools( put("window", dump.windowId) put("heapDumpPath", dump.heapDumpPath) put("dumped", true) - put("next", "Call $LIST_LEAKS with this window to see what the dump says about itself.") + put("next", NEXT_WITH_A_NEW_DUMP) } } @@ -792,6 +794,17 @@ internal class AgentTools( const val HOW_TO_REPRODUCE = "howToReproduce" const val NOT_CHECKED = "notChecked" + /** + * What to do with a heap dump that has just been opened, whichever tool opened it. + * + * Both questions, because a dump is not always a leak. A dump somebody took because the app was using a + * gigabyte is a dominator tree, and being pointed only at the leaks is being pointed away from the + * question — while a dump with `KeyedWeakReference`s in it has an answer waiting in `list_leaks` that + * walking a tree would take an hour to reach. + */ + const val NEXT_WITH_A_NEW_DUMP = "Call $LIST_LEAKS with this window to see what the dump says about " + + "itself, or dominator_tree to see where its memory has gone." + /** * How many objects a list comes back with by default, well under * [HeapDominatorTreemap.MAX_LISTED_OBJECTS]: an agent reads the whole answer, so 500 rows of JSON is From 102eb34412ee05480e56acecab72c7d0fadbc483 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Wed, 26 Aug 2026 10:01:11 +0200 Subject: [PATCH 23/27] Link the thing a call was about, and give every call one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A row of the Agent logs screen is a sentence with one link in it, and the link was the whole sentence whenever the call named nothing: "Listed the leaks" and "Read the agent log" were underlined verbs. A verb now stops where the thing starts — "Listed the" and then *leaks* — which needs the words for a screen to live beside the tool names rather than being taken from the tab titles, since a tab says `Leaks` and a sentence says leaks. Two calls had no link at all, and both are things a person does in this window: dominator_tree with no object is the tree from its root, and find_objects with no className is the object list unfiltered, which also stops that row reading as "Searched for in com.squareup.hprof". AgentScreen is the words and the place as one value, read by AgentTools.placeOrNull and by the screen, so a screen cannot be reachable and unnamed or named and unreachable. Sessions already on disk have no link for those two, so the place is worked out from the tool when the file says none. The call that asks which heap dumps are open answers with a list, so its row unfolds into them: each dump a window away, the one this window has open marked and leading nowhere. Which needs the answer recorded, like a conclusion — the run that had them open has ended by the time anybody reads it. And every session row leads somewhere now. One about another heap dump opens in a window of that dump as before; the ones with no such window to be had — a dump that has been deleted, or a client that read none — are read here rather than being dead rows, since an address whose file has gone resolves to nothing in any window there is, and the verbs, the reasons and the refusals are still what the agent said. The group of a dump that has gone says so. "Described" is gone: it read as the agent having written a description rather than having asked what an object is. Every read now starts with Read, Looked, Listed, Searched or Asked, and every write with Recorded, Took, Wrote, Showed, Concluded, Opened or Dumped. --- docs/shark-explorer.md | 12 +- .../shark-explorer-agent/AGENTS.md | 27 ++- .../shark/explorer/agent/AgentSessionFile.kt | 131 ++++++++++-- .../java/shark/explorer/agent/AgentTools.kt | 32 ++- .../java/shark/explorer/agent/McpSession.kt | 6 +- .../explorer/agent/AgentSessionFileTest.kt | 65 +++++- .../shark/explorer/agent/McpSessionTest.kt | 35 +++- .../shark/explorer/app/AgentLogsScreen.kt | 186 +++++++++++++----- .../shark/explorer/app/AgentLogsScreenTest.kt | 168 ++++++++++++++-- 9 files changed, 562 insertions(+), 100 deletions(-) diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 5baf4dd2b6..08f9563830 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -393,19 +393,25 @@ and there is every call that agent made, in order and in words — what it did, the sentence it gave for doing it: ``` +08:23:04 ▸ Asked which heap dumps are open + because: Seeing what there is to read before asking anything about it. 08:23:11 Listed the leaks because: Starting from what the heap dump already says shouldn't be here. 08:23:18 Read the chain to 0x12d368b8 because: This is the one App leak: a MainActivity the app watched and whose mDestroyed is true. Reading the chain from a GC root. -08:23:27 Described 0x12d00c30 +08:23:27 Looked at 0x12d00c30 because: The FutureTask in the middle of the chain: checking whether it is really running. 08:23:34 Looked for every way of holding 0x12d368b8 because: Checking whether anything else holds the activity, or only this one chain. ``` -**A row leads where the call went**: click *Read the chain to 0x12d368b8* and the window opens that object, -so reading what an agent did and going to look at it are one move. +**A row leads where the call went**, and what leads there is the thing rather than the verb: click +*0x12d368b8* on *Read the chain to 0x12d368b8* and the window opens that object, so reading what an agent did +and going to look at it are one move. A call that named nothing went somewhere all the same — *leaks* on +*Listed the leaks* is the leaks screen, and *dominator tree* on *Read the dominator tree* is the tree from its +root. The one row that leads to several places unfolds instead: *Asked which heap dumps are open* opens into +the dumps that were open, each of them a window away. **A refused call is a row too**, in red, under the reason the agent gave for making it — and those are the half of a session worth reading, since a refusal is where the method sent an agent back to the heap dump diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index 88dc6a05a3..5b5b3771bd 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -62,14 +62,17 @@ connection, a JSON object per line, the newest `KEEP_SESSION_COUNT` kept. What r reading half lives here beside the writing half and is tested with it. A field written and never read back is a row of that screen saying nothing. -Three things follow that reading the code won't tell you. +What follows from that, and reading the code won't tell you: **A call is described before it is answered, not after.** `McpSession.callTool` asks `AgentTools.target` what the call is about and only then invokes the handler, so **a refused call still records its place** and its row is still clickable. That is deliberate: the refusals are the half of a session worth reading afterwards, and a refusal nobody can follow up on is a dead end on the screen. `target` derives the place from the argument -*names* rather than from a second list of tool names — one exception, `list_leaks`, which takes no argument -saying where it is. +*names* rather than from a second list of tool names — except for the four tools that take no argument saying +where they are, which are named in `placeOrNull` because **every call that goes somewhere in the window has to +lead there**: the leaks, the agent log, and `dominator_tree` and `find_objects` given nothing, which are the +tree from its root and the object list unfiltered. Anything left with no place is a call about the app rather +than about a heap dump. **A session records addresses, and is read in the window of its heap dump.** What an agent types is `0x12d368b8` and what the screen shows is `MainActivity 0x12d368b8`, so somebody has to resolve it — and @@ -82,16 +85,26 @@ heap dump read on every call to answer a question the reader already has the dum can open a second dump, and a call about one this window hasn't got is a row it leaves as the address, saying which file, and opens that dump when clicked. -**One field comes off the answer instead: `outcome`.** What an agent asked is what it typed, and what it -concluded is what the heap dump *agreed to* — so `outcomeOfTool` reads the reference out of `conclude`'s -answer, and nothing else records an answer. Both readers need it and neither can work it out: the screen's -last row is what a session came to, and the eval has nothing to mark against its answer key without it. +**Two fields come off the answer instead.** What an agent asked is what it typed, and what it concluded is +what the heap dump *agreed to* — so `outcomeOfTool` reads the reference out of `conclude`'s answer. Both +readers need that one and neither can work it out: the screen's last row is what a session came to, and the +eval has nothing to mark against its answer key without it. `openHeapDumpsOfTool` is the other, and the reason +is the same shape: `open_heap_dumps` is the one call whose subject is the app, and the dumps it heard about are +in the answer alone. Nothing else reads an answer — a row saying what a read came back with would be the +answer printed twice. **The verbs are here rather than in the app.** `verbOfTool` is beside the tool names, so that a screen never spells them itself and drift is one list rather than two. `AgentSessionFileTest` asserts every tool in the registry has one; a tool added without a verb reads as its own name, which is the protocol showing through on the screen that exists to not show it. +**A verb stops where the thing it was about starts**, which is why several of them end mid-sentence: a row of +that screen is prose with one link in it, and the link is the thing. So `list_leaks` is "Listed the" and +`screenOfTool` is the *leaks* after it, in lower case because it is inside a sentence rather than a tab title. +Every tool `placeOrNull` names has words there, and only those — `AgentSessionFileTest` fails on either half +of that being added without the other, since a place with no words is a call that went somewhere the reader is +never shown, and words with no place are a link to nothing. + **Writing a session never throws and never blocks the answer.** A bad line is skipped on read with a `SharkLog.d` saying which, a file whose header is missing falls back to the id in its name, and a truncated last line — an app killed mid-write — keeps every call before it. An agent's call must not fail because the diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt index 262e9bbfb2..206c0e428b 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt @@ -7,12 +7,14 @@ import java.time.ZoneId import java.time.format.DateTimeFormatter import kotlinx.serialization.json.Json import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.add import kotlinx.serialization.json.JsonElement import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.JsonPrimitive import kotlinx.serialization.json.buildJsonObject import kotlinx.serialization.json.jsonObject import kotlinx.serialization.json.put +import kotlinx.serialization.json.putJsonArray import kotlinx.serialization.json.putJsonObject import shark.SharkLog import shark.explorer.DeepLink @@ -256,6 +258,9 @@ class AgentSessionFile private constructor( link()?.let { put(LINK_KEY, it) } refusal?.let { put(REFUSAL_KEY, it) } outcome?.let { put(OUTCOME_KEY, it) } + if (openHeapDumps.isNotEmpty()) { + putJsonArray(OPEN_HEAP_DUMPS_KEY) { openHeapDumps.forEach { add(it) } } + } put(MILLIS_KEY, millis) if (arguments.isNotEmpty()) { putJsonObject(ARGUMENTS_KEY) { @@ -275,16 +280,21 @@ class AgentSessionFile private constructor( return null } val link = text(LINK_KEY) + val arguments = this[ARGUMENTS_KEY]?.asStringMap().orEmpty() return AgentSessionCall( at = at, tool = tool, reason = text(REASON_KEY), windowId = text(WINDOW_KEY), heapDumpPath = text(HEAP_DUMP_KEY), - place = link?.let { placeOfLinkOrNull(it, file, lineNumber) }, - arguments = this[ARGUMENTS_KEY]?.asStringMap().orEmpty(), + // From the tool for a line with no link, which is a session written by a build that recorded no + // place for a call that named nothing — and it went to the same screen then as it would now. + place = link?.let { placeOfLinkOrNull(it, file, lineNumber) } + ?: screenOfTool(tool, arguments)?.place, + arguments = arguments, refusal = text(REFUSAL_KEY), outcome = text(OUTCOME_KEY), + openHeapDumps = this[OPEN_HEAP_DUMPS_KEY].asStrings(), millis = text(MILLIS_KEY)?.toLongOrNull() ?: 0L ) } @@ -318,6 +328,9 @@ class AgentSessionFile private constructor( } } + private fun JsonElement?.asStrings(): List = + (this as? JsonArray)?.mapNotNull { (it as? JsonPrimitive)?.content }.orEmpty() + private fun JsonElement.asStringMap(): Map = (this as? JsonObject)?.mapValues { (_, value) -> (value as? JsonPrimitive)?.content ?: value.toString() @@ -379,6 +392,7 @@ class AgentSessionFile private constructor( private const val LINK_KEY = "link" private const val REFUSAL_KEY = "refused" private const val OUTCOME_KEY = "outcome" + private const val OPEN_HEAP_DUMPS_KEY = "openHeapDumps" private const val MILLIS_KEY = "millis" private const val ARGUMENTS_KEY = "arguments" } @@ -438,6 +452,16 @@ class AgentSessionCall( * the eval scores against the answer key. Null for a call whose answer is data rather than a conclusion. */ val outcome: String?, + /** + * Which heap dumps the answer said were open, for the one call that asks about the app. See + * [openHeapDumpsOfTool]. + * + * Empty for every other call, and that is the whole of what it means: a row with these is a row whose + * answer was a list of dumps, which the window unfolds and makes each of them somewhere to go. Recorded + * because it is a list of what *was* open — the run has usually ended by the time anybody reads it, so + * nothing can be asked again. + */ + val openHeapDumps: List = emptyList(), /** How long the app took to answer, which is mostly how long the heap dump read took. */ val millis: Long ) { @@ -451,24 +475,43 @@ class AgentSessionCall( } /** - * What the call did, as a couple of words. + * What the call did, as a couple of words, and never the thing it did it to. * * Here rather than in the window that draws it because this is where the tool names are: a screen spelling * them itself would be a second list of them to keep in step. Every tool has one, which `AgentSessionFileTest` * is what keeps true — a tool added without a verb reads as its own name, which is the raw protocol showing * through on a screen that exists to not show it. + * + * **Prose, so it can be drawn as prose.** What a row leads to is [subject] or [screen], and a verb that + * swallowed the thing it was about — "Listed the leaks" — leaves a row with no part of it to be the link + * except the whole sentence. So a verb ends where the thing begins, even when that makes it "Listed the". */ val AgentSessionCall.verb: String get() = verbOfTool(tool, arguments) ?: tool.replace('_', ' ') /** * What the call was about, in the words the window uses for it: an address, a class name, a place. * - * Null for a call whose subject is the whole heap dump or the app itself, where the verb says all of it. + * Null for a call whose subject is a screen of the heap dump, which is [screen], or the app itself, where + * the verb says all of it. */ val AgentSessionCall.subject: String? get() = arguments[SUBJECT_OBJECT] ?: arguments[SUBJECT_PLACE] ?: arguments[SUBJECT_CLASS_NAME] ?: arguments[SUBJECT_SESSION] +/** + * What the call was about where that is a whole screen of the heap dump rather than something in it. + * + * The other half of [verb], and the reason a verb is allowed to end mid-sentence: a call that named nothing + * still went somewhere, so it still has one part of the row that leads there. "Read the" + "dominator tree", + * where the second words are the link and the first are not. + * + * Spelled beside the tool names rather than taken from what the window calls that screen, because these read + * inside a sentence: the tab says `Leaks` and the row says "Listed the leaks", and a row built from the tab + * titles reads as "Listed the Leaks". Null for a call that named an object — [subject] is that — and for the + * calls about the app rather than about a heap dump, which have no place of one to go to. + */ +val AgentSessionCall.screen: String? get() = screenOfTool(tool, arguments)?.words + /** * What the answer to a call came to, as a couple of words, and null when the answer is data rather than a * conclusion. @@ -487,21 +530,42 @@ internal fun outcomeOfTool( else -> null } +/** + * Which heap dumps an answer said were open, which is the second thing read off an answer rather than off + * the arguments. See [AgentSessionCall.openHeapDumps]. + * + * Only `open_heap_dumps`, and for the same reason `outcomeOfTool` is only `conclude`: this is the one call + * whose answer is not about a heap dump but *is* a list of them, and a row saying "asked which dumps are + * open" without saying which is a row that withholds the answer it is a record of. The paths, since a + * window is opened on a path — the window ids beside them in that answer belong to a run that has usually + * ended by the time anybody reads this. + */ +internal fun openHeapDumpsOfTool( + tool: String, + answer: JsonObject +): List = when (tool) { + "open_heap_dumps" -> (answer[ANSWER_HEAP_DUMPS] as? JsonArray).orEmpty() + .mapNotNull { ((it as? JsonObject)?.get(ANSWER_HEAP_DUMP_PATH) as? JsonPrimitive)?.content } + else -> emptyList() +} + /** Null for a tool this build has no verb for, which is what a test asserts never happens. */ internal fun verbOfTool( tool: String, arguments: Map ): String? = when (tool) { "open_heap_dumps" -> "Asked which heap dumps are open" - "list_leaks" -> "Listed the leaks" - "describe_object" -> "Described" + // Ending on "the", because what follows it is the link. See [AgentSessionCall.screen]. + "list_leaks" -> "Listed the" + // Not "Described", which reads as the agent having written a description of something rather than having + // asked what it is. Every tool here is a read unless it says otherwise, and the verbs have to say which. + "describe_object" -> "Looked at" "chain_from_gc_root" -> "Read the chain to" "ways_held" -> "Looked for every way of holding" - "find_objects" -> "Searched for" - // Both of these are about the whole heap dump when they name nothing, so the verb has to stand on its - // own: a row reads as the verb and then the subject, and "Read the notes on" alone says nothing. - "dominator_tree" -> - if (SUBJECT_OBJECT in arguments) "Read the dominator tree under" else "Read the dominator tree" + // Which is a search of the whole dump when it names no class, and that is the list of the biggest + // objects rather than a search for nothing. + "find_objects" -> if (SUBJECT_CLASS_NAME in arguments) "Searched for" else "Listed the" + "dominator_tree" -> if (SUBJECT_OBJECT in arguments) "Read the dominator tree under" else "Read the" "set_verdict" -> "Recorded ${arguments[SUBJECT_VERDICT] ?: "a verdict"} on" "clear_verdict" -> "Took the verdict off" "read_notes" -> if (SUBJECT_PLACE in arguments) "Read the notes on" else "Read what has been written" @@ -509,11 +573,12 @@ internal fun verbOfTool( // which is the one thing an agent does here that a reader can't get back. "take_note" -> if (arguments[SUBJECT_REPLACE] == "true") "Rewrote the note on" else "Wrote a note on" // Reading what other agents did, which is the one call whose subject is another session of this screen. - "agent_log" -> if (SUBJECT_SESSION in arguments) "Read what an agent did in" else "Read the agent log" + "agent_log" -> if (SUBJECT_SESSION in arguments) "Read what an agent did in" else "Read the" "show" -> "Showed" "conclude" -> "Concluded about" - // The app rather than a heap dump, so each of these says the whole of what it did: there is no subject - // to put after it, the heap dump it opens not existing as a place until it is open. + // The app rather than a heap dump, so each of these says the whole of what it did: there is no place of + // an open dump to go to, the file one of them opens and the file another one writes not being one until + // the call has been answered. "open_heap_dump" -> "Opened ${arguments[SUBJECT_PATH] ?: "a heap dump"}" "list_devices" -> arguments[SUBJECT_DEVICE] ?.let { "Listed the processes of $it" } @@ -522,10 +587,46 @@ internal fun verbOfTool( else -> null } -/** What `conclude` answers with the reference under, which is the one answer this file records. */ +/** + * A screen of the heap dump a whole call was about: what to call it inside a sentence, and where it is. + * + * One thing rather than two because the words and the place cannot be allowed to disagree — words with no + * place are a link to nothing, and a place with no words is a call that went somewhere the reader is never + * shown. `AgentTools.placeOrNull` reads the place off this, and the *Agent logs* screen draws the words. + */ +internal class AgentScreen( + val words: String, + val place: Place +) + +/** + * Which screen a call that named nothing was about, and null for a call that named something. + * + * The calls that name nothing are the ones where naming nothing *means* something: the leaks, the agent log + * as a list, and the two tools that mean the whole heap dump when they are given no object — the tree from + * its root, and the list of every object. + */ +internal fun screenOfTool( + tool: String, + arguments: Map +): AgentScreen? = when (tool) { + "list_leaks" -> AgentScreen("leaks", Place.Leaks()) + "agent_log" -> if (SUBJECT_SESSION in arguments) null else AgentScreen("agent log", Place.AgentLogs) + "find_objects" -> + if (SUBJECT_CLASS_NAME in arguments) null else AgentScreen("biggest objects", Place.Objects()) + "dominator_tree" -> + if (SUBJECT_OBJECT in arguments) null else AgentScreen("dominator tree", Place.wholeHeapDump()) + else -> null +} + +/** What `conclude` answers with the reference under, which is one of the two answers this file records. */ private const val ANSWER_FAULTY_REFERENCE = "faultyReference" private const val ANSWER_REFERENCE = "reference" +/** And what `open_heap_dumps` answers with the dumps under. See `AgentJson.heapDump`. */ +private const val ANSWER_HEAP_DUMPS = "heapDumps" +private const val ANSWER_HEAP_DUMP_PATH = "heapDumpPath" + private const val SUBJECT_OBJECT = "object" private const val SUBJECT_PLACE = "place" private const val SUBJECT_CLASS_NAME = "className" diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index a6e07edfd2..c0c2a7eaa7 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -233,13 +233,13 @@ internal class AgentTools( } private fun findObjects() = AgentTool( - name = "find_objects", + name = FIND_OBJECTS, description = "The objects of this heap dump whose class name matches, largest retained size first, " + "with how many matched in total. Use it on a class you have assumed something about: two instances " + "of a class you took for a singleton is the answer to a surprising number of leaks, because the " + "object on the chain then isn't the instance you thought it was. **With no className it is every " + "object, so it is also the answer to \"what are the biggest things in this heap\"** — one object at " + - "a time, where dominator_tree is what holds them.", + "a time, where $DOMINATOR_TREE is what holds them.", schema = schema( WINDOW to window(), CLASS_NAME to string("Matched against the class name.").optional(), @@ -272,7 +272,7 @@ internal class AgentTools( } private fun dominatorTree() = AgentTool( - name = "dominator_tree", + name = DOMINATOR_TREE, description = "Where the memory has gone: what holds the most of it, what holds the most of that, and " + "so on. The tree the window draws as a treemap, without the pixels. Start at the whole heap dump and " + "give `object` to walk down from one node. This answers \"why is this app using 400 MB\" — for " + @@ -671,17 +671,27 @@ internal class AgentTools( * * By argument name, so that a tool added here is described by this without being listed in it: everything * about an object takes `object`, everything about a place takes `place`, the search takes a class name and - * one session of the log takes its id. The two tools whose subject is in none of them name a screen and - * take nothing at all — the leaks, and the log read as a list. + * one session of the log takes its id. + * + * The tools whose subject is in none of them go through [screenOfTool], which is where the screens an + * agent names by naming nothing live: the leaks, the log as a list, and the two that mean the whole heap + * dump when they are given no object — the tree from its root, and the object list unfiltered. Every one of + * them has to be there, because **anything an agent can do that the window can do leads somewhere in the + * window**: a call with no place is a row of the *Agent logs* screen that shows a reader what was looked at + * and then declines to show them the thing. The words that row draws come off the same list, so a screen + * cannot be reachable and unnamed or named and unreachable. + * + * What is left with no place is the calls about the app rather than about a heap dump — which dumps are + * open, which devices are connected, opening a file, taking a dump — and `read_notes` with no place, + * whose answer is the list of places that have notes, which is the tab strip rather than a screen. */ private fun AgentArguments.placeOrNull(name: String): Place? = when { optionalString(PLACE) != null -> place() optionalString(OBJECT) != null -> Place.Object(objectId(OBJECT)) optionalString(CLASS_NAME) != null -> Place.Objects(ObjectListFilter(query = string(CLASS_NAME))) optionalString(SESSION) != null -> Place.AgentLog(string(SESSION)) - name == LIST_LEAKS -> Place.Leaks() - name == AGENT_LOG -> Place.AgentLogs - else -> null + // No arguments, since nothing above matched: what is left is what the tool means on its own. + else -> screenOfTool(name, emptyMap())?.place } /** Which heap dump a call is about, or a refusal naming the ones that are open. */ @@ -768,9 +778,11 @@ internal class AgentTools( const val SET_VERDICT = "set_verdict" const val CONCLUDE = "conclude" - /** Named because [placeOrNull] is the one description of a call that has to know which tool it is. */ + /** Named because another tool's description tells an agent to call it, or its own says what it is. */ const val LIST_LEAKS = "list_leaks" const val AGENT_LOG = "agent_log" + const val FIND_OBJECTS = "find_objects" + const val DOMINATOR_TREE = "dominator_tree" const val WINDOW = "window" const val SESSION = "session" @@ -803,7 +815,7 @@ internal class AgentTools( * walking a tree would take an hour to reach. */ const val NEXT_WITH_A_NEW_DUMP = "Call $LIST_LEAKS with this window to see what the dump says about " + - "itself, or dominator_tree to see where its memory has gone." + "itself, or $DOMINATOR_TREE to see where its memory has gone." /** * How many objects a list comes back with by default, well under diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt index f1e7c70409..feaa967624 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/McpSession.kt @@ -160,13 +160,15 @@ internal class McpSession( val startedAt = System.nanoTime() return try { val answer = tool.call(arguments) - // The answer as well as the arguments, because one of them is a conclusion: see [outcomeOfTool]. + // The answer as well as the arguments, because two of them are things the arguments don't say: what + // was concluded, and which heap dumps were open. See [outcomeOfTool] and [openHeapDumpsOfTool]. record( name, arguments, target, refusal = null, outcome = outcomeOfTool(name, answer), + openHeapDumps = openHeapDumpsOfTool(name, answer), at = at, startedAt = startedAt ) @@ -202,6 +204,7 @@ internal class McpSession( target: AgentTarget, refusal: String?, outcome: String?, + openHeapDumps: List = emptyList(), at: Instant, startedAt: Long ) { @@ -216,6 +219,7 @@ internal class McpSession( arguments = arguments.recorded(), refusal = refusal, outcome = outcome, + openHeapDumps = openHeapDumps, millis = (System.nanoTime() - startedAt) / NANOS_PER_MILLI ) ) diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt index e8a35ddf6b..1dc85702f3 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentSessionFileTest.kt @@ -149,6 +149,67 @@ class AgentSessionFileTest { assertThat(withoutAVerb).isEmpty() } + @Test + fun `a call that named nothing goes where the tool means, whether or not the file says so`() { + val tools = agentTools(FakeAgentHeapDumps()) + + // Both sides read [AgentScreen], so a screen cannot end up reachable and unnamed or named and + // unreachable: what a call with no arguments is about is the place, and the words are what the row of + // the *Agent logs* screen draws as the link to it. + tools.all.map { it.name }.forEach { name -> + assertThat(tools.target(name, buildJsonObject { }).place) + .describedAs(name) + .isEqualTo(screenOfTool(name, emptyMap())?.place) + } + } + + @Test + fun `a session written before a call like that had a link still leads where it went`() { + directory.mkdirs() + // Which is every session on this machine, since the place of these was worked out from the arguments + // and they have none: a row of one that leads nowhere is the bug this fixed, kept fixed for the + // sessions that were already on disk. + File(directory, "agent-2026-08-25_18-19-48_035-older.jsonl").writeText( + """{"agentSession":"older","startedAt":"$STARTED_AT","sharkExplorer":"1.0.0"}""" + "\n" + + """{"at":"$STARTED_AT","tool":"dominator_tree","reason":"Where the memory went.",""" + + """"window":"$WINDOW_ID","heapDump":"/dumps/leak.hprof","millis":3}""" + "\n" + ) + + val call = AgentSessionFile.sessionsIn(directory).single().calls.single() + assertThat(call.place).isEqualTo(Place.wholeHeapDump()) + assertThat(call.screen).isEqualTo("dominator tree") + } + + @Test + fun `which heap dumps were open is read off the answer, and nothing else is`() { + val answered = buildJsonObject { + putJsonArray("heapDumps") { + addJsonObject { + put("window", WINDOW_ID) + put("heapDumpPath", "/dumps/leak.hprof") + } + } + } + + assertThat(openHeapDumpsOfTool("open_heap_dumps", answered)).containsExactly("/dumps/leak.hprof") + // Every other call is about a heap dump rather than about which ones there are, and a row of them + // listing the dumps would be the window's own state printed against somebody's investigation. + assertThat(openHeapDumpsOfTool("list_leaks", answered)).isEmpty() + } + + @Test + fun `the heap dumps a call was answered with are read back as somewhere to go`() { + val file = AgentSessionFile.starting(directory, SERVER_VERSION) + file.called( + call(tool = "open_heap_dumps", openHeapDumps = listOf("/dumps/leak.hprof", "/dumps/other.hprof")) + ) + + // In the order they were open in, because that is the order the window unfolds them in — and paths, + // since the window ids beside them belonged to a run that has ended by the time this is read. + assertThat(AgentSessionFile.sessionsIn(directory).single().calls.single().openHeapDumps) + .containsExactly("/dumps/leak.hprof", "/dumps/other.hprof") + } + @Test fun `a directory no agent has ever connected through is no sessions rather than a failure`() { assertThat(AgentSessionFile.sessionsIn(File(temporaryFolder.root, "never-used"))).isEmpty() @@ -160,7 +221,8 @@ class AgentSessionFileTest { place: Place? = null, arguments: Map = emptyMap(), refusal: String? = null, - outcome: String? = null + outcome: String? = null, + openHeapDumps: List = emptyList() ) = AgentSessionCall( at = STARTED_AT, tool = tool, @@ -171,6 +233,7 @@ class AgentSessionFileTest { arguments = arguments, refusal = refusal, outcome = outcome, + openHeapDumps = openHeapDumps, millis = 12L ) diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index 1dcbebcb8c..47df271291 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -207,7 +207,7 @@ class McpSessionTest { assertThat(session.client).isEqualTo("claude-code 9.9.9") assertThat(session.serverVersion).isEqualTo(SERVER_VERSION) val call = session.calls.single() - assertThat(call.verb).isEqualTo("Described") + assertThat(call.verb).isEqualTo("Looked at") assertThat(call.subject).isEqualTo(hex(heapDump.holderObjectId)) assertThat(call.reason).isEqualTo("Checking whether the holder is the singleton it looks like.") assertThat(call.refusal).isNull() @@ -237,14 +237,41 @@ class McpSessionTest { fun `a call that named no place is written down with somewhere to go all the same`() { callTool("""{"name":"list_leaks","arguments":{"reason":"Starting with what the dump says."}}""") - // The leaks screen to go to, and nothing after the verb, which already says the whole of what this call - // did. The one place a call is about without naming it in an argument. See [AgentTools.target]. + // The leaks screen to go to, and the words for it, so that a row of the window reads as a sentence with + // one link in it: "Listed the" and then *leaks*. See [AgentTools.target] and [screenOfTool]. val call = sessions().single().calls.single() - assertThat(call.verb).isEqualTo("Listed the leaks") + assertThat(call.verb).isEqualTo("Listed the") + assertThat(call.screen).isEqualTo("leaks") assertThat(call.place).isEqualTo(Place.Leaks()) assertThat(call.subject).isNull() } + @Test + fun `a call about the whole heap dump goes to the whole heap dump`() { + callTool("""{"name":"dominator_tree","arguments":{"reason":"Where has the memory gone."}}""") + callTool("""{"name":"find_objects","arguments":{"reason":"What the biggest objects are."}}""") + + // Both of these mean the whole heap dump when they name nothing in it, and both are something a person + // does in this window — so both lead there, rather than being the two rows of a session that show a + // reader what was looked at and then decline to show them the thing. + val calls = sessions().single().calls + assertThat(calls.map { it.verb }).containsExactly("Read the", "Listed the") + assertThat(calls.map { it.screen }).containsExactly("dominator tree", "biggest objects") + assertThat(calls.map { it.place }).containsExactly(Place.wholeHeapDump(), Place.Objects()) + } + + @Test + fun `the call that asks which heap dumps are open is written down with the ones that were`() { + callTool("""{"name":"open_heap_dumps","arguments":{"reason":"Seeing what there is."}}""") + + // Off the answer, like a conclusion and unlike everything else: this is the one call whose subject is + // the app rather than a heap dump, and a row saying it asked without saying what it heard is a row that + // withholds the answer it is a record of. See [openHeapDumpsOfTool]. + val call = sessions().single().calls.single() + assertThat(call.openHeapDumps).containsExactly(window.heapDumpPath) + assertThat(call.place).isNull() + } + @Test fun `a call that concluded is written down with the reference it concluded on`() { callTool( diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt index 10e4390b6d..98bb16107f 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt @@ -14,6 +14,10 @@ import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Surface import androidx.compose.material3.Text import androidx.compose.runtime.Composable +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue import androidx.compose.ui.Modifier import androidx.compose.ui.unit.dp import java.io.File @@ -24,6 +28,7 @@ import shark.SharkLog import shark.explorer.Place import shark.explorer.agent.AgentSession import shark.explorer.agent.AgentSessionCall +import shark.explorer.agent.screen import shark.explorer.agent.subject import shark.explorer.agent.verb @@ -83,11 +88,14 @@ internal fun AgentLogsScreen( } /** - * One agent's session: read in this window when the heap dump it read is the one open here, and otherwise a - * way to that dump. + * One agent's session: read in a window of the heap dump it read, which is this one when it read this one. * - * A session that read no dump at all — a client that connected and asked nothing — leads nowhere, and neither - * does one whose dump has been deleted: a session outlives the files it was about. + * **Every session on this screen leads to itself**, and where it opens is the only question. A session of + * another dump opens in a window of that dump, because its addresses are that file's; the ones with no such + * window to be had — this window's own, one whose dump has been deleted, one that read no dump at all — are + * read here. Reading a session against the wrong heap dump costs the names of the objects in it and nothing + * else: the verbs, the reasons and the refusals are what the agent said, and an address whose dump has gone + * resolves to nothing in any window there is. */ @Composable private fun SessionRow( @@ -100,29 +108,22 @@ private fun SessionRow( val place = Place.AgentLog(session.sessionId) val title = session.title() val summary = session.summary() - if (group.isThisWindow) { - val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } - OpenTarget(open, { onCopyLink(place) }) { - Column(Modifier.openable(open)) { - Text(title, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) - Text(summary, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) - } + val opensHeapDump = group.heapDumpFile?.takeIf { !group.isThisWindow && it.isFile } + if (opensHeapDump != null) { + // No tab to choose and no link to copy: what a link names is a window, and the window this session was + // read in belongs to a run that has usually ended. The heap dump is what outlived it. + Column(Modifier.openable { onOpenHeapDump(opensHeapDump, place) }) { + Text(title, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) + Text(summary, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) } return } - val opens = group.heapDumpFile?.takeIf { it.isFile } - if (opens == null) { - Column { - Text(title, style = MaterialTheme.typography.bodyMedium) + val open: (OpenIn) -> Unit = { openIn -> onOpen(place, openIn) } + OpenTarget(open, { onCopyLink(place) }) { + Column(Modifier.openable(open)) { + Text(title, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) Text(summary, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) } - return - } - // No tab to choose and no link to copy: what a link names is a window, and the window this session was - // read in belongs to a run that has usually ended. The heap dump is what outlived it. - Column(Modifier.openable { onOpenHeapDump(opens, place) }) { - Text(title, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) - Text(summary, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) } } @@ -160,8 +161,13 @@ private fun List.byHeapDump(heapDumpFile: File): List 1) " ($seen)" else "") + - (if (isThisWindow) " ($THIS_HEAP_DUMP)" else ""), + // And said to be gone where it is, since that is why the objects in those sessions have no names: + // there is no window that can resolve an address of a file nobody has any more. + label = file.name + (if (seen > 1) " ($seen)" else "") + when { + isThisWindow -> " ($THIS_HEAP_DUMP)" + !file.isFile -> " ($MISSING_HEAP_DUMP)" + else -> "" + }, heapDumpFile = file, isThisWindow = isThisWindow, sessions = filter { path in it.heapDumpPaths } @@ -253,10 +259,15 @@ internal fun AgentLogScreen( /** * One call: when, what it did, and why the agent said it was doing it. * - * **What leads somewhere is the object, not the verb.** A row is a sentence about a thing — "Described - * MainActivity 0x12d368b8" — and the thing is what a reader wants to go and look at, so it is the only part - * that is a link. Where the call named nothing, the verb is the whole of what it was about and is the link - * itself: "Listed the leaks" is the leaks screen. + * **What leads somewhere is the thing, never the verb.** A row is a sentence about something — "Looked at + * MainActivity 0x12d368b8", "Listed the leaks" — and the thing is what a reader wants to go and look at, so + * it is the only part that is a link. Which is why a verb ends where the thing begins even when that leaves + * it hanging: "Listed the" is prose and *leaks* is the leaks screen. See `shark.explorer.agent.verb`. + * + * **And there is a thing for every call that went anywhere.** An object the agent named is named back by + * this window; a call that named nothing went to a screen of the dump all the same, and the words for that + * come with the verb. A row with no link is a call about the app rather than about a heap dump — which dumps + * are open, which devices are connected — or one about a heap dump that has since been deleted. */ @Composable private fun AgentCallRow( @@ -275,12 +286,16 @@ private fun AgentCallRow( // And whether that is still possible. A session outlives the heap dumps it was about, so a row naming one // that has been deleted says which and leads nowhere. val opens = elsewhere?.takeIf { it.isFile } - // What the call itself said it was about, and null for the calls that named nothing — where the verb says - // the whole of it. Only those are named by this window: naming a place derived from which tool it is would - // put "Leaks" after "Listed the leaks", and a call about another dump names a file this window never read. + // What the call was about: the object it named, in this window's words for it, or the screen it went to in + // the words that came with the verb. A place this window named would be the wrong words for a sentence — + // "Listed the Leaks" — and a call about another dump names a file this window has never read. val target = call.subject?.let { subject -> if (elsewhere == null) place?.let { placeTitles[it] } ?: subject else subject - } + } ?: call.screen + // The heap dumps the answer named, for the one call that asks the app which are open: several rows rather + // than one, so they are behind the verb until somebody asks for them. + val openHeapDumps = call.openHeapDumps + var isUnfolded by remember { mutableStateOf(false) } Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) { Text( call.at.clockTime(), @@ -289,30 +304,33 @@ private fun AgentCallRow( color = MUTED_TEXT ) Column { - // Nowhere to go for a call about the app rather than about a heap dump — which dumps are open — or - // about one that has since been deleted. + // Nowhere to go for a call about the app rather than about a heap dump, or about one that has since + // been deleted. val leadsTo = place?.takeIf { elsewhere == null || opens != null } // Wrapped rather than truncated, since a class name is as long as it is and the reason under it is a // sentence: this row is read, not scanned past. FlowRow(horizontalArrangement = Arrangement.spacedBy(4.dp)) { - // The link is the target, or the verb where the call named no target: what a reader clicks is the - // thing, and a verb that is the whole sentence is the thing. - val linked = target ?: call.verb - if (target != null) { - Text(call.verb, style = MaterialTheme.typography.bodyMedium) - } when { - leadsTo == null -> Text(linked, style = MaterialTheme.typography.bodyMedium) - opens != null -> LinkText(linked, Modifier.openable { onOpenHeapDump(opens, leadsTo) }) + openHeapDumps.isNotEmpty() -> UnfoldableVerb(call.verb, isUnfolded) { isUnfolded = !isUnfolded } + target == null -> Text(call.verb, style = MaterialTheme.typography.bodyMedium) else -> { - val open: (OpenIn) -> Unit = { openIn -> onOpen(leadsTo, openIn) } - OpenTarget(open, { onCopyLink(leadsTo) }) { LinkText(linked, Modifier.openable(open)) } + Text(call.verb, style = MaterialTheme.typography.bodyMedium) + when { + leadsTo == null -> Text(target, style = MaterialTheme.typography.bodyMedium) + opens != null -> LinkText(target, Modifier.openable { onOpenHeapDump(opens, leadsTo) }) + else -> { + val open: (OpenIn) -> Unit = { openIn -> onOpen(leadsTo, openIn) } + OpenTarget(open, { onCopyLink(leadsTo) }) { LinkText(target, Modifier.openable(open)) } + } + } } } // What the answer came to, and — for a row that opens another dump when clicked — which dump that - // is: worth knowing before rather than after. + // is: worth knowing before rather than after. Not on the row whose answer is a list of dumps: the + // file that one was recorded against is whichever was open, and what it came back with is below it. call.outcome?.let { Text("$LEADS_TO $it", style = MaterialTheme.typography.bodyMedium) } - elsewhere?.let { Text("$IN ${it.name}", style = MaterialTheme.typography.bodyMedium) } + elsewhere?.takeIf { openHeapDumps.isEmpty() } + ?.let { Text("$IN ${it.name}", style = MaterialTheme.typography.bodyMedium) } } call.reason?.let { reason -> // The agent's own sentence, indented under what it did: read down the column of these and a session @@ -326,10 +344,77 @@ private fun AgentCallRow( color = MaterialTheme.colorScheme.error ) } + if (isUnfolded) { + openHeapDumps.forEach { path -> + OpenHeapDumpRow(path, heapDumpFile, onOpenHeapDump) + } + } } } } +/** + * A verb with what is behind it, for the call whose answer is a list rather than a thing. + * + * The verb itself is what opens it, since there is no thing on that row to be a link — and the arrow is the + * same one the leaks screen folds its sections with, because it is the same gesture on a screen somebody + * reads straight after that one. + */ +@Composable +private fun UnfoldableVerb( + verb: String, + isUnfolded: Boolean, + onToggle: () -> Unit +) { + Row( + Modifier.clickableRow(onClick = onToggle), + horizontalArrangement = Arrangement.spacedBy(4.dp) + ) { + Text( + if (isUnfolded) EXPANDED_ARROW else FOLDED_ARROW, + style = MaterialTheme.typography.bodyMedium, + color = MUTED_TEXT + ) + Text(verb, style = MaterialTheme.typography.bodyMedium) + } +} + +/** + * One of the heap dumps a call was answered with, as somewhere to go. + * + * Which is why unfolding the row is worth anything: the answer to "which dumps are open" is a list of the + * files an investigation could have been about, and the run that had them open has ended by the time + * anybody reads this — so a name with nothing behind it would be the one part of a session a reader is shown + * and cannot follow. The dump this window has open is the exception, and says so rather than leading to the + * window it already is. + */ +@Composable +private fun OpenHeapDumpRow( + path: String, + heapDumpFile: File, + onOpenHeapDump: (File, Place) -> Unit +) { + val file = File(path) + val name = file.name + val style = MaterialTheme.typography.bodySmall + val indent = Modifier.padding(start = UNFOLDED_INSET) + when { + path == heapDumpFile.absolutePath -> + Text("$name ($THIS_HEAP_DUMP)", indent, style = style, color = MUTED_TEXT) + // Gone, which a list of what *was* open is exactly where somebody finds out. + !file.isFile -> Text("$name ($MISSING_HEAP_DUMP)", indent, style = style, color = MUTED_TEXT) + // The whole heap dump, since a dump named without a place in it is the window that dump opens on. No + // tab to choose and no link to copy, for the reason a session of another dump has neither: what a link + // names is a window, and this is a file that has to be opened in one first. + else -> Text( + name, + indent.openable { onOpenHeapDump(file, Place.wholeHeapDump()) }, + style = style, + color = LINK_COLOR + ) + } +} + /** * The heap dump the call was about when it is one this window hasn't got open, and null when it has. * @@ -395,6 +480,15 @@ private const val A_CLIENT_THAT_DID_NOT_SAY = "An agent" /** After the heap dump this window has open, which is the one group of sessions that is read here. */ private const val THIS_HEAP_DUMP = "this heap dump" +/** + * And after one that isn't on this machine any more, which is why the objects in those sessions have no + * names: an address is an address of a file, and that file has gone. + */ +private const val MISSING_HEAP_DUMP = "missing" + +/** How far the rows behind a verb sit in from it, which is the arrow's width and the gap after it. */ +private val UNFOLDED_INSET = 20.dp + /** And over the sessions of a client that connected and read nothing, which no window can be about. */ private const val NO_HEAP_DUMP_READ = "No heap dump" diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt index f152cdfc66..685bfba09a 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt @@ -9,7 +9,9 @@ import androidx.compose.ui.test.SemanticsMatcher import androidx.compose.ui.test.assertHasClickAction import androidx.compose.ui.test.assertHasNoClickAction import androidx.compose.ui.test.assertIsDisplayed +import androidx.compose.ui.test.assertTextContains import androidx.compose.ui.test.hasText +import androidx.compose.ui.test.isSelected import androidx.compose.ui.test.onAllNodesWithText import androidx.compose.ui.test.onNodeWithText import androidx.compose.ui.test.performClick @@ -24,6 +26,7 @@ import org.junit.rules.TemporaryFolder import shark.explorer.Adb import shark.explorer.AdbOutput import shark.explorer.DeviceHeapDumps +import shark.explorer.HeapDominatorTreemap import shark.explorer.Place import shark.explorer.agent.AgentSession import shark.explorer.agent.AgentSessionCall @@ -64,7 +67,7 @@ class AgentLogsScreenTest { // The verb, the object named the way a tab on it is named, and the agent's own sentence for why it // asked: no JSON and no bare address on any of it. waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) - waitUntilAtLeastOneExists(hasText(DESCRIBED), OPEN_TIMEOUT_MILLIS) + waitUntilAtLeastOneExists(hasText(LOOKED_AT), OPEN_TIMEOUT_MILLIS) waitUntilAtLeastOneExists(hasText(activityName()), OPEN_TIMEOUT_MILLIS) } } @@ -76,29 +79,66 @@ class AgentLogsScreenTest { waitUntilAtLeastOneExists(hasText(activityName()), OPEN_TIMEOUT_MILLIS) // What a reader wants to go and look at is the object, so that is the whole of what leads anywhere: - // a row where clicking the word "Described" navigates is a row with a hand cursor over prose. + // a row where clicking the word "Looked at" navigates is a row with a hand cursor over prose. onNodeWithText(activityName()).assertHasClickAction() - onNodeWithText(DESCRIBED).assertHasNoClickAction() + onNodeWithText(LOOKED_AT).assertHasNoClickAction() } } - @Test fun `a call that named nothing is the verb itself, and leads where it went`() { + @Test fun `a call that named nothing links the words for where it went, and not the verb`() { explorerUiTest { openAgentLogs(listOf(session(calls = listOf(leaksCall())))) onNodeWithText(CLIENT, substring = true).performClick() - waitUntilAtLeastOneExists(hasText(LISTED_THE_LEAKS), OPEN_TIMEOUT_MILLIS) + waitUntilAtLeastOneExists(hasText(LISTED_THE), OPEN_TIMEOUT_MILLIS) - // "Listed the leaks" is the whole sentence and the leaks screen is where it went, so the verb is the - // link — and nothing follows it. A screen name after it, from the tool rather than from an argument, - // read as "Listed the leaks Leaks". - onNodeWithText(LISTED_THE_LEAKS).assertHasClickAction() - onNodeWithText(LISTED_THE_LEAKS).performClick() + // "Listed the leaks" is a sentence about the leaks screen, so *leaks* is the link and what comes + // before it is prose. Linking the whole of it was underlining a verb; naming the place from the tool + // instead read as "Listed the leaks Leaks". + onNodeWithText(LISTED_THE).assertHasNoClickAction() + onNodeWithText(LEAKS).assertHasClickAction() + onNodeWithText(LEAKS).performClick() // The leaks screen, named by the reference each leak is: the same screen the agent was reading. waitUntilAtLeastOneExists(hasText(ACTIVITY_LEAK_NAME, substring = true), OPEN_TIMEOUT_MILLIS) } } + @Test fun `a call that read the tree from its root leads to the whole heap dump`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(wholeDumpCall(tool = "dominator_tree"))))) + onNodeWithText(CLIENT, substring = true).performClick() + waitUntilAtLeastOneExists(hasText(DOMINATOR_TREE), OPEN_TIMEOUT_MILLIS) + + // Reading the tree from its root is what this window opens on, so a row saying an agent did it leads + // there. Anything an agent can do that a person can do here is a row that goes where they went. + onNodeWithText(READ_THE).assertHasNoClickAction() + // The tab is on the agent's log until the row moves it, and then on the tree from its root, which is + // where a window opens and what the row said the agent read. + selectedTab().assertTextContains(Place.AgentLog(SESSION_ID).title) + onNodeWithText(DOMINATOR_TREE).performClick() + + waitUntilAtLeastOneExists( + hasText(HeapDominatorTreemap.ROOT_LABEL) and isTab() and isSelected(), + OPEN_TIMEOUT_MILLIS + ) + } + } + + @Test fun `a search of the whole heap dump leads to the list of every object`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(wholeDumpCall(tool = "find_objects"))))) + onNodeWithText(CLIENT, substring = true).performClick() + waitUntilAtLeastOneExists(hasText(BIGGEST_OBJECTS), OPEN_TIMEOUT_MILLIS) + + // A search with no class name is the biggest objects of the dump, which is the object list unfiltered + // — and reads as that rather than as "Searched for" with nothing after it. + onNode(hasText(Place.OBJECTS_LABEL) and isTab()).assertDoesNotExist() + onNodeWithText(BIGGEST_OBJECTS).performClick() + + waitUntilAtLeastOneExists(hasText(Place.OBJECTS_LABEL) and isTab(), OPEN_TIMEOUT_MILLIS) + } + } + @Test fun `a refused call says so, and still says what it was about`() { explorerUiTest { openAgentLogs(listOf(session(calls = listOf(call(tool = "conclude", refusal = REFUSAL))))) @@ -189,9 +229,51 @@ class AgentLogsScreenTest { assertThat(opened).isEqualTo(otherHeapDump to Place.Object(activityObjectId())) } + @Test fun `the call that asked which heap dumps are open unfolds into them`() { + val otherHeapDump = testFolder.newFile("another.hprof") + var opened: Pair? = null + explorerUiTest { + openAgentLogs( + sessions = listOf(session(calls = listOf(openHeapDumpsCall(otherHeapDump)))), + onOpenHeapDump = { file, place -> opened = file to place } + ) + thisWindowsSession().performClick() + waitUntilAtLeastOneExists(hasText(ASKED_WHICH_ARE_OPEN), OPEN_TIMEOUT_MILLIS) + + // Behind the verb until somebody asks, because what this one came back with is a list where every + // other row of a session is a sentence. + onNodeWithText(otherHeapDump.name).assertDoesNotExist() + onNodeWithText(ASKED_WHICH_ARE_OPEN).performClick() + + // The dump this window has open says so and leads nowhere — it is already here — and the other one is + // a window away, which is what makes a list of what *was* open worth keeping. + waitUntilAtLeastOneExists(hasText(thisHeapDumpRow()), OPEN_TIMEOUT_MILLIS) + onNodeWithText(thisHeapDumpRow()).assertHasNoClickAction() + onNodeWithText(otherHeapDump.name).performClick() + } + + assertThat(opened).isEqualTo(otherHeapDump to Place.wholeHeapDump()) + } + + @Test fun `a session about a heap dump that has gone is read here, there being no window for it`() { + explorerUiTest { + openAgentLogs(listOf(session(calls = listOf(call(heapDumpPath = DELETED_HEAP_DUMP))))) + + // Headed as gone, which is the answer to why the objects in it have no names: an address is an + // address of a file, and that file isn't here. + onNodeWithText("deleted.hprof ($MISSING)").assertIsDisplayed() + onNodeWithText(CLIENT, substring = true).performClick() + + // And read here rather than nowhere. There is no window that could name those addresses, so what is + // left is what the agent said — the verbs, the reasons and the refusals — which is worth reading. + waitUntilAtLeastOneExists(hasText(REASON, substring = true), OPEN_TIMEOUT_MILLIS) + onNodeWithText("in deleted.hprof").assertIsDisplayed() + } + } + @Test fun `a call about a heap dump that has gone leads nowhere`() { explorerUiTest { - openAgentLogs(listOf(session(calls = listOf(call(), call(heapDumpPath = "/dumps/deleted.hprof"))))) + openAgentLogs(listOf(session(calls = listOf(call(), call(heapDumpPath = DELETED_HEAP_DUMP))))) thisWindowsSession().performClick() waitUntilAtLeastOneExists(hasText(activityName()), OPEN_TIMEOUT_MILLIS) @@ -281,6 +363,42 @@ class AgentLogsScreenTest { millis = 12L ) + /** + * A call that named nothing in the dump and so was about the whole of it: the tree from its root, or the + * list of every object. See `AgentTools.placeOrNull`. + */ + private fun wholeDumpCall(tool: String) = AgentSessionCall( + at = STARTED_AT, + tool = tool, + reason = REASON, + windowId = "zvphq4r3", + heapDumpPath = heapDump.file.absolutePath, + place = if (tool == "find_objects") Place.Objects() else Place.wholeHeapDump(), + arguments = emptyMap(), + refusal = null, + outcome = null, + millis = 12L + ) + + /** The first call of most sessions: which dumps are open, answered with the ones that were. */ + private fun openHeapDumpsCall(otherHeapDump: File) = AgentSessionCall( + at = STARTED_AT, + tool = "open_heap_dumps", + reason = REASON, + windowId = "zvphq4r3", + heapDumpPath = heapDump.file.absolutePath, + // Nowhere to go: this one asks the app rather than a heap dump. What it came back with is where it goes. + place = null, + arguments = emptyMap(), + refusal = null, + outcome = null, + openHeapDumps = listOf(heapDump.file.absolutePath, otherHeapDump.absolutePath), + millis = 12L + ) + + /** How the dump this window has open reads in a list of the dumps that were open. */ + private fun thisHeapDumpRow() = "${heapDump.file.name} (this heap dump)" + private fun activityObjectId() = heapDump.activityObjectIds.first() /** How the window names the activity: the same title the tab a row opens carries. */ @@ -298,6 +416,13 @@ class AgentLogsScreenTest { private fun isButton(): SemanticsMatcher = SemanticsMatcher.expectValue(SemanticsProperties.Role, Role.Button) + /** Where the window is, since a row of this screen moves the tab it is read in. */ + private fun ComposeUiTest.selectedTab() = onNode(isTab() and isSelected()) + + /** The tab a row opened, as against the button of the same name that would have opened it too. */ + private fun isTab(): SemanticsMatcher = + SemanticsMatcher.expectValue(SemanticsProperties.Role, Role.Tab) + private companion object { const val SESSION_ID = "1a2b3c4d" @@ -307,10 +432,27 @@ class AgentLogsScreenTest { const val FAULTY_REFERENCE = "Holder.activity" const val NO_AGENT_YET = "No agent has worked on this heap dump" + /** A heap dump a session was about and nobody has any more. */ + const val DELETED_HEAP_DUMP = "/dumps/deleted.hprof" + + /** After the name of one, wherever this screen names a dump that isn't there. */ + const val MISSING = "missing" + /** The verbs the rows read as, which are [shark.explorer.agent.verb]'s and not this screen's. */ - const val DESCRIBED = "Described" + const val LOOKED_AT = "Looked at" const val CONCLUDED_ABOUT = "Concluded about" - const val LISTED_THE_LEAKS = "Listed the leaks" + const val ASKED_WHICH_ARE_OPEN = "Asked which heap dumps are open" + + /** + * And the verbs of the calls that named nothing, which stop where the link starts: the words after each + * of these are [shark.explorer.agent.screen]'s. See [LEAKS], [DOMINATOR_TREE] and [BIGGEST_OBJECTS]. + */ + const val LISTED_THE = "Listed the" + const val READ_THE = "Read the" + + const val LEAKS = "leaks" + const val DOMINATOR_TREE = "dominator tree" + const val BIGGEST_OBJECTS = "biggest objects" val STARTED_AT: Instant = Instant.parse("2026-08-25T18:19:48.035Z") From c0297d6020c850545762cd4d62c35f36f1c8eb43 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Thu, 27 Aug 2026 16:02:42 +0200 Subject: [PATCH 24/27] Name the heap dump in a link, and the window only as a refinement MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A link was `shark:///`, which made it die with the window — and that is most links a day later, and most links in an agent's session log, since a session outlives the run that wrote it. Every place a link can name belongs to the heap dump rather than to whatever is showing it, so the dump is now the identity: shark:///?[&dump=][&window=] The file name is the authority because it is what a person reads and types and what every agent answer already carries; `dump=` is the normalized path, since two `com.squareup.hprof` off two devices are two investigations and a path is the only thing that can open a dump nothing has open. `window=` is honoured while that window exists and ignored once it doesn't, rather than turning the link into an error — which keeps the case the window id was for, the same dump open twice being two readings of it compared. Links already written into notes and session files still resolve: the last step of `ExplorerWindows.windowFor` matches the authority against a window id. A run claims a link only for a window it already has, never for a file it could open, or every run would claim every link. Two things this unlocks, and the reason to reverse it rather than live with it. A `--no-ui` run answers `show` with a link now — no window and the file all the same — instead of saying there was nothing to hand back. And every *Agent logs* row about another heap dump has a link to copy, where before there was nothing to send. On the agent surface the argument is `heapDump`, taking a file name, with a window id only where a name cannot answer. --- .claude/skills/shark-explorer/SKILL.md | 9 +- docs/shark-explorer-changelog.md | 16 +- docs/shark-explorer.md | 52 ++++--- shark/shark-explorer/AGENTS.md | 19 ++- shark/shark-explorer/notes/decisions.md | 41 ++++- .../shark-explorer-agent/AGENTS.md | 7 +- .../shark/explorer/agent/AgentCommandLine.kt | 2 +- .../shark/explorer/agent/AgentHeapDump.kt | 41 +++-- .../java/shark/explorer/agent/AgentJson.kt | 9 +- .../java/shark/explorer/agent/AgentMethod.kt | 5 +- .../java/shark/explorer/agent/AgentPlace.kt | 6 +- .../shark/explorer/agent/AgentSessionFile.kt | 13 +- .../java/shark/explorer/agent/AgentTools.kt | 96 +++++++----- .../shark/explorer/agent/AgentToolsTest.kt | 57 ++++++- .../shark/explorer/agent/FakeAgentHeapDump.kt | 8 +- .../shark/explorer/agent/McpSessionTest.kt | 11 +- .../shark/explorer/app/AgentLogsScreen.kt | 63 +++++--- .../java/shark/explorer/app/DeepLinkPeers.kt | 27 ++-- .../java/shark/explorer/app/ExplorerAgents.kt | 5 +- .../java/shark/explorer/app/ExplorerWindow.kt | 88 ++++++++--- .../explorer/app/HeadlessAgentHeapDumps.kt | 15 +- .../shark/explorer/app/HeapDumpExplorer.kt | 19 ++- .../shark/explorer/app/AgentLogsScreenTest.kt | 26 +++- .../shark/explorer/app/ExplorerAppTest.kt | 13 +- .../shark/explorer/app/ExplorerWindowTest.kt | 121 ++++++++++++-- .../app/HeadlessAgentHeapDumpsTest.kt | 12 +- .../shark/explorer/app/ObjectsScreenTest.kt | 14 +- .../java/shark/explorer/app/TabStripTest.kt | 21 ++- .../src/main/java/shark/explorer/DeepLink.kt | 105 +++++++++++-- .../main/java/shark/explorer/HeapDumpFiles.kt | 6 +- .../test/java/shark/explorer/DeepLinkTest.kt | 147 +++++++++++++----- 31 files changed, 811 insertions(+), 263 deletions(-) diff --git a/.claude/skills/shark-explorer/SKILL.md b/.claude/skills/shark-explorer/SKILL.md index 263c1406e9..84cd4a8290 100644 --- a/.claude/skills/shark-explorer/SKILL.md +++ b/.claude/skills/shark-explorer/SKILL.md @@ -17,7 +17,7 @@ is leaking still has a biggest object. ## Start by working out which case you are in -**Something is already open.** Ask, and the answer carries the method to follow, the window ids every other +**Something is already open.** Ask, and the answer carries the method to follow, the file names every other tool names a dump by, and any verdicts somebody has already reached: ```bash @@ -76,6 +76,10 @@ An investigation somebody already ran is either the answer or the half of the du stderr is the next thing to do, not an error to retry. **1** means nothing was there to answer it. - **Addresses are `0x…`, exactly as the surface writes them.** Never decimal: a heap dump's addresses do not survive a JSON number. +- **A call is about one heap dump**, and `heapDump=` says which — needed once more than one is + open, and the window id instead in the one case a name cannot answer, which is the same file open twice. + The `shark://` link `show` and `conclude` answer with names the dump too, so it still opens after this run + has ended: **put those links in your reply** rather than describing which screen to open. - `--agent-run=` picks between several open runs. `--agent-session=` says which investigation these calls are one of; by default one shell is one session, so what you did reads as one row of that screen rather than a row per call. @@ -90,7 +94,8 @@ An investigation somebody already ran is either the answer or the half of the du ``` Add `--no-ui` for a machine with no screen — a build server, or a dump at the far end of an ssh session. -Everything works the same except `show`, which has nowhere to put a tab. +Everything works the same except `show`, which has nowhere to put a tab and says so; it still answers with the +link, since a link names the heap dump rather than a window. ## What to do with it diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index 5ee88cf193..fd6f989b43 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -22,7 +22,9 @@ uses, without the one for a newly recognized library leak: * ✨ Right click anything the window can take you to — a tab, a rectangle, a row, a field — and copy a `shark://` link to it, beside opening it in a new tab. Clicking one brings the app to the front and opens that place in a new tab: an object, a filtered object list, the leaks with the same groups - unfolded. See [Link to a tab](shark-explorer.md#link-to-a-tab). + unfolded. A link names the **heap dump**, so it goes on working after the window it was copied from has + gone: it opens the place in a window that has that dump, and opens the file in a new window when none + has. See [Link to a tab](shark-explorer.md#link-to-a-tab). * ✨ **Notes**: every location takes a markdown note, kept between runs, and the tab strip marks the tabs whose location has one. A note belongs to the location rather than to the tab, so two tabs on one location are one note. Class names, addresses and `shark://` links written in a note become links back @@ -55,9 +57,10 @@ uses, without the one for a newly recognized library leak: opens one — on the heap dump its command line named, if it named one — and leaves it open for whoever comes back to it. And with `--no-ui`, the tools are served from that process with no window anywhere, for a build server or a heap dump at the end of an ssh session: everything works the same except `show`, which says it - has nowhere to put a tab rather than answering that it showed you something, and hands back no link since a - link names a window and this run has none. Notes and verdicts were never - on the screen, so a heap dump investigated with no window opens in one later with all of it on. + has nowhere to put a tab rather than answering that it showed you something — and hands back the link all + the same, since a link names the heap dump, so whoever reads the answer can open the place nobody saw. + Notes and verdicts were never on the screen, so a heap dump investigated with no window opens in one later + with all of it on. See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). * ✨ **The method sends an agent to the code, at the version the heap dump is of.** Isolating the reference says where the problem is and not how it happened, so the method that comes with the tools also says how to @@ -72,8 +75,9 @@ uses, without the one for a newly recognized library leak: * ✨ **Agent logs**: every agent that has connected to the app is a row on a screen of its own, and opening one is everything that agent did — what each call did, which object it did it to, and the sentence it gave for making it, with the refusals in red. A row leads where the call went, so reading what an agent did and - going to look at it are one move. Kept in `~/.shark-explorer/agents/sessions`, one file per session and the - newest hundred kept, so a session outlives the window it was worked in. + going to look at it are one move — including a row about a heap dump this window hasn't got, which opens + that dump, and whose link is there to copy like every other. Kept in `~/.shark-explorer/agents/sessions`, + one file per session and the newest hundred kept, so a session outlives the window it was worked in. See [Hand it to an agent](shark-explorer.md#hand-it-to-an-agent). * ✨ **The chain marks the faulty reference**: the one step going from an `Expected` object straight to a `Stuck` one reads `Holder.activity · faulty reference`, which is the leak itself rather than one of the diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 08f9563830..54b12e491c 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -85,9 +85,9 @@ of the map, a row of the object list or of the leaks, a step of a chain, a field starred object. Wherever the window will take you somewhere, it will also hand you the link to it. ``` -shark://vugs93jp/object?id=0x7f2a4b18 -shark://vugs93jp/objects?query=Bitmap&exact=true -shark://vugs93jp/leaks +shark://bug-4821.hprof/object?id=0x7f2a4b18 +shark://bug-4821.hprof/objects?query=Bitmap&exact=true +shark://bug-4821.hprof/leaks ``` Anywhere a tab can be is a link: an object, the object list with its search and filters filled in, the @@ -95,10 +95,23 @@ leaks with the same groups unfolded, the starred objects. So "look at this" is a paragraph of directions, which is also how a tool or an agent that has read your heap dump can point you straight at what it found. -The part after `shark://` is **the window, not the heap dump** — the same dump open twice is two windows, -and a link leads to the one it was copied from. Which means a link works while that window is open and -stops working once it is closed or the app is restarted; following one then opens an empty window saying -so. A link never replaces what you were reading: it always opens a tab of its own. +The part after `shark://` is **the heap dump**, because every place a link can name belongs to the dump +rather than to the window showing it. So a link goes on working: following one opens that place in a window +that has the dump open, and opens the file in a new window when none has — the run it was copied from can +be long gone. A link never replaces what you were reading: it always opens a tab of its own. + +A copied link carries two more things after the place, and this is one in full: + +``` +shark://bug-4821.hprof/leaks?dump=%2FUsers%2Fyou%2Fdumps%2Fbug-4821.hprof&window=vugs93jp +``` + +`dump` is the file's full path, which is what makes the link exact — two `com.example.hprof` off two +devices are two investigations — and what lets it open the dump again months later. `window` is the window +it was copied from, honoured while that window is open and ignored once it isn't, so that the same dump +open twice, which is two readings of it side by side, lands where you meant. Neither is needed to type one +by hand: `shark://bug-4821.hprof/leaks` finds whichever window has a `bug-4821.hprof` open. What a link +can't do is open a dump it doesn't have the path of and nobody has open — that opens a window saying so. Links reach the app from an installed build — the installer is what tells the OS that `shark://` is this app's. A copy run from source can still be linked to from another one, but the OS won't start it for a @@ -137,7 +150,7 @@ this heap dump recognises becomes a way back into the window**: | --- | --- | --- | | `com.example.MyApp$Cache` | `MyApp$Cache` | Opens that class in a new tab | | `0x7f2a4b18` | `Cache instance (0x7f2a4b18)` | Opens that object in a new tab | -| `shark://vugs93jp/leaks` | `Leaks` | Follows the link, like clicking it anywhere else | +| `shark://bug-4821.hprof/leaks` | `Leaks` | Follows the link, like clicking it anywhere else | | `https://github.com/square/leakcanary/issues/2841` | `square/leakcanary#2841` | Opens it in your browser | A name or an address this dump has nothing for is left exactly as you typed it: a class this heap dump has @@ -151,9 +164,9 @@ GitHub reads markdown. Nothing inside a fenced code block is linked or shortened A location is *where* you are rather than how it is arranged, so searching in the object list, unfolding a leak or resizing the window stays on the same note rather than starting a new one. -Since a `shark://` link names a window, a link written into a note stops working once that window is closed -— see above. Copy one for the tab you want to come back to *while you are writing about it*, and it will -take you there for as long as that window is open. +A `shark://` link written into a note keeps working, since it names the heap dump rather than the window it +was copied from — see above. So a note that links to three places an investigation turned on still leads to +all three next week, in whatever window has that dump open by then. ## The verdict @@ -276,8 +289,9 @@ instead of piped to a window: ``` Everything works the same except `show`, which has nowhere to put a tab and says so rather than answering that -it showed you something — and hands back no `shark://` link either, since a link names a window and this run -has none. Nothing else changes, because **notes and verdicts were never on the screen** — they are files +it showed you something. It still hands back the `shark://` link, which names the heap dump: nobody saw the +place, and the link opens it for whoever reads the answer. Nothing else changes, because **notes and verdicts +were never on the screen** — they are files beside the heap dump, so a dump investigated over ssh today opens in a window tomorrow with the verdicts, the reasons and the conclusion already on it. @@ -339,7 +353,7 @@ press, because a surface with less than that is one whose answer is "ask your hu | Tool | What it is | | --- | --- | -| `open_heap_dumps` | Every window and what is open in it, with the method to follow. | +| `open_heap_dumps` | Every heap dump open, by the file name the other tools take, with the method to follow. | | `list_leaks` | The **Leaks** screen: what this heap dump says shouldn't be there. | | `agent_log` | The **Agent logs** screen: what has already been tried on this dump, and what it came to. | | `chain_from_gc_root` | One chain, every step with its labels and its verdict. | @@ -349,7 +363,7 @@ press, because a surface with less than that is one whose answer is "ask your hu | `dominator_tree` | The treemap, without the pixels: where the memory has gone, a level at a time. | | `set_verdict`, `clear_verdict` | The pencil, with the reason required the same way. | | `read_notes`, `take_note` | The notes: where somebody has been, what they wrote, and adding to or replacing it. | -| `show` | Opens a tab in your window and brings it to the front, and answers with the `shark://` link to it. The one tool a `--no-ui` run can't do. | +| `show` | Opens a tab in your window and brings it to the front, and answers with the `shark://` link to it. The one tool a `--no-ui` run can only half do — no tab, and the link all the same. | | `conclude` | The root cause, and the only way to finish. | | `open_heap_dump` | **Open heap dump…**, for a file nobody has open yet. | | `list_devices`, `dump_heap` | **Take heap dump…**: which device, which process, and the dump itself. | @@ -462,11 +476,11 @@ and the method tells an agent to put those links in its reply — so a sentence request comment or a bug report ends up carrying a way in: > The leak is `MainActivity$2.this$0`, a non-static inner class holding the activity it was declared in: -> shark://zvphq4r3/0x12d368b8 +> shark://leak_asynctask_o.hprof/object?id=0x12d368b8&dump=%2FUsers%2Fyou%2Fdumps%2Fleak_asynctask_o.hprof&window=zvphq4r3 -Clicking it opens that object in that window, with the reasoning on its tabs. Once the window is closed the -link says which window it was rather than opening the wrong one, so an answer worth keeping is worth -[copying the heap dump's path](#open-a-heap-dump) beside it. +Clicking it opens that object with the reasoning on its tabs — in the window it was written from while that +window is up, and by opening the heap dump again once it isn't. So an answer worth keeping keeps working: it +carries the path of the dump it is about, and the only thing that stops it is deleting the file. An agent's verdicts are verdicts like any other: they say `set by hand` on every chain that runs through the object, the reason is the one it gave, and the pencil takes one off if you disagree with it. Which is the diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index 3b91c59a56..4d64d76198 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -237,18 +237,25 @@ cp -R "shark/shark-explorer/shark-explorer-app/build/compose/binaries/main/app/S /System/Library/Frameworks/CoreServices.framework/Frameworks/LaunchServices.framework/Support/lsregister \ -f ~/Applications/"Shark Explorer.app" open -a ~/Applications/"Shark Explorer.app" --args --title="Links" path/to/dump.hprof -grep "Windows of this run" ~/.shark-explorer/logs/$(ls -t ~/.shark-explorer/logs | head -1) -open "shark:///leaks" +open "shark://dump.hprof/leaks" ``` +A link names the heap dump, so that is the whole recipe — no id to read out of the log first, and the same +line works after the run that opened the dump has ended, which is the case worth trying. To try the window +half of it, the `window=` a copied link carries, `grep "Windows of this run"` in the newest file under +`~/.shark-explorer/logs` for the ids of the run. + **Read the result in the log rather than off the screen.** Following a link raises the app over whatever the person at the machine was doing, so a screenshot to check it worked costs them their window and shows -you theirs. `The OS handed this run`, `A link asked window for ` and `A link asked this window -for ` are the three lines that say a link was delivered, routed and opened as a tab. +you theirs. `The OS handed this run`, `A link asked window for of `, `A link asked for + of , which is not open yet` and `A link asked this window for ` are the lines that say +a link was delivered, routed and opened as a tab. A run from source is still *reachable*: every run publishes a loopback port under `~/.shark-explorer/runs`, -and the installed app hands on any link naming a window it doesn't have. That is what makes a link to a -`./gradlew run` window work — the installed app is the courier, so there has to be one. +and the installed app hands on any link it has no window for. That is what makes a link to a +`./gradlew run` window work — the installed app is the courier, so there has to be one. It claims a link +only for a heap dump it *already has open*, never for a file it could open, or every run would claim every +link. **Deliberately not single instance.** Several explorers open at once is how this app is used, so a run holding a link asks each of the others in turn rather than the second run handing its command line to the diff --git a/shark/shark-explorer/notes/decisions.md b/shark/shark-explorer/notes/decisions.md index 5736126f01..3c7592b9f8 100644 --- a/shark/shark-explorer/notes/decisions.md +++ b/shark/shark-explorer/notes/decisions.md @@ -814,10 +814,10 @@ recognise stays exactly as typed: a class this dump has never heard of is a clas - **An address is two `Long`s.** A 32 bit dump's ids are four bytes widened by sign, so `0x82182c00` is either that or the negative id `hexObjectId` prints as `0x82182c00`, and only the dump says which. - **A `shark://` link keeps its link and gains a name.** Resolving a mention never replaces a link that is - already there, so a link to an object reads as that object and still leads to the window it names — - followed exactly the way one arriving from the OS is, which is what makes the same link work in a note, a - chat message and an issue. A note outlives the window, so most of them are dead links a run later: that is - the honest answer, and it is the same empty window a stale link opens anywhere else. + already there, so a link to an object reads as that object and still leads where it names — followed + exactly the way one arriving from the OS is, which is what makes the same link work in a note, a chat + message and an issue. A note outlives the run it was written in, which is why a link names the heap dump + and not the window: see below. - **Inline code is still read for mentions, a fenced block is not.** `` `com.example.Thing` `` is how anyone who writes markdown writes a class name; a fenced block is quoted rather than written. - **One line is one block.** No two-space line endings, no blank line above a list. A note is written in @@ -847,6 +847,39 @@ inside it, a file per `noteKey` rather than one document with a section per plac the note that was typed into, nothing has to be parsed back out of a document that also holds somebody's own headings, and the listing is the index. +## A link names the heap dump, and a window only as a refinement + +`shark:///?[&dump=][&window=]`. The first version of this +named the window — `shark:///` — and it was wrong for the reason a link exists: every place +there is belongs to the heap dump, not to whatever is showing it, so a link that named a window died with the +window. Which is most links a day later, and most links in an agent's session log, since a session outlives +the run that wrote it. A link that mostly doesn't work is a link nobody sends. + +So the dump is the identity and the window is honoured while it exists and **ignored once it doesn't**, rather +than turning the link into an error. Being right about which window is worth a lot while the window is there +and nothing at all afterwards. + +- **The authority is the file name**, because it is the part a person reads and types, and it is in every + answer an agent has already been given. `dump=` carries the normalized absolute path beside it, since two + dumps called `com.squareup.hprof` off two devices are two investigations — and because a path is the only + thing that can open a dump nothing has open. A link with a name and no path is one somebody typed, and it + resolves against what is open. +- **Not `heapDumpFileKey`**, the `-` the notes and statuses are filed under. It is + one-way and nothing on disk maps a key back to a path, so a key-only link could never reopen a dump. +- **Window ids stay random.** A counted id repeats across runs *and* within one as windows close and open, so + it would be honoured against the wrong reading of the dump — silently, which is worse than being ignored. A + file name plus a number fixes neither half: the number would have to be handed out across runs that cannot + see each other's windows. +- **Resolution order is windowId, then path, then file name, then the authority as a window id** — that last + step for the `shark:///` links already sitting in notes and in session files on disk. +- **A run claims a link only for a window it already has**, never for a file it could open, or every run of + the app would claim every link. Whoever is left holding it opens the dump. `DeepLinkPeers`. +- **The agent surface converged on the same choice**: the tool argument is `heapDump`, taking a file name, and + a window id only in the one case a name cannot answer, which is the same file open twice. `AgentTools`. +- **What it unlocked**, and the reason to reverse it rather than live with it: a `--no-ui` run answers `show` + with a link now — it has no window and the file all the same — and every *Agent logs* row about another + heap dump has a link to copy, where before there was nothing to send. + ## A leaking status is the heap dump's answer until a hand overrules it Every chain already carried a `LeakStatus` per object, worked out by Shark's inspectors and then propagated diff --git a/shark/shark-explorer/shark-explorer-agent/AGENTS.md b/shark/shark-explorer/shark-explorer-agent/AGENTS.md index 5b5b3771bd..d7e7cb207c 100644 --- a/shark/shark-explorer/shark-explorer-agent/AGENTS.md +++ b/shark/shark-explorer/shark-explorer-agent/AGENTS.md @@ -192,8 +192,8 @@ here, and the same the other way round. Two consequences worth knowing before adding one. **A tool that makes a window is answered once the dump is *readable*.** `AgentHeapDumps.open` and `dumpHeap` -hand back an `AgentHeapDump`, not a path or a window id, because everything else on this surface is a read: an -id handed over while the dump is still being indexed is one that refuses every call made with it. The app's +hand back an `AgentHeapDump`, not a path or a name, because everything else on this surface is a read: a dump +named back while it is still being indexed is one that refuses every call made with it. The app's side waits on three outcomes — open, failed to open, window closed — which is why `ExplorerWindow` publishes `openProblem` beside `openHeapDump`. Waiting on "opened" alone means a file that was never a heap dump is a call that never comes back. @@ -237,7 +237,8 @@ the reads happen on the heap dump's thread and the tests run headless. # What the surface is, from a shell, with nothing open and no Gradle. Then one call at a window. "Shark Explorer.app/Contents/MacOS/Shark Explorer" --agent-help -"Shark Explorer.app/Contents/MacOS/Shark Explorer" --agent list_leaks window= reason="Trying it" +"Shark Explorer.app/Contents/MacOS/Shark Explorer" \ + --agent list_leaks heapDump= reason="Trying it" # The whole surface end to end, in a real window, with an agent that has never seen this repository. shark/shark-explorer/shark-explorer-agent/harness/start-harness.sh [heap-dump.hprof] diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt index 0c861e446c..ddf23df829 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentCommandLine.kt @@ -298,7 +298,7 @@ object AgentCommandLine { | $command $AGENT_OPTION open_heap_dumps reason="Finding out which heap dump is open" | $command $AGENT_OPTION describe_object object=0x7205 reason="Reading the holder's fields" | - |Start with open_heap_dumps: its answer carries the method to follow, the window ids every other tool + |Start with open_heap_dumps: its answer carries the method to follow, the file names every other tool |names a heap dump by, and whatever verdicts somebody has already recorded about that dump. | |Every tool takes `reason`, which is why you are making the call. It is logged beside the reads it causes diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt index 6ca76605fb..d13c8355c6 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentHeapDump.kt @@ -15,18 +15,27 @@ import shark.explorer.Place * dump and nothing else: the app's implementation carries a `HeapDumpSession`, the statuses set by hand and * the tabs, none of which a test of what a tool answers needs. * - * **A window and not a heap dump file**, matching `shark.explorer.DeepLink`: the same dump is often open - * twice — that is what comparing two of them is — so a path would be ambiguous exactly when it matters, and - * a verdict set through one of two windows has to be the verdict the other one draws. + * **One of these is a window and not a heap dump file**, even though an agent names it by the file: the same + * dump is open twice whenever two readings of it are being compared, and a verdict set through one of those + * windows has to be the verdict the other one draws. See [AgentTools.HEAP_DUMP] for how one is asked for, and + * `shark.explorer.DeepLink` for the same split in a link. */ interface AgentHeapDump { - /** What a link names this window by, and what an agent addresses it by. See [AgentTools]. */ + /** + * Which window this is, for the one thing the file name can't say: which of two windows on one dump. + * + * What a link names as its window and what this run's log calls it, so it is also how a person watching + * finds the window an agent was in. See [AgentTools.HEAP_DUMP]. + */ val windowId: String /** Which heap dump is open here, absolute, so that an agent can check it is the one it was asked about. */ val heapDumpPath: String + /** How an agent names this dump, which is the file name. See [AgentTools.HEAP_DUMP]. */ + val heapDumpName: String get() = File(heapDumpPath).name + /** * Runs [block] against the open heap dump, wherever the implementation reads one. * @@ -100,11 +109,12 @@ interface AgentHeapDump { } /** - * What came of putting a place in front of the person watching: the link to it, or why there was nowhere. + * What came of putting a place in front of the person watching: the link to it, and why there was nowhere. * - * **One answer rather than two calls**, because the two questions have one answer. A link names a window, so - * whether there is a link and whether anything was shown are the same fact — and a run with no window that - * handed out a `shark://` link anyway would be handing out an address nothing answers to. + * **One answer rather than two calls**, because a call to `show` raises two questions with one cause and an + * agent needs both answers: whether anybody saw it, and what to write down so that somebody can. They are not + * the same fact, since a `shark://` link names the *heap dump* — so a run with no window has nothing on screen + * to point at and a link worth passing on all the same. * * The link matters as much as the showing does: it is what an agent puts in its *reply* so that whoever asked * can open the place themselves, later, from wherever the conversation is. Showing raises a window over @@ -112,7 +122,7 @@ interface AgentHeapDump { * time. See [AgentTools] `show`. */ class ShownPlace private constructor( - /** The `shark://` link a person can click to open it, and null when nothing was shown. */ + /** The `shark://` link a person can click to open it, and null when there is no heap dump to link to. */ val link: String?, /** Why it wasn't shown, and null when it was. */ val problem: String? @@ -123,9 +133,18 @@ class ShownPlace private constructor( fun at(link: String) = ShownPlace(link = link, problem = null) /** - * Nothing was shown, and [problem] says why — which is worth answering rather than logging: an agent that - * told its human to look at something they cannot see has said the one thing worse than nothing. + * Nothing was shown and [problem] says why, with [link] to the place it would have been. + * + * Both, because either alone is misleading: an agent that told its human to look at something they cannot + * see has said the one thing worse than nothing, and one that dropped the link would leave them with a + * description of a place they could have been taken to. */ + fun onlyAsALink( + link: String, + problem: String + ) = ShownPlace(link = link, problem = problem) + + /** Nothing was shown and there is nowhere to link to either, which [problem] has to account for. */ fun nowhere(problem: String) = ShownPlace(link = null, problem = problem) } } diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt index ed2fa6b1ab..cfbf311de0 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentJson.kt @@ -46,13 +46,20 @@ import shark.explorer.leakLabel */ internal object AgentJson { - /** Which window, which dump, how big it is, and what has been concluded about it so far. */ + /** + * Which heap dump, in which window, how big it is, and what has been concluded about it so far. + * + * The name first because it is what every other call names this dump by — [AgentTools.HEAP_DUMP] — and the + * window id after it, for the one thing the name can't say: which of two windows on one file. + */ fun heapDump( + heapDumpName: String, windowId: String, heapDumpPath: String, sizes: HeapSizes, verdicts: LeakStatusOverrides ): JsonObject = buildJsonObject { + put("heapDump", heapDumpName) put("window", windowId) put("heapDumpPath", heapDumpPath) putJsonObject("sizes") { diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt index 06b1934c7b..81c504f5af 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentMethod.kt @@ -130,8 +130,9 @@ internal object AgentMethod { - **`show` puts what you are looking at on screen.** Use it when you reach something that matters. The window is how the person watching follows the work, and it costs you one call. - **Put the `shark://` links you are answered with in your reply.** `show` and `conclude` hand one back: - it opens that exact object, in that window, with your notes on it. Whoever asked you can click it - while reading your answer, and again next week. So write "the leak is + it opens that exact object, in this heap dump, with your notes on it. A link names the dump rather than + the window, so it still works once this run has ended — it opens the file again. Whoever asked you can + click it while reading your answer, and again next week. So write "the leak is `Holder.activity`(shark://…)" rather than describing which screen to open and what to click — a link is the difference between an answer they have to take your word for and one they can go and look at. """.trimIndent() diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentPlace.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentPlace.kt index 0c7d788117..258b081d29 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentPlace.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentPlace.kt @@ -12,9 +12,9 @@ import shark.explorer.exactHexObjectId * directions in one file because they are one spelling — a place written one way and read another is a place * an agent can be told about and then cannot go to. * - * Deliberately not `shark://` links, which name a place too: a link names a *window* as well, so it is the - * thing to hand to a person rather than the thing to pass back over this protocol. See - * [AgentTools] `show`. + * Deliberately not `shark://` links, which name a place too: a link names the *heap dump* as well, and often + * the window, so it is the thing to hand to a person — who has neither in front of them — rather than the + * thing to pass back over a protocol where both are already known. See [AgentTools] `show`. */ internal fun AgentArguments.place(): Place { val text = string(PLACE) diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt index 206c0e428b..898543d97a 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt @@ -466,11 +466,18 @@ class AgentSessionCall( val millis: Long ) { - /** The link to [place] in the window the call was made against, for a call that was about one. */ + /** + * The link to [place] in the heap dump the call was about, for a call that was about one. + * + * The window as well, since it was open when the line was written, and a link is a request to look at + * something in the window somebody was watching while that is still possible. It stops being possible + * about as soon as anybody reads this — an agent's session outlives its run — and a link that names the + * heap dump goes on working after that. See [DeepLink]. + */ fun link(): String? { val place = place ?: return null - val windowId = windowId ?: return null - return DeepLink(windowId, place).toUri() + val heapDumpPath = heapDumpPath ?: return null + return DeepLink(File(heapDumpPath), place, windowId = windowId).toUri() } } diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt index c0c2a7eaa7..f2f8b0484e 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentTools.kt @@ -80,7 +80,7 @@ internal class AgentTools( private fun openHeapDumps() = AgentTool( name = OPEN_HEAP_DUMPS, description = "Every heap dump open in Shark Explorer right now, with the method to investigate it. " + - "Call this first: the window ids it hands back are what every other tool names a heap dump by, and " + + "Call this first: the file names it hands back are what every other tool names a heap dump by, and " + "the verdicts it lists are the conclusions somebody has already reached about that dump.", schema = schema() ) { _ -> @@ -90,6 +90,7 @@ internal class AgentTools( // don't take a suspending block. val described = dumps.map { dump -> AgentJson.heapDump( + heapDumpName = dump.heapDumpName, windowId = dump.windowId, heapDumpPath = dump.heapDumpPath, sizes = dump.read("its sizes, for an agent") { it.sizes }, @@ -118,7 +119,7 @@ internal class AgentTools( "are instances of. The heap dump's own answer and the place to start: objects the app itself handed " + "to LeakCanary and said it was done with are the strongest evidence a dump carries. Sections marked " + "isOnTheWayOut are objects the garbage collector will take on its own — not leaks to fix.", - schema = schema(WINDOW to window()) + schema = schema(HEAP_DUMP to heapDumpArgument()) ) { arguments -> val dump = arguments.heapDump() val leaks = dump.read("the leaks, for an agent") { it.tree.findLeaks(dump.verdicts) } @@ -134,7 +135,7 @@ internal class AgentTools( "starting: an investigation somebody already ran is either the answer or the half of the dump not " + "worth doing again. Sessions of earlier runs of the app are in it, and so is this one.", schema = schema( - WINDOW to window(), + HEAP_DUMP to heapDumpArgument(), SESSION to string("Optional: one session's id, from the list, to read every call it made.").optional() ) ) { arguments -> @@ -169,7 +170,7 @@ internal class AgentTools( description = "What one object is: its class, what the inspectors made of it, its verdict and the " + "reason under it, what it retains, what dominates it, and every field with the address of each " + "field's value. Reading fields is how a guess about an object becomes evidence.", - schema = schema(WINDOW to window(), OBJECT to objectId("The object to describe.")) + schema = schema(HEAP_DUMP to heapDumpArgument(), OBJECT to objectId("The object to describe.")) ) { arguments -> val dump = arguments.heapDump() val objectId = arguments.objectId(OBJECT) @@ -192,7 +193,7 @@ internal class AgentTools( "chain in the window. It is null while the verdicts don't yet cross from EXPECTED to STUCK at a " + "single reference, which is the state an investigation works towards and what conclude requires. " + "Steps marked isDominator are the ones every path to the object goes through.", - schema = schema(WINDOW to window(), OBJECT to objectId("The object to walk up from.")) + schema = schema(HEAP_DUMP to heapDumpArgument(), OBJECT to objectId("The object to walk up from.")) ) { arguments -> val dump = arguments.heapDump() val objectId = arguments.objectId(OBJECT) @@ -210,7 +211,7 @@ internal class AgentTools( "and one that decides whether clearing a field would free anything at all. Give `from` to ask only " + "about the ways between that object and this one.", schema = schema( - WINDOW to window(), + HEAP_DUMP to heapDumpArgument(), OBJECT to objectId("The object being held."), FROM to objectId("Optional: only the ways this object holds it, rather than from the GC roots.") .optional() @@ -241,7 +242,7 @@ internal class AgentTools( "object, so it is also the answer to \"what are the biggest things in this heap\"** — one object at " + "a time, where $DOMINATOR_TREE is what holds them.", schema = schema( - WINDOW to window(), + HEAP_DUMP to heapDumpArgument(), CLASS_NAME to string("Matched against the class name.").optional(), EXACT_MATCH to boolean( "Whether className has to be the whole name — `android.graphics.Bitmap` or `Bitmap` — rather " + @@ -278,7 +279,7 @@ internal class AgentTools( "give `object` to walk down from one node. This answers \"why is this app using 400 MB\" — for " + "\"why is this object still here\", read its chain instead.", schema = schema( - WINDOW to window(), + HEAP_DUMP to heapDumpArgument(), OBJECT to objectId("Optional: the node to walk down from, the whole heap dump by default.") .optional(), MAX_DEPTH to integer( @@ -318,7 +319,7 @@ internal class AgentTools( "or a line of source rather than a hunch. Refuses a verdict that contradicts one already set " + "unless solveConflicts is true, in which case the ones it disagrees with are flipped and say so.", schema = schema( - WINDOW to window(), + HEAP_DUMP to heapDumpArgument(), OBJECT to objectId("The object to record a verdict about."), VERDICT to enumString( "STUCK for an object that should be gone, EXPECTED for one that is meant to be here.", @@ -384,7 +385,7 @@ internal class AgentTools( description = "Takes a verdict off an object, so the heap dump says what it says about it again. For " + "a verdict of yours that the evidence turned out not to support — leaving a wrong one in place is " + "worse than never setting it, because everything below it reads as stuck because of it.", - schema = schema(WINDOW to window(), OBJECT to objectId("The object to take the verdict off.")) + schema = schema(HEAP_DUMP to heapDumpArgument(), OBJECT to objectId("The object to take the verdict off.")) ) { arguments -> val dump = arguments.heapDump() val objectId = arguments.objectId(OBJECT) @@ -406,7 +407,7 @@ internal class AgentTools( "you earlier, or by whoever read it last. Without `$PLACE`, every place that has a note, so that an " + "investigation starts from what is known rather than on top of it. With one, that note in full. " + "Notes outlive the window and are where a conclusion is kept.", - schema = schema(WINDOW to window(), PLACE to place().optional()) + schema = schema(HEAP_DUMP to heapDumpArgument(), PLACE to place().optional()) ) { arguments -> val dump = arguments.heapDump() val place = arguments.optionalString(PLACE)?.let { arguments.place() } @@ -438,7 +439,7 @@ internal class AgentTools( "something you wrote earlier is — read it first with read_notes. Notes are kept between runs of the " + "app. Write what you found and where you looked, not what you are about to do.", schema = schema( - WINDOW to window(), + HEAP_DUMP to heapDumpArgument(), PLACE to place(), TEXT to string("Markdown. `0x…` addresses in it become links to those objects."), REPLACE to boolean( @@ -468,7 +469,7 @@ internal class AgentTools( "that matters rather than for every step. It answers with a `shark://` link to that place: put that " + "link in your reply to whoever asked you, because clicking it opens the place again, later, without " + "you.", - schema = schema(WINDOW to window(), PLACE to place()) + schema = schema(HEAP_DUMP to heapDumpArgument(), PLACE to place()) ) { arguments -> val dump = arguments.heapDump() val place = arguments.place() @@ -491,7 +492,7 @@ internal class AgentTools( "which is a sequence of events rather than a line. The conclusion is written into the notes of the " + "object and shown in the window.", schema = schema( - WINDOW to window(), + HEAP_DUMP to heapDumpArgument(), OBJECT to objectId("The stuck object whose being in memory is being explained."), ROOT_CAUSE to string( "How the faulty reference came to still be set: what assigned it, what should have cleared it, " + @@ -574,6 +575,7 @@ internal class AgentTools( } val dump = heapDumps.open(file) buildJsonObject { + put("heapDump", dump.heapDumpName) put("window", dump.windowId) put("heapDumpPath", dump.heapDumpPath) put("opened", true) @@ -630,6 +632,7 @@ internal class AgentTools( processName = arguments.string(PROCESS) ) buildJsonObject { + put("heapDump", dump.heapDumpName) put("window", dump.windowId) put("heapDumpPath", dump.heapDumpPath) put("dumped", true) @@ -657,7 +660,7 @@ internal class AgentTools( arguments: JsonObject ): AgentTarget { val read = AgentArguments(name, arguments) - val dump = read.orNull { resolvedDump(optionalString(WINDOW)) } + val dump = read.orNull { resolvedDump(optionalString(HEAP_DUMP)) } val place = read.orNull { placeOrNull(name) } return AgentTarget( windowId = dump?.windowId, @@ -696,39 +699,51 @@ internal class AgentTools( /** Which heap dump a call is about, or a refusal naming the ones that are open. */ private fun AgentArguments.heapDump(): AgentHeapDump { - val windowId = optionalString(WINDOW) - val asked = resolvedDump(windowId) - if (asked != null) { - return asked + val asked = optionalString(HEAP_DUMP) + val resolved = resolvedDump(asked) + if (resolved != null) { + return resolved } val open = heapDumps.openHeapDumps() - val windows = open.joinToString(", ") { "${it.windowId} (${it.heapDumpPath})" } + // The window id beside each, since that is what tells two windows of one file apart and this is one of + // the two moments an agent needs it. The other is being told a place was shown. + val dumps = open.joinToString(", ") { "${it.heapDumpName} (${it.windowId}) at ${it.heapDumpPath}" } throw AgentRefusal( when { open.isEmpty() -> "No heap dump is open in Shark Explorer, so there is nothing to read. Call $OPEN_HEAP_DUMPS." - windowId == null -> - "${open.size} heap dumps are open, so say which with `$WINDOW`: $windows" + asked == null -> + "${open.size} heap dumps are open, so say which with `$HEAP_DUMP`: $dumps" + open.count { it.heapDumpName == asked } > 1 -> + "${open.count { it.heapDumpName == asked }} windows have \"$asked\" open, which is how two " + + "readings of one dump are compared, so `$HEAP_DUMP` has to be the window id of the one you " + + "mean: $dumps" else -> - "No window is called \"$windowId\". A window id names one window of one run of this app, so it " + - "stops being valid when that window is closed. Open windows: $windows. Call $OPEN_HEAP_DUMPS." + "No open heap dump is called \"$asked\", and no window is either. Open heap dumps: $dumps. " + + "Call $OPEN_HEAP_DUMPS." } ) } /** - * The window a call names, and null for one that names none of the open ones. + * The heap dump a call names, and null for one that names none of the open ones. * - * One open dump needs no naming, which is most sessions. Two of them always do: the same file open twice is - * how two readings of it are compared, so guessing would be answering about the wrong one. + * One open dump needs no naming, which is most sessions. More than one always does, and **the file name is + * what it is named by**: that is what an agent has read in every answer and what it can say back without + * copying an id, and it goes on meaning the same thing after the window it was opened in has closed. + * + * A window id is taken too, and has to be: the same file open twice is how two readings of it are compared, + * so the name is ambiguous exactly there, and answering about either of them would be answering about the + * wrong one half the time. Ids first, because a file called `abcd2345` is a heap dump somebody has and a + * window id is ours to hand out. */ - private fun resolvedDump(windowId: String?): AgentHeapDump? { + private fun resolvedDump(asked: String?): AgentHeapDump? { val open = heapDumps.openHeapDumps() - return if (windowId == null) { - open.singleOrNull() - } else { - open.firstOrNull { it.windowId == windowId } + if (asked == null) { + return open.singleOrNull() } + return open.firstOrNull { it.windowId == asked } + ?: open.filter { it.heapDumpName == asked }.singleOrNull() } /** @@ -784,7 +799,15 @@ internal class AgentTools( const val FIND_OBJECTS = "find_objects" const val DOMINATOR_TREE = "dominator_tree" - const val WINDOW = "window" + /** + * Which heap dump a call is about: a file name, or a window id where a file name can't say. + * + * Named after the dump rather than after the window it is open in, because that is what it is — every + * place an agent can ask about belongs to the heap dump, the file name is in every answer it has read, + * and a window id is eight characters it would have to copy. See [resolvedDump] for the ordering, and + * `shark.explorer.DeepLink`, which is the same choice made for a link. + */ + const val HEAP_DUMP = "heapDump" const val SESSION = "session" const val OBJECT = "object" const val FROM = "from" @@ -814,7 +837,7 @@ internal class AgentTools( * question — while a dump with `KeyedWeakReference`s in it has an answer waiting in `list_leaks` that * walking a tree would take an hour to reach. */ - const val NEXT_WITH_A_NEW_DUMP = "Call $LIST_LEAKS with this window to see what the dump says about " + + const val NEXT_WITH_A_NEW_DUMP = "Call $LIST_LEAKS with this heap dump to see what it says about " + "itself, or $DOMINATOR_TREE to see where its memory has gone." /** @@ -824,8 +847,9 @@ internal class AgentTools( */ const val DEFAULT_LISTED_OBJECTS = 30 - fun window() = string( - "Which open heap dump, from ${OPEN_HEAP_DUMPS}. Optional while only one is open." + fun heapDumpArgument() = string( + "Which open heap dump, by file name from ${OPEN_HEAP_DUMPS}. Optional while only one is open, and " + + "the window id instead when two windows have the same file open." ).optional() fun objectId(description: String) = diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index 3f035b7b88..ffb17fa425 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -20,6 +20,7 @@ import org.junit.Rule import org.junit.Test import org.junit.rules.TemporaryFolder import shark.explorer.AndroidDevice +import shark.explorer.DeepLink import shark.explorer.DeviceProcess import shark.explorer.HeapObjectKind import shark.explorer.LeakStatus @@ -173,14 +174,37 @@ class AgentToolsTest { .hasMessageContaining(window.windowId) .hasMessageContaining(other.windowId) - assertThat(call("list_leaks", "window" to other.windowId).text("objectCount")).isNotEmpty() + assertThat(call("list_leaks", HEAP_DUMP to other.windowId).text("objectCount")).isNotEmpty() } @Test - fun `a window that is not open is refused by name`() { - assertThatThrownBy { call("list_leaks", "window" to "closedwindow") } + fun `a heap dump is named by its file name`() { + // Which is the whole point of naming it that way: the name is in every answer an agent has been given, + // and it goes on meaning this dump after the window it was opened in has gone. + assertThat(call("list_leaks", HEAP_DUMP to window.heapDumpName).text("objectCount")).isNotEmpty() + } + + @Test + fun `two windows on one heap dump have to be named by window id`() { + val other = FakeAgentHeapDump(heapDump.explorer, windowId = "otherwindow") + tools = agentTools(FakeAgentHeapDumps(listOf(window, other))) + + // The one case a file name cannot answer, and so the reason a window id is still on this surface: the + // same file open twice is two readings of it being compared, and either answer would be the wrong one + // half the time. + assertThatThrownBy { call("list_leaks", HEAP_DUMP to window.heapDumpName) } .isInstanceOf(AgentRefusal::class.java) - .hasMessageContaining("No window is called \"closedwindow\"") + .hasMessageContaining("2 windows have \"${window.heapDumpName}\" open") + .hasMessageContaining(window.windowId) + .hasMessageContaining(other.windowId) + } + + @Test + fun `a heap dump that is not open is refused by name`() { + assertThatThrownBy { call("list_leaks", HEAP_DUMP to "closed.hprof") } + .isInstanceOf(AgentRefusal::class.java) + .hasMessageContaining("No open heap dump is called \"closed.hprof\", and no window is either") + .hasMessageContaining(window.heapDumpName) .hasMessageContaining(window.windowId) } @@ -341,8 +365,7 @@ class AgentToolsTest { assertThat(faulty.text("heldClassName")).isEqualTo(ACTIVITY_CLASS_NAME) // The one link most worth handing back, so it comes with the conclusion rather than needing a show call // after it: it opens the object this conclusion is about, with the conclusion in its notes. - assertThat(answer.text("link")) - .isEqualTo("shark://${window.windowId}/${hex(heapDump.activityObjectId)}") + assertThatLinkOpens(answer, heapDump.activityObjectId) } @Test @@ -530,8 +553,7 @@ class AgentToolsTest { // The half of showing that outlives the call: an agent writing its answer somewhere else has this to // point at, where "open the window and click the activity" is a set of instructions. - assertThat(answer.text("link")) - .isEqualTo("shark://${window.windowId}/${hex(heapDump.activityObjectId)}") + assertThatLinkOpens(answer, heapDump.activityObjectId) } @Test @@ -823,6 +845,24 @@ class AgentToolsTest { private fun hex(objectId: Long) = exactHexObjectId(objectId) + /** + * The link an answer hands back, read as a link rather than compared as text. + * + * What matters about it here is what it opens — this object, of this heap dump, in the window the call was + * made against — and how it is spelled is `DeepLinkTest`'s. It names the heap dump rather than only the + * window so that an agent can put it in an answer somebody reads after this run has ended. + */ + private fun assertThatLinkOpens( + answer: JsonObject, + objectId: Long + ) { + val link = DeepLink.parse(answer.text("link")) + assertThat(link.heapDumpName).isEqualTo(window.heapDumpName) + assertThat(link.heapDumpPath?.path).isEqualTo(window.heapDumpPath) + assertThat(link.windowId).isEqualTo(window.windowId) + assertThat(link.place).isEqualTo(Place.Object(objectId)) + } + private companion object { const val OPEN_HEAP_DUMPS = "open_heap_dumps" @@ -830,6 +870,7 @@ class AgentToolsTest { const val AGENT_LOG = "agent_log" const val SET_VERDICT = "set_verdict" const val CONCLUDE = "conclude" + const val HEAP_DUMP = "heapDump" const val OBJECT = "object" const val PLACE_LEAKS = "leaks" diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt index b2fc4da080..fb0ef18669 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt @@ -4,6 +4,7 @@ import java.io.Closeable import java.io.File import shark.SharkLog import shark.explorer.AndroidDevice +import shark.explorer.DeepLink import shark.explorer.DeviceProcess import shark.explorer.HeapExplorer import shark.explorer.LeakStatusOverride @@ -80,9 +81,10 @@ internal class FakeAgentHeapDump( override fun show(place: Place): ShownPlace { shown += place - // A window's answer, which is a link. What a run with no window answers is `HeadlessAgentHeapDumpsTest`'s, - // since it is that run's one difference from this one. - return ShownPlace.at("shark://$windowId/${placeText(place)}") + // A window's answer, which is a link — built the way the window builds one, since a fake that spelled it + // itself would be a test passing on a link nobody could follow. What a run with no window answers is + // `HeadlessAgentHeapDumpsTest`'s, since it is that run's one difference from this one. + return ShownPlace.at(DeepLink(File(heapDumpPath), place, windowId = windowId).toUri()) } override fun close() { diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index 47df271291..c8b7e755c2 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -13,6 +13,7 @@ import org.junit.Before import org.junit.Rule import org.junit.Test import org.junit.rules.TemporaryFolder +import shark.explorer.DeepLink import shark.explorer.Place import shark.explorer.exactHexObjectId @@ -211,10 +212,16 @@ class McpSessionTest { assertThat(call.subject).isEqualTo(hex(heapDump.holderObjectId)) assertThat(call.reason).isEqualTo("Checking whether the holder is the singleton it looks like.") assertThat(call.refusal).isNull() - // Which is what makes the row clickable: the place, in the window the call was made against. + // Which is what makes the row clickable: the place, in the heap dump the call was made against — named + // by the dump so that the link still opens it once this run has ended, with the window it was made in as + // a refinement, honoured while that window is open. assertThat(call.place).isEqualTo(Place.Object(heapDump.holderObjectId)) - assertThat(call.link()).isEqualTo("shark://${window.windowId}/object?id=${hex(heapDump.holderObjectId)}") assertThat(call.heapDumpPath).isEqualTo(window.heapDumpPath) + val link = DeepLink.parse(call.link()!!) + assertThat(link.heapDumpName).isEqualTo(window.heapDumpName) + assertThat(link.heapDumpPath?.path).isEqualTo(window.heapDumpPath) + assertThat(link.windowId).isEqualTo(window.windowId) + assertThat(link.place).isEqualTo(Place.Object(heapDump.holderObjectId)) } @Test diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt index 98bb16107f..22682f8ca5 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/AgentLogsScreen.kt @@ -62,6 +62,10 @@ internal fun AgentLogsScreen( onOpenHeapDump: (File, Place) -> Unit = { file, place -> SharkLog.d { "Nothing here to open $place of $file with" } }, + /** And the link to it, which names that dump: a row about another one is worth sending, not only clicking. */ + onCopyHeapDumpLink: (File, Place) -> Unit = { file, place -> + SharkLog.d { "Nothing here to link to $place of $file with" } + }, modifier: Modifier = Modifier ) { val groups = sessions.byHeapDump(heapDumpFile) @@ -80,7 +84,7 @@ internal fun AgentLogsScreen( Text(NO_SESSIONS, style = MaterialTheme.typography.bodyMedium) } group.sessions.forEach { session -> - SessionRow(session, group, onOpen, onCopyLink, onOpenHeapDump) + SessionRow(session, group, onOpen, onCopyLink, onOpenHeapDump, onCopyHeapDumpLink) } } } @@ -103,18 +107,22 @@ private fun SessionRow( group: HeapDumpSessions, onOpen: (Place, OpenIn) -> Unit, onCopyLink: (Place) -> Unit, - onOpenHeapDump: (File, Place) -> Unit + onOpenHeapDump: (File, Place) -> Unit, + onCopyHeapDumpLink: (File, Place) -> Unit ) { val place = Place.AgentLog(session.sessionId) val title = session.title() val summary = session.summary() val opensHeapDump = group.heapDumpFile?.takeIf { !group.isThisWindow && it.isFile } if (opensHeapDump != null) { - // No tab to choose and no link to copy: what a link names is a window, and the window this session was - // read in belongs to a run that has usually ended. The heap dump is what outlived it. - Column(Modifier.openable { onOpenHeapDump(opensHeapDump, place) }) { - Text(title, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) - Text(summary, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) + // No tab to choose, since this opens a window of its own heap dump rather than a tab of this one — and a + // link all the same, naming that dump: the run this session was read in has usually ended, and the file + // is what outlived it. + CopyLinkTarget({ onCopyHeapDumpLink(opensHeapDump, place) }) { + Column(Modifier.openable { onOpenHeapDump(opensHeapDump, place) }) { + Text(title, style = MaterialTheme.typography.bodyMedium, color = LINK_COLOR) + Text(summary, style = MaterialTheme.typography.bodySmall, color = MUTED_TEXT) + } } return } @@ -219,6 +227,10 @@ internal fun AgentLogScreen( onOpenHeapDump: (File, Place) -> Unit = { file, place -> SharkLog.d { "Nothing here to open $place of $file with" } }, + /** And the link to it, which names that dump rather than this window. See [AgentLogsScreen]. */ + onCopyHeapDumpLink: (File, Place) -> Unit = { file, place -> + SharkLog.d { "Nothing here to link to $place of $file with" } + }, modifier: Modifier = Modifier ) { Surface(modifier, color = MaterialTheme.colorScheme.surface) { @@ -249,7 +261,8 @@ internal fun AgentLogScreen( placeTitles = placeTitles, onOpen = onOpen, onCopyLink = onCopyLink, - onOpenHeapDump = onOpenHeapDump + onOpenHeapDump = onOpenHeapDump, + onCopyHeapDumpLink = onCopyHeapDumpLink ) } } @@ -276,7 +289,8 @@ private fun AgentCallRow( placeTitles: Map, onOpen: (Place, OpenIn) -> Unit, onCopyLink: (Place) -> Unit, - onOpenHeapDump: (File, Place) -> Unit + onOpenHeapDump: (File, Place) -> Unit, + onCopyHeapDumpLink: (File, Place) -> Unit ) { val place = call.place // Which heap dump the row is about when it isn't this window's, and null when it is. An address is an @@ -317,7 +331,11 @@ private fun AgentCallRow( Text(call.verb, style = MaterialTheme.typography.bodyMedium) when { leadsTo == null -> Text(target, style = MaterialTheme.typography.bodyMedium) - opens != null -> LinkText(target, Modifier.openable { onOpenHeapDump(opens, leadsTo) }) + // Another dump: one place to go, and a link that names that dump for somebody to go there + // without this window. + opens != null -> CopyLinkTarget({ onCopyHeapDumpLink(opens, leadsTo) }) { + LinkText(target, Modifier.openable { onOpenHeapDump(opens, leadsTo) }) + } else -> { val open: (OpenIn) -> Unit = { openIn -> onOpen(leadsTo, openIn) } OpenTarget(open, { onCopyLink(leadsTo) }) { LinkText(target, Modifier.openable(open)) } @@ -346,7 +364,7 @@ private fun AgentCallRow( } if (isUnfolded) { openHeapDumps.forEach { path -> - OpenHeapDumpRow(path, heapDumpFile, onOpenHeapDump) + OpenHeapDumpRow(path, heapDumpFile, onOpenHeapDump, onCopyHeapDumpLink) } } } @@ -392,7 +410,8 @@ private fun UnfoldableVerb( private fun OpenHeapDumpRow( path: String, heapDumpFile: File, - onOpenHeapDump: (File, Place) -> Unit + onOpenHeapDump: (File, Place) -> Unit, + onCopyHeapDumpLink: (File, Place) -> Unit ) { val file = File(path) val name = file.name @@ -403,15 +422,17 @@ private fun OpenHeapDumpRow( Text("$name ($THIS_HEAP_DUMP)", indent, style = style, color = MUTED_TEXT) // Gone, which a list of what *was* open is exactly where somebody finds out. !file.isFile -> Text("$name ($MISSING_HEAP_DUMP)", indent, style = style, color = MUTED_TEXT) - // The whole heap dump, since a dump named without a place in it is the window that dump opens on. No - // tab to choose and no link to copy, for the reason a session of another dump has neither: what a link - // names is a window, and this is a file that has to be opened in one first. - else -> Text( - name, - indent.openable { onOpenHeapDump(file, Place.wholeHeapDump()) }, - style = style, - color = LINK_COLOR - ) + // The whole heap dump, since a dump named without a place in it is the window that dump opens on. No tab + // to choose, for the reason a session of another dump has none — it opens a window of its own — and a + // link to that dump all the same, which is a file anybody can be sent. + else -> CopyLinkTarget({ onCopyHeapDumpLink(file, Place.wholeHeapDump()) }) { + Text( + name, + indent.openable { onOpenHeapDump(file, Place.wholeHeapDump()) }, + style = style, + color = LINK_COLOR + ) + } } } diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt index 6f7a5d8f52..0f2add1022 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt @@ -74,11 +74,16 @@ internal object DeepLinkPeers { } /** - * Follows [link] in this run, or in whichever other run has the window it names. + * Follows [link] in this run, or in whichever other run has a window of the heap dump it names. * * Which is what a run handed a link by the OS does with it, and the macOS half of the reason this exists * at all: there, one installed app is handed every `shark://` link on the machine, including the ones - * naming a window of a run from source that the OS knows nothing about. + * about a heap dump open in a run from source that the OS knows nothing about. + * + * **A run claims a link only for a window it already has**, never for a heap dump it could open, or every + * run on the machine would claim every link. Whoever is left with it opens the dump — see + * [ExplorerWindows.open] — so a link with nobody to take it lands in the run the OS chose, which is the + * installed app. * * Asking the others is a connection each, so it happens off the caller's thread — a link arrives on the * event thread there, and a run that has been killed is only found out about by waiting for it. @@ -87,12 +92,12 @@ internal object DeepLinkPeers { link: DeepLink, windows: ExplorerWindows ) { - if (windows.holds(link.windowId)) { + if (windows.windowFor(link) != null) { windows.open(link) return } Thread({ - // Nobody else's, so this run answers for it, which is an empty window saying the window has gone. + // Nobody else's, so this run answers for it, which is opening that heap dump here. if (deliver(listOf(link)).isNotEmpty()) { windows.open(link) } @@ -103,11 +108,11 @@ internal object DeepLinkPeers { } /** - * Hands each of [links] to whichever other run has the window it names, and returns the ones nobody - * claimed. + * Hands each of [links] to whichever other run has a window of the heap dump it names, and returns the + * ones nobody claimed. * - * The leftovers are the caller's to answer for, which is what makes a link to a window that has gone an - * empty window saying so rather than a process that started and exited without a word. + * The leftovers are the caller's to answer for, which is what makes a link whose window has gone a window + * of that heap dump here rather than a process that started and exited without a word. */ fun deliver(links: List): List { if (links.isEmpty()) { @@ -163,8 +168,8 @@ internal object DeepLinkPeers { } /** - * One line in, one line out, per connection: the token and the link, answered with whether the window - * this link names belongs to this run. + * One line in, one line out, per connection: the token and the link, answered with whether a window of the + * heap dump this link names belongs to this run. */ private fun accept( serverSocket: ServerSocket, @@ -217,7 +222,7 @@ internal object DeepLinkPeers { } // Answered before the window is asked to go anywhere, because the run on the other end is waiting to // find out whether to keep looking, and going somewhere is a frame away rather than a read away. - if (windows.holds(link.windowId)) { + if (windows.windowFor(link) != null) { writer.println(ACCEPTED) windows.open(link) } else { diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt index fd32d2f7de..7bd7af43c0 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt @@ -387,8 +387,9 @@ private fun ExplorerWindow.agentHeapDump(open: OpenHeapDump): AgentHeapDump = goToLinked(place) bringToFront() // And the link itself, which is the same one the right click menu copies: an agent's answer can then - // point at this place rather than describe how to get to it. - ShownPlace.at(DeepLink(deepLinkId, place).toUri()) + // point at this place rather than describe how to get to it. Naming this window as well as the dump, + // since a reader following it while this run is up should land on the window they watched it happen in. + ShownPlace.at(DeepLink(open.session.heapDumpFile, place, windowId = deepLinkId).toUri()) } /** diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt index c8fa2ce62e..81e4c94df6 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt @@ -153,42 +153,92 @@ internal class ExplorerWindows( private val windows: SnapshotStateList = mutableStateListOf() ) : MutableList by windows { - /** Whether a window of this run answers to [windowId], which is what a link asks before it is handed over. */ - fun holds(windowId: String): Boolean = any { it.deepLinkId == windowId } + /** + * The window of this run [link] leads to, or null for one no window here can answer. + * + * Which is what a run asks before handing a link on to the others, so it is deliberately *only* about + * windows that exist: a run that answered "I could open that file" would claim every link on the machine. + * Opening the dump is what whoever ends up answering does, in [open]. + * + * In order, because each step is right about something the next one isn't: + * + * - **The window the link was made from**, while it is still open. Two windows on one dump are two + * readings of it, and this is the only thing that tells them apart. + * - **A window of that heap dump**, by path. The window is gone or belonged to another run, and the dump + * is what the link was really about. + * - **A window of a dump with that file name**, for a link somebody typed, which carries no path. + * - **A window with that id**, for links written before a link named a heap dump: `shark:///…` is + * in notes on disk and in agent sessions, and one of those pasted back into the run it came from still + * goes where it went. Last, so that a heap dump called like a window id — which is a file called + * `abcd2345` — is read as the heap dump. + */ + fun windowFor(link: DeepLink): ExplorerWindow? { + val ofWindowId = link.windowId?.let { id -> firstOrNull { it.deepLinkId == id } } + if (ofWindowId != null) { + return ofWindowId + } + val path = link.heapDumpPath + if (path != null) { + return firstOrNull { it.heapDumpFile?.absoluteFile?.normalize() == path } + } + return firstOrNull { it.heapDumpFile?.name == link.heapDumpName } + ?: firstOrNull { it.deepLinkId == link.heapDumpName } + } /** - * Follows [link]: the window it names opens the place in a new tab and comes to the front. + * Follows [link]: the place opens as a new tab in a window of that heap dump, which comes to the front. * - * A link whose window has gone gets an empty window saying so rather than silence. Silence is the one + * **A link outlives the window it was made from**, so one whose window has gone opens the heap dump it + * names — that is the whole point of naming the dump — and only a link naming a file that isn't there any + * more has nowhere to go. That gets an empty window saying so rather than silence: silence is the one * answer that can't be told from the app having failed to start, and a link is usually followed from * somewhere that cannot see whether this app did anything at all. */ fun open(link: DeepLink) { - val window = firstOrNull { it.deepLinkId == link.windowId } - if (window == null) { - SharkLog.d { "No window of this run is ${link.windowId}: opening one to say so" } + val window = windowFor(link) + if (window != null) { + SharkLog.d { "A link asked window ${window.deepLinkId} for ${link.place} of ${link.heapDumpName}" } + window.goToLinked(link.place) + window.bringToFront() + return + } + val heapDumpFile = link.heapDumpPath?.takeIf { it.isFile } + if (heapDumpFile == null) { + SharkLog.d { "No window of this run has ${link.heapDumpName} open: opening one to say so" } add( ExplorerWindow( cascade = freeCascade(), titlePrefix = titlePrefix, - deepLinkProblem = noSuchWindow(link.windowId) + deepLinkProblem = noSuchHeapDump(link) ) ) return } - SharkLog.d { "A link asked window ${link.windowId} for ${link.place}" } - window.goToLinked(link.place) - window.bringToFront() + SharkLog.d { "A link asked for ${link.place} of ${link.heapDumpName}, which is not open yet" } + goToHeapDump(heapDumpFile, link.place) } /** The first step of the cascade no window is at, which is where the next window goes. */ fun freeCascade(): Int = generateSequence(0, Int::inc).first { step -> none { it.cascade == step } } companion object { - /** What an empty window opened by a link naming a window that has gone says in the middle of it. */ - fun noSuchWindow(windowId: String): String = - "No window called $windowId is open. A link leads to one window of one run of this app, so it " + - "stops working when that window is closed or the app is restarted." + /** + * What an empty window opened by a link with nowhere to go says in the middle of it. + * + * Two ways to get here, and they need different things done about them: a heap dump that has been moved + * or deleted since the link was made, and a link that never said where the dump was — which is one + * somebody typed or shortened, since every link this app writes carries the path. + */ + fun noSuchHeapDump(link: DeepLink): String { + val path = link.heapDumpPath + return if (path == null) { + "No heap dump called ${link.heapDumpName} is open, and this link doesn't say where that file is, " + + "so there is nothing to open. A link copied from a window carries the path." + } else { + "${link.heapDumpName} is not open and there is no file at $path to open, so this link has nowhere " + + "to go. A link outlives the window it was copied from, but not the heap dump it is about." + } + } } } @@ -206,8 +256,8 @@ internal fun explorerWindows(arguments: ExplorerArguments): ExplorerWindows = add(ExplorerWindow(file, cascade = index, titlePrefix = titlePrefix)) } } - // Which window is which, for reading a link out of a log afterwards: a link names one of these and - // nothing else in the file says what the name stands for. + // Which window is which, for reading a link out of a log afterwards: a link carries the window it was + // copied from, and nothing else in the file says what that id stands for. SharkLog.d { "Windows of this run: ${joinToString { "${it.deepLinkId} ${it.title}" }}" } } @@ -278,8 +328,8 @@ internal fun ExplorerWindows.openHeapDump( * second one on the same file — the same rule [openHeapDump] follows, one window per heap dump — and the * window that has just been opened for it is in front already, so only an existing one is brought forward. * - * Not [DeepLink]: a link names the window it was copied from, and the window a session recorded is usually - * one from a run that has since ended. The heap dump outlives it, which is why this goes by the file. + * Which is also where [ExplorerWindows.open] ends up for a link whose window has gone, a link being about a + * heap dump for the same reason a row of that screen is. */ internal fun ExplorerWindows.goToHeapDump( heapDumpFile: File, diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt index 221938dc46..bc2866f729 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt @@ -122,12 +122,15 @@ internal class HeadlessAgentHeapDumps( open = open, agent = OpenAgentHeapDump(windowId = windowId, open = open) { place -> SharkLog.d { "Nowhere to show $place: this run was started with $NO_UI_OPTION" } - // And no link either, deliberately: a link names a window, so one from here would be an address - // nothing answers to, handed to somebody who would click it. - ShownPlace.nowhere( - "This Shark Explorer was started with $NO_UI_OPTION, so it has no window and nothing was shown. " + - "Say what you found in your answer instead. Whoever opens ${file.name} in a window later will " + - "find your notes and verdicts on it, since those are on disk rather than on screen." + // A link all the same, and it works: a link names the heap dump rather than a window, so this one + // opens the file at that place in whatever Shark Explorer whoever clicks it has. Which is the whole + // of what a run with no screen can offer, and more than nothing. + ShownPlace.onlyAsALink( + link = DeepLink(file, place).toUri(), + problem = "This Shark Explorer was started with $NO_UI_OPTION, so it has no window and nobody " + + "saw this. Put the link in your answer instead: it opens ${file.name} at that place for " + + "whoever reads it, with your notes and verdicts on it, since those are on disk rather than on " + + "screen." ) } ) diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt index 896f011bf6..8d5129c57d 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt @@ -648,12 +648,24 @@ internal fun HeapDumpExplorer( * rectangle, a row, a field, a button and a tab is one thing, and five of them would drift. */ val copyLink: (Place) -> Unit = { destination -> - val link = DeepLink(deepLinkId, destination).toUri() + val link = DeepLink(session.heapDumpFile, destination, windowId = deepLinkId).toUri() // In the log as well as on the clipboard, so that a link someone reports as not working can be compared // against the one this window actually handed out. SharkLog.d { "Copied $link" } copyToClipboard(link) } + /** + * And for a place of a heap dump this window hasn't got, which the *Agent logs* screens are full of. + * + * No window id on it: this window is not one of that dump's, so there is no window to prefer and the link + * is the file plus the place — which is all a link needs, and is why a row about somebody else's dump is + * something to send rather than only something to click. See [shark.explorer.DeepLink]. + */ + val copyHeapDumpLink: (File, Place) -> Unit = { heapDumpFile, destination -> + val link = DeepLink(heapDumpFile, destination).toUri() + SharkLog.d { "Copied $link" } + copyToClipboard(link) + } /** The same, for everything that names an object by its id. */ val copyObjectLink: (Long) -> Unit = { objectId -> copyLink(Place.Object(objectId)) } /** And for the view's right click menu, which is on whatever the pointer is on. */ @@ -774,6 +786,7 @@ internal fun HeapDumpExplorer( heapDumpFile = session.heapDumpFile, agentPlaceTitles = agentPlaceTitles, onOpenHeapDump = onOpenHeapDump, + onCopyHeapDumpLink = copyHeapDumpLink, sizes = sizes, onOpen = openObject, onCopyLink = copyObjectLink, @@ -1119,6 +1132,8 @@ private fun ListPlace( agentPlaceTitles: Map, /** And where a session or a row about another heap dump leads: that dump. See [AgentLogsScreen]. */ onOpenHeapDump: (File, Place) -> Unit, + /** And the link to it, which names that dump rather than this window. */ + onCopyHeapDumpLink: (File, Place) -> Unit, sizes: HeapSizes, onOpen: (Long, OpenIn) -> Unit, onCopyLink: (Long) -> Unit, @@ -1173,6 +1188,7 @@ private fun ListPlace( onOpen = onOpenPlace, onCopyLink = onCopyPlaceLink, onOpenHeapDump = onOpenHeapDump, + onCopyHeapDumpLink = onCopyHeapDumpLink, modifier = modifier ) is Place.AgentLog -> AgentLogScreen( @@ -1183,6 +1199,7 @@ private fun ListPlace( onOpen = onOpenPlace, onCopyLink = onCopyPlaceLink, onOpenHeapDump = onOpenHeapDump, + onCopyHeapDumpLink = onCopyHeapDumpLink, modifier = modifier ) // The places with a view of their own are drawn by the panes, not here. diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt index 685bfba09a..17e850c872 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/AgentLogsScreenTest.kt @@ -15,6 +15,8 @@ import androidx.compose.ui.test.isSelected import androidx.compose.ui.test.onAllNodesWithText import androidx.compose.ui.test.onNodeWithText import androidx.compose.ui.test.performClick +import androidx.compose.ui.test.performMouseInput +import androidx.compose.ui.test.rightClick import androidx.compose.ui.test.waitUntilAtLeastOneExists import java.io.File import java.time.Instant @@ -25,6 +27,7 @@ import org.junit.Test import org.junit.rules.TemporaryFolder import shark.explorer.Adb import shark.explorer.AdbOutput +import shark.explorer.DeepLink import shark.explorer.DeviceHeapDumps import shark.explorer.HeapDominatorTreemap import shark.explorer.Place @@ -204,6 +207,25 @@ class AgentLogsScreenTest { assertThat(opened).isEqualTo(otherHeapDump to Place.AgentLog(SESSION_ID)) } + @Test fun `a session about another heap dump is worth sending as well as opening`() { + val otherHeapDump = testFolder.newFile("another.hprof") + val copied = mutableListOf() + explorerUiTest { + openAgentLogs( + sessions = listOf(session(calls = listOf(call(heapDumpPath = otherHeapDump.absolutePath)))), + copyToClipboard = { copied += it } + ) + + onNodeWithText(CLIENT, substring = true).performMouseInput { rightClick() } + onNodeWithText(COPY_LINK).performClick() + } + + // That dump, and no window id: this window is not one of that file's, so there is none to prefer — and + // a link names the heap dump, which is what makes a row about somebody else's dump something to send + // rather than only something to click. See [DeepLink]. + assertThat(copied).containsExactly(DeepLink(otherHeapDump, Place.AgentLog(SESSION_ID)).toUri()) + } + @Test fun `a call that went on to another heap dump reads as the address, and leads to that dump`() { val otherHeapDump = testFolder.newFile("another.hprof") var opened: Pair? = null @@ -294,7 +316,8 @@ class AgentLogsScreenTest { /** Opens the window on [leakyHeapDump] with [sessions] as the agents that have worked through it. */ private fun ComposeUiTest.openAgentLogs( sessions: List, - onOpenHeapDump: (File, Place) -> Unit = { _, _ -> } + onOpenHeapDump: (File, Place) -> Unit = { _, _ -> }, + copyToClipboard: (String) -> Unit = {} ) { setContent { MaterialTheme { @@ -307,6 +330,7 @@ class AgentLogsScreenTest { // Where a row about another heap dump goes, which is a question about every window of the run and // so answered outside this one. See [ExplorerWindowTest]. onOpenHeapDump = onOpenHeapDump, + copyToClipboard = copyToClipboard, deviceHeapDumps = DeviceHeapDumps(NO_DEVICE_ADB) ) } diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerAppTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerAppTest.kt index ad93a64dab..6bef635e41 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerAppTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerAppTest.kt @@ -211,7 +211,7 @@ class ExplorerAppTest { @Test fun `the map's menu copies a link to the rectangle under the pointer`() { val copied = mutableListOf() explorerUiTest { - openHeapDump(copyToClipboard = { copied += it }) + val heapDumpFile = openHeapDump(copyToClipboard = { copied += it }) hoverView(TREEMAP_X, TREEMAP_Y) // The card naming the rectangle, which is how a test knows the pointer has settled on one: the menu // acts on what is hovered, and nothing is until then. @@ -221,8 +221,11 @@ class ExplorerAppTest { onNodeWithText(COPY_LINK).performClick() // Beside opening the rectangle in a tab of its own, which is the menu's other item: a link is the - // same move made somewhere else, so wherever one is offered so is the other. - assertThat(copied).containsExactly(DeepLink(WINDOW_ID, Place.Object(payloadObjectId)).toUri()) + // same move made somewhere else, so wherever one is offered so is the other. Named after the heap + // dump, with this window as the refinement it is while the window is open. See [DeepLink]. + assertThat(copied).containsExactly( + DeepLink(heapDumpFile, Place.Object(payloadObjectId), windowId = WINDOW_ID).toUri() + ) } } @@ -809,12 +812,14 @@ class ExplorerAppTest { } } + /** Opens a heap dump and hands back the file, which is what a link to a place in it names. */ private fun ComposeUiTest.openHeapDump( heapDumpFile: File = testHeapDump(), copyToClipboard: (String) -> Unit = {} - ) { + ): File { setExplorerContent(heapDumpFile, copyToClipboard = copyToClipboard) waitForTheTree(OPEN_TIMEOUT_MILLIS) + return heapDumpFile } /** diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt index f0666a4129..5d811d5d6e 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt @@ -7,6 +7,7 @@ import org.assertj.core.api.Assertions.assertThat import org.assertj.core.api.Assertions.assertThatThrownBy import org.junit.Rule import org.junit.Test +import org.junit.rules.TemporaryFolder import shark.explorer.Adb import shark.explorer.AdbOutput import shark.explorer.DeepLink @@ -27,6 +28,9 @@ class ExplorerWindowTest { */ @get:Rule val logged = RecordedLog() + /** For the one thing a link needs a real file for: opening a heap dump no window has. */ + @get:Rule val temporaryFolder = TemporaryFolder() + @Test fun `an app started with no heap dump has one window to open one from`() { val windows = explorerWindows(noHeapDumps()) @@ -128,26 +132,76 @@ class ExplorerWindowTest { @Test fun `every window answers to an id of its own`() { val windows = explorerWindows(opening(FIRST_DUMP, FIRST_DUMP)) - // The whole reason a link names a window rather than a heap dump: the same dump open twice is two - // places to be, and a link has to lead to the one it was copied from. + // The whole reason a link says which window as well as which heap dump: the same dump open twice is two + // places to be, and while both are open a link leads to the one it was copied from. assertThat(windows.map { it.deepLinkId }).doesNotHaveDuplicates() } - @Test fun `a link goes to the window it names and to no other`() { + @Test fun `a link goes to the window of the heap dump it names and to no other`() { val windows = explorerWindows(opening(FIRST_DUMP, SECOND_DUMP)) val (first, second) = windows - windows.open(DeepLink(second.deepLinkId, Place.Starred)) + windows.open(DeepLink(SECOND_DUMP, Place.Starred)) assertThat(second.linkedPlaces).containsExactly(Place.Starred) assertThat(first.linkedPlaces).isEmpty() assertThat(windows).hasSize(2) } + /** + * The case the window id is there for, and the only one: which of two readings of one dump. Both windows + * answer to the heap dump, so without it a link would land on whichever came first. + */ + @Test fun `a link to one of two windows on the same heap dump goes to that one`() { + val windows = explorerWindows(opening(FIRST_DUMP, FIRST_DUMP)) + val (first, second) = windows + + windows.open(DeepLink(FIRST_DUMP, Place.Starred, windowId = second.deepLinkId)) + + assertThat(second.linkedPlaces).containsExactly(Place.Starred) + assertThat(first.linkedPlaces).isEmpty() + } + + /** + * Which is most links a day later: the run they were copied from has been closed and started again, and + * every window id it handed out went with it. + */ + @Test fun `a link whose window has gone goes to a window of its heap dump`() { + val windows = explorerWindows(opening(FIRST_DUMP)) + + windows.open(DeepLink(FIRST_DUMP, Place.Starred, windowId = CLOSED_WINDOW_ID)) + + assertThat(windows.single().linkedPlaces).containsExactly(Place.Starred) + assertThat(windows).hasSize(1) + } + + /** A link somebody typed or shortened, which names the dump the way a person would. */ + @Test fun `a link with a file name and no path goes to the window of that file`() { + val windows = explorerWindows(opening(FIRST_DUMP)) + + windows.open(DeepLink.parse("shark://${FIRST_DUMP.name}/starred")) + + assertThat(windows.single().linkedPlaces).containsExactly(Place.Starred) + } + + /** + * Links written before a link named a heap dump are on disk in notes and in agent sessions, and one of + * those pasted back into the run it came from still goes where it went. + */ + @Test fun `a link that names only a window id still finds that window`() { + val windows = explorerWindows(opening(FIRST_DUMP)) + val window = windows.single() + + windows.open(DeepLink.parse("shark://${window.deepLinkId}/leaks")) + + assertThat(window.linkedPlaces).containsExactly(Place.Leaks()) + assertThat(windows).hasSize(1) + } + @Test fun `a place a link asked for is dropped once a tab has opened it`() { val windows = explorerWindows(opening(FIRST_DUMP)) val window = windows.single() - windows.open(DeepLink(window.deepLinkId, Place.Leaks())) + windows.open(DeepLink(FIRST_DUMP, Place.Leaks())) window.linkedPlaceOpened(Place.Leaks()) @@ -160,36 +214,65 @@ class ExplorerWindowTest { val windows = explorerWindows(opening(FIRST_DUMP)) val window = windows.single() - windows.open(DeepLink(window.deepLinkId, Place.Starred)) - windows.open(DeepLink(window.deepLinkId, Place.Leaks())) + windows.open(DeepLink(FIRST_DUMP, Place.Starred)) + windows.open(DeepLink(FIRST_DUMP, Place.Leaks())) assertThat(window.linkedPlaces).containsExactly(Place.Starred, Place.Leaks()) } - @Test fun `a link to a window that has gone opens one saying so`() { + /** + * The payoff of a link naming the heap dump: the run it was copied from can be gone, and the link still + * puts the reader in front of what it names. + */ + @Test fun `a link to a heap dump nothing has open opens it`() { + val heapDumpFile = temporaryFolder.newFile("third.hprof") val windows = explorerWindows(opening(FIRST_DUMP)) - windows.open(DeepLink(CLOSED_WINDOW_ID, Place.Starred)) + windows.open(DeepLink(heapDumpFile, Place.Starred, windowId = CLOSED_WINDOW_ID)) + + val opened = windows.last() + assertThat(windows).hasSize(2) + assertThat(opened.heapDumpFile).isEqualTo(heapDumpFile.absoluteFile) + assertThat(opened.linkedPlaces).containsExactly(Place.Starred) + } + + @Test fun `a link to a heap dump that has been deleted opens a window saying so`() { + val windows = explorerWindows(opening(FIRST_DUMP)) + + windows.open(DeepLink(SECOND_DUMP, Place.Starred)) // Rather than nothing at all, which is the one answer that can't be told from the app having failed // to start — and a link is usually followed from somewhere that can't see either way. assertThat(windows).hasSize(2) - assertThat(windows.last().deepLinkProblem).contains(CLOSED_WINDOW_ID) + assertThat(windows.last().deepLinkProblem) + .contains(SECOND_DUMP.name) + .contains(SECOND_DUMP.absolutePath) assertThat(windows.last().heapDumpFile).isNull() - assertThat(logged).anyMatch { CLOSED_WINDOW_ID in it } + assertThat(logged).anyMatch { SECOND_DUMP.name in it } + } + + /** A link shortened to the file name, which only works while something has that file open. */ + @Test fun `a link with no path to a heap dump nothing has open says what is missing`() { + val windows = explorerWindows(opening(FIRST_DUMP)) + + windows.open(DeepLink.parse("shark://${SECOND_DUMP.name}/starred")) + + assertThat(windows.last().deepLinkProblem) + .contains(SECOND_DUMP.name) + .contains("doesn't say where that file is") } @Test fun `a window opened by a link that found nothing lands beside the others`() { val windows = explorerWindows(opening(FIRST_DUMP, SECOND_DUMP)) - windows.open(DeepLink(CLOSED_WINDOW_ID, Place.Starred)) + windows.open(DeepLink.parse("shark://third.hprof/starred")) assertThat(windows.map { it.cascade }).doesNotHaveDuplicates() } @Test fun `a window that gets a heap dump stops saying a link found nothing`() { val windows = explorerWindows(noHeapDumps()) - windows.open(DeepLink(CLOSED_WINDOW_ID, Place.Starred)) + windows.open(DeepLink.parse("shark://third.hprof/starred")) val empty = windows.last() windows.openHeapDump(empty, FIRST_DUMP) @@ -225,12 +308,16 @@ class ExplorerWindowTest { assertThat(opened.linkedPlaces).containsExactly(Place.Starred) } - @Test fun `a run knows which windows are its own`() { + /** + * What another run of this app asks before handing a link over — and it is about windows that exist and + * never about a file this run could open, or every run on the machine would claim every link. See + * [DeepLinkPeers]. + */ + @Test fun `a run claims a link only for a heap dump it has open`() { val windows = explorerWindows(opening(FIRST_DUMP)) - // What another run of this app asks before handing a link over. See [DeepLinkPeers]. - assertThat(windows.holds(windows.single().deepLinkId)).isTrue() - assertThat(windows.holds(CLOSED_WINDOW_ID)).isFalse() + assertThat(windows.windowFor(DeepLink(FIRST_DUMP, Place.Starred))).isEqualTo(windows.single()) + assertThat(windows.windowFor(DeepLink(SECOND_DUMP, Place.Starred))).isNull() } @Test fun `an agent asking for a heap dump a window already has gets that window`() { diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt index 2f1d1eb428..813e4f858c 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt @@ -9,6 +9,7 @@ import org.junit.Test import org.junit.rules.TemporaryFolder import shark.explorer.Adb import shark.explorer.AdbOutput +import shark.explorer.DeepLink import shark.explorer.DeviceHeapDumps import shark.explorer.LeakStatus import shark.explorer.LeakStatusOverride @@ -83,9 +84,14 @@ class HeadlessAgentHeapDumpsTest { assertThat(shown.problem) .contains(NO_UI_OPTION) .contains(file.name) - // And no link, which is the half of it an agent would otherwise pass on: a `shark://` link names a - // window, so one from a run that has none is an address nothing answers to. - assertThat(shown.link).isNull() + // And a link all the same, which is the half of it an agent passes on: a `shark://` link names the heap + // dump rather than a window, so one from a run that has no window opens this file for whoever clicks + // it. No window id on it, since there is no window of this run to prefer. + val link = DeepLink.parse(shown.link!!) + assertThat(link.heapDumpName).isEqualTo(file.name) + assertThat(link.heapDumpPath).isEqualTo(file.absoluteFile) + assertThat(link.windowId).isNull() + assertThat(link.place).isEqualTo(Place.Leaks()) } } diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ObjectsScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ObjectsScreenTest.kt index e7e0eb96af..37d79101b1 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ObjectsScreenTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ObjectsScreenTest.kt @@ -148,20 +148,23 @@ class ObjectsScreenTest { @Test fun `a listed object's menu copies a link to it`() { val copied = mutableListOf() explorerUiTest { - openHeapDump(copyToClipboard = { copied += it }) + val heapDumpFile = openHeapDump(copyToClipboard = { copied += it }) listObjects() onNodeWithText("java.lang.Object[] array").performMouseInput { rightClick() } onNodeWithText(COPY_LINK).performClick() // Beside "open in a new tab" wherever that is, this row included: the two are the same thought a - // step apart, and a link is how the object leaves this window at all. - assertThat(copied) - .containsExactly(DeepLink(WINDOW_ID, Place.Object(payloadObjectId)).toUri()) + // step apart, and a link is how the object leaves this window at all. It names the heap dump, so it + // outlives the window it was copied from. See [DeepLink]. + assertThat(copied).containsExactly( + DeepLink(heapDumpFile, Place.Object(payloadObjectId), windowId = WINDOW_ID).toUri() + ) } } - private fun ComposeUiTest.openHeapDump(copyToClipboard: (String) -> Unit = {}) { + /** Opens a heap dump and hands back the file, which is what a link to a place in it names. */ + private fun ComposeUiTest.openHeapDump(copyToClipboard: (String) -> Unit = {}): File { val heapDumpFile = testHeapDump() setContent { MaterialTheme { @@ -179,6 +182,7 @@ class ObjectsScreenTest { } } waitForTheTree(OPEN_TIMEOUT_MILLIS) + return heapDumpFile } /** Opens a tab on the list and waits for the pass over the heap dump that fills it. */ diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/TabStripTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/TabStripTest.kt index 961ccb82ae..aefdf8388e 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/TabStripTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/TabStripTest.kt @@ -174,14 +174,17 @@ class TabStripTest { @Test fun `right clicking a tab copies a link to where that tab is`() { val copied = mutableListOf() explorerUiTest { - openHeapDump(copyToClipboard = { copied += it }) + val heapDumpFile = openHeapDump(copyToClipboard = { copied += it }) tab(HeapDominatorTreemap.ROOT_LABEL).performMouseInput { rightClick() } onNodeWithText(COPY_LINK).performClick() - // The window rather than the heap dump, because the same dump open twice is two places to be — - // and the tab's own place, so that following it lands where it was copied from. See [DeepLink]. - assertThat(copied).containsExactly(DeepLink(WINDOW_ID, Place.wholeHeapDump()).toUri()) + // The heap dump, so that the link outlives this window, and this window with it, because the same + // dump open twice is two places to be — plus the tab's own place, so that following it lands where + // it was copied from. See [DeepLink]. + assertThat(copied).containsExactly( + DeepLink(heapDumpFile, Place.wholeHeapDump(), windowId = WINDOW_ID).toUri() + ) } } @@ -204,14 +207,16 @@ class TabStripTest { @Test fun `right clicking a button on the bar copies a link to the screen it opens`() { val copied = mutableListOf() explorerUiTest { - openHeapDump(copyToClipboard = { copied += it }) + val heapDumpFile = openHeapDump(copyToClipboard = { copied += it }) screenButton(Place.LEAKS_LABEL).performMouseInput { rightClick() } onNodeWithText(COPY_LINK).performClick() // A button opens a screen nobody has been to yet, and a link to it is that screen as it opens: no // tab has to be opened first to have something to copy. - assertThat(copied).containsExactly(DeepLink(WINDOW_ID, Place.Leaks()).toUri()) + assertThat(copied).containsExactly( + DeepLink(heapDumpFile, Place.Leaks(), windowId = WINDOW_ID).toUri() + ) // And nothing was opened by asking for the link, which a menu that clicked the button would have. assertThat(tabs().fetchSemanticsNodes()).hasSize(1) } @@ -261,12 +266,13 @@ class TabStripTest { private fun isButton(): SemanticsMatcher = SemanticsMatcher.expectValue(SemanticsProperties.Role, Role.Button) + /** Opens a heap dump and hands back the file, which is what a link to a place in it names. */ private fun ComposeUiTest.openHeapDump( /** Read inside the composition, so that a test can ask for a place once the window is up. */ linkedPlaces: () -> List = { emptyList() }, onLinkedPlaceOpened: (Place) -> Unit = {}, copyToClipboard: (String) -> Unit = {} - ) { + ): File { // Written before the composition rather than in it: every recomposition would write it again, and // the second one fails rather than returning the file the window is already reading. val heapDumpFile = testHeapDump() @@ -287,6 +293,7 @@ class TabStripTest { // A tab is named by a read of the heap dump, so a window whose strip has not caught up yet is one // where every assertion about a title would be about the placeholder. waitUntilAtLeastOneExists(hasText(HeapDominatorTreemap.ROOT_LABEL) and isTab(), OPEN_TIMEOUT_MILLIS) + return heapDumpFile } /** diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt index a7c7320782..9de31e5659 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt @@ -1,40 +1,81 @@ package shark.explorer +import java.io.File import java.net.URLDecoder import java.net.URLEncoder import kotlin.random.Random /** - * A link to one place in one open window: `shark:///?`. + * A link to one place in one heap dump: `shark:///?`. * * The point of it is that anything on screen can be handed to someone else — or printed by a script or an * agent — as one line of text that puts them in front of it. So every [Place] has a spelling here, and the * spellings carry the whole of the place rather than a shorthand for it: a filtered list of objects arrives * filtered, and a page of leaks arrives with the same ones unfolded. * - * **It names a window and not a heap dump.** The same dump is often open twice — that is what comparing two - * of them is — so a path would be ambiguous exactly when it matters. A [windowId] is not ambiguous, and it - * also settles what a link means once the window is gone: nothing, which is the honest answer, rather than - * the same object in whichever other window happened to have that file open. + * **It names the heap dump, and a window only as a refinement.** Every place there is belongs to the dump + * rather than to whatever is showing it — an address, a leak, a filter over the object list, a note — so a + * link that named a window was a link that died with the window, which is most links a day later. This one + * survives: the run it was made from can be gone, and it still opens what it names, in a window of that dump + * if there is one and in a new window if there isn't. + * + * Which leaves the case the window id was there for. The same dump *is* often open twice — that is what + * comparing two readings of it is — so [windowId] says which of them a link was made from, and it is + * honoured while that window is open and **ignored once it isn't**, rather than turning the link into an + * error. Being right about which window is worth a lot while the window exists and nothing at all + * afterwards. + * + * [heapDumpName] is the authority because it is the part a person reads and types; [heapDumpPath] is what + * makes the link exact, since two dumps called `com.squareup.hprof` off two devices are two investigations, + * and it is also the only way to open a dump nothing has open. A link with the name and no path is what + * somebody typed by hand, and it resolves against what is open. * * Immutable and in this module rather than in the UI, so that what a link means is unit tested rather than - * found out by clicking one. See [Place]. + * found out by clicking one. See [Place] and `ExplorerWindows.windowFor`. */ data class DeepLink( - /** Which open window answers to this link. See [newWindowId]. */ - val windowId: String, - val place: Place + /** The heap dump's file name: what a link is read as, and all of it that has to be typed. */ + val heapDumpName: String, + val place: Place, + /** + * Where that dump is, so that a link outlives every window of it and can open one. + * + * Absolute and normalized, since it is compared against what a window has open and read months later. + * Null for a link somebody typed, which names a dump only by [heapDumpName]. + */ + val heapDumpPath: File? = null, + /** + * Which window of that dump the link was made from, or null for one nobody made from a window. + * + * A refinement and never a requirement: see the class comment. [newWindowId] is where these come from. + */ + val windowId: String? = null ) { + /** A link to a place in a heap dump this app has open, which is every link the app itself writes. */ + constructor( + heapDumpFile: File, + place: Place, + windowId: String? = null + ) : this( + heapDumpName = heapDumpFile.name, + place = place, + heapDumpPath = normalizedHeapDumpPath(heapDumpFile), + windowId = windowId + ) + /** The link as text, which is what gets copied, printed and pasted. */ fun toUri(): String { - val parameters = place.linkParameters() + val parameters = place.linkParameters() + listOfNotNull( + heapDumpPath?.let { DUMP_PARAMETER to it.path }, + windowId?.let { WINDOW_PARAMETER to it } + ) val query = if (parameters.isEmpty()) { "" } else { "?" + parameters.joinToString("&") { (name, value) -> "${encode(name)}=${encode(value)}" } } - return "$SCHEME://$windowId/${place.linkPath()}$query" + return "$SCHEME://${encodeSegment(heapDumpName)}/${place.linkPath()}$query" } companion object { @@ -52,10 +93,15 @@ data class DeepLink( * A window id: eight lowercase characters, from an alphabet with no `l`, `1`, `o` or `0` in it so that a * link read off a screen and typed back in is the link that was read. * - * Random rather than counted up, which is not a detail. Ids handed out in order would repeat across - * runs, so a link copied yesterday would open *something* today — the second window of this run rather - * than the window it was made from — and be wrong without saying so. A random id is either the window it - * names or no window at all, and the second of those is an error message. + * Random rather than counted up, which is not a detail even now that a link works without one. Ids + * handed out in order repeat across runs, and they repeat *within* one as windows close and open, so a + * link copied yesterday would be honoured today against the second window of whatever is running — + * silently the wrong reading of the dump, which is worse than being ignored. A random id is either the + * window it was made from or no window at all, and the second of those falls back to the heap dump. + * + * A file name and a number would not fix that. The number would have to be handed out across runs that + * cannot see each other's windows, and it would be reused the moment a window closed, so it would be + * exactly the id that opens *something*. */ fun newWindowId(random: Random = Random.Default): String = (1..WINDOW_ID_LENGTH).map { ID_ALPHABET[random.nextInt(ID_ALPHABET.length)] }.joinToString("") @@ -75,10 +121,16 @@ data class DeepLink( val query = afterScheme.substringAfter('?', "") val segments = afterScheme.substringBefore('?').split('/').filter { it.isNotEmpty() } require(segments.size == 2) { - "A link is a window and a place, \"$PREFIX/\", and \"$uri\" names " + + "A link is a heap dump and a place, \"$PREFIX/\", and \"$uri\" names " + "${segments.size} of the two. ${usage()}" } - return DeepLink(windowId = segments[0], place = placeOf(segments[1], parseQuery(query), uri)) + val parameters = parseQuery(query) + return DeepLink( + heapDumpName = decode(segments[0]), + place = placeOf(segments[1], parameters, uri), + heapDumpPath = parameters.firstOrNull(DUMP_PARAMETER)?.let { File(it) }, + windowId = parameters.firstOrNull(WINDOW_PARAMETER) + ) } private fun placeOf( @@ -185,6 +237,15 @@ data class DeepLink( private fun encode(value: String): String = URLEncoder.encode(value, CHARSET) + /** + * A file name as it goes in front of the first `/`, where a `+` is a `+` rather than a space. + * + * [URLEncoder] writes a form field, which is the query and not this: a heap dump called `my dump.hprof` + * would come out as `my+dump.hprof`, and every reader of a URL outside this file — the OS handing one + * over, a terminal, a browser — reads that as a plus sign. + */ + private fun encodeSegment(value: String): String = encode(value).replace("+", "%20") + private fun decode(value: String): String = URLDecoder.decode(value, CHARSET) /** Spelled as a name because the [java.nio.charset.Charset] overloads are Java 10, and this is Java 8. */ @@ -215,6 +276,16 @@ data class DeepLink( AGENT_LOG_PATH ) + /** + * Where the heap dump is, and which window it was read in: the two parameters that are about the link + * rather than about the place. + * + * Which is why no [Place] may spell a parameter either of these names — they are read off the same query + * — and none does. `DeepLinkTest` holds them apart. + */ + internal const val DUMP_PARAMETER = "dump" + internal const val WINDOW_PARAMETER = "window" + internal const val ID_PARAMETER = "id" internal const val PARENT_PARAMETER = "parent" internal const val COUNT_PARAMETER = "count" diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpFiles.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpFiles.kt index 561bf90822..c037a99a13 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpFiles.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpFiles.kt @@ -33,15 +33,15 @@ internal fun heapDumpFileKey(heapDumpFile: File): String { } /** - * One spelling of a heap dump's path, so that two ways of naming one file are one set of notes and one set - * of statuses. + * One spelling of a heap dump's path, so that two ways of naming one file are one set of notes, one set of + * statuses, and one dump for a [DeepLink] to be about. * * Absolute, since what is written about a dump outlives the working directory the app was started in, and * with the `.` and `..` steps taken out, since `./heap.hprof` and `heap.hprof` are what the same dump gets * called on a command line. Not the canonical path: that resolves symlinks, which means asking the * filesystem and getting a different answer once the dump has been deleted. */ -private fun normalizedHeapDumpPath(heapDumpFile: File): File = heapDumpFile.absoluteFile.normalize() +internal fun normalizedHeapDumpPath(heapDumpFile: File): File = heapDumpFile.absoluteFile.normalize() /** * Puts [text] in [file], through a file of its own and a rename, so that a run killed halfway through a diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt index 7c0cbcdb6e..d0d3a9686b 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt @@ -1,5 +1,6 @@ package shark.explorer +import java.io.File import kotlin.random.Random import org.assertj.core.api.Assertions.assertThat import org.assertj.core.api.Assertions.assertThatThrownBy @@ -8,16 +9,16 @@ import org.junit.Test class DeepLinkTest { @Test - fun `link to an object names the window and the object`() { - val link = DeepLink("abcd2345", Place.Object(0x12ab34cd)) + fun `link to an object names the heap dump and the object`() { + val link = DeepLink("leak.hprof", Place.Object(0x12ab34cd)) - assertThat(link.toUri()).isEqualTo("shark://abcd2345/object?id=0x12ab34cd") + assertThat(link.toUri()).isEqualTo("shark://leak.hprof/object?id=0x12ab34cd") } @Test fun `link to an object is read back as the object`() { - assertThat(DeepLink.parse("shark://abcd2345/object?id=0x12ab34cd")) - .isEqualTo(DeepLink("abcd2345", Place.Object(0x12ab34cd))) + assertThat(DeepLink.parse("shark://leak.hprof/object?id=0x12ab34cd")) + .isEqualTo(DeepLink("leak.hprof", Place.Object(0x12ab34cd))) } /** @@ -39,7 +40,7 @@ class DeepLinkTest { ) nodeIds.forEach { nodeId -> - val link = DeepLink("abcd2345", Place.Object(nodeId)) + val link = DeepLink("leak.hprof", Place.Object(nodeId)) assertThat(DeepLink.parse(link.toUri()).place) .describedAs(link.toUri()) .isEqualTo(Place.Object(nodeId)) @@ -52,9 +53,9 @@ class DeepLinkTest { */ @Test fun `a negative object id is not written the way a label writes it`() { - val link = DeepLink("abcd2345", Place.Object(-2112345088L)).toUri() + val link = DeepLink("leak.hprof", Place.Object(-2112345088L)).toUri() - assertThat(link).isEqualTo("shark://abcd2345/object?id=0xffffffff82182c00") + assertThat(link).isEqualTo("shark://leak.hprof/object?id=0xffffffff82182c00") // Which is 0x82182c00 on a label, and that is a different node of the tree. assertThat(link).doesNotContain(hexObjectId(-2112345088L)) } @@ -63,14 +64,14 @@ class DeepLinkTest { fun `link to a pile of smaller objects carries what the pile is`() { val place = Place.SmallerObjects(parentObjectId = 0x40, nodeCount = 42, byteCount = 9876) - assertThat(DeepLink("abcd2345", place).toUri()) - .isEqualTo("shark://abcd2345/smaller-objects?parent=0x40&count=42&bytes=9876") - assertThat(DeepLink.parse(DeepLink("abcd2345", place).toUri()).place).isEqualTo(place) + assertThat(DeepLink("leak.hprof", place).toUri()) + .isEqualTo("shark://leak.hprof/smaller-objects?parent=0x40&count=42&bytes=9876") + assertThat(DeepLink.parse(DeepLink("leak.hprof", place).toUri()).place).isEqualTo(place) } @Test fun `link to an unfiltered object list is the list and nothing else`() { - assertThat(DeepLink("abcd2345", Place.Objects()).toUri()).isEqualTo("shark://abcd2345/objects") + assertThat(DeepLink("leak.hprof", Place.Objects()).toUri()).isEqualTo("shark://leak.hprof/objects") } @Test @@ -83,10 +84,10 @@ class DeepLinkTest { ) ) - val uri = DeepLink("abcd2345", place).toUri() + val uri = DeepLink("leak.hprof", place).toUri() assertThat(uri).isEqualTo( - "shark://abcd2345/objects?query=android.graphics.Bitmap&exact=true&kinds=CLASS%2CINSTANCE" + "shark://leak.hprof/objects?query=android.graphics.Bitmap&exact=true&kinds=CLASS%2CINSTANCE" ) assertThat(DeepLink.parse(uri).place).isEqualTo(place) } @@ -96,9 +97,9 @@ class DeepLinkTest { fun `an object list filtered to no kind at all is not an unfiltered one`() { val place = Place.Objects(ObjectListFilter(kinds = emptySet())) - val uri = DeepLink("abcd2345", place).toUri() + val uri = DeepLink("leak.hprof", place).toUri() - assertThat(uri).isEqualTo("shark://abcd2345/objects?kinds=") + assertThat(uri).isEqualTo("shark://leak.hprof/objects?kinds=") assertThat(DeepLink.parse(uri).place).isEqualTo(place) } @@ -108,30 +109,30 @@ class DeepLinkTest { val query = "com.example a&b=c?d/e#f+g%h" val place = Place.Objects(ObjectListFilter(query = query)) - assertThat(DeepLink.parse(DeepLink("abcd2345", place).toUri()).place).isEqualTo(place) + assertThat(DeepLink.parse(DeepLink("leak.hprof", place).toUri()).place).isEqualTo(place) } @Test fun `leaks arrive with the same ones unfolded`() { val place = Place.Leaks(expandedGroups = setOf("APPLICATION 12ab", "LIBRARY 34cd")) - val uri = DeepLink("abcd2345", place).toUri() + val uri = DeepLink("leak.hprof", place).toUri() assertThat(uri) - .isEqualTo("shark://abcd2345/leaks?expanded=APPLICATION+12ab&expanded=LIBRARY+34cd") + .isEqualTo("shark://leak.hprof/leaks?expanded=APPLICATION+12ab&expanded=LIBRARY+34cd") assertThat(DeepLink.parse(uri).place).isEqualTo(place) } @Test fun `leaks with nothing unfolded is the page and nothing else`() { - assertThat(DeepLink("abcd2345", Place.Leaks()).toUri()).isEqualTo("shark://abcd2345/leaks") - assertThat(DeepLink.parse("shark://abcd2345/leaks").place).isEqualTo(Place.Leaks()) + assertThat(DeepLink("leak.hprof", Place.Leaks()).toUri()).isEqualTo("shark://leak.hprof/leaks") + assertThat(DeepLink.parse("shark://leak.hprof/leaks").place).isEqualTo(Place.Leaks()) } @Test fun `starred is a place with nothing to say about it`() { - assertThat(DeepLink("abcd2345", Place.Starred).toUri()).isEqualTo("shark://abcd2345/starred") - assertThat(DeepLink.parse("shark://abcd2345/starred").place).isEqualTo(Place.Starred) + assertThat(DeepLink("leak.hprof", Place.Starred).toUri()).isEqualTo("shark://leak.hprof/starred") + assertThat(DeepLink.parse("shark://leak.hprof/starred").place).isEqualTo(Place.Starred) } /** @@ -142,14 +143,14 @@ class DeepLinkTest { fun `one agent's session is named by the session`() { val place = Place.AgentLog("1a2b3c4d") - assertThat(DeepLink("abcd2345", place).toUri()) - .isEqualTo("shark://abcd2345/agent-log?session=1a2b3c4d") - assertThat(DeepLink.parse("shark://abcd2345/agent-log?session=1a2b3c4d").place).isEqualTo(place) + assertThat(DeepLink("leak.hprof", place).toUri()) + .isEqualTo("shark://leak.hprof/agent-log?session=1a2b3c4d") + assertThat(DeepLink.parse("shark://leak.hprof/agent-log?session=1a2b3c4d").place).isEqualTo(place) } @Test fun `an agent log link with no session says what is missing`() { - assertThatThrownBy { DeepLink.parse("shark://abcd2345/agent-log") } + assertThatThrownBy { DeepLink.parse("shark://leak.hprof/agent-log") } .isInstanceOf(IllegalArgumentException::class.java) .hasMessageContaining("needs a \"session\"") } @@ -174,7 +175,7 @@ class DeepLinkTest { ) places.forEach { place -> - val uri = DeepLink("abcd2345", place).toUri() + val uri = DeepLink("leak.hprof", place).toUri() assertThat(DeepLink.parse(uri).place).describedAs(uri).isEqualTo(place) } } @@ -185,8 +186,8 @@ class DeepLinkTest { val place = Place.Leaks(expandedGroups = setOf("LIBRARY 34cd", "APPLICATION 12ab")) val sameOtherWayRound = Place.Leaks(expandedGroups = setOf("APPLICATION 12ab", "LIBRARY 34cd")) - assertThat(DeepLink("abcd2345", place).toUri()) - .isEqualTo(DeepLink("abcd2345", sameOtherWayRound).toUri()) + assertThat(DeepLink("leak.hprof", place).toUri()) + .isEqualTo(DeepLink("leak.hprof", sameOtherWayRound).toUri()) } @Test @@ -198,35 +199,35 @@ class DeepLinkTest { @Test fun `a link naming no place says which places there are`() { - assertThatThrownBy { DeepLink.parse("shark://abcd2345") } + assertThatThrownBy { DeepLink.parse("shark://leak.hprof") } .isInstanceOf(IllegalArgumentException::class.java) .hasMessageContaining("object, smaller-objects, objects, leaks, starred") } @Test fun `a link to a place this app has no screen for says so`() { - assertThatThrownBy { DeepLink.parse("shark://abcd2345/dominators") } + assertThatThrownBy { DeepLink.parse("shark://leak.hprof/dominators") } .isInstanceOf(IllegalArgumentException::class.java) .hasMessageContaining("\"dominators\" is no place") } @Test fun `an object link with no object says what is missing`() { - assertThatThrownBy { DeepLink.parse("shark://abcd2345/object") } + assertThatThrownBy { DeepLink.parse("shark://leak.hprof/object") } .isInstanceOf(IllegalArgumentException::class.java) .hasMessageContaining("needs a \"id\"") } @Test fun `an object id that is not one says so`() { - assertThatThrownBy { DeepLink.parse("shark://abcd2345/object?id=the+big+one") } + assertThatThrownBy { DeepLink.parse("shark://leak.hprof/object?id=the+big+one") } .isInstanceOf(IllegalArgumentException::class.java) .hasMessageContaining("which is no object id") } @Test fun `an object kind this app has never heard of says which kinds there are`() { - assertThatThrownBy { DeepLink.parse("shark://abcd2345/objects?kinds=WIDGETS") } + assertThatThrownBy { DeepLink.parse("shark://leak.hprof/objects?kinds=WIDGETS") } .isInstanceOf(IllegalArgumentException::class.java) .hasMessageContaining("Kinds are CLASS, INSTANCE, OBJECT_ARRAY, PRIMITIVE_ARRAY") } @@ -246,9 +247,83 @@ class DeepLinkTest { assertThat(ids.toSet()).hasSize(ids.size) } + /** + * The link the app itself writes, and the whole of what it is for: a heap dump, a place in it, where that + * dump is, and which window it was copied from. + */ + @Test + fun `a link from a window carries the dump, the file and the window`() { + val link = DeepLink(File("/dumps/leak.hprof"), Place.Leaks(), windowId = "abcd2345") + + assertThat(link.toUri()) + .isEqualTo("shark://leak.hprof/leaks?dump=%2Fdumps%2Fleak.hprof&window=abcd2345") + assertThat(DeepLink.parse(link.toUri())).isEqualTo(link) + } + + /** Because a link is read out of a heap dump's notes months later, from a machine with another home. */ + @Test + fun `the path in a link is absolute and has no dots in it`() { + val link = DeepLink(File("dumps/./over/../leak.hprof"), Place.Starred) + + assertThat(link.heapDumpPath).isEqualTo(File(File("").absoluteFile, "dumps/leak.hprof")) + } + + /** + * Which is a link somebody typed, or shortened by hand to the two things worth reading. It resolves + * against whatever is open, so it is worth being able to write. See `ExplorerWindows.windowFor`. + */ + @Test + fun `a link is a heap dump and a place and needs nothing else`() { + val link = DeepLink.parse("shark://leak.hprof/leaks") + + assertThat(link.heapDumpName).isEqualTo("leak.hprof") + assertThat(link.place).isEqualTo(Place.Leaks()) + assertThat(link.heapDumpPath).isNull() + assertThat(link.windowId).isNull() + } + + /** + * `+` is a space to [java.net.URLEncoder], which writes a form field rather than the part of a URL in + * front of the first `/` — so a dump with a space in its name has to be written the other way, or every + * reader of the link outside this class reads a plus sign. + */ + @Test + fun `a heap dump whose name needs escaping survives the trip`() { + val link = DeepLink(File("/dumps/my dump (2).hprof"), Place.Starred) + + assertThat(link.toUri()).startsWith("shark://my%20dump%20%282%29.hprof/starred") + assertThat(DeepLink.parse(link.toUri())).isEqualTo(link) + } + + /** + * The two parameters that are about the link rather than about the place share the query with the ones that + * are, so a [Place] spelling a parameter `dump` or `window` would quietly take one of them over. This is + * what would fail. + */ + @Test + fun `no place takes the dump or the window off a link`() { + val places = listOf( + Place.wholeHeapDump(), + Place.Object(0x12ab34cd), + Place.SmallerObjects(parentObjectId = 0x40, nodeCount = 42, byteCount = 9876), + Place.Objects(), + Place.Objects(ObjectListFilter(query = "Bitmap", isExactMatch = true, kinds = emptySet())), + Place.Leaks(), + Place.Leaks(expandedGroups = setOf("APPLICATION 12ab", "LIBRARY 34cd")), + Place.Starred, + Place.AgentLogs, + Place.AgentLog("1a2b3c4d") + ) + + places.forEach { place -> + val link = DeepLink(File("/dumps/leak.hprof"), place, windowId = "abcd2345") + assertThat(DeepLink.parse(link.toUri())).describedAs(link.toUri()).isEqualTo(link) + } + } + @Test fun `a link tells itself apart from a heap dump path`() { - assertThat(DeepLink.looksLikeOne("shark://abcd2345/starred")).isTrue() + assertThat(DeepLink.looksLikeOne("shark://leak.hprof/starred")).isTrue() assertThat(DeepLink.looksLikeOne("/Users/me/dumps/shark.hprof")).isFalse() assertThat(DeepLink.looksLikeOne("--title=Reading a leak")).isFalse() } From 2370e15ae95ab10eb20a6bab7d220226fc8b0676 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Thu, 27 Aug 2026 16:50:36 +0200 Subject: [PATCH 25/27] Look up where a heap dump is rather than say it in the link MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A copied link was 220 characters, four fifths of them a percent encoded path that says nothing to whoever reads it: shark://leak.hprof/object?id=0x12d368b8&dump=%2FUsers%2F…&window=zvphq4r3 The path was in there because a link has to outlive the run it was copied from, and nothing else remembered where the file was. So something else does: HeapDumpPaths writes down the path of every heap dump that opens, under the id of the window it opened in, the newest 200 kept, and following a link is a lookup. The same link is now shark://leak.hprof/object?id=0x12d368b8&window=zvphq4r3 What it costs is that a link works for as long as this machine remembers the file rather than for as long as the file exists — and a link that has been forgotten says so, and can still be given &dump= by hand, which is also the answer for a dump this machine has never opened. Recorded under the window id rather than the dump, because that answers both questions with one file per open: a window id resolves to the dump it was showing, and a file name to the newest record with that name. So `shark:///` — the shortest a link can be, and what this app used to write — opens the right heap dump again after every window of it has gone, instead of finding nothing. A run with no window records its dumps too, which is what makes the link `show` hands back from a --no-ui run resolve for the next reader. --- docs/shark-explorer-changelog.md | 9 +- docs/shark-explorer.md | 28 +-- shark/shark-explorer/AGENTS.md | 7 + shark/shark-explorer/notes/decisions.md | 37 ++-- .../shark/explorer/agent/AgentToolsTest.kt | 5 +- .../shark/explorer/agent/McpSessionTest.kt | 5 +- .../java/shark/explorer/app/DeepLinkPeers.kt | 21 ++- .../explorer/app/ExplorerHeapDumpPaths.kt | 19 +++ .../java/shark/explorer/app/ExplorerWindow.kt | 28 +-- .../explorer/app/HeadlessAgentHeapDumps.kt | 15 +- .../src/main/java/shark/explorer/app/Main.kt | 10 +- .../shark/explorer/app/ExplorerWindowTest.kt | 30 +++- .../app/HeadlessAgentHeapDumpsTest.kt | 24 ++- .../src/main/java/shark/explorer/DeepLink.kt | 36 ++-- .../main/java/shark/explorer/HeapDumpFiles.kt | 10 +- .../main/java/shark/explorer/HeapDumpPaths.kt | 139 +++++++++++++++ .../test/java/shark/explorer/DeepLinkTest.kt | 30 ++-- .../java/shark/explorer/HeapDumpPathsTest.kt | 161 ++++++++++++++++++ 18 files changed, 522 insertions(+), 92 deletions(-) create mode 100644 shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerHeapDumpPaths.kt create mode 100644 shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpPaths.kt create mode 100644 shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapDumpPathsTest.kt diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index fd6f989b43..f65f210cfd 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -22,9 +22,12 @@ uses, without the one for a newly recognized library leak: * ✨ Right click anything the window can take you to — a tab, a rectangle, a row, a field — and copy a `shark://` link to it, beside opening it in a new tab. Clicking one brings the app to the front and opens that place in a new tab: an object, a filtered object list, the leaks with the same groups - unfolded. A link names the **heap dump**, so it goes on working after the window it was copied from has - gone: it opens the place in a window that has that dump, and opens the file in a new window when none - has. See [Link to a tab](shark-explorer.md#link-to-a-tab). + unfolded. A link names the **heap dump** — `shark://bug-4821.hprof/leaks?window=vugs93jp`, short enough to + read in a sentence — so it goes on working after the window it was copied from has gone: it opens the place + in a window that has that dump, and opens the file in a new window when none has. Where that file is never + goes in the link: every heap dump opened is written down in `~/.shark-explorer/heap-dump-paths`, the last + 200 kept, and following a link looks it up there. See + [Link to a tab](shark-explorer.md#link-to-a-tab). * ✨ **Notes**: every location takes a markdown note, kept between runs, and the tab strip marks the tabs whose location has one. A note belongs to the location rather than to the tab, so two tabs on one location are one note. Class names, addresses and `shark://` links written in a note become links back diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 54b12e491c..10976fdaaf 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -100,18 +100,22 @@ rather than to the window showing it. So a link goes on working: following one o that has the dump open, and opens the file in a new window when none has — the run it was copied from can be long gone. A link never replaces what you were reading: it always opens a tab of its own. -A copied link carries two more things after the place, and this is one in full: +A copied link carries one more thing after the place, and this is one in full: ``` -shark://bug-4821.hprof/leaks?dump=%2FUsers%2Fyou%2Fdumps%2Fbug-4821.hprof&window=vugs93jp +shark://bug-4821.hprof/leaks?window=vugs93jp ``` -`dump` is the file's full path, which is what makes the link exact — two `com.example.hprof` off two -devices are two investigations — and what lets it open the dump again months later. `window` is the window -it was copied from, honoured while that window is open and ignored once it isn't, so that the same dump -open twice, which is two readings of it side by side, lands where you meant. Neither is needed to type one -by hand: `shark://bug-4821.hprof/leaks` finds whichever window has a `bug-4821.hprof` open. What a link -can't do is open a dump it doesn't have the path of and nobody has open — that opens a window saying so. +`window` is the window it was copied from, honoured while that window is open and ignored once it isn't, so +that the same dump open twice, which is two readings of it side by side, lands where you meant. It isn't +needed to type one by hand: `shark://bug-4821.hprof/leaks` is a link. + +**Where the file is doesn't travel in the link.** Every heap dump this app opens is written down in +`~/.shark-explorer/heap-dump-paths`, the last 200 kept, so following a link is a lookup rather than a path +pasted into a URL — which is what keeps a link short enough to read in a sentence. A link about a dump this +machine has no record of opening, which is one from somebody else's machine, opens a window saying so; open +that file and the link works, or add the path to the link yourself as +`&dump=/Users/you/dumps/bug-4821.hprof`. Links reach the app from an installed build — the installer is what tells the OS that `shark://` is this app's. A copy run from source can still be linked to from another one, but the OS won't start it for a @@ -290,7 +294,7 @@ instead of piped to a window: Everything works the same except `show`, which has nowhere to put a tab and says so rather than answering that it showed you something. It still hands back the `shark://` link, which names the heap dump: nobody saw the -place, and the link opens it for whoever reads the answer. Nothing else changes, because **notes and verdicts +place, and the link opens it for the next reader on the machine the dump is on. Nothing else changes, because **notes and verdicts were never on the screen** — they are files beside the heap dump, so a dump investigated over ssh today opens in a window tomorrow with the verdicts, the reasons and the conclusion already on it. @@ -476,11 +480,11 @@ and the method tells an agent to put those links in its reply — so a sentence request comment or a bug report ends up carrying a way in: > The leak is `MainActivity$2.this$0`, a non-static inner class holding the activity it was declared in: -> shark://leak_asynctask_o.hprof/object?id=0x12d368b8&dump=%2FUsers%2Fyou%2Fdumps%2Fleak_asynctask_o.hprof&window=zvphq4r3 +> shark://leak_asynctask_o.hprof/object?id=0x12d368b8&window=zvphq4r3 Clicking it opens that object with the reasoning on its tabs — in the window it was written from while that -window is up, and by opening the heap dump again once it isn't. So an answer worth keeping keeps working: it -carries the path of the dump it is about, and the only thing that stops it is deleting the file. +window is up, and by opening the heap dump again once it isn't. So an answer worth keeping keeps working, and +it is short enough to read: it names the heap dump, and where that file is, is looked up. An agent's verdicts are verdicts like any other: they say `set by hand` on every chain that runs through the object, the reason is the one it gave, and the pencil takes one off if you disagree with it. Which is the diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index 4d64d76198..a7a82bc00e 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -245,6 +245,13 @@ line works after the run that opened the dump has ended, which is the case worth half of it, the `window=` a copied link carries, `grep "Windows of this run"` in the newest file under `~/.shark-explorer/logs` for the ids of the run. +**A link says no path, so a dump this machine has never opened has nowhere to go.** Where each dump was is +written down under `~/.shark-explorer/heap-dump-paths`, one file per window that opened one, and the +`HeapDumpPaths` lookup is what turns a file name back into a file. So a link to a dump opened only in a run +started before this feature existed, or one whose record has been evicted, opens a window saying so — `ls` +that directory before concluding the routing is broken, and `&dump=` is the way to try a link about a +dump nothing here has opened. + **Read the result in the log rather than off the screen.** Following a link raises the app over whatever the person at the machine was doing, so a screenshot to check it worked costs them their window and shows you theirs. `The OS handed this run`, `A link asked window for of `, `A link asked for diff --git a/shark/shark-explorer/notes/decisions.md b/shark/shark-explorer/notes/decisions.md index 3c7592b9f8..5bcc06aad6 100644 --- a/shark/shark-explorer/notes/decisions.md +++ b/shark/shark-explorer/notes/decisions.md @@ -849,29 +849,40 @@ headings, and the listing is the index. ## A link names the heap dump, and a window only as a refinement -`shark:///?[&dump=][&window=]`. The first version of this -named the window — `shark:///` — and it was wrong for the reason a link exists: every place -there is belongs to the heap dump, not to whatever is showing it, so a link that named a window died with the -window. Which is most links a day later, and most links in an agent's session log, since a session outlives -the run that wrote it. A link that mostly doesn't work is a link nobody sends. +`shark:///?[&window=]`. The first version of this named the window +— `shark:///` — and it was wrong for the reason a link exists: every place there is belongs +to the heap dump, not to whatever is showing it, so a link that named a window died with the window. Which is +most links a day later, and most links in an agent's session log, since a session outlives the run that wrote +it. A link that mostly doesn't work is a link nobody sends. So the dump is the identity and the window is honoured while it exists and **ignored once it doesn't**, rather than turning the link into an error. Being right about which window is worth a lot while the window is there and nothing at all afterwards. - **The authority is the file name**, because it is the part a person reads and types, and it is in every - answer an agent has already been given. `dump=` carries the normalized absolute path beside it, since two - dumps called `com.squareup.hprof` off two devices are two investigations — and because a path is the only - thing that can open a dump nothing has open. A link with a name and no path is one somebody typed, and it - resolves against what is open. -- **Not `heapDumpFileKey`**, the `-` the notes and statuses are filed under. It is - one-way and nothing on disk maps a key back to a path, so a key-only link could never reopen a dump. + answer an agent has already been given. +- **Where the file is doesn't travel in the link.** The version between these two carried + `dump=%2FUsers%2F…`, which was four fifths of the characters of a link and the fifth nobody could read. So + `HeapDumpPaths` writes down the path of every dump that opens, under the id of the window it opened in, the + newest 200 kept, and following a link is a lookup. What that costs is honest and small: a link works for as + long as this machine remembers the file rather than for as long as the file exists, and a link that has been + forgotten says so and can be given `&dump=` by hand. +- **Recorded under the window id, not the dump.** One record per open is a single whole-file write by one run + with nothing to merge, and it answers both questions at once: a window id resolves to the dump it was + showing, and a file name resolves to the newest record with that name. Two dumps called `com.squareup.hprof` + off two devices are two investigations, and the `window=` on a copied link is what tells them apart. +- **Not `heapDumpFileKey`**, the `-` the notes and statuses are filed under: it is + one-way, so a key on its own can name a dump but never find one. Which was also the objection to a window id + as the authority, and `HeapDumpPaths` answers it — a link that is nothing but an id resolves now. It still + isn't what the app writes, because an id says nothing to whoever reads the link, is not what a window's + title shows, and is not the same for the same place twice. - **Window ids stay random.** A counted id repeats across runs *and* within one as windows close and open, so it would be honoured against the wrong reading of the dump — silently, which is worse than being ignored. A file name plus a number fixes neither half: the number would have to be handed out across runs that cannot see each other's windows. -- **Resolution order is windowId, then path, then file name, then the authority as a window id** — that last - step for the `shark:///` links already sitting in notes and in session files on disk. +- **Resolution order is windowId, then path, then file name, then the authority as a window id**, in the + windows of the run and again in the records on disk. That last step is what keeps a `shark:///…` + link working — the ones already sitting in notes and session files, and the shortest link anyone can write. - **A run claims a link only for a window it already has**, never for a file it could open, or every run of the app would claim every link. Whoever is left holding it opens the dump. `DeepLinkPeers`. - **The agent surface converged on the same choice**: the tool argument is `heapDump`, taking a file name, and diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index ffb17fa425..bd5cfccd00 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -850,7 +850,8 @@ class AgentToolsTest { * * What matters about it here is what it opens — this object, of this heap dump, in the window the call was * made against — and how it is spelled is `DeepLinkTest`'s. It names the heap dump rather than only the - * window so that an agent can put it in an answer somebody reads after this run has ended. + * window so that an agent can put it in an answer somebody reads after this run has ended, and it names it + * by file name alone: where that file is, is looked up by whoever follows the link. */ private fun assertThatLinkOpens( answer: JsonObject, @@ -858,7 +859,7 @@ class AgentToolsTest { ) { val link = DeepLink.parse(answer.text("link")) assertThat(link.heapDumpName).isEqualTo(window.heapDumpName) - assertThat(link.heapDumpPath?.path).isEqualTo(window.heapDumpPath) + assertThat(link.heapDumpPath).isNull() assertThat(link.windowId).isEqualTo(window.windowId) assertThat(link.place).isEqualTo(Place.Object(objectId)) } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index c8b7e755c2..9c77329620 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -214,12 +214,13 @@ class McpSessionTest { assertThat(call.refusal).isNull() // Which is what makes the row clickable: the place, in the heap dump the call was made against — named // by the dump so that the link still opens it once this run has ended, with the window it was made in as - // a refinement, honoured while that window is open. + // a refinement, honoured while that window is open. The session line records the dump's path as well, + // which the link doesn't have to: a row leads to a file, and a link is looked up. assertThat(call.place).isEqualTo(Place.Object(heapDump.holderObjectId)) assertThat(call.heapDumpPath).isEqualTo(window.heapDumpPath) val link = DeepLink.parse(call.link()!!) assertThat(link.heapDumpName).isEqualTo(window.heapDumpName) - assertThat(link.heapDumpPath?.path).isEqualTo(window.heapDumpPath) + assertThat(link.heapDumpPath).isNull() assertThat(link.windowId).isEqualTo(window.windowId) assertThat(link.place).isEqualTo(Place.Object(heapDump.holderObjectId)) } diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt index 0f2add1022..e534459a9e 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt @@ -86,7 +86,9 @@ internal object DeepLinkPeers { * installed app. * * Asking the others is a connection each, so it happens off the caller's thread — a link arrives on the - * event thread there, and a run that has been killed is only found out about by waiting for it. + * event thread there, and a run that has been killed is only found out about by waiting for it. A window of + * this run is answered before any of that, on the thread the link came in on: it takes no disk and no + * socket to see that a link is about a heap dump on screen here. */ fun follow( link: DeepLink, @@ -97,10 +99,9 @@ internal object DeepLinkPeers { return } Thread({ - // Nobody else's, so this run answers for it, which is opening that heap dump here. - if (deliver(listOf(link)).isNotEmpty()) { - windows.open(link) - } + // Nobody else's, so this run answers for it, which is opening that heap dump here — at the path + // [deliver] looked up, since a link says the dump's name and not where it is. + deliver(listOf(link)).forEach { windows.open(it) } }, THREAD_NAME).apply { isDaemon = true start() @@ -113,13 +114,21 @@ internal object DeepLinkPeers { * * The leftovers are the caller's to answer for, which is what makes a link whose window has gone a window * of that heap dump here rather than a process that started and exited without a word. + * + * **Where the heap dump is gets looked up first**, and it is looked up once for every run: a link carries a + * file name, and a run asked about a link it has no window for should be answering about the file rather + * than about the name — two dumps called `com.squareup.hprof` off two devices are two investigations. So + * what goes out on the socket is the link with the path filled in, and what comes back to the caller is the + * same, ready to open. See [HeapDumpPaths.resolve]. */ fun deliver(links: List): List { if (links.isEmpty()) { return emptyList() } + val heapDumpPaths = explorerHeapDumpPaths() + val resolved = links.map { heapDumpPaths.resolve(it) } val peers = peers() - return links.filter { link -> peers.none { peer -> peer.deliver(link) } } + return resolved.filter { link -> peers.none { peer -> peer.deliver(link) } } } /** Every other run that has published itself, stale files cleared out on the way past. */ diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerHeapDumpPaths.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerHeapDumpPaths.kt new file mode 100644 index 0000000000..b8844f05d1 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerHeapDumpPaths.kt @@ -0,0 +1,19 @@ +package shark.explorer.app + +import java.io.File +import shark.explorer.HeapDumpPaths + +/** + * Where every run of this app writes down the heap dumps it opens, so that a link can find one afterwards. + * + * One directory for the whole machine rather than one per run, which is the point of it: the run that wrote a + * link is usually not the run that follows it. Beside the notes and the verdicts, which are the other things + * kept about a heap dump between runs. + * + * A function rather than an object held somewhere, because this keeps nothing in memory — it is a directory + * and two ways of reading it — and the three callers are as far apart as a window, a run with no window, and + * a socket answering another run. + */ +internal fun explorerHeapDumpPaths(): HeapDumpPaths = HeapDumpPaths(HEAP_DUMP_PATHS_DIRECTORY) + +private val HEAP_DUMP_PATHS_DIRECTORY = File(SHARK_EXPLORER_DIRECTORY, "heap-dump-paths") diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt index 81e4c94df6..24dcfdab47 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt @@ -164,13 +164,14 @@ internal class ExplorerWindows( * * - **The window the link was made from**, while it is still open. Two windows on one dump are two * readings of it, and this is the only thing that tells them apart. - * - **A window of that heap dump**, by path. The window is gone or belonged to another run, and the dump - * is what the link was really about. - * - **A window of a dump with that file name**, for a link somebody typed, which carries no path. - * - **A window with that id**, for links written before a link named a heap dump: `shark:///…` is - * in notes on disk and in agent sessions, and one of those pasted back into the run it came from still - * goes where it went. Last, so that a heap dump called like a window id — which is a file called - * `abcd2345` — is read as the heap dump. + * - **A window of that heap dump**, by path, when the link says where the file is — which is one handed + * over by another run, since that run looked it up. Exact, so nothing is tried after it: two dumps of one + * name off two devices are two investigations. + * - **A window of a dump with that file name**, which is what a link says about the dump and all it says. + * - **A window with that id**, for a link whose whole authority is a window: `shark:///…` is in + * notes on disk and in agent sessions, and one of those pasted back into the run it came from still goes + * where it went. Last, so that a heap dump called like a window id — which is a file called `abcd2345` — + * is read as the heap dump. */ fun windowFor(link: DeepLink): ExplorerWindow? { val ofWindowId = link.windowId?.let { id -> firstOrNull { it.deepLinkId == id } } @@ -214,7 +215,8 @@ internal class ExplorerWindows( ) return } - SharkLog.d { "A link asked for ${link.place} of ${link.heapDumpName}, which is not open yet" } + // The file rather than what the link called it, which for a link named by a window id is that id. + SharkLog.d { "A link asked for ${link.place} of ${heapDumpFile.name}, which is not open yet" } goToHeapDump(heapDumpFile, link.place) } @@ -226,14 +228,16 @@ internal class ExplorerWindows( * What an empty window opened by a link with nowhere to go says in the middle of it. * * Two ways to get here, and they need different things done about them: a heap dump that has been moved - * or deleted since the link was made, and a link that never said where the dump was — which is one - * somebody typed or shortened, since every link this app writes carries the path. + * or deleted since the link was made, and one this machine has no record of ever opening — which is a + * link from somebody else's machine, or one about a dump opened so long ago that where it was has been + * forgotten. Both are answered by opening the file, so both messages end by saying so. */ fun noSuchHeapDump(link: DeepLink): String { val path = link.heapDumpPath return if (path == null) { - "No heap dump called ${link.heapDumpName} is open, and this link doesn't say where that file is, " + - "so there is nothing to open. A link copied from a window carries the path." + "No heap dump called ${link.heapDumpName} is open here, and this machine has no record of opening " + + "one by that name. Open that file and follow the link again, or say where it is in the link: " + + "&${DeepLink.DUMP_PARAMETER}=/path/to/${link.heapDumpName}" } else { "${link.heapDumpName} is not open and there is no file at $path to open, so this link has nowhere " + "to go. A link outlives the window it was copied from, but not the heap dump it is about." diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt index bc2866f729..3c4cc9b159 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt @@ -11,6 +11,7 @@ import kotlinx.coroutines.cancel import shark.SharkLog import shark.explorer.DeepLink import shark.explorer.DeviceHeapDumps +import shark.explorer.HeapDumpPaths import shark.explorer.agent.AgentHeapDump import shark.explorer.agent.AgentRefusal import shark.explorer.agent.ShownPlace @@ -33,7 +34,9 @@ internal class HeadlessAgentHeapDumps( heapDumpFiles: List = emptyList(), /** The same notes a window keeps, in the same directory: a test passes its own. See [ExplorerNotes]. */ private val notes: ExplorerNotes = ExplorerNotes(), - private val leakStatuses: ExplorerLeakStatuses = ExplorerLeakStatuses() + private val leakStatuses: ExplorerLeakStatuses = ExplorerLeakStatuses(), + /** And the same record of where a heap dump was, which is what makes the links below resolve. */ + private val heapDumpPaths: HeapDumpPaths = explorerHeapDumpPaths() ) : RunAgentHeapDumps(deviceHeapDumps), Closeable { /** @@ -118,13 +121,17 @@ internal class HeadlessAgentHeapDumps( // agent does with it is name a dump, and a vocabulary that changes with whether there is a screen is one // nobody can carry between the two. val windowId = DeepLink.newWindowId() + // Written down the same way a window's dump is, and here it is the whole of what makes the links this + // hands back work: nobody watching a run with no screen can be told where the file was. + heapDumpPaths.record(windowId, file) val dump = HeadlessHeapDump( open = open, agent = OpenAgentHeapDump(windowId = windowId, open = open) { place -> SharkLog.d { "Nowhere to show $place: this run was started with $NO_UI_OPTION" } - // A link all the same, and it works: a link names the heap dump rather than a window, so this one - // opens the file at that place in whatever Shark Explorer whoever clicks it has. Which is the whole - // of what a run with no screen can offer, and more than nothing. + // A link all the same, and it works: a link names the heap dump rather than a window, and where this + // dump is has just been written down, so this opens the file at that place in whatever Shark Explorer + // reads it on this machine. Which is the whole of what a run with no screen can offer, and more than + // nothing. ShownPlace.onlyAsALink( link = DeepLink(file, place).toUri(), problem = "This Shark Explorer was started with $NO_UI_OPTION, so it has no window and nobody " + diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt index b80169bbce..7ca2a3dd04 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt @@ -164,6 +164,9 @@ private fun explorerApplication( // And one set of statuses set by hand per heap dump, for the same reason: a status is a conclusion about // the dump rather than about a window, so both windows on one dump read the heap through the same ones. val leakStatuses = remember { ExplorerLeakStatuses() } + // And where the heap dumps of every window are written down, so that a link into one of them works after + // this run has gone — which is what lets a link name the dump without carrying its path. + val heapDumpPaths = remember { explorerHeapDumpPaths() } // Once per run, not once per window, and off the UI thread: this is a network request, and a window that // waits for GitHub to answer before it draws is a window that hangs when GitHub is unreachable. LaunchedEffect(updateNotice) { @@ -208,7 +211,12 @@ private fun explorerApplication( leakStatuses = leakStatuses, // What this window has open, for the agent surface: a socket thread has to be able to find it, // and it is a composable's state. See [ExplorerWindow.openHeapDump]. - onHeapDumpOpen = { open -> window.openHeapDump = open }, + onHeapDumpOpen = { open -> + window.openHeapDump = open + // Once it is open rather than as the window is given the file: a dump that turns out not to be + // one is not a heap dump for a link to be sent to. See [HeapDumpPaths]. + open?.let { heapDumpPaths.record(window.deepLinkId, it.session.heapDumpFile) } + }, onHeapDumpProblem = { problem -> window.openProblem = problem }, deepLinkId = window.deepLinkId, // The same way a link arriving from the OS is followed, which is what makes a `shark://` link diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt index 5d811d5d6e..49ddd67e35 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt @@ -228,7 +228,7 @@ class ExplorerWindowTest { val heapDumpFile = temporaryFolder.newFile("third.hprof") val windows = explorerWindows(opening(FIRST_DUMP)) - windows.open(DeepLink(heapDumpFile, Place.Starred, windowId = CLOSED_WINDOW_ID)) + windows.open(lookedUp(heapDumpFile, Place.Starred, windowId = CLOSED_WINDOW_ID)) val opened = windows.last() assertThat(windows).hasSize(2) @@ -239,7 +239,7 @@ class ExplorerWindowTest { @Test fun `a link to a heap dump that has been deleted opens a window saying so`() { val windows = explorerWindows(opening(FIRST_DUMP)) - windows.open(DeepLink(SECOND_DUMP, Place.Starred)) + windows.open(lookedUp(SECOND_DUMP, Place.Starred)) // Rather than nothing at all, which is the one answer that can't be told from the app having failed // to start — and a link is usually followed from somewhere that can't see either way. @@ -251,15 +251,20 @@ class ExplorerWindowTest { assertThat(logged).anyMatch { SECOND_DUMP.name in it } } - /** A link shortened to the file name, which only works while something has that file open. */ - @Test fun `a link with no path to a heap dump nothing has open says what is missing`() { + /** + * A link about a heap dump this machine has no record of ever opening, which is one from somebody else's + * machine: nothing looked its path up, because there was nothing to look up. See [HeapDumpPaths]. + */ + @Test fun `a link to a heap dump nothing knows where to find says what is missing`() { val windows = explorerWindows(opening(FIRST_DUMP)) windows.open(DeepLink.parse("shark://${SECOND_DUMP.name}/starred")) assertThat(windows.last().deepLinkProblem) .contains(SECOND_DUMP.name) - .contains("doesn't say where that file is") + .contains("no record of opening one by that name") + // And what to type instead, since a link from another machine can carry the path. + .contains("&dump=/path/to/${SECOND_DUMP.name}") } @Test fun `a window opened by a link that found nothing lands beside the others`() { @@ -357,6 +362,21 @@ class ExplorerWindowTest { }) ) + /** + * A link as [ExplorerWindows] is handed one: where the heap dump is has been looked up already, by the run + * that took the link off the OS. A link itself says the dump's file name and no more — see [HeapDumpPaths]. + */ + private fun lookedUp( + heapDumpFile: File, + place: Place, + windowId: String? = null + ) = DeepLink( + heapDumpName = heapDumpFile.name, + place = place, + heapDumpPath = heapDumpFile.absoluteFile.normalize(), + windowId = windowId + ) + private fun noHeapDumps(titlePrefix: String? = null) = ExplorerArguments(heapDumpFiles = emptyList(), titlePrefix = titlePrefix) diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt index 813e4f858c..2d7f01b7b8 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt @@ -11,6 +11,7 @@ import shark.explorer.Adb import shark.explorer.AdbOutput import shark.explorer.DeepLink import shark.explorer.DeviceHeapDumps +import shark.explorer.HeapDumpPaths import shark.explorer.LeakStatus import shark.explorer.LeakStatusOverride import shark.explorer.Place @@ -76,7 +77,8 @@ class HeadlessAgentHeapDumpsTest { @Test fun `showing a place says there is no window rather than that it was shown`() { val file = temporaryFolder.leakyHeapDump().file - headless().use { heapDumps -> + val paths = temporaryFolder.newFolder("paths-of-the-shown-place") + headless(paths = paths).use { heapDumps -> val dump = runBlocking { heapDumps.open(file) } val shown = dump.show(Place.Leaks()) @@ -84,14 +86,16 @@ class HeadlessAgentHeapDumpsTest { assertThat(shown.problem) .contains(NO_UI_OPTION) .contains(file.name) - // And a link all the same, which is the half of it an agent passes on: a `shark://` link names the heap - // dump rather than a window, so one from a run that has no window opens this file for whoever clicks - // it. No window id on it, since there is no window of this run to prefer. + // And a link all the same, which is the half of it an agent passes on: a link names the heap dump rather + // than a window, so one from a run with no window opens this file for whoever clicks it. Nothing but the + // dump and the place on it — no window of this run to prefer, and no path, because opening the dump + // wrote down where it is. val link = DeepLink.parse(shown.link!!) assertThat(link.heapDumpName).isEqualTo(file.name) - assertThat(link.heapDumpPath).isEqualTo(file.absoluteFile) - assertThat(link.windowId).isNull() assertThat(link.place).isEqualTo(Place.Leaks()) + assertThat(link.windowId).isNull() + assertThat(link.heapDumpPath).isNull() + assertThat(HeapDumpPaths(paths).resolve(link).heapDumpPath).isEqualTo(file.absoluteFile) } } @@ -141,14 +145,18 @@ class HeadlessAgentHeapDumpsTest { private fun headless( vararg heapDumpFiles: File, statuses: File = temporaryFolder.newFolder("statuses-${heapDumpFiles.size}"), - notes: File = temporaryFolder.newFolder("notes-${heapDumpFiles.size}") + notes: File = temporaryFolder.newFolder("notes-${heapDumpFiles.size}"), + paths: File = temporaryFolder.newFolder("paths-${heapDumpFiles.size}") ) = HeadlessAgentHeapDumps( // Nothing here reaches a device, and an `adb` that answers nothing is what proves it: a test that took // the machine's would have whatever is plugged in to answer for. deviceHeapDumps = DeviceHeapDumps(NoAdb), heapDumpFiles = heapDumpFiles.toList(), notes = ExplorerNotes(notes), - leakStatuses = ExplorerLeakStatuses(statuses) + leakStatuses = ExplorerLeakStatuses(statuses), + // This machine's own is where the app writes these, and a test writing there would leave records of heap + // dumps that only ever existed in a temporary folder. + heapDumpPaths = HeapDumpPaths(paths) ) /** An `adb` that isn't there, which is what a build server running this has. */ diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt index 9de31e5659..c293356eeb 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt @@ -25,10 +25,11 @@ import kotlin.random.Random * error. Being right about which window is worth a lot while the window exists and nothing at all * afterwards. * - * [heapDumpName] is the authority because it is the part a person reads and types; [heapDumpPath] is what - * makes the link exact, since two dumps called `com.squareup.hprof` off two devices are two investigations, - * and it is also the only way to open a dump nothing has open. A link with the name and no path is what - * somebody typed by hand, and it resolves against what is open. + * [heapDumpName] is the authority because it is the part a person reads and types, and **it is all a link + * says about which file**: where that file is, is looked up on the machine following the link rather than + * carried in it, since a path is most of the characters of a link and the least readable part of one. See + * [HeapDumpPaths], which is what remembers it, and [heapDumpPath], which is where a link that does carry one + * puts it. * * Immutable and in this module rather than in the UI, so that what a link means is unit tested rather than * found out by clicking one. See [Place] and `ExplorerWindows.windowFor`. @@ -38,10 +39,14 @@ data class DeepLink( val heapDumpName: String, val place: Place, /** - * Where that dump is, so that a link outlives every window of it and can open one. + * Where that dump is, for the links that say: null in every link this app writes. * - * Absolute and normalized, since it is compared against what a window has open and read months later. - * Null for a link somebody typed, which names a dump only by [heapDumpName]. + * Filled in by [HeapDumpPaths.resolve] as a link is followed, which is how a link finds the file without + * carrying it, and passed on in the query when one run hands a link to another so that the second doesn't + * have to look it up again. Written by hand in a link about a heap dump this machine has never opened, + * which is the one case a name cannot answer. + * + * Absolute and normalized when this app put it there, since it is compared against what a window has open. */ val heapDumpPath: File? = null, /** @@ -52,7 +57,13 @@ data class DeepLink( val windowId: String? = null ) { - /** A link to a place in a heap dump this app has open, which is every link the app itself writes. */ + /** + * A link to a place in a heap dump this app has open, which is every link the app itself writes. + * + * Names it by its file name and nothing else — where the file is doesn't travel in the link, see + * [heapDumpPath] — so this takes the [File] to save every caller writing `.name`, and to be the one + * spelling of "a link to what this window is showing". + */ constructor( heapDumpFile: File, place: Place, @@ -60,7 +71,6 @@ data class DeepLink( ) : this( heapDumpName = heapDumpFile.name, place = place, - heapDumpPath = normalizedHeapDumpPath(heapDumpFile), windowId = windowId ) @@ -278,13 +288,15 @@ data class DeepLink( /** * Where the heap dump is, and which window it was read in: the two parameters that are about the link - * rather than about the place. + * rather than about the place. Only `window` is written into a link this app copies — see [heapDumpPath] + * for when the other one is there. * * Which is why no [Place] may spell a parameter either of these names — they are read off the same query * — and none does. `DeepLinkTest` holds them apart. */ - internal const val DUMP_PARAMETER = "dump" - internal const val WINDOW_PARAMETER = "window" + /** Public because a message telling somebody to add one to a link has to spell it the way this does. */ + const val DUMP_PARAMETER = "dump" + const val WINDOW_PARAMETER = "window" internal const val ID_PARAMETER = "id" internal const val PARENT_PARAMETER = "parent" diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpFiles.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpFiles.kt index c037a99a13..57aff88049 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpFiles.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpFiles.kt @@ -69,5 +69,11 @@ internal fun writeWholeFile( Files.move(partial.toPath(), file.toPath(), StandardCopyOption.REPLACE_EXISTING) } -/** A save in flight, which is never a file anything reads. */ -private const val PARTIAL_SUFFIX = ".partial" +/** + * A save in flight, which is never a file anything reads. + * + * Named here rather than hidden in [writeWholeFile] because a directory these are written into is also a + * directory something lists — and one that took a save in flight for a file of its own would delete it out + * from under the run writing it. See [HeapDumpPaths]. + */ +internal const val PARTIAL_SUFFIX = ".partial" diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpPaths.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpPaths.kt new file mode 100644 index 0000000000..22624a5523 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpPaths.kt @@ -0,0 +1,139 @@ +package shark.explorer + +import java.io.File +import shark.SharkLog + +/** + * Where the heap dumps opened on this machine are, under the ids a [DeepLink] names them by. + * + * **This is what lets a link be short.** A link is a line of text someone reads — in a note, a pull request + * comment, an agent's answer — and a path is most of the characters of one while saying the least: it is + * unreadable at a glance, it says nothing a reader can act on, and it is the part that a link outliving the + * run it was copied from has to carry only because nothing else remembers it. So nothing else is where it + * stops being: a heap dump opening writes down where it was, and a link says the dump's file name and the + * window it was copied from — short, readable, and enough to find the file again here. + * + * One file per heap dump opened, named after the window that opened it and holding that dump's path. A file + * each rather than one file of all of them, because several runs of this app open heap dumps at the same time + * and none of them coordinates with the others: a whole file written and renamed into place cannot be read as + * half of one, and two runs opening two dumps write two files instead of racing over one. + * + * The newest [keepCount] are kept, so this is a directory that stops growing rather than a record of every + * heap dump ever opened. Which is the one thing a link loses by not carrying the path: it goes on working for + * as long as this machine remembers the file, rather than for as long as the file exists. A link that has been + * forgotten says so and can still be given the path by hand — see [DeepLink.heapDumpPath]. + * + * Machine local, and no worse than the path would have been: a link followed on another machine could never + * have used this one's paths. What it uses there is the file name, against the dumps that machine has open or + * has opened. + */ +class HeapDumpPaths( + /** This app's directory for these, which the caller decides, the way [NoteDirectory] takes its root. */ + private val directory: File, + private val keepCount: Int = KEEP_COUNT +) { + + init { + require(keepCount >= 1) { + "Expected to keep at least the heap dump being opened, not $keepCount of them" + } + } + + /** + * Writes down that the window called [windowId] has [heapDumpFile] open, and forgets the oldest of these + * beyond [keepCount]. + * + * Called as a heap dump finishes opening, in a window or in a run that has none: a dump that failed to open + * is not one a link should be sent to. + */ + fun record( + windowId: String, + heapDumpFile: File + ) { + val path = normalizedHeapDumpPath(heapDumpFile) + try { + writeWholeFile(File(directory, windowId), path.path) + } catch (throwable: Throwable) { + // Not a reason to fail the open: what stops working is links to this dump once every window of it has + // gone, which is worth a line in the log rather than a window that refuses to show a heap dump. + SharkLog.d(throwable) { "Could not record where $path is: links to it will need its path" } + return + } + forgetOldest() + } + + /** + * [link] with the heap dump's path filled in from what this machine remembers, or [link] as it is when + * nothing here has that dump on record. + * + * What a link says is tried in the order that is right about the most: the window it was copied from, since + * that window's dump is the one its reader was looking at; then a dump of that file name, newest first, + * since a name is what a link and a person both call a heap dump; then the name as a window id, for a link + * whose whole authority is one — `shark://abcd2345/leaks`, which is what this app used to write and what + * anything can still write, since a window id is enough to find the dump it was showing. + * + * A link that already carries a path is left alone. That path was either put there by hand or filled in by + * another run of this app, and either way it is more specific than a name. + */ + fun resolve(link: DeepLink): DeepLink { + if (link.heapDumpPath != null) { + return link + } + val records = records() + val recorded = link.windowId?.let { id -> records.firstOrNull { it.windowId == id } } + ?: records.firstOrNull { it.path.name == link.heapDumpName } + ?: records.firstOrNull { it.windowId == link.heapDumpName } + ?: return link + // Worded to read for a link named by a window id as well as by a file name, since both land here. + SharkLog.d { + "${link.heapDumpName} is ${recorded.path}, which was last open as ${recorded.windowId}" + } + return link.copy(heapDumpPath = recorded.path) + } + + /** Every heap dump on record, most recently opened first, which is the order all three lookups want. */ + private fun records(): List = + files().sortedByDescending { it.lastModified() }.mapNotNull { file -> + val path = try { + file.readText().trim() + } catch (throwable: Throwable) { + SharkLog.d(throwable) { "Could not read $file, so it names no heap dump" } + return@mapNotNull null + } + if (path.isEmpty()) null else Record(windowId = file.name, path = File(path)) + } + + private fun forgetOldest() { + val forgotten = files().sortedByDescending { it.lastModified() }.drop(keepCount) + if (forgotten.isEmpty()) { + return + } + SharkLog.d { + "Forgetting where ${forgotten.size} heap dump(s) opened before the last $keepCount were" + } + forgotten.forEach { it.delete() } + } + + /** + * The records, and only those: a write in flight is a file in here too, and deleting another run's would + * make its write fail. See [writeWholeFile]. + */ + private fun files(): List = + directory.listFiles { file -> file.isFile && !file.name.endsWith(PARTIAL_SUFFIX) } + .orEmpty() + .toList() + + /** One heap dump this machine has opened, and the window it was open in. */ + private class Record( + val windowId: String, + val path: File + ) + + companion object { + /** + * How many heap dumps are remembered. Enough that a link written weeks ago still opens the dump it names, + * few enough that this stays a directory somebody can read rather than search. + */ + const val KEEP_COUNT = 200 + } +} diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt index d0d3a9686b..9a2ce7ba56 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt @@ -248,24 +248,29 @@ class DeepLinkTest { } /** - * The link the app itself writes, and the whole of what it is for: a heap dump, a place in it, where that - * dump is, and which window it was copied from. + * The link the app itself writes, and the whole of what it is for: a heap dump, a place in it, and which + * window it was copied from. **Not where the file is** — that is looked up by whoever follows the link, see + * [HeapDumpPaths] — because a path is most of the characters of a link and the least readable part of it. */ @Test - fun `a link from a window carries the dump, the file and the window`() { + fun `a link from a window is a heap dump, a place and a window`() { val link = DeepLink(File("/dumps/leak.hprof"), Place.Leaks(), windowId = "abcd2345") - assertThat(link.toUri()) - .isEqualTo("shark://leak.hprof/leaks?dump=%2Fdumps%2Fleak.hprof&window=abcd2345") + assertThat(link.toUri()).isEqualTo("shark://leak.hprof/leaks?window=abcd2345") + assertThat(link.heapDumpPath).isNull() assertThat(DeepLink.parse(link.toUri())).isEqualTo(link) } - /** Because a link is read out of a heap dump's notes months later, from a machine with another home. */ + /** + * The one case a file name cannot answer: a heap dump this machine has never opened, which is a link from + * somebody else's. Nothing writes one of these — a person does, into a link that had nowhere to go. + */ @Test - fun `the path in a link is absolute and has no dots in it`() { - val link = DeepLink(File("dumps/./over/../leak.hprof"), Place.Starred) + fun `a link can say where the heap dump is`() { + val link = DeepLink("leak.hprof", Place.Starred, heapDumpPath = File("/dumps/leak.hprof")) - assertThat(link.heapDumpPath).isEqualTo(File(File("").absoluteFile, "dumps/leak.hprof")) + assertThat(link.toUri()).isEqualTo("shark://leak.hprof/starred?dump=%2Fdumps%2Fleak.hprof") + assertThat(DeepLink.parse(link.toUri())).isEqualTo(link) } /** @@ -316,7 +321,12 @@ class DeepLinkTest { ) places.forEach { place -> - val link = DeepLink(File("/dumps/leak.hprof"), place, windowId = "abcd2345") + val link = DeepLink( + heapDumpName = "leak.hprof", + place = place, + heapDumpPath = File("/dumps/leak.hprof"), + windowId = "abcd2345" + ) assertThat(DeepLink.parse(link.toUri())).describedAs(link.toUri()).isEqualTo(link) } } diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapDumpPathsTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapDumpPathsTest.kt new file mode 100644 index 0000000000..a34ebee4de --- /dev/null +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapDumpPathsTest.kt @@ -0,0 +1,161 @@ +package shark.explorer + +import java.io.File +import org.assertj.core.api.Assertions.assertThat +import org.assertj.core.api.Assertions.assertThatThrownBy +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder + +/** + * Where a heap dump was, remembered by the id a link names it with. Which is what a link not carrying the + * path rests on, so what is tested here is every way a link asks for one. + */ +class HeapDumpPathsTest { + + @get:Rule val temporaryFolder = TemporaryFolder() + + private val paths by lazy { HeapDumpPaths(temporaryFolder.newFolder("heap-dump-paths")) } + + @Test fun `a link naming a heap dump gets the path it was opened from`() { + paths.record(WINDOW_ID, File("/dumps/leak.hprof")) + + val resolved = paths.resolve(DeepLink.parse("shark://leak.hprof/starred")) + + assertThat(resolved.heapDumpPath).isEqualTo(File("/dumps/leak.hprof")) + // The rest of the link is what it was: this fills in the one thing a link doesn't say. + assertThat(resolved.place).isEqualTo(Place.Starred) + assertThat(resolved.heapDumpName).isEqualTo("leak.hprof") + } + + /** + * The whole point of it, and the case the path in a link used to be there for: the run that copied the link + * has been closed, and where its heap dump was is on disk rather than in the link. + */ + @Test fun `a link outlives the run that copied it`() { + val fromAWindow = DeepLink(File("/dumps/leak.hprof"), Place.Leaks(), windowId = WINDOW_ID) + paths.record(WINDOW_ID, File("/dumps/leak.hprof")) + + val resolved = paths.resolve(DeepLink.parse(fromAWindow.toUri())) + + assertThat(resolved.heapDumpPath).isEqualTo(File("/dumps/leak.hprof")) + } + + /** Two dumps of one name off two devices are two investigations, so which window it was copied from wins. */ + @Test fun `a link says which of two heap dumps of the same name`() { + paths.record(WINDOW_ID, File("/dumps/pixel/app.hprof")) + paths.record(OTHER_WINDOW_ID, File("/dumps/emulator/app.hprof")) + + val resolved = paths.resolve(DeepLink("app.hprof", Place.Starred, windowId = WINDOW_ID)) + + assertThat(resolved.heapDumpPath).isEqualTo(File("/dumps/pixel/app.hprof")) + } + + /** + * A link whose window is not on record — typed by hand, or copied from a run whose record has been + * forgotten — is still about a heap dump of that name, and the last one opened is the one being worked on. + */ + @Test fun `a name with no window falls back to the heap dump opened last`() { + paths.record(WINDOW_ID, File("/dumps/pixel/app.hprof")) + paths.record(OTHER_WINDOW_ID, File("/dumps/emulator/app.hprof")) + // Recorded in the same millisecond otherwise, which is not an order to read them in. + val directory = File(temporaryFolder.root, "heap-dump-paths") + File(directory, WINDOW_ID).setLastModified(FIRST_MODIFIED) + File(directory, OTHER_WINDOW_ID).setLastModified(LATER) + + val resolved = paths.resolve(DeepLink("app.hprof", Place.Starred, windowId = "qrst6789")) + + assertThat(resolved.heapDumpPath).isEqualTo(File("/dumps/emulator/app.hprof")) + } + + /** + * `shark:///`, which is a link with nothing in it but an id. Nothing writes one now, and it + * is the shortest a link can be, so it goes on working: a window id names a heap dump too. + */ + @Test fun `a link that is only a window id finds that window's heap dump`() { + paths.record(WINDOW_ID, File("/dumps/leak.hprof")) + + val resolved = paths.resolve(DeepLink.parse("shark://$WINDOW_ID/leaks")) + + assertThat(resolved.heapDumpPath).isEqualTo(File("/dumps/leak.hprof")) + } + + /** Which is what tells the reader to open the file, rather than a window that says nothing. */ + @Test fun `a heap dump nothing here has opened stays unresolved`() { + paths.record(WINDOW_ID, File("/dumps/leak.hprof")) + + val resolved = paths.resolve(DeepLink.parse("shark://another.hprof/starred")) + + assertThat(resolved.heapDumpPath).isNull() + } + + @Test fun `nothing recorded at all resolves nothing`() { + assertThat(paths.resolve(DeepLink.parse("shark://leak.hprof/starred")).heapDumpPath).isNull() + } + + /** Handed over by another run, or written by hand: more specific than a name, so it is left alone. */ + @Test fun `a link that already says where the dump is keeps that path`() { + paths.record(WINDOW_ID, File("/dumps/leak.hprof")) + val link = DeepLink("leak.hprof", Place.Starred, heapDumpPath = File("/elsewhere/leak.hprof")) + + assertThat(paths.resolve(link).heapDumpPath).isEqualTo(File("/elsewhere/leak.hprof")) + } + + /** Because a link is read months later, from a run started in another directory. */ + @Test fun `a recorded path is absolute and has no dots in it`() { + paths.record(WINDOW_ID, File("dumps/./over/../leak.hprof")) + + assertThat(paths.resolve(DeepLink.parse("shark://leak.hprof/starred")).heapDumpPath) + .isEqualTo(File(File("").absoluteFile, "dumps/leak.hprof")) + } + + @Test fun `only the newest heap dumps are remembered`() { + val directory = temporaryFolder.newFolder("keep-two") + val paths = HeapDumpPaths(directory, keepCount = 2) + + listOf("first", "second", "third").forEachIndexed { index, name -> + paths.record(name, File("/dumps/$name.hprof")) + // Written in the same millisecond otherwise, which is not an order to evict by. + File(directory, name).setLastModified(FIRST_MODIFIED + index * MINUTE) + } + + // A directory that stops growing, which is the one thing a link loses by not carrying the path: it works + // for as long as this machine remembers the file rather than for as long as the file exists. + assertThat(directory.list()).containsExactlyInAnyOrder("second", "third") + } + + @Test fun `a record nobody can read names no heap dump`() { + val directory = temporaryFolder.newFolder("unreadable") + File(directory, WINDOW_ID).writeText("") + + assertThat(HeapDumpPaths(directory).resolve(DeepLink.parse("shark://$WINDOW_ID/leaks")).heapDumpPath) + .isNull() + } + + /** A save in flight is a file in this directory too, and one nothing may read or delete. */ + @Test fun `a write in flight is not a heap dump on record`() { + val directory = temporaryFolder.newFolder("in-flight") + val paths = HeapDumpPaths(directory, keepCount = 1) + File(directory, "$WINDOW_ID.partial").writeText("/dumps/half-written.hprof") + + paths.record(OTHER_WINDOW_ID, File("/dumps/leak.hprof")) + + assertThat(File(directory, "$WINDOW_ID.partial")).exists() + assertThat(paths.resolve(DeepLink.parse("shark://half-written.hprof/starred")).heapDumpPath).isNull() + } + + @Test fun `remembering none of them is not something to ask for`() { + assertThatThrownBy { HeapDumpPaths(temporaryFolder.newFolder("none"), keepCount = 0) } + .hasMessageContaining("not 0 of them") + } + + companion object { + private const val WINDOW_ID = "abcd2345" + private const val OTHER_WINDOW_ID = "wxyz6789" + + /** Any time at all, since what is read off these is their order. */ + private const val FIRST_MODIFIED = 1_600_000_000_000L + private const val MINUTE = 60_000L + private const val LATER = FIRST_MODIFIED + MINUTE + } +} From 5716c2fdb7f51a2787f97742a7a8e4f87d8019fc Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Thu, 27 Aug 2026 19:32:35 +0200 Subject: [PATCH 26/27] Say only which heap dump a link is about, and ask when that isn't enough MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A link was carrying the window it was copied from, then the heap dump's whole path, and both were answers to a question that hardly ever comes up: which of the heap dumps called that. Heap dump file names are as good as unique — this app names every dump it pulls off a device after the process, its pid and a random number, and LeakCanary names its own after the time of the dump — so a link that says `shark://bug-4821.hprof/leaks` names one file on any machine that has it, and the long form was paying for the rare case in every link. So the name is the whole of what a link says, and the rare cases are asked about rather than encoded: - A window has that heap dump open, which is nearly always. That window is where the link goes. - None has, but this machine has had it open. It opens, from where it was last seen. - Two heap dumps of that name, open or on record. The reader picks, by path. - Nothing here knows the name — a link from another machine, a dump deleted or opened long ago. The reader is asked for the file, since silence can't be told from the app having failed to start. One dialog puts both questions, because both answers are a path. `&dump=` is still there for a link that wants to say where the file is, and it does the picking for the reader when it does. --- docs/shark-explorer-changelog.md | 5 +- docs/shark-explorer.md | 42 ++- shark/shark-explorer/AGENTS.md | 26 +- shark/shark-explorer/notes/decisions.md | 81 ++--- .../shark/explorer/agent/AgentSessionFile.kt | 11 +- .../shark/explorer/agent/AgentToolsTest.kt | 1 - .../shark/explorer/agent/FakeAgentHeapDump.kt | 2 +- .../shark/explorer/agent/McpSessionTest.kt | 7 +- .../java/shark/explorer/app/DeepLinkPeers.kt | 20 +- .../java/shark/explorer/app/ExplorerAgents.kt | 15 +- .../java/shark/explorer/app/ExplorerWindow.kt | 303 +++++++++++++----- .../explorer/app/HeadlessAgentHeapDumps.kt | 4 +- .../shark/explorer/app/HeapDumpExplorer.kt | 25 +- .../explorer/app/LinkedHeapDumpDialog.kt | 87 +++++ .../src/main/java/shark/explorer/app/Main.kt | 58 +++- .../shark/explorer/app/ExplorerAppTest.kt | 6 +- .../shark/explorer/app/ExplorerWindowTest.kt | 197 ++++++++---- .../app/HeadlessAgentHeapDumpsTest.kt | 3 +- .../shark/explorer/app/LinkedHeapDumpTest.kt | 138 ++++++++ .../shark/explorer/app/NoteSectionTest.kt | 13 +- .../shark/explorer/app/ObjectsScreenTest.kt | 6 +- .../java/shark/explorer/app/TabStripTest.kt | 8 +- .../src/main/java/shark/explorer/DeepLink.kt | 93 ++---- .../main/java/shark/explorer/HeapDumpPaths.kt | 78 ++--- .../test/java/shark/explorer/DeepLinkTest.kt | 43 +-- .../java/shark/explorer/HeapDumpPathsTest.kt | 129 +++----- 26 files changed, 872 insertions(+), 529 deletions(-) create mode 100644 shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/LinkedHeapDumpDialog.kt create mode 100644 shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LinkedHeapDumpTest.kt diff --git a/docs/shark-explorer-changelog.md b/docs/shark-explorer-changelog.md index f65f210cfd..a1ac2e30ce 100644 --- a/docs/shark-explorer-changelog.md +++ b/docs/shark-explorer-changelog.md @@ -22,11 +22,12 @@ uses, without the one for a newly recognized library leak: * ✨ Right click anything the window can take you to — a tab, a rectangle, a row, a field — and copy a `shark://` link to it, beside opening it in a new tab. Clicking one brings the app to the front and opens that place in a new tab: an object, a filtered object list, the leaks with the same groups - unfolded. A link names the **heap dump** — `shark://bug-4821.hprof/leaks?window=vugs93jp`, short enough to + unfolded. A link names the **heap dump** and nothing else — `shark://bug-4821.hprof/leaks`, short enough to read in a sentence — so it goes on working after the window it was copied from has gone: it opens the place in a window that has that dump, and opens the file in a new window when none has. Where that file is never goes in the link: every heap dump opened is written down in `~/.shark-explorer/heap-dump-paths`, the last - 200 kept, and following a link looks it up there. See + 200 kept, and following a link looks it up there. A link about a heap dump this machine can't find asks for + the file, and one about a name two heap dumps share asks which of them. See [Link to a tab](shark-explorer.md#link-to-a-tab). * ✨ **Notes**: every location takes a markdown note, kept between runs, and the tab strip marks the tabs whose location has one. A note belongs to the location rather than to the tab, so two tabs on one diff --git a/docs/shark-explorer.md b/docs/shark-explorer.md index 10976fdaaf..bb57975b14 100644 --- a/docs/shark-explorer.md +++ b/docs/shark-explorer.md @@ -95,27 +95,25 @@ leaks with the same groups unfolded, the starred objects. So "look at this" is a paragraph of directions, which is also how a tool or an agent that has read your heap dump can point you straight at what it found. -The part after `shark://` is **the heap dump**, because every place a link can name belongs to the dump -rather than to the window showing it. So a link goes on working: following one opens that place in a window -that has the dump open, and opens the file in a new window when none has — the run it was copied from can -be long gone. A link never replaces what you were reading: it always opens a tab of its own. - -A copied link carries one more thing after the place, and this is one in full: - -``` -shark://bug-4821.hprof/leaks?window=vugs93jp -``` - -`window` is the window it was copied from, honoured while that window is open and ignored once it isn't, so -that the same dump open twice, which is two readings of it side by side, lands where you meant. It isn't -needed to type one by hand: `shark://bug-4821.hprof/leaks` is a link. +The part after `shark://` is **the heap dump**, and it is the whole of what a link says about which one, +because every place a link can name belongs to the dump rather than to the window showing it. So a link goes +on working: following one opens that place in a window that has the dump open, and opens the file in a new +window when none has — the run it was copied from can be long gone. A link never replaces what you were +reading: it always opens a tab of its own. What you copy is what you can type, and nothing more. **Where the file is doesn't travel in the link.** Every heap dump this app opens is written down in `~/.shark-explorer/heap-dump-paths`, the last 200 kept, so following a link is a lookup rather than a path -pasted into a URL — which is what keeps a link short enough to read in a sentence. A link about a dump this -machine has no record of opening, which is one from somebody else's machine, opens a window saying so; open -that file and the link works, or add the path to the link yourself as -`&dump=/Users/you/dumps/bug-4821.hprof`. +pasted into a URL — which is what keeps a link short enough to read in a sentence. + +Two links can't be sorted out on their own, and both ask rather than guess: + +* **A heap dump this machine can't find**, which is a link from somebody else's machine, or about a dump + deleted or moved since the link was written. A window opens saying which, and asks for the file. You can + also put the path in the link yourself, as `&dump=/Users/you/dumps/bug-4821.hprof`. +* **Two heap dumps of one name**, which is one app dumped on two devices, or a dump copied somewhere. The + places they are in are offered, and the one you pick is where the link goes. Uncommon: a dump this app + takes is named after the process, its pid and a random number, and LeakCanary names its own after the + time of the dump. Links reach the app from an installed build — the installer is what tells the OS that `shark://` is this app's. A copy run from source can still be linked to from another one, but the OS won't start it for a @@ -480,11 +478,11 @@ and the method tells an agent to put those links in its reply — so a sentence request comment or a bug report ends up carrying a way in: > The leak is `MainActivity$2.this$0`, a non-static inner class holding the activity it was declared in: -> shark://leak_asynctask_o.hprof/object?id=0x12d368b8&window=zvphq4r3 +> shark://leak_asynctask_o.hprof/object?id=0x12d368b8 -Clicking it opens that object with the reasoning on its tabs — in the window it was written from while that -window is up, and by opening the heap dump again once it isn't. So an answer worth keeping keeps working, and -it is short enough to read: it names the heap dump, and where that file is, is looked up. +Clicking it opens that object with the reasoning on its tabs — in a window that has the heap dump while one +is up, and by opening the file again once none is. So an answer worth keeping keeps working, and it is short +enough to read: it names the heap dump, and where that file is, is looked up. An agent's verdicts are verdicts like any other: they say `set by hand` on every chain that runs through the object, the reason is the one it gave, and the pencil takes one off if you disagree with it. Which is the diff --git a/shark/shark-explorer/AGENTS.md b/shark/shark-explorer/AGENTS.md index a7a82bc00e..1baf37459a 100644 --- a/shark/shark-explorer/AGENTS.md +++ b/shark/shark-explorer/AGENTS.md @@ -240,23 +240,23 @@ open -a ~/Applications/"Shark Explorer.app" --args --title="Links" path/to/dump. open "shark://dump.hprof/leaks" ``` -A link names the heap dump, so that is the whole recipe — no id to read out of the log first, and the same -line works after the run that opened the dump has ended, which is the case worth trying. To try the window -half of it, the `window=` a copied link carries, `grep "Windows of this run"` in the newest file under -`~/.shark-explorer/logs` for the ids of the run. - -**A link says no path, so a dump this machine has never opened has nowhere to go.** Where each dump was is -written down under `~/.shark-explorer/heap-dump-paths`, one file per window that opened one, and the -`HeapDumpPaths` lookup is what turns a file name back into a file. So a link to a dump opened only in a run -started before this feature existed, or one whose record has been evicted, opens a window saying so — `ls` -that directory before concluding the routing is broken, and `&dump=` is the way to try a link about a -dump nothing here has opened. +A link names the heap dump and nothing else, so that is the whole recipe — no id to read out of the log +first, and the same line works after the run that opened the dump has ended, which is the case worth trying. + +**A link says no path, so a dump this machine has never opened is a question rather than a jump.** Where each +dump was is written down under `~/.shark-explorer/heap-dump-paths`, one file per heap dump opened, and the +`HeapDumpPaths` lookup is what turns a file name back into a file. A link about a name nothing there has, or +one whose record has been evicted, opens a window asking for the file — and a name two records share asks +which of them. Both are dialogs waiting on somebody, so a link that "did nothing" is worth a look at the +screen: `ls ~/.shark-explorer/heap-dump-paths` says which of the two it will be, and `&dump=` skips the +question for a dump nothing here has opened. **Read the result in the log rather than off the screen.** Following a link raises the app over whatever the person at the machine was doing, so a screenshot to check it worked costs them their window and shows you theirs. `The OS handed this run`, `A link asked window for of `, `A link asked for - of , which is not open yet` and `A link asked this window for ` are the lines that say -a link was delivered, routed and opened as a tab. + of , which is not open yet`, `A link to of is asking: ` and `A link +asked this window for ` are the lines that say a link was delivered, routed, asked about and opened +as a tab. A run from source is still *reachable*: every run publishes a loopback port under `~/.shark-explorer/runs`, and the installed app hands on any link it has no window for. That is what makes a link to a diff --git a/shark/shark-explorer/notes/decisions.md b/shark/shark-explorer/notes/decisions.md index 5bcc06aad6..41210993e9 100644 --- a/shark/shark-explorer/notes/decisions.md +++ b/shark/shark-explorer/notes/decisions.md @@ -847,49 +847,52 @@ inside it, a file per `noteKey` rather than one document with a section per plac the note that was typed into, nothing has to be parsed back out of a document that also holds somebody's own headings, and the listing is the index. -## A link names the heap dump, and a window only as a refinement - -`shark:///?[&window=]`. The first version of this named the window -— `shark:///` — and it was wrong for the reason a link exists: every place there is belongs -to the heap dump, not to whatever is showing it, so a link that named a window died with the window. Which is -most links a day later, and most links in an agent's session log, since a session outlives the run that wrote -it. A link that mostly doesn't work is a link nobody sends. - -So the dump is the identity and the window is honoured while it exists and **ignored once it doesn't**, rather -than turning the link into an error. Being right about which window is worth a lot while the window is there -and nothing at all afterwards. - -- **The authority is the file name**, because it is the part a person reads and types, and it is in every - answer an agent has already been given. -- **Where the file is doesn't travel in the link.** The version between these two carried - `dump=%2FUsers%2F…`, which was four fifths of the characters of a link and the fifth nobody could read. So - `HeapDumpPaths` writes down the path of every dump that opens, under the id of the window it opened in, the +## A link names the heap dump and nothing else + +`shark:///?`. Two things were tried in front of that and both were +taken back out. The first version named the window — `shark:///` — and it was wrong for the +reason a link exists: every place there is belongs to the heap dump, not to whatever is showing it, so a link +that named a window died with the window. Which is most links a day later, and most links in an agent's +session log, since a session outlives the run that wrote it. A link that mostly doesn't work is a link nobody +sends. The second carried the dump's path and then, briefly, the window as a refinement — `…/leaks?window=…` +— and both were paid for on every link that didn't need them, which is nearly all of them. + +**Heap dump names are unique in practice**, which is what makes a name enough: a dump this app takes is +`--.hprof` and LeakCanary's are `yyyy-MM-dd_HH-mm-ss_SSS.hprof`. So the cases a name +can't settle are rare enough to *ask* about, and asking is better than a link that carries an answer to a +question nobody had. + +- **The authority is the file name**, because it is the part a person reads and types, it is what the window's + title shows, and it is in every answer an agent has already been given. +- **Where the file is doesn't travel in the link.** `dump=%2FUsers%2F…` was four fifths of the characters of a + link and the fifth nobody could read. So `HeapDumpPaths` writes down the path of every dump that opens, the newest 200 kept, and following a link is a lookup. What that costs is honest and small: a link works for as - long as this machine remembers the file rather than for as long as the file exists, and a link that has been - forgotten says so and can be given `&dump=` by hand. -- **Recorded under the window id, not the dump.** One record per open is a single whole-file write by one run - with nothing to merge, and it answers both questions at once: a window id resolves to the dump it was - showing, and a file name resolves to the newest record with that name. Two dumps called `com.squareup.hprof` - off two devices are two investigations, and the `window=` on a copied link is what tells them apart. -- **Not `heapDumpFileKey`**, the `-` the notes and statuses are filed under: it is - one-way, so a key on its own can name a dump but never find one. Which was also the objection to a window id - as the authority, and `HeapDumpPaths` answers it — a link that is nothing but an id resolves now. It still - isn't what the app writes, because an id says nothing to whoever reads the link, is not what a window's - title shows, and is not the same for the same place twice. -- **Window ids stay random.** A counted id repeats across runs *and* within one as windows close and open, so - it would be honoured against the wrong reading of the dump — silently, which is worse than being ignored. A - file name plus a number fixes neither half: the number would have to be handed out across runs that cannot - see each other's windows. -- **Resolution order is windowId, then path, then file name, then the authority as a window id**, in the - windows of the run and again in the records on disk. That last step is what keeps a `shark:///…` - link working — the ones already sitting in notes and session files, and the shortest link anyone can write. + long as this machine remembers the file rather than for as long as the file exists, and a link about a dump + that has been forgotten asks for the file — or can be given `&dump=` by hand, which is also the answer + for a dump this machine has never opened. +- **One record per heap dump**, named `heapDumpFileKey` — the `-` the notes and statuses + are filed under — with the path inside it. The key is one-way, so the file name of the record can name a + dump but never find one; the path it holds is what makes the lookup work. A file each rather than one file + of all of them, because several runs open dumps at once and none of them coordinates: a whole-file write and + a rename cannot be read as half of one. +- **Four outcomes, and the first is nearly always the one.** A window of this run has that dump: that window. + None has, but the machine has had it open: the file opens. Two dumps of that name: ask which, by path. Name + unknown here: ask for the file. `ExplorerWindows.open`. +- **The two questions are one dialog**, because both answers are a path — the places on record as rows, and + the file picker under them. It is hosted in a window already showing one of the dumps in question when there + is one, so asking which costs no window, and in an empty window otherwise, which is where the dump picked + opens and which says why it is empty if the question is dismissed. +- **Window ids stay, and stay out of links.** They are what an agent calls a window, since one heap dump open + in two of them is two places to be told about, and they stay random for that: a counted id repeats across + runs and within one as windows close, which is an id that names the wrong window rather than none. - **A run claims a link only for a window it already has**, never for a file it could open, or every run of - the app would claim every link. Whoever is left holding it opens the dump. `DeepLinkPeers`. + the app would claim every link. The link is passed on exactly as it arrived, so what to do about a dump no + window has — open it, ask which, ask where — belongs to whoever ends up holding it. `DeepLinkPeers`. - **The agent surface converged on the same choice**: the tool argument is `heapDump`, taking a file name, and a window id only in the one case a name cannot answer, which is the same file open twice. `AgentTools`. -- **What it unlocked**, and the reason to reverse it rather than live with it: a `--no-ui` run answers `show` - with a link now — it has no window and the file all the same — and every *Agent logs* row about another - heap dump has a link to copy, where before there was nothing to send. +- **What it unlocked**, and the reason to reverse the first version rather than live with it: a `--no-ui` run + answers `show` with a link now — it has no window and the file all the same — and every *Agent logs* row + about another heap dump has a link to copy, where before there was nothing to send. ## A leaking status is the heap dump's answer until a hand overrules it diff --git a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt index 898543d97a..8eae4be186 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/main/java/shark/explorer/agent/AgentSessionFile.kt @@ -225,7 +225,7 @@ class AgentSessionFile private constructor( /** * Eight hexadecimal characters, from [SecureRandom] like the token beside it. * - * Random rather than counted, for the reason `DeepLink.newWindowId` is: ids handed out in order repeat + * Random rather than counted, for the reason a window's id is: ids handed out in order repeat * across runs of the app, and a session log named the same as one from yesterday is two investigations * that read as one. */ @@ -469,15 +469,14 @@ class AgentSessionCall( /** * The link to [place] in the heap dump the call was about, for a call that was about one. * - * The window as well, since it was open when the line was written, and a link is a request to look at - * something in the window somebody was watching while that is still possible. It stops being possible - * about as soon as anybody reads this — an agent's session outlives its run — and a link that names the - * heap dump goes on working after that. See [DeepLink]. + * The heap dump and not [windowId], even though the window was open when the line was written: an agent's + * session outlives its run, so by the time anybody reads this the window has almost always gone while the + * heap dump is still there to open. See [DeepLink]. */ fun link(): String? { val place = place ?: return null val heapDumpPath = heapDumpPath ?: return null - return DeepLink(File(heapDumpPath), place, windowId = windowId).toUri() + return DeepLink(File(heapDumpPath), place).toUri() } } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt index bd5cfccd00..332803df67 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/AgentToolsTest.kt @@ -860,7 +860,6 @@ class AgentToolsTest { val link = DeepLink.parse(answer.text("link")) assertThat(link.heapDumpName).isEqualTo(window.heapDumpName) assertThat(link.heapDumpPath).isNull() - assertThat(link.windowId).isEqualTo(window.windowId) assertThat(link.place).isEqualTo(Place.Object(objectId)) } diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt index fb0ef18669..d2d6d2357f 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/FakeAgentHeapDump.kt @@ -84,7 +84,7 @@ internal class FakeAgentHeapDump( // A window's answer, which is a link — built the way the window builds one, since a fake that spelled it // itself would be a test passing on a link nobody could follow. What a run with no window answers is // `HeadlessAgentHeapDumpsTest`'s, since it is that run's one difference from this one. - return ShownPlace.at(DeepLink(File(heapDumpPath), place, windowId = windowId).toUri()) + return ShownPlace.at(DeepLink(File(heapDumpPath), place).toUri()) } override fun close() { diff --git a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt index 9c77329620..c990550c0a 100644 --- a/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt +++ b/shark/shark-explorer/shark-explorer-agent/src/test/java/shark/explorer/agent/McpSessionTest.kt @@ -213,15 +213,14 @@ class McpSessionTest { assertThat(call.reason).isEqualTo("Checking whether the holder is the singleton it looks like.") assertThat(call.refusal).isNull() // Which is what makes the row clickable: the place, in the heap dump the call was made against — named - // by the dump so that the link still opens it once this run has ended, with the window it was made in as - // a refinement, honoured while that window is open. The session line records the dump's path as well, - // which the link doesn't have to: a row leads to a file, and a link is looked up. + // by the dump and nothing else, so that the link still opens it once the window it was made in has gone. + // The session line records the dump's path as well, which the link doesn't have to: a row leads to a + // file, and a link is looked up. assertThat(call.place).isEqualTo(Place.Object(heapDump.holderObjectId)) assertThat(call.heapDumpPath).isEqualTo(window.heapDumpPath) val link = DeepLink.parse(call.link()!!) assertThat(link.heapDumpName).isEqualTo(window.heapDumpName) assertThat(link.heapDumpPath).isNull() - assertThat(link.windowId).isEqualTo(window.windowId) assertThat(link.place).isEqualTo(Place.Object(heapDump.holderObjectId)) } diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt index e534459a9e..9239849bae 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/DeepLinkPeers.kt @@ -94,13 +94,13 @@ internal object DeepLinkPeers { link: DeepLink, windows: ExplorerWindows ) { - if (windows.windowFor(link) != null) { + if (windows.windowsFor(link).isNotEmpty()) { windows.open(link) return } Thread({ - // Nobody else's, so this run answers for it, which is opening that heap dump here — at the path - // [deliver] looked up, since a link says the dump's name and not where it is. + // Nobody else's, so this run answers for it, which is opening that heap dump here — or asking where it + // is, since a link says the dump's name and not where it is. See [ExplorerWindows.open]. deliver(listOf(link)).forEach { windows.open(it) } }, THREAD_NAME).apply { isDaemon = true @@ -115,20 +115,16 @@ internal object DeepLinkPeers { * The leftovers are the caller's to answer for, which is what makes a link whose window has gone a window * of that heap dump here rather than a process that started and exited without a word. * - * **Where the heap dump is gets looked up first**, and it is looked up once for every run: a link carries a - * file name, and a run asked about a link it has no window for should be answering about the file rather - * than about the name — two dumps called `com.squareup.hprof` off two devices are two investigations. So - * what goes out on the socket is the link with the path filled in, and what comes back to the caller is the - * same, ready to open. See [HeapDumpPaths.resolve]. + * The link goes out exactly as it arrived: what a run does about a heap dump it has no window for is that + * run's own business — open it, or ask which one, or ask where it is — and every one of those answers + * belongs to whoever ends up with the link rather than to whoever passed it on. See [ExplorerWindows.open]. */ fun deliver(links: List): List { if (links.isEmpty()) { return emptyList() } - val heapDumpPaths = explorerHeapDumpPaths() - val resolved = links.map { heapDumpPaths.resolve(it) } val peers = peers() - return resolved.filter { link -> peers.none { peer -> peer.deliver(link) } } + return links.filter { link -> peers.none { peer -> peer.deliver(link) } } } /** Every other run that has published itself, stale files cleared out on the way past. */ @@ -231,7 +227,7 @@ internal object DeepLinkPeers { } // Answered before the window is asked to go anywhere, because the run on the other end is waiting to // find out whether to keep looking, and going somewhere is a frame away rather than a read away. - if (windows.windowFor(link) != null) { + if (windows.windowsFor(link).isNotEmpty()) { writer.println(ACCEPTED) windows.open(link) } else { diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt index 7bd7af43c0..aa7ba4feed 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerAgents.kt @@ -152,9 +152,9 @@ internal class WindowAgentHeapDumps( val window = already ?: windows.openHeapDump(file) SharkLog.d { if (already == null) { - "An agent opened ${file.absolutePath} in window ${window.deepLinkId}" + "An agent opened ${file.absolutePath} in window ${window.windowId}" } else { - "An agent asked for ${file.absolutePath}, which window ${window.deepLinkId} already has" + "An agent asked for ${file.absolutePath}, which window ${window.windowId} already has" } } // Three ways this ends and only one of them is an answer — the dump opens, it fails to open, or the @@ -170,7 +170,7 @@ internal class WindowAgentHeapDumps( } throw AgentRefusal( window.openProblem?.let { "${file.name} could not be opened as a heap dump: $it" } - ?: "Window ${window.deepLinkId} was closed before ${file.name} had finished opening, so there is " + + ?: "Window ${window.windowId} was closed before ${file.name} had finished opening, so there is " + "nothing to read. Opening it again is a call away." ) } @@ -380,16 +380,15 @@ internal class OpenHeapDump( /** This window's heap dump as an agent sees it: shown by going to a tab, the way a link does. */ private fun ExplorerWindow.agentHeapDump(open: OpenHeapDump): AgentHeapDump = - OpenAgentHeapDump(windowId = deepLinkId, open = open) { place -> - SharkLog.d { "An agent asked window $deepLinkId for $place" } + OpenAgentHeapDump(windowId = windowId, open = open) { place -> + SharkLog.d { "An agent asked window $windowId for $place" } // The same two steps following a link takes, which is what makes an agent showing something and a // person clicking a link land in the same place. See [ExplorerWindows.open]. goToLinked(place) bringToFront() // And the link itself, which is the same one the right click menu copies: an agent's answer can then - // point at this place rather than describe how to get to it. Naming this window as well as the dump, - // since a reader following it while this run is up should land on the window they watched it happen in. - ShownPlace.at(DeepLink(open.session.heapDumpFile, place, windowId = deepLinkId).toUri()) + // point at this place rather than describe how to get to it. + ShownPlace.at(DeepLink(open.session.heapDumpFile, place).toUri()) } /** diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt index 24dcfdab47..41fc6eff0e 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt @@ -8,8 +8,10 @@ import androidx.compose.runtime.snapshots.SnapshotStateList import java.awt.EventQueue import java.awt.Frame import java.io.File +import kotlin.random.Random import shark.SharkLog import shark.explorer.DeepLink +import shark.explorer.HeapDumpPaths import shark.explorer.NativeBitmapPixels import shark.explorer.Place @@ -31,12 +33,11 @@ internal class ExplorerWindow( /** What this run calls its windows, in front of the heap dump. See [ExplorerArguments.titlePrefix]. */ val titlePrefix: String? = null, /** - * What a link to a place in this window names it by, and the whole of what makes deep linking work: a - * link is answered by the window it was copied from and by no other, however many windows have the same - * heap dump open. See [DeepLink]. + * What an agent calls this window, since a heap dump open in two of them is two places to be told about. + * Never in a link — see [DeepLink] — and it is not what makes one work. */ - val deepLinkId: String = DeepLink.newWindowId(), - /** Why this window is empty, for one opened by a link naming a window that had already gone. */ + val windowId: String = newWindowId(), + /** Why this window is empty, for one a link opened having nowhere to go. See [ExplorerWindows.open]. */ deepLinkProblem: String? = null ) { @@ -55,6 +56,15 @@ internal class ExplorerWindow( */ var deepLinkProblem: String? by mutableStateOf(deepLinkProblem) + /** + * What a link is waiting on an answer about, and null whenever nothing is being asked. + * + * The other way a link ends up here: two heap dumps of the name it says, or none this machine can find. + * Set from whatever thread the link arrived on, for the reason [linkedPlaces] is, and taken by the window + * on the next frame as a dialog. See [ExplorerWindows.open] and [ExplorerWindows.chooseLinkedHeapDump]. + */ + var linkedHeapDump: LinkedHeapDump? by mutableStateOf(null) + /** * The heap dump this window has open, once it is open, for everything that isn't drawing the window. * @@ -122,7 +132,7 @@ internal class ExplorerWindow( val frame = frame if (frame == null) { // Which is what a link that raised nothing at all looks like from here. - SharkLog.d { "Window $deepLinkId has no frame yet, so nothing was brought to the front" } + SharkLog.d { "Window $windowId has no frame yet, so nothing was brought to the front" } return } EventQueue.invokeLater { @@ -139,6 +149,34 @@ internal class ExplorerWindow( } } +/** + * A new name for a window: eight characters of an alphabet nothing can be misread in. + * + * Read out of a log and typed back into an agent's call — see [ExplorerWindow.windowId] — so the alphabet + * leaves out `l`, `1`, `o` and `0`, and eight characters is short enough to retype. Random rather than + * counted, because two runs of this app count from the same place and an agent talking to both would be told + * about two windows called `1`. + */ +internal fun newWindowId(random: Random = Random.Default): String = + (1..WINDOW_ID_LENGTH).map { WINDOW_ID_ALPHABET.random(random) }.joinToString("") + +/** + * A heap dump a link named that this run could not pin down on its own, and what is being asked about it. + * + * Two ways one link gets here — see [ExplorerWindows.open] — and one question answers both, because both + * answers are a path: which of the heap dumps called that, or where the one called that is. + */ +internal class LinkedHeapDump( + /** What the link called the heap dump, which is a file name and all a link says. */ + val heapDumpName: String, + /** Why this is being asked, said in the dialog and in the middle of a window opened to ask it. */ + val question: String, + /** The paths of that name this machine knows of, which is what there is to pick from. Can be empty. */ + val choices: List, + /** Where the link was going, which is where whichever heap dump is picked opens. */ + val place: Place +) + /** * Every window of this run, and the way in for anything that isn't drawing one. * @@ -147,6 +185,11 @@ internal class ExplorerWindow( * composing nothing. See [ExplorerWindow.linkedPlaces]. */ internal class ExplorerWindows( + /** + * Where the heap dumps this machine has opened are, which is how a link about one that no window of this + * run has open finds the file. See [open]. + */ + val heapDumpPaths: HeapDumpPaths, /** Put in front of every window title of this run. See [ExplorerArguments.titlePrefix]. */ val titlePrefix: String? = null, /** One Compose window is drawn per entry, so a window opening or closing is an edit of this. */ @@ -154,94 +197,180 @@ internal class ExplorerWindows( ) : MutableList by windows { /** - * The window of this run [link] leads to, or null for one no window here can answer. + * The windows of this run [link] leads to, which is every window showing the heap dump it names. * * Which is what a run asks before handing a link on to the others, so it is deliberately *only* about * windows that exist: a run that answered "I could open that file" would claim every link on the machine. - * Opening the dump is what whoever ends up answering does, in [open]. - * - * In order, because each step is right about something the next one isn't: + * Opening the file, and asking where it is, is what whoever ends up answering does. See [open]. * - * - **The window the link was made from**, while it is still open. Two windows on one dump are two - * readings of it, and this is the only thing that tells them apart. - * - **A window of that heap dump**, by path, when the link says where the file is — which is one handed - * over by another run, since that run looked it up. Exact, so nothing is tried after it: two dumps of one - * name off two devices are two investigations. - * - **A window of a dump with that file name**, which is what a link says about the dump and all it says. - * - **A window with that id**, for a link whose whole authority is a window: `shark:///…` is in - * notes on disk and in agent sessions, and one of those pasted back into the run it came from still goes - * where it went. Last, so that a heap dump called like a window id — which is a file called `abcd2345` — - * is read as the heap dump. + * By file name, which is the whole of what a link says about the heap dump — and by path for the rare link + * that says where the file is, since that one is exact: two dumps called `com.squareup.hprof` off two + * devices are two investigations, and a link carrying a path has already said which. */ - fun windowFor(link: DeepLink): ExplorerWindow? { - val ofWindowId = link.windowId?.let { id -> firstOrNull { it.deepLinkId == id } } - if (ofWindowId != null) { - return ofWindowId + fun windowsFor(link: DeepLink): List { + val path = link.heapDumpPath?.normalizedPath() + return filter { window -> + val heapDumpFile = window.heapDumpFile ?: return@filter false + if (path == null) heapDumpFile.name == link.heapDumpName else heapDumpFile.normalizedPath() == path } - val path = link.heapDumpPath - if (path != null) { - return firstOrNull { it.heapDumpFile?.absoluteFile?.normalize() == path } - } - return firstOrNull { it.heapDumpFile?.name == link.heapDumpName } - ?: firstOrNull { it.deepLinkId == link.heapDumpName } } /** - * Follows [link]: the place opens as a new tab in a window of that heap dump, which comes to the front. + * Follows [link]: the place opens as a new tab in a window of the heap dump it names, which comes to the + * front. + * + * **A link outlives the window it was copied from**, and naming the heap dump rather than the window is + * what makes that work. Four ways it goes, and the first is nearly always the one: * - * **A link outlives the window it was made from**, so one whose window has gone opens the heap dump it - * names — that is the whole point of naming the dump — and only a link naming a file that isn't there any - * more has nowhere to go. That gets an empty window saying so rather than silence: silence is the one - * answer that can't be told from the app having failed to start, and a link is usually followed from - * somewhere that cannot see whether this app did anything at all. + * - **A window of this run has that heap dump open.** That window is where the link goes, and nothing else + * is looked at. + * - **None has, but this machine has had it open.** The file opens in a window of its own, wherever it was + * last seen. See [HeapDumpPaths]. + * - **There are two heap dumps of that name.** Two windows on two files, or two paths on record: a link + * says nothing that tells them apart, so the reader is asked which, by path. + * - **Nothing here knows that name.** Which is a link from somebody else's machine, or about a heap dump + * opened too long ago to still be on record, or one that has been deleted: the reader is asked for the + * file. Asked, rather than left with silence — silence is the one answer that can't be told from the app + * having failed to start, and a link is usually followed from somewhere that cannot see either way. */ fun open(link: DeepLink) { - val window = windowFor(link) + val openWindows = windowsFor(link) + val openPaths = openWindows.mapNotNull { it.heapDumpFile?.normalizedPath() }.distinct() + if (openPaths.size > 1) { + // Which of them is a question only the reader can answer: nothing in the link tells the two apart, and + // guessing would be picking somebody's investigation for them. + ask(link, whichHeapDump(link, openPaths, areOpen = true), openPaths, host = openWindows.first()) + return + } + val window = openWindows.firstOrNull() if (window != null) { - SharkLog.d { "A link asked window ${window.deepLinkId} for ${link.place} of ${link.heapDumpName}" } + // The first of them when one heap dump is open in two windows, which is two readings of one file: a + // link says nothing that tells those apart either, but they show the same dump, so there is nothing + // worth asking. + SharkLog.d { "A link asked window ${window.windowId} for ${link.place} of ${link.heapDumpName}" } window.goToLinked(link.place) window.bringToFront() return } - val heapDumpFile = link.heapDumpPath?.takeIf { it.isFile } - if (heapDumpFile == null) { - SharkLog.d { "No window of this run has ${link.heapDumpName} open: opening one to say so" } - add( - ExplorerWindow( - cascade = freeCascade(), - titlePrefix = titlePrefix, - deepLinkProblem = noSuchHeapDump(link) - ) - ) + // Nothing here has it open, so where the file is comes off the link when it says, and off what this + // machine remembers opening when it doesn't. + val remembered = link.heapDumpPath?.let { listOf(it.normalizedPath()) } + ?: heapDumpPaths.pathsNamed(link.heapDumpName).map { it.normalizedPath() }.distinct() + val found = remembered.filter { it.isFile } + when { + found.size == 1 -> { + SharkLog.d { "A link asked for ${link.place} of ${found.single()}, which is not open yet" } + goToHeapDump(found.single(), link.place) + } + found.size > 1 -> ask(link, whichHeapDump(link, found, areOpen = false), found, host = null) + // Nowhere to go on its own, so the question is the whole of what this link gets: a window of its own, + // saying what is missing, over a dialog asking for the file. + else -> ask(link, noSuchHeapDump(link, remembered), choices = emptyList(), host = null) + } + } + + /** + * Puts a question about [link]'s heap dump to whoever followed it, in [host] or in a window of its own. + * + * A window already showing one of the heap dumps in question when there is one, so that asking which of + * them costs no window: the dialog is over what its reader was looking at either way. Failing that the + * question needs a window, which is also where the heap dump picked will open — and that window says the + * question in the middle of it as well, so that a question dismissed leaves the reason on screen rather + * than a window with nothing in it and nothing to explain it. + * + * Brought to the front either way, for the same reason following a link raises a window: a dialog drawn + * inside a window that is behind another one is a link that did nothing, as far as its reader can tell. + */ + private fun ask( + link: DeepLink, + question: String, + choices: List, + host: ExplorerWindow? + ) { + SharkLog.d { "A link to ${link.place} of ${link.heapDumpName} is asking: $question" } + val window = host ?: emptyWindow().also { it.deepLinkProblem = question } + window.linkedHeapDump = LinkedHeapDump( + heapDumpName = link.heapDumpName, + question = question, + choices = choices, + place = link.place + ) + window.bringToFront() + } + + /** + * What an answer to [ExplorerWindow.linkedHeapDump] does: the link goes where it was going, in the heap + * dump that was picked. + * + * [chosen] is null for a question dismissed, which is a link not followed — the window keeps the reason it + * was asked, and there is nothing else to do about it. + */ + fun chooseLinkedHeapDump( + window: ExplorerWindow, + chosen: File? + ) { + val asked = window.linkedHeapDump ?: return + window.linkedHeapDump = null + if (chosen == null) { + SharkLog.d { "Nothing was picked for ${asked.heapDumpName}, so its link goes nowhere" } return } - // The file rather than what the link called it, which for a link named by a window id is that id. - SharkLog.d { "A link asked for ${link.place} of ${heapDumpFile.name}, which is not open yet" } - goToHeapDump(heapDumpFile, link.place) + SharkLog.d { "$chosen was picked for ${asked.heapDumpName}, so its link goes to ${asked.place}" } + goToHeapDump(chosen, asked.place) } /** The first step of the cascade no window is at, which is where the next window goes. */ fun freeCascade(): Int = generateSequence(0, Int::inc).first { step -> none { it.cascade == step } } + /** + * A window with nothing in it for a link to say something in: the one this run started with when it was + * started with no heap dump, and a new one otherwise. + * + * Never one that is already asking about another link. Two links that both need an answer are two + * questions, and the second one taking this window would take the first one's question with it. + */ + private fun emptyWindow(): ExplorerWindow = + firstOrNull { it.heapDumpFile == null && it.linkedHeapDump == null } + ?: ExplorerWindow(cascade = freeCascade(), titlePrefix = titlePrefix).also { add(it) } + companion object { /** - * What an empty window opened by a link with nowhere to go says in the middle of it. + * What a link about a heap dump with more than one place to be asks, which is: which of these? * - * Two ways to get here, and they need different things done about them: a heap dump that has been moved - * or deleted since the link was made, and one this machine has no record of ever opening — which is a - * link from somebody else's machine, or one about a dump opened so long ago that where it was has been - * forgotten. Both are answered by opening the file, so both messages end by saying so. + * Rare, and worth wording rather than guessing at. Every heap dump this app takes off a device is named + * after the process, its pid and a random number, and LeakCanary names its own after the time of the + * dump — so two files of one name are a dump named by hand, or one app dumped on two devices, which are + * exactly the two cases where picking one for the reader would be picking wrong. */ - fun noSuchHeapDump(link: DeepLink): String { - val path = link.heapDumpPath - return if (path == null) { - "No heap dump called ${link.heapDumpName} is open here, and this machine has no record of opening " + - "one by that name. Open that file and follow the link again, or say where it is in the link: " + - "&${DeepLink.DUMP_PARAMETER}=/path/to/${link.heapDumpName}" - } else { - "${link.heapDumpName} is not open and there is no file at $path to open, so this link has nowhere " + - "to go. A link outlives the window it was copied from, but not the heap dump it is about." - } + fun whichHeapDump( + link: DeepLink, + choices: List, + areOpen: Boolean + ): String = if (areOpen) { + "${choices.size} heap dumps called ${link.heapDumpName} are open." + } else { + "${choices.size} heap dumps called ${link.heapDumpName} have been opened here, and none is open now." + } + + /** + * What a link about a heap dump this run cannot find says, which ends in the two ways to say where it is. + * + * Two ways to get here, and they mean different things to whoever reads it: [gone] is where this machine + * remembers the heap dump being, so an empty one is a name nothing here has ever opened — a link from + * somebody else's machine, or about a dump opened so long ago that where it was has been forgotten — and + * a full one is a heap dump moved or deleted since the link was written. See [HeapDumpPaths]. + */ + fun noSuchHeapDump( + link: DeepLink, + gone: List + ): String = if (gone.isEmpty()) { + "No heap dump called ${link.heapDumpName} is open here, and this machine has no record of opening one " + + "by that name. Choose the file, or say where it is in the link: " + + "&${DeepLink.DUMP_PARAMETER}=/path/to/${link.heapDumpName}" + } else { + "${link.heapDumpName} is not open, and there is no file at ${gone.joinToString(" or ")} any more. A " + + "link outlives the window it was copied from, but not the heap dump it is about. Choose the file if " + + "it has moved." } } } @@ -250,8 +379,12 @@ internal class ExplorerWindows( * A window per heap dump named on the command line, or one window with none — something has to carry * the button that opens the first one. */ -internal fun explorerWindows(arguments: ExplorerArguments): ExplorerWindows = - ExplorerWindows(arguments.titlePrefix).apply { +internal fun explorerWindows( + arguments: ExplorerArguments, + /** Handed in rather than made here, because a run records the heap dumps it opens into the same one. */ + heapDumpPaths: HeapDumpPaths +): ExplorerWindows = + ExplorerWindows(heapDumpPaths, arguments.titlePrefix).apply { val titlePrefix = arguments.titlePrefix if (arguments.heapDumpFiles.isEmpty()) { add(ExplorerWindow(null, titlePrefix = titlePrefix)) @@ -260,9 +393,9 @@ internal fun explorerWindows(arguments: ExplorerArguments): ExplorerWindows = add(ExplorerWindow(file, cascade = index, titlePrefix = titlePrefix)) } } - // Which window is which, for reading a link out of a log afterwards: a link carries the window it was - // copied from, and nothing else in the file says what that id stands for. - SharkLog.d { "Windows of this run: ${joinToString { "${it.deepLinkId} ${it.title}" }}" } + // Which window is which, for reading an agent's session out of a log afterwards: a call names the window + // it was answered by, and nothing else in the file says what that id stands for. + SharkLog.d { "Windows of this run: ${joinToString { "${it.windowId} ${it.title}" }}" } } /** @@ -332,24 +465,38 @@ internal fun ExplorerWindows.openHeapDump( * second one on the same file — the same rule [openHeapDump] follows, one window per heap dump — and the * window that has just been opened for it is in front already, so only an existing one is brought forward. * - * Which is also where [ExplorerWindows.open] ends up for a link whose window has gone, a link being about a - * heap dump for the same reason a row of that screen is. + * Which is also where [ExplorerWindows.open] ends up for a link about a heap dump no window has open, and + * where an answer to [ExplorerWindows.chooseLinkedHeapDump] goes: a link is about a heap dump for the same + * reason a row of that screen is. */ internal fun ExplorerWindows.goToHeapDump( heapDumpFile: File, place: Place ) { - // By absolute path, because a window opened from a command line holds the relative path it was given while - // a session recorded the absolute one, and those are the same heap dump. - val showing = firstOrNull { it.heapDumpFile?.absoluteFile == heapDumpFile.absoluteFile } + val showing = firstOrNull { it.heapDumpFile?.normalizedPath() == heapDumpFile.normalizedPath() } SharkLog.d { - val where = if (showing == null) "a window it is not open in yet" else "window ${showing.deepLinkId}" - "A row of an agent's session asked $where for $place of ${heapDumpFile.name}" + val where = if (showing == null) "a window it is not open in yet" else "window ${showing.windowId}" + "Something outside a window asked $where for $place of ${heapDumpFile.name}" } val window = showing ?: openHeapDump(heapDumpFile) window.goToLinked(place) showing?.bringToFront() } +/** + * One spelling of a heap dump's path, so that two names for one file are one heap dump: a window opened from + * a command line holds the relative path it was given, while a link, a session and [HeapDumpPaths] carry the + * absolute one. + * + * The same spelling `normalizedHeapDumpPath` gives what shark-explorer-core writes about a heap dump, copied + * rather than shared: a line of this belongs in whichever module needs it, not in a published API. + */ +private fun File.normalizedPath(): File = absoluteFile.normalize() + /** Between what a run is called and which heap dump a window shows, as elsewhere in this window. */ private const val TITLE_SEPARATOR = " · " + +private const val WINDOW_ID_LENGTH = 8 + +/** Lowercase and digits without `l`, `1`, `o` and `0`, which is what nothing can be misread in. */ +private const val WINDOW_ID_ALPHABET = "abcdefghijkmnpqrstuvwxyz23456789" diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt index 3c4cc9b159..d22ae98a4b 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeadlessAgentHeapDumps.kt @@ -120,10 +120,10 @@ internal class HeadlessAgentHeapDumps( // `window` on the surface even here, rather than growing a second word for a run that has none — what an // agent does with it is name a dump, and a vocabulary that changes with whether there is a screen is one // nobody can carry between the two. - val windowId = DeepLink.newWindowId() + val windowId = newWindowId() // Written down the same way a window's dump is, and here it is the whole of what makes the links this // hands back work: nobody watching a run with no screen can be told where the file was. - heapDumpPaths.record(windowId, file) + heapDumpPaths.record(file) val dump = HeadlessHeapDump( open = open, agent = OpenAgentHeapDump(windowId = windowId, open = open) { place -> diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt index 8d5129c57d..eaaef7c3da 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/HeapDumpExplorer.kt @@ -123,8 +123,6 @@ internal fun HeapDumpExplorer( notes: HeapDumpNotes, /** And what has been decided about its objects by hand, shared the same way. See [LeakStatusDetail]. */ leakStatuses: HeapDumpLeakStatuses, - /** What a link to one of these tabs names this window by. See [DeepLink]. */ - deepLinkId: String = remember { DeepLink.newWindowId() }, /** Places a link has asked for, opened as tabs. See [ExplorerWindow.linkedPlaces]. */ linkedPlaces: List = emptyList(), onLinkedPlaceOpened: (Place) -> Unit = {}, @@ -644,27 +642,24 @@ internal fun HeapDumpExplorer( */ val openHovered: (OpenIn) -> Unit = { openIn -> hovered?.place?.let { open(it, openIn) } } /** - * Where every "copy link" in this window ends up, for the same reason [open] is one function: a link to a - * rectangle, a row, a field, a button and a tab is one thing, and five of them would drift. + * Where every "copy link" about another heap dump ends up, which the *Agent logs* screens are full of: a + * link is a heap dump and a place, so a row about a dump this window hasn't got is something to send as + * well as something to click. See [shark.explorer.DeepLink]. */ - val copyLink: (Place) -> Unit = { destination -> - val link = DeepLink(session.heapDumpFile, destination, windowId = deepLinkId).toUri() + val copyHeapDumpLink: (File, Place) -> Unit = { heapDumpFile, destination -> + val link = DeepLink(heapDumpFile, destination).toUri() // In the log as well as on the clipboard, so that a link someone reports as not working can be compared // against the one this window actually handed out. SharkLog.d { "Copied $link" } copyToClipboard(link) } /** - * And for a place of a heap dump this window hasn't got, which the *Agent logs* screens are full of. - * - * No window id on it: this window is not one of that dump's, so there is no window to prefer and the link - * is the file plus the place — which is all a link needs, and is why a row about somebody else's dump is - * something to send rather than only something to click. See [shark.explorer.DeepLink]. + * And for this window's own heap dump, which is every other "copy link" there is, for the same reason + * [open] is one function: a link to a rectangle, a row, a field, a button and a tab is one thing, and five + * of them would drift. */ - val copyHeapDumpLink: (File, Place) -> Unit = { heapDumpFile, destination -> - val link = DeepLink(heapDumpFile, destination).toUri() - SharkLog.d { "Copied $link" } - copyToClipboard(link) + val copyLink: (Place) -> Unit = { destination -> + copyHeapDumpLink(session.heapDumpFile, destination) } /** The same, for everything that names an object by its id. */ val copyObjectLink: (Long) -> Unit = { objectId -> copyLink(Place.Object(objectId)) } diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/LinkedHeapDumpDialog.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/LinkedHeapDumpDialog.kt new file mode 100644 index 0000000000..52aa37301c --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/LinkedHeapDumpDialog.kt @@ -0,0 +1,87 @@ +package shark.explorer.app + +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.heightIn +import androidx.compose.foundation.rememberScrollState +import androidx.compose.foundation.verticalScroll +import androidx.compose.material3.AlertDialog +import androidx.compose.material3.HorizontalDivider +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.Text +import androidx.compose.material3.TextButton +import androidx.compose.runtime.Composable +import androidx.compose.ui.Modifier +import androidx.compose.ui.unit.dp +import java.io.File + +/** + * Asks which heap dump a link is about, or where the one it names is — see [LinkedHeapDump]. + * + * One dialog for both because the answer is the same either way: a path. So the rows are the paths this + * machine knows of, and the button under them is the file picker for a heap dump that is at neither — or, + * where there are no rows at all, for the one thing left to try. + * + * The rows are directories rather than paths. Everything being picked between has the file name the link + * says, so the name on every row would be the same word repeated down the dialog with the answer hidden + * inside it, and it is in the title anyway. + */ +@Composable +internal fun LinkedHeapDumpDialog( + asked: LinkedHeapDump, + /** The platform file picker, as anywhere else a heap dump is chosen. Overridden by tests. */ + chooseHeapDumpFile: () -> File?, + /** The heap dump picked, or null for a question dismissed. See [ExplorerWindows.chooseLinkedHeapDump]. */ + onChosen: (File?) -> Unit +) { + AlertDialog( + onDismissRequest = { onChosen(null) }, + title = { + Text( + if (asked.choices.isEmpty()) { + whereIsHeapDumpTitle(asked.heapDumpName) + } else { + whichHeapDumpTitle(asked.heapDumpName) + } + ) + }, + text = { + Column( + Modifier.heightIn(max = DIALOG_MAX_HEIGHT), + verticalArrangement = Arrangement.spacedBy(8.dp) + ) { + Text(asked.question, style = MaterialTheme.typography.bodyMedium) + if (asked.choices.isNotEmpty()) { + HorizontalDivider() + // Scrolled, and only this part of it: what is being asked stays put while the places go past. + Column(Modifier.weight(1f, fill = false).verticalScroll(rememberScrollState())) { + asked.choices.forEach { path -> + PickerRow(name = path.parent ?: path.path, onClick = { onChosen(path) }) + } + } + } + } + }, + confirmButton = { + // A file at neither of the places offered, which for a link from another machine is every heap dump. + // Not worded like the button in the bar behind this, which opens a heap dump without answering the + // question: two buttons saying `Open heap dump…` with one of them doing something else is a trap. + TextButton(onClick = { chooseHeapDumpFile()?.let(onChosen) }) { + Text(CHOOSE_HEAP_DUMP_FILE) + } + }, + dismissButton = { + TextButton(onClick = { onChosen(null) }) { + Text(CANCEL_LINK) + } + } + ) +} + +internal fun whichHeapDumpTitle(heapDumpName: String): String = "Which $heapDumpName?" + +internal fun whereIsHeapDumpTitle(heapDumpName: String): String = "Where is $heapDumpName?" + +internal const val CHOOSE_HEAP_DUMP_FILE = "Choose file…" + +internal const val CANCEL_LINK = "Cancel" diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt index 7ca2a3dd04..d8fbce0d25 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/Main.kt @@ -7,6 +7,7 @@ import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.widthIn import androidx.compose.material3.Button import androidx.compose.material3.CircularProgressIndicator import androidx.compose.material3.MaterialTheme @@ -24,6 +25,7 @@ import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.res.painterResource +import androidx.compose.ui.text.style.TextAlign import androidx.compose.ui.unit.dp import androidx.compose.ui.window.Window import androidx.compose.ui.window.WindowPosition @@ -84,7 +86,9 @@ fun main(args: Array) { // reads as one on the line above. SharkLog.d { "Read that as $arguments" } nameThisRun(arguments.titlePrefix ?: APP_NAME) - val windows = explorerWindows(arguments) + // With where this machine's heap dumps are, which is what a link naming one this run has not opened is + // looked up in. One per run, and the same one every window records into. See [HeapDumpPaths]. + val windows = explorerWindows(arguments, explorerHeapDumpPaths()) // One per run rather than one per window, because an agent has no window: opening a heap dump and taking // one off a device are what make a window, so whatever does them has to outlive every window there is. val deviceHeapDumps = commandLineDeviceHeapDumps() @@ -96,8 +100,8 @@ fun main(args: Array) { // Published before the first window too, so that an agent whose client started it while the heap // dumps were still opening finds this run and waits for a dump rather than finding nothing. listenForAgents(windows, deviceHeapDumps).use { - // Whatever no other run claimed, which for a link naming a window that has gone is an empty window - // saying so. Ours to answer for now: nobody else is going to. + // Whatever no other run claimed, which for a link about a heap dump nothing has open is this run + // opening it, or asking where it is. Ours to answer for now: nobody else is going to. DeepLinkPeers.deliver(arguments.deepLinks).forEach { windows.open(it) } // Heap dump paths on the command line open straight away, which is how this is usually run. explorerApplication(windows, deviceHeapDumps) @@ -164,9 +168,6 @@ private fun explorerApplication( // And one set of statuses set by hand per heap dump, for the same reason: a status is a conclusion about // the dump rather than about a window, so both windows on one dump read the heap through the same ones. val leakStatuses = remember { ExplorerLeakStatuses() } - // And where the heap dumps of every window are written down, so that a link into one of them works after - // this run has gone — which is what lets a link name the dump without carrying its path. - val heapDumpPaths = remember { explorerHeapDumpPaths() } // Once per run, not once per window, and off the UI thread: this is a network request, and a window that // waits for GitHub to answer before it draws is a window that hangs when GitHub is unreachable. LaunchedEffect(updateNotice) { @@ -214,11 +215,11 @@ private fun explorerApplication( onHeapDumpOpen = { open -> window.openHeapDump = open // Once it is open rather than as the window is given the file: a dump that turns out not to be - // one is not a heap dump for a link to be sent to. See [HeapDumpPaths]. - open?.let { heapDumpPaths.record(window.deepLinkId, it.session.heapDumpFile) } + // one is not a heap dump for a link to be sent to. Which is also what makes a link work after + // this run has gone, since it is where the path is written down. See [HeapDumpPaths]. + open?.let { windows.heapDumpPaths.record(it.session.heapDumpFile) } }, onHeapDumpProblem = { problem -> window.openProblem = problem }, - deepLinkId = window.deepLinkId, // The same way a link arriving from the OS is followed, which is what makes a `shark://` link // written in a note work wherever it is read from. followDeepLink = { link -> DeepLinkPeers.follow(link, windows) }, @@ -228,6 +229,10 @@ private fun explorerApplication( linkedPlaces = window.linkedPlaces, onLinkedPlaceOpened = { place -> window.linkedPlaceOpened(place) }, deepLinkProblem = window.deepLinkProblem, + // And what a link is waiting to be told about the heap dump it names: which of the ones called + // that, or where it is. See [ExplorerWindows.open]. + linkedHeapDump = window.linkedHeapDump, + onLinkedHeapDumpChosen = { chosen -> windows.chooseLinkedHeapDump(window, chosen) }, // The run's rather than this window's, because an agent reaches the same one through no window // at all — and because two windows asking `adb` at once is two `adb` processes. deviceHeapDumps = deviceHeapDumps @@ -287,8 +292,6 @@ internal fun ExplorerApp( * under whoever is running it would be a test of their investigations rather than of this window. */ agentSessions: () -> List = ::agentSessions, - /** What a link to a place in this window names it by. See [shark.explorer.DeepLink]. */ - deepLinkId: String = remember { DeepLink.newWindowId() }, /** Places a link has asked this window for, which its tabs open. See [ExplorerWindow.linkedPlaces]. */ linkedPlaces: List = emptyList(), onLinkedPlaceOpened: (Place) -> Unit = {}, @@ -311,6 +314,15 @@ internal fun ExplorerApp( * way, and the two buttons above are what to do about it in both cases. */ deepLinkProblem: String? = null, + /** + * What a link is waiting to be told about the heap dump it names, and null while nothing is being asked. + * + * A dialog rather than something in the window, because it is a question with an answer only the reader + * has: which of the heap dumps called that, or where the one called that is. See [LinkedHeapDumpDialog]. + */ + linkedHeapDump: LinkedHeapDump? = null, + /** The heap dump picked for it, or null for a question dismissed. See [ExplorerWindows.open]. */ + onLinkedHeapDumpChosen: (File?) -> Unit = {}, /** Overridden by tests, which have no system clipboard and want to read what would have been copied. */ copyToClipboard: (String) -> Unit = ::copyTextToClipboard, /** Overridden by tests, which have no browser to open a link written in the notes in. */ @@ -388,6 +400,15 @@ internal fun ExplorerApp( ) } + if (linkedHeapDump != null) { + LinkedHeapDumpDialog( + asked = linkedHeapDump, + // The same picker the button in the bar opens: choosing a heap dump is choosing a heap dump. + chooseHeapDumpFile = chooseHeapDumpFile, + onChosen = onLinkedHeapDumpChosen + ) + } + Column(Modifier.fillMaxSize()) { // Above the heap dump bar, because it is about the app rather than about what is open in it, and // because a bar that pushes the map down is one nobody can miss and nobody has to act on. @@ -417,7 +438,6 @@ internal fun ExplorerApp( notes = notes.of(currentState.session.heapDumpFile), leakStatuses = leakStatuses.of(currentState.session.heapDumpFile), agentSessions = agentSessions, - deepLinkId = deepLinkId, linkedPlaces = linkedPlaces, onLinkedPlaceOpened = onLinkedPlaceOpened, followDeepLink = followDeepLink, @@ -429,18 +449,23 @@ internal fun ExplorerApp( } else { Box(Modifier.weight(1f).fillMaxWidth(), contentAlignment = Alignment.Center) { Column( + // Narrow enough for a sentence to wrap into lines the eye can come back to, since what a link + // asking for a heap dump says is longer than the words the rest of this window puts here. + Modifier.widthIn(max = CENTER_MESSAGE_MAX_WIDTH), horizontalAlignment = Alignment.CenterHorizontally, verticalArrangement = Arrangement.spacedBy(16.dp) ) { if (currentState is HeapDumpState.Opening) { CircularProgressIndicator() } - // A window a link opened to say the window it named has gone says that instead of the invitation - // to open a heap dump: it was opened to carry a message, and the invitation is under it anyway. + // A window a link opened to ask which heap dump it meant, or where the one it named is, says that + // instead of the invitation to open a heap dump: it was opened to carry the question, and the + // invitation is under it anyway. See [ExplorerWindows.ask]. Text( deepLinkProblem?.takeIf { currentState is HeapDumpState.None } ?: currentState.centerMessage(), - style = MaterialTheme.typography.bodyLarge + style = MaterialTheme.typography.bodyLarge, + textAlign = TextAlign.Center ) } } @@ -613,6 +638,9 @@ internal val WINDOW_HEIGHT = 900.dp /** How far a window opens from the one before it, which is about the height of a title bar. */ private val CASCADE_STEP = 28.dp +/** As wide as what a window with nothing in it says gets, which is a line length rather than a window. */ +private val CENTER_MESSAGE_MAX_WIDTH = 480.dp + internal const val OPEN_HEAP_DUMP = "Open heap dump…" internal const val NO_HEAP_DUMP = "Open an Android heap dump to see what retains its memory." diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerAppTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerAppTest.kt index 6bef635e41..e157077aca 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerAppTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerAppTest.kt @@ -224,7 +224,7 @@ class ExplorerAppTest { // same move made somewhere else, so wherever one is offered so is the other. Named after the heap // dump, with this window as the refinement it is while the window is open. See [DeepLink]. assertThat(copied).containsExactly( - DeepLink(heapDumpFile, Place.Object(payloadObjectId), windowId = WINDOW_ID).toUri() + DeepLink(heapDumpFile, Place.Object(payloadObjectId)).toUri() ) } } @@ -801,7 +801,6 @@ class ExplorerAppTest { var shown: File? by remember { mutableStateOf(heapDumpFile) } ExplorerApp( heapDumpFile = shown, - deepLinkId = WINDOW_ID, // No pixels to keep track of: nothing here takes a dump off a device, which is the only way // any come with one. onHeapDumpChosen = { file, _ -> shown = file }, @@ -1019,9 +1018,6 @@ class ExplorerAppTest { /** Opening a heap dump and rebuilding a tree both happen on another thread. */ private const val OPEN_TIMEOUT_MILLIS = 10_000L - /** What a link copied here names this window by, fixed so that the copied link can be spelled out. */ - private const val WINDOW_ID = "abcd2345" - /** How the log says a treemap was laid out, and what it calls the node at the top of the tree. */ private const val TREEMAP_LAID_OUT = "Read the treemap rooted at" private const val WHOLE_HEAP_DUMP_NODE = "the whole heap dump" diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt index 49ddd67e35..fac46b31e3 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt @@ -12,6 +12,7 @@ import shark.explorer.Adb import shark.explorer.AdbOutput import shark.explorer.DeepLink import shark.explorer.DeviceHeapDumps +import shark.explorer.HeapDumpPaths import shark.explorer.Place import shark.explorer.agent.AgentRefusal @@ -31,6 +32,9 @@ class ExplorerWindowTest { /** For the one thing a link needs a real file for: opening a heap dump no window has. */ @get:Rule val temporaryFolder = TemporaryFolder() + /** Where this test's runs remember the heap dumps they opened, which is never this machine's own. */ + private val heapDumpPaths by lazy { HeapDumpPaths(temporaryFolder.newFolder("heap-dump-paths")) } + @Test fun `an app started with no heap dump has one window to open one from`() { val windows = explorerWindows(noHeapDumps()) @@ -132,9 +136,9 @@ class ExplorerWindowTest { @Test fun `every window answers to an id of its own`() { val windows = explorerWindows(opening(FIRST_DUMP, FIRST_DUMP)) - // The whole reason a link says which window as well as which heap dump: the same dump open twice is two - // places to be, and while both are open a link leads to the one it was copied from. - assertThat(windows.map { it.deepLinkId }).doesNotHaveDuplicates() + // Which is what an agent calls a window, and the same heap dump open twice is what it is for: two windows + // on one file are two places to be told about. Never in a link, which names the heap dump. + assertThat(windows.map { it.windowId }).doesNotHaveDuplicates() } @Test fun `a link goes to the window of the heap dump it names and to no other`() { @@ -148,33 +152,6 @@ class ExplorerWindowTest { assertThat(windows).hasSize(2) } - /** - * The case the window id is there for, and the only one: which of two readings of one dump. Both windows - * answer to the heap dump, so without it a link would land on whichever came first. - */ - @Test fun `a link to one of two windows on the same heap dump goes to that one`() { - val windows = explorerWindows(opening(FIRST_DUMP, FIRST_DUMP)) - val (first, second) = windows - - windows.open(DeepLink(FIRST_DUMP, Place.Starred, windowId = second.deepLinkId)) - - assertThat(second.linkedPlaces).containsExactly(Place.Starred) - assertThat(first.linkedPlaces).isEmpty() - } - - /** - * Which is most links a day later: the run they were copied from has been closed and started again, and - * every window id it handed out went with it. - */ - @Test fun `a link whose window has gone goes to a window of its heap dump`() { - val windows = explorerWindows(opening(FIRST_DUMP)) - - windows.open(DeepLink(FIRST_DUMP, Place.Starred, windowId = CLOSED_WINDOW_ID)) - - assertThat(windows.single().linkedPlaces).containsExactly(Place.Starred) - assertThat(windows).hasSize(1) - } - /** A link somebody typed or shortened, which names the dump the way a person would. */ @Test fun `a link with a file name and no path goes to the window of that file`() { val windows = explorerWindows(opening(FIRST_DUMP)) @@ -185,17 +162,18 @@ class ExplorerWindowTest { } /** - * Links written before a link named a heap dump are on disk in notes and in agent sessions, and one of - * those pasted back into the run it came from still goes where it went. + * Two readings of one heap dump, which is one heap dump: a link says nothing that tells them apart, and + * they are showing the same file, so there is nothing worth asking about. */ - @Test fun `a link that names only a window id still finds that window`() { - val windows = explorerWindows(opening(FIRST_DUMP)) - val window = windows.single() + @Test fun `a link to a heap dump open in two windows goes to the first of them`() { + val windows = explorerWindows(opening(FIRST_DUMP, FIRST_DUMP)) + val (first, second) = windows - windows.open(DeepLink.parse("shark://${window.deepLinkId}/leaks")) + windows.open(DeepLink(FIRST_DUMP, Place.Starred)) - assertThat(window.linkedPlaces).containsExactly(Place.Leaks()) - assertThat(windows).hasSize(1) + assertThat(first.linkedPlaces).containsExactly(Place.Starred) + assertThat(second.linkedPlaces).isEmpty() + assertThat(windows.mapNotNull { it.linkedHeapDump }).isEmpty() } @Test fun `a place a link asked for is dropped once a tab has opened it`() { @@ -222,21 +200,34 @@ class ExplorerWindowTest { /** * The payoff of a link naming the heap dump: the run it was copied from can be gone, and the link still - * puts the reader in front of what it names. + * puts the reader in front of what it names, because opening the file wrote down where it was. */ - @Test fun `a link to a heap dump nothing has open opens it`() { + @Test fun `a link to a heap dump this machine has opened before opens it`() { val heapDumpFile = temporaryFolder.newFile("third.hprof") + heapDumpPaths.record(heapDumpFile) val windows = explorerWindows(opening(FIRST_DUMP)) - windows.open(lookedUp(heapDumpFile, Place.Starred, windowId = CLOSED_WINDOW_ID)) + windows.open(DeepLink(heapDumpFile, Place.Starred)) val opened = windows.last() assertThat(windows).hasSize(2) assertThat(opened.heapDumpFile).isEqualTo(heapDumpFile.absoluteFile) assertThat(opened.linkedPlaces).containsExactly(Place.Starred) + assertThat(windows.mapNotNull { it.linkedHeapDump }).isEmpty() + } + + /** The one case a file name cannot answer, and the reason a link can still carry a path. */ + @Test fun `a link that says where the heap dump is opens it without looking anything up`() { + val heapDumpFile = temporaryFolder.newFile("fourth.hprof") + val windows = explorerWindows(opening(FIRST_DUMP)) + + windows.open(lookedUp(heapDumpFile, Place.Starred)) + + assertThat(windows.last().heapDumpFile).isEqualTo(heapDumpFile.absoluteFile) + assertThat(windows.last().linkedPlaces).containsExactly(Place.Starred) } - @Test fun `a link to a heap dump that has been deleted opens a window saying so`() { + @Test fun `a link to a heap dump that has been deleted asks where it is`() { val windows = explorerWindows(opening(FIRST_DUMP)) windows.open(lookedUp(SECOND_DUMP, Place.Starred)) @@ -244,27 +235,110 @@ class ExplorerWindowTest { // Rather than nothing at all, which is the one answer that can't be told from the app having failed // to start — and a link is usually followed from somewhere that can't see either way. assertThat(windows).hasSize(2) - assertThat(windows.last().deepLinkProblem) + val asked = windows.last() + assertThat(asked.deepLinkProblem) .contains(SECOND_DUMP.name) .contains(SECOND_DUMP.absolutePath) - assertThat(windows.last().heapDumpFile).isNull() + assertThat(asked.heapDumpFile).isNull() + // With nothing to pick from, so the question is where the file is rather than which of them it is. + assertThat(asked.linkedHeapDump?.choices).isEmpty() + assertThat(asked.linkedHeapDump?.place).isEqualTo(Place.Starred) assertThat(logged).anyMatch { SECOND_DUMP.name in it } } /** * A link about a heap dump this machine has no record of ever opening, which is one from somebody else's - * machine: nothing looked its path up, because there was nothing to look up. See [HeapDumpPaths]. + * machine: there was nothing here to look its path up in. See [HeapDumpPaths]. */ @Test fun `a link to a heap dump nothing knows where to find says what is missing`() { val windows = explorerWindows(opening(FIRST_DUMP)) windows.open(DeepLink.parse("shark://${SECOND_DUMP.name}/starred")) - assertThat(windows.last().deepLinkProblem) + val asked = windows.last() + assertThat(asked.deepLinkProblem) .contains(SECOND_DUMP.name) .contains("no record of opening one by that name") // And what to type instead, since a link from another machine can carry the path. .contains("&dump=/path/to/${SECOND_DUMP.name}") + // The same sentence in the dialog that asks for the file, so the question and the reason for it are one. + assertThat(asked.linkedHeapDump?.question).isEqualTo(asked.deepLinkProblem) + } + + /** + * Which nothing can answer for the reader: two dumps of one name are an app dumped on two devices, or a + * dump copied somewhere, and picking one would be picking somebody's investigation for them. + */ + @Test fun `a link about a name two open windows share asks which of them`() { + val pixel = temporaryFolder.newFolder("pixel").resolve("app.hprof") + val emulator = temporaryFolder.newFolder("emulator").resolve("app.hprof") + val windows = explorerWindows(opening(pixel, emulator)) + + windows.open(DeepLink.parse("shark://app.hprof/starred")) + + // In the window of the first of them rather than in a window of its own: the question is a dialog over + // what its reader was looking at either way, and a third window would be one nobody asked for. + val asked = windows.first().linkedHeapDump + assertThat(windows).hasSize(2) + assertThat(asked?.choices).containsExactly(pixel.absoluteFile, emulator.absoluteFile) + assertThat(asked?.question).contains("2 heap dumps called app.hprof are open") + assertThat(windows.none { it.linkedPlaces.isNotEmpty() }).isTrue() + } + + @Test fun `a link about a name two heap dumps on record share asks which of them`() { + val pixel = temporaryFolder.newFolder("pixel").resolve("app.hprof").apply { writeText("") } + val emulator = temporaryFolder.newFolder("emulator").resolve("app.hprof").apply { writeText("") } + heapDumpPaths.record(pixel) + heapDumpPaths.record(emulator) + val windows = explorerWindows(opening(FIRST_DUMP)) + + windows.open(DeepLink.parse("shark://app.hprof/starred")) + + // A window of its own this time, since no window here is showing either of them: it is where whichever + // one is picked will open, and it says why it is empty in the meantime. + val asked = windows.last() + assertThat(asked.linkedHeapDump?.choices) + .containsExactlyInAnyOrder(pixel.absoluteFile, emulator.absoluteFile) + assertThat(asked.deepLinkProblem).contains("have been opened here, and none is open now") + } + + @Test fun `the heap dump picked for a link is where the link goes`() { + val heapDumpFile = temporaryFolder.newFile("picked.hprof") + val windows = explorerWindows(noHeapDumps()) + windows.open(DeepLink.parse("shark://picked.hprof/leaks")) + val asked = windows.single() + + windows.chooseLinkedHeapDump(asked, heapDumpFile) + + // The window that asked, since it had nothing in it, and the place the link was going all along. + assertThat(asked.heapDumpFile).isEqualTo(heapDumpFile.absoluteFile) + assertThat(asked.linkedPlaces).containsExactly(Place.Leaks()) + assertThat(asked.linkedHeapDump).isNull() + } + + @Test fun `a question dismissed leaves the reason it was asked on screen`() { + val windows = explorerWindows(noHeapDumps()) + windows.open(DeepLink.parse("shark://picked.hprof/leaks")) + val asked = windows.single() + + windows.chooseLinkedHeapDump(asked, chosen = null) + + // A link not followed, which is the reader's answer: the dialog goes and what it was about stays. + assertThat(asked.linkedHeapDump).isNull() + assertThat(asked.heapDumpFile).isNull() + assertThat(asked.deepLinkProblem).contains("picked.hprof") + assertThat(logged).anyMatch { "Nothing was picked" in it } + } + + /** Two links with nowhere to go are two questions, and the second must not take the first one's window. */ + @Test fun `a second link that needs an answer gets a window of its own`() { + val windows = explorerWindows(noHeapDumps()) + + windows.open(DeepLink.parse("shark://one.hprof/leaks")) + windows.open(DeepLink.parse("shark://two.hprof/starred")) + + assertThat(windows.mapNotNull { it.linkedHeapDump?.heapDumpName }) + .containsExactly("one.hprof", "two.hprof") } @Test fun `a window opened by a link that found nothing lands beside the others`() { @@ -320,9 +394,12 @@ class ExplorerWindowTest { */ @Test fun `a run claims a link only for a heap dump it has open`() { val windows = explorerWindows(opening(FIRST_DUMP)) + heapDumpPaths.record(temporaryFolder.newFile(SECOND_DUMP.name)) - assertThat(windows.windowFor(DeepLink(FIRST_DUMP, Place.Starred))).isEqualTo(windows.single()) - assertThat(windows.windowFor(DeepLink(SECOND_DUMP, Place.Starred))).isNull() + assertThat(windows.windowsFor(DeepLink(FIRST_DUMP, Place.Starred))).containsExactly(windows.single()) + // On record here and not on screen here, which is a link for whoever has it open — and this run's to + // answer for only once nobody else has claimed it. + assertThat(windows.windowsFor(DeepLink(SECOND_DUMP, Place.Starred))).isEmpty() } @Test fun `an agent asking for a heap dump a window already has gets that window`() { @@ -338,7 +415,7 @@ class ExplorerWindowTest { // A second window on it would be a second index of the same gigabyte, and a window nobody asked for — // unlike the button above the map, where a person opening one dump twice is comparing two readings of it. assertThat(windows).hasSize(1) - assertThat(logged).anyMatch { "already has" in it && window.deepLinkId in it } + assertThat(logged).anyMatch { "already has" in it && window.windowId in it } } @Test fun `an agent asking for a heap dump nobody has open gets a window of its own`() { @@ -363,18 +440,16 @@ class ExplorerWindowTest { ) /** - * A link as [ExplorerWindows] is handed one: where the heap dump is has been looked up already, by the run - * that took the link off the OS. A link itself says the dump's file name and no more — see [HeapDumpPaths]. + * A link that says where the heap dump is, which is one written by hand about a dump this machine has never + * opened. Every link this app writes says the file name and no more — see [HeapDumpPaths]. */ private fun lookedUp( heapDumpFile: File, - place: Place, - windowId: String? = null + place: Place ) = DeepLink( heapDumpName = heapDumpFile.name, place = place, - heapDumpPath = heapDumpFile.absoluteFile.normalize(), - windowId = windowId + heapDumpPath = heapDumpFile.absoluteFile.normalize() ) private fun noHeapDumps(titlePrefix: String? = null) = @@ -385,15 +460,21 @@ class ExplorerWindowTest { titlePrefix: String? = null ) = ExplorerArguments(heapDumpFiles = heapDumpFiles.toList(), titlePrefix = titlePrefix) + /** + * The run's windows, with a record of this machine's heap dumps that belongs to this test. + * + * Never the real one: what a link about a dump no window has open does is look in it, so a test reading the + * directory under whoever is running it would pass or fail on which heap dumps they last opened. + */ + private fun explorerWindows(arguments: ExplorerArguments) = + explorerWindows(arguments, heapDumpPaths) + companion object { /** Never opened, so these don't have to exist. */ private val FIRST_DUMP = File("first.hprof") private val SECOND_DUMP = File("second.hprof") private const val TITLE = "Hover previews" - /** Shaped like one this run could have handed out, and belonging to no window of it. */ - private const val CLOSED_WINDOW_ID = "qrst6789" - /** Long enough for a window to be added and short enough not to be a pause anybody notices. */ private const val WAIT_MILLIS = 200L } diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt index 2d7f01b7b8..d9746cdf74 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/HeadlessAgentHeapDumpsTest.kt @@ -93,9 +93,8 @@ class HeadlessAgentHeapDumpsTest { val link = DeepLink.parse(shown.link!!) assertThat(link.heapDumpName).isEqualTo(file.name) assertThat(link.place).isEqualTo(Place.Leaks()) - assertThat(link.windowId).isNull() assertThat(link.heapDumpPath).isNull() - assertThat(HeapDumpPaths(paths).resolve(link).heapDumpPath).isEqualTo(file.absoluteFile) + assertThat(HeapDumpPaths(paths).pathsNamed(link.heapDumpName)).containsExactly(file.absoluteFile) } } diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LinkedHeapDumpTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LinkedHeapDumpTest.kt new file mode 100644 index 0000000000..86097f1080 --- /dev/null +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/LinkedHeapDumpTest.kt @@ -0,0 +1,138 @@ +package shark.explorer.app + +import androidx.compose.material3.MaterialTheme +import androidx.compose.ui.test.ComposeUiTest +import androidx.compose.ui.test.ExperimentalTestApi +import androidx.compose.ui.test.hasText +import androidx.compose.ui.test.onAllNodesWithText +import androidx.compose.ui.test.onNodeWithText +import androidx.compose.ui.test.performClick +import androidx.compose.ui.test.v2.runComposeUiTest +import androidx.compose.ui.test.waitUntilAtLeastOneExists +import java.io.File +import org.assertj.core.api.Assertions.assertThat +import org.junit.Test +import shark.explorer.Place + +/** + * What a link that couldn't be followed on its own asks, and what the answer does. See [LinkedHeapDump]. + * + * The question itself is [ExplorerWindows]' — which link gets asked what is `ExplorerWindowTest` — so what + * is here is the dialog: that both questions can be answered, and that the answer is the path the link then + * opens. + */ +@OptIn(ExperimentalTestApi::class) +class LinkedHeapDumpTest { + + private val chosen = mutableListOf() + + @Test fun `a link about two heap dumps of one name offers the places they are in`() { + runComposeUiTest { + setContentAsking( + LinkedHeapDump( + heapDumpName = HEAP_DUMP_NAME, + question = "2 heap dumps called $HEAP_DUMP_NAME are open.", + choices = listOf(PIXEL_DUMP, EMULATOR_DUMP), + place = Place.Starred + ) + ) + + waitUntilAtLeastOneExists(hasText(whichHeapDumpTitle(HEAP_DUMP_NAME)), TIMEOUT_MILLIS) + // The directories, since the file name is the same on every row and is in the title above them. + onNodeWithText(EMULATOR_DUMP.parent).performClick() + + assertThat(chosen).containsExactly(EMULATOR_DUMP) + } + } + + @Test fun `a link about a heap dump nothing can find asks for the file`() { + runComposeUiTest { + setContentAsking( + LinkedHeapDump( + heapDumpName = HEAP_DUMP_NAME, + question = "No heap dump called $HEAP_DUMP_NAME is open here.", + choices = emptyList(), + place = Place.Starred + ), + chooseHeapDumpFile = { PIXEL_DUMP } + ) + + waitUntilAtLeastOneExists(hasText(whereIsHeapDumpTitle(HEAP_DUMP_NAME)), TIMEOUT_MILLIS) + // Nothing to pick between, so the file picker is the whole of the answer. + onNodeWithText(CHOOSE_HEAP_DUMP_FILE).performClick() + + assertThat(chosen).containsExactly(PIXEL_DUMP) + } + } + + @Test fun `why it is asking is said in the dialog and behind it`() { + val question = "No heap dump called $HEAP_DUMP_NAME is open here." + runComposeUiTest { + setContentAsking( + LinkedHeapDump( + heapDumpName = HEAP_DUMP_NAME, + question = question, + choices = emptyList(), + place = Place.Starred + ) + ) + + waitUntilAtLeastOneExists(hasText(question), TIMEOUT_MILLIS) + + // Once in the dialog and once in the middle of the window under it, which is what a question + // dismissed leaves on screen: a window with nothing in it and no reason for it is worse. + assertThat(onAllNodesWithText(question).fetchSemanticsNodes()).hasSize(2) + } + } + + @Test fun `a question dismissed picks nothing`() { + runComposeUiTest { + setContentAsking( + LinkedHeapDump( + heapDumpName = HEAP_DUMP_NAME, + question = "2 heap dumps called $HEAP_DUMP_NAME are open.", + choices = listOf(PIXEL_DUMP, EMULATOR_DUMP), + place = Place.Starred + ) + ) + + waitUntilAtLeastOneExists(hasText(CANCEL_LINK), TIMEOUT_MILLIS) + onNodeWithText(CANCEL_LINK).performClick() + + // Answered, and the answer is "not this link": null is what closes the question. See + // [ExplorerWindows.chooseLinkedHeapDump]. + assertThat(chosen).containsExactly(null) + } + } + + /** + * A window with no heap dump in it, being asked about one: which is where a question with nothing to pick + * from is put, since the heap dump picked opens in that window. See [ExplorerWindows.open]. + */ + private fun ComposeUiTest.setContentAsking( + asked: LinkedHeapDump, + chooseHeapDumpFile: () -> File? = { null } + ) = setContent { + MaterialTheme { + ExplorerApp( + heapDumpFile = null, + onHeapDumpChosen = { _, _ -> }, + deepLinkProblem = asked.question, + linkedHeapDump = asked, + onLinkedHeapDumpChosen = { chosen += it }, + chooseHeapDumpFile = chooseHeapDumpFile + ) + } + } + + private companion object { + /** One name, two heap dumps: an app dumped on two devices, which is what a link cannot tell apart. */ + const val HEAP_DUMP_NAME = "com.example.hprof" + + val PIXEL_DUMP = File("/dumps/pixel/$HEAP_DUMP_NAME") + val EMULATOR_DUMP = File("/dumps/emulator/$HEAP_DUMP_NAME") + + /** Long enough for the dialog to be composed, and it draws nothing that has to be read off disk. */ + const val TIMEOUT_MILLIS = 5_000L + } +} diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/NoteSectionTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/NoteSectionTest.kt index 1506d159fd..a4448f1ba6 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/NoteSectionTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/NoteSectionTest.kt @@ -185,15 +185,15 @@ class NoteSectionTest { startNote() // Starred rather than the leaks or the object list, because the bar's button for those says the same // word as the link would: "★ 0 starred" doesn't, so this text is the note's and nothing else. - write("shark://$WINDOW_ID/starred") + write("shark://$LINKED_HEAP_DUMP/starred") save() waitUntilAtLeastOneExists(hasText(Place.STARRED_LABEL), RENDER_TIMEOUT_MILLIS) onNodeWithText(Place.STARRED_LABEL).performClick() - // Handed to whatever routes links rather than opened here: a link names one window of one run, and - // which window that is, is not a question this one can answer. See [DeepLinkPeers.follow]. - assertThat(followed).containsExactly(DeepLink(WINDOW_ID, Place.Starred)) + // Handed to whatever routes links rather than opened here: a link names a heap dump, and where that + // heap dump is open is not a question one window can answer. See [DeepLinkPeers.follow]. + assertThat(followed).containsExactly(DeepLink(LINKED_HEAP_DUMP, Place.Starred)) } } @@ -358,7 +358,6 @@ class NoteSectionTest { MaterialTheme { ExplorerApp( heapDumpFile = heapDumpFile, - deepLinkId = WINDOW_ID, // A directory of this test's, never `~/.shark-explorer`: a test that saved into the real one // would write into the notes of whoever is running it. notes = ExplorerNotes(notesRoot), @@ -474,8 +473,8 @@ class NoteSectionTest { /** Saving is a file written, on another thread. */ private const val SAVE_TIMEOUT_MILLIS = 10_000L - /** What a link here names this window by, fixed so that a link can be spelled out in a test. */ - private const val WINDOW_ID = "abcd2345" + /** The heap dump a link written in a note names, which is not the one this window has open. */ + private const val LINKED_HEAP_DUMP = "another.hprof" private val NO_DEVICE_ADB = Adb { AdbOutput(exitCode = 0, text = "List of devices attached\n") } } diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ObjectsScreenTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ObjectsScreenTest.kt index 37d79101b1..a3d2a69190 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ObjectsScreenTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ObjectsScreenTest.kt @@ -158,7 +158,7 @@ class ObjectsScreenTest { // step apart, and a link is how the object leaves this window at all. It names the heap dump, so it // outlives the window it was copied from. See [DeepLink]. assertThat(copied).containsExactly( - DeepLink(heapDumpFile, Place.Object(payloadObjectId), windowId = WINDOW_ID).toUri() + DeepLink(heapDumpFile, Place.Object(payloadObjectId)).toUri() ) } } @@ -170,7 +170,6 @@ class ObjectsScreenTest { MaterialTheme { ExplorerApp( heapDumpFile = heapDumpFile, - deepLinkId = WINDOW_ID, copyToClipboard = copyToClipboard, // Nothing here opens a second heap dump, and which window one would land in is // `ExplorerWindowTest`'s. @@ -229,9 +228,6 @@ class ObjectsScreenTest { private const val TREEMAP_LAID_OUT = "Read the treemap rooted at" - /** What a link copied here names this window by, fixed so that the copied link can be spelled out. */ - private const val WINDOW_ID = "abcd2345" - /** An `adb` that answers as if nothing were plugged in, so no test here reaches a real device. */ private val NO_DEVICE_ADB = Adb { AdbOutput(exitCode = 0, text = "List of devices attached\n") } } diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/TabStripTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/TabStripTest.kt index aefdf8388e..e37ebe1a76 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/TabStripTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/TabStripTest.kt @@ -183,7 +183,7 @@ class TabStripTest { // dump open twice is two places to be — plus the tab's own place, so that following it lands where // it was copied from. See [DeepLink]. assertThat(copied).containsExactly( - DeepLink(heapDumpFile, Place.wholeHeapDump(), windowId = WINDOW_ID).toUri() + DeepLink(heapDumpFile, Place.wholeHeapDump()).toUri() ) } } @@ -215,7 +215,7 @@ class TabStripTest { // A button opens a screen nobody has been to yet, and a link to it is that screen as it opens: no // tab has to be opened first to have something to copy. assertThat(copied).containsExactly( - DeepLink(heapDumpFile, Place.Leaks(), windowId = WINDOW_ID).toUri() + DeepLink(heapDumpFile, Place.Leaks()).toUri() ) // And nothing was opened by asking for the link, which a menu that clicked the button would have. assertThat(tabs().fetchSemanticsNodes()).hasSize(1) @@ -282,7 +282,6 @@ class TabStripTest { heapDumpFile = heapDumpFile, onHeapDumpChosen = { _, _ -> }, deviceHeapDumps = DeviceHeapDumps(NO_DEVICE_ADB), - deepLinkId = WINDOW_ID, linkedPlaces = linkedPlaces(), onLinkedPlaceOpened = onLinkedPlaceOpened, copyToClipboard = copyToClipboard @@ -334,9 +333,6 @@ class TabStripTest { */ private const val TABS_PAST_ONE_LINE = 20 - /** What a link copied here names this window by, fixed so that the copied link can be spelled out. */ - private const val WINDOW_ID = "abcd2345" - /** An `adb` that answers as if nothing were plugged in, so no test here reaches a real device. */ private val NO_DEVICE_ADB = Adb { AdbOutput(exitCode = 0, text = "List of devices attached\n") } } diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt index c293356eeb..1f0692c4b1 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/DeepLink.kt @@ -3,7 +3,6 @@ package shark.explorer import java.io.File import java.net.URLDecoder import java.net.URLEncoder -import kotlin.random.Random /** * A link to one place in one heap dump: `shark:///?`. @@ -13,48 +12,33 @@ import kotlin.random.Random * spellings carry the whole of the place rather than a shorthand for it: a filtered list of objects arrives * filtered, and a page of leaks arrives with the same ones unfolded. * - * **It names the heap dump, and a window only as a refinement.** Every place there is belongs to the dump - * rather than to whatever is showing it — an address, a leak, a filter over the object list, a note — so a - * link that named a window was a link that died with the window, which is most links a day later. This one - * survives: the run it was made from can be gone, and it still opens what it names, in a window of that dump - * if there is one and in a new window if there isn't. + * **A heap dump and a place, and nothing else.** Every place there is belongs to the dump rather than to + * whatever is showing it — an address, a leak, a filter over the object list, a note — so a link that named a + * window was a link that died with the window, which is most links a day later. This one survives: the run it + * was made from can be gone, and it still opens what it names, in a window of that dump if there is one and in + * a new window if there isn't. * - * Which leaves the case the window id was there for. The same dump *is* often open twice — that is what - * comparing two readings of it is — so [windowId] says which of them a link was made from, and it is - * honoured while that window is open and **ignored once it isn't**, rather than turning the link into an - * error. Being right about which window is worth a lot while the window exists and nothing at all - * afterwards. - * - * [heapDumpName] is the authority because it is the part a person reads and types, and **it is all a link - * says about which file**: where that file is, is looked up on the machine following the link rather than - * carried in it, since a path is most of the characters of a link and the least readable part of one. See - * [HeapDumpPaths], which is what remembers it, and [heapDumpPath], which is where a link that does carry one - * puts it. + * [heapDumpName] is the whole of what it says about which file, because a file name is what a person reads, + * what a window's title shows, and what every heap dump this app takes is given something unique for. + * **Where that file is doesn't travel in the link**: it is looked up by whoever follows one, since a path is + * most of the characters of a link and the least readable part of it. See [HeapDumpPaths], which is what + * remembers it, and [heapDumpPath], for the link that does say. * * Immutable and in this module rather than in the UI, so that what a link means is unit tested rather than - * found out by clicking one. See [Place] and `ExplorerWindows.windowFor`. + * found out by clicking one. See [Place] and `ExplorerWindows.open`. */ data class DeepLink( /** The heap dump's file name: what a link is read as, and all of it that has to be typed. */ val heapDumpName: String, val place: Place, /** - * Where that dump is, for the links that say: null in every link this app writes. - * - * Filled in by [HeapDumpPaths.resolve] as a link is followed, which is how a link finds the file without - * carrying it, and passed on in the query when one run hands a link to another so that the second doesn't - * have to look it up again. Written by hand in a link about a heap dump this machine has never opened, - * which is the one case a name cannot answer. - * - * Absolute and normalized when this app put it there, since it is compared against what a window has open. - */ - val heapDumpPath: File? = null, - /** - * Which window of that dump the link was made from, or null for one nobody made from a window. + * Where that dump is, for the link that says: null in every link this app writes. * - * A refinement and never a requirement: see the class comment. [newWindowId] is where these come from. + * Written by hand — or by a script — about a heap dump this machine has never opened, which is the one case + * a file name cannot answer. A link that has one is matched by path rather than by name, so it is also how + * to say *which* `com.squareup.hprof` when two of them off two devices have been opened here. */ - val windowId: String? = null + val heapDumpPath: File? = null ) { /** @@ -66,19 +50,16 @@ data class DeepLink( */ constructor( heapDumpFile: File, - place: Place, - windowId: String? = null + place: Place ) : this( heapDumpName = heapDumpFile.name, - place = place, - windowId = windowId + place = place ) /** The link as text, which is what gets copied, printed and pasted. */ fun toUri(): String { val parameters = place.linkParameters() + listOfNotNull( - heapDumpPath?.let { DUMP_PARAMETER to it.path }, - windowId?.let { WINDOW_PARAMETER to it } + heapDumpPath?.let { DUMP_PARAMETER to it.path } ) val query = if (parameters.isEmpty()) { "" @@ -99,23 +80,6 @@ data class DeepLink( /** Whether [argument] is a link rather than a heap dump path, which is all a command line has to ask. */ fun looksLikeOne(argument: String): Boolean = argument.startsWith(PREFIX) - /** - * A window id: eight lowercase characters, from an alphabet with no `l`, `1`, `o` or `0` in it so that a - * link read off a screen and typed back in is the link that was read. - * - * Random rather than counted up, which is not a detail even now that a link works without one. Ids - * handed out in order repeat across runs, and they repeat *within* one as windows close and open, so a - * link copied yesterday would be honoured today against the second window of whatever is running — - * silently the wrong reading of the dump, which is worse than being ignored. A random id is either the - * window it was made from or no window at all, and the second of those falls back to the heap dump. - * - * A file name and a number would not fix that. The number would have to be handed out across runs that - * cannot see each other's windows, and it would be reused the moment a window closed, so it would be - * exactly the id that opens *something*. - */ - fun newWindowId(random: Random = Random.Default): String = - (1..WINDOW_ID_LENGTH).map { ID_ALPHABET[random.nextInt(ID_ALPHABET.length)] }.joinToString("") - /** * Reads a link, or throws [IllegalArgumentException] saying what is wrong with it. * @@ -138,8 +102,7 @@ data class DeepLink( return DeepLink( heapDumpName = decode(segments[0]), place = placeOf(segments[1], parameters, uri), - heapDumpPath = parameters.firstOrNull(DUMP_PARAMETER)?.let { File(it) }, - windowId = parameters.firstOrNull(WINDOW_PARAMETER) + heapDumpPath = parameters.firstOrNull(DUMP_PARAMETER)?.let { File(it) } ) } @@ -263,11 +226,6 @@ data class DeepLink( private const val PREFIX = "$SCHEME://" - private const val WINDOW_ID_LENGTH = 8 - - /** No `l`, `1`, `o` or `0`: a window id is read off a screen and typed back in often enough to care. */ - private const val ID_ALPHABET = "abcdefghijkmnpqrstuvwxyz23456789" - internal const val OBJECT_PATH = "object" internal const val SMALLER_OBJECTS_PATH = "smaller-objects" internal const val OBJECTS_PATH = "objects" @@ -287,16 +245,13 @@ data class DeepLink( ) /** - * Where the heap dump is, and which window it was read in: the two parameters that are about the link - * rather than about the place. Only `window` is written into a link this app copies — see [heapDumpPath] - * for when the other one is there. + * Where the heap dump is: the one parameter that is about the link rather than about the place, and the + * one no [Place] may spell — they are read off the same query, and none of them does. `DeepLinkTest` + * holds them apart. * - * Which is why no [Place] may spell a parameter either of these names — they are read off the same query - * — and none does. `DeepLinkTest` holds them apart. + * Public because a message telling somebody to add one to a link has to spell it the way this does. */ - /** Public because a message telling somebody to add one to a link has to spell it the way this does. */ const val DUMP_PARAMETER = "dump" - const val WINDOW_PARAMETER = "window" internal const val ID_PARAMETER = "id" internal const val PARENT_PARAMETER = "parent" diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpPaths.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpPaths.kt index 22624a5523..11ce268cae 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpPaths.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapDumpPaths.kt @@ -4,28 +4,29 @@ import java.io.File import shark.SharkLog /** - * Where the heap dumps opened on this machine are, under the ids a [DeepLink] names them by. + * Where the heap dumps opened on this machine are, under the file names a [DeepLink] names them by. * * **This is what lets a link be short.** A link is a line of text someone reads — in a note, a pull request * comment, an agent's answer — and a path is most of the characters of one while saying the least: it is * unreadable at a glance, it says nothing a reader can act on, and it is the part that a link outliving the * run it was copied from has to carry only because nothing else remembers it. So nothing else is where it - * stops being: a heap dump opening writes down where it was, and a link says the dump's file name and the - * window it was copied from — short, readable, and enough to find the file again here. + * stops being: a heap dump opening writes down where it was, and a link says only the dump's file name — + * short, readable, and enough to find the file again here. * - * One file per heap dump opened, named after the window that opened it and holding that dump's path. A file - * each rather than one file of all of them, because several runs of this app open heap dumps at the same time - * and none of them coordinates with the others: a whole file written and renamed into place cannot be read as - * half of one, and two runs opening two dumps write two files instead of racing over one. + * One file per heap dump opened, named after the dump's path and holding it. A file each rather than one file + * of all of them, because several runs of this app open heap dumps at the same time and none of them + * coordinates with the others: a whole file written and renamed into place cannot be read as half of one, and + * two runs opening two dumps write two files instead of racing over one. * * The newest [keepCount] are kept, so this is a directory that stops growing rather than a record of every * heap dump ever opened. Which is the one thing a link loses by not carrying the path: it goes on working for - * as long as this machine remembers the file, rather than for as long as the file exists. A link that has been - * forgotten says so and can still be given the path by hand — see [DeepLink.heapDumpPath]. + * as long as this machine remembers the file, rather than for as long as the file exists. A link about a dump + * that has been forgotten asks where it is — see `ExplorerWindows.open` — and can also be given the path by + * hand, see [DeepLink.heapDumpPath]. * * Machine local, and no worse than the path would have been: a link followed on another machine could never * have used this one's paths. What it uses there is the file name, against the dumps that machine has open or - * has opened. + * has opened, and failing that the reader is asked for the file. */ class HeapDumpPaths( /** This app's directory for these, which the caller decides, the way [NoteDirectory] takes its root. */ @@ -40,19 +41,16 @@ class HeapDumpPaths( } /** - * Writes down that the window called [windowId] has [heapDumpFile] open, and forgets the oldest of these - * beyond [keepCount]. + * Writes down where [heapDumpFile] is, and forgets the oldest of these beyond [keepCount]. * * Called as a heap dump finishes opening, in a window or in a run that has none: a dump that failed to open - * is not one a link should be sent to. + * is not one a link should be sent to. Opening the same dump again rewrites its record, which is what keeps + * a heap dump somebody keeps coming back to from being forgotten. */ - fun record( - windowId: String, - heapDumpFile: File - ) { + fun record(heapDumpFile: File) { val path = normalizedHeapDumpPath(heapDumpFile) try { - writeWholeFile(File(directory, windowId), path.path) + writeWholeFile(File(directory, heapDumpFileKey(path)), path.path) } catch (throwable: Throwable) { // Not a reason to fail the open: what stops working is links to this dump once every window of it has // gone, which is worth a line in the log rather than a window that refuses to show a heap dump. @@ -63,36 +61,18 @@ class HeapDumpPaths( } /** - * [link] with the heap dump's path filled in from what this machine remembers, or [link] as it is when - * nothing here has that dump on record. + * Every path this machine remembers for a heap dump called [heapDumpName], most recently opened first. * - * What a link says is tried in the order that is right about the most: the window it was copied from, since - * that window's dump is the one its reader was looking at; then a dump of that file name, newest first, - * since a name is what a link and a person both call a heap dump; then the name as a window id, for a link - * whose whole authority is one — `shark://abcd2345/leaks`, which is what this app used to write and what - * anything can still write, since a window id is enough to find the dump it was showing. - * - * A link that already carries a path is left alone. That path was either put there by hand or filled in by - * another run of this app, and either way it is more specific than a name. + * More than one when heap dumps off two devices are both called `com.squareup.hprof`, which is what a link + * naming only the file has no answer for and asks about. Empty for a name nothing here has opened, or has + * opened recently enough to still be on record. The files themselves may be gone — this says where a dump + * was, and whoever follows a link is the one that cares whether it is still there. */ - fun resolve(link: DeepLink): DeepLink { - if (link.heapDumpPath != null) { - return link - } - val records = records() - val recorded = link.windowId?.let { id -> records.firstOrNull { it.windowId == id } } - ?: records.firstOrNull { it.path.name == link.heapDumpName } - ?: records.firstOrNull { it.windowId == link.heapDumpName } - ?: return link - // Worded to read for a link named by a window id as well as by a file name, since both land here. - SharkLog.d { - "${link.heapDumpName} is ${recorded.path}, which was last open as ${recorded.windowId}" - } - return link.copy(heapDumpPath = recorded.path) - } + fun pathsNamed(heapDumpName: String): List = + records().filter { it.name == heapDumpName } - /** Every heap dump on record, most recently opened first, which is the order all three lookups want. */ - private fun records(): List = + /** Every heap dump on record, most recently opened first, which is the order a link wants them tried in. */ + private fun records(): List = files().sortedByDescending { it.lastModified() }.mapNotNull { file -> val path = try { file.readText().trim() @@ -100,7 +80,7 @@ class HeapDumpPaths( SharkLog.d(throwable) { "Could not read $file, so it names no heap dump" } return@mapNotNull null } - if (path.isEmpty()) null else Record(windowId = file.name, path = File(path)) + if (path.isEmpty()) null else File(path) } private fun forgetOldest() { @@ -123,12 +103,6 @@ class HeapDumpPaths( .orEmpty() .toList() - /** One heap dump this machine has opened, and the window it was open in. */ - private class Record( - val windowId: String, - val path: File - ) - companion object { /** * How many heap dumps are remembered. Enough that a link written weeks ago still opens the dump it names, diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt index 9a2ce7ba56..48c6137919 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/DeepLinkTest.kt @@ -1,7 +1,6 @@ package shark.explorer import java.io.File -import kotlin.random.Random import org.assertj.core.api.Assertions.assertThat import org.assertj.core.api.Assertions.assertThatThrownBy import org.junit.Test @@ -232,31 +231,16 @@ class DeepLinkTest { .hasMessageContaining("Kinds are CLASS, INSTANCE, OBJECT_ARRAY, PRIMITIVE_ARRAY") } - @Test - fun `a window id is eight characters of an alphabet nothing can be misread in`() { - val ids = (1..200).map { DeepLink.newWindowId(Random(it)) } - - assertThat(ids).allMatch { it.length == 8 } - assertThat(ids.joinToString("")).matches("[abcdefghijkmnpqrstuvwxyz23456789]+") - } - - @Test - fun `two windows do not get one id`() { - val ids = (1..1000).map { DeepLink.newWindowId() } - - assertThat(ids.toSet()).hasSize(ids.size) - } - /** - * The link the app itself writes, and the whole of what it is for: a heap dump, a place in it, and which - * window it was copied from. **Not where the file is** — that is looked up by whoever follows the link, see - * [HeapDumpPaths] — because a path is most of the characters of a link and the least readable part of it. + * The link the app itself writes, and the whole of what it is for: a heap dump and a place in it. **Not + * where the file is** — that is looked up by whoever follows the link, see [HeapDumpPaths] — because a path + * is most of the characters of a link and the least readable part of it. */ @Test - fun `a link from a window is a heap dump, a place and a window`() { - val link = DeepLink(File("/dumps/leak.hprof"), Place.Leaks(), windowId = "abcd2345") + fun `a link from a window is a heap dump and a place`() { + val link = DeepLink(File("/dumps/leak.hprof"), Place.Leaks()) - assertThat(link.toUri()).isEqualTo("shark://leak.hprof/leaks?window=abcd2345") + assertThat(link.toUri()).isEqualTo("shark://leak.hprof/leaks") assertThat(link.heapDumpPath).isNull() assertThat(DeepLink.parse(link.toUri())).isEqualTo(link) } @@ -274,8 +258,8 @@ class DeepLinkTest { } /** - * Which is a link somebody typed, or shortened by hand to the two things worth reading. It resolves - * against whatever is open, so it is worth being able to write. See `ExplorerWindows.windowFor`. + * Which is also a link somebody typed, since there is nothing else to type. It resolves against whatever + * heap dump of that name is open, or has been. See `ExplorerWindows.open`. */ @Test fun `a link is a heap dump and a place and needs nothing else`() { @@ -284,7 +268,6 @@ class DeepLinkTest { assertThat(link.heapDumpName).isEqualTo("leak.hprof") assertThat(link.place).isEqualTo(Place.Leaks()) assertThat(link.heapDumpPath).isNull() - assertThat(link.windowId).isNull() } /** @@ -301,12 +284,11 @@ class DeepLinkTest { } /** - * The two parameters that are about the link rather than about the place share the query with the ones that - * are, so a [Place] spelling a parameter `dump` or `window` would quietly take one of them over. This is - * what would fail. + * The one parameter that is about the link rather than about the place shares the query with the ones that + * are, so a [Place] spelling a parameter `dump` would quietly take it over. This is what would fail. */ @Test - fun `no place takes the dump or the window off a link`() { + fun `no place takes the dump off a link`() { val places = listOf( Place.wholeHeapDump(), Place.Object(0x12ab34cd), @@ -324,8 +306,7 @@ class DeepLinkTest { val link = DeepLink( heapDumpName = "leak.hprof", place = place, - heapDumpPath = File("/dumps/leak.hprof"), - windowId = "abcd2345" + heapDumpPath = File("/dumps/leak.hprof") ) assertThat(DeepLink.parse(link.toUri())).describedAs(link.toUri()).isEqualTo(link) } diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapDumpPathsTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapDumpPathsTest.kt index a34ebee4de..8ab07219a6 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapDumpPathsTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapDumpPathsTest.kt @@ -8,24 +8,21 @@ import org.junit.Test import org.junit.rules.TemporaryFolder /** - * Where a heap dump was, remembered by the id a link names it with. Which is what a link not carrying the - * path rests on, so what is tested here is every way a link asks for one. + * Where the heap dumps this machine has opened are, under the file names a link names them by. Which is what + * a link not carrying a path rests on, so what is tested here is what a link gets to ask. */ class HeapDumpPathsTest { @get:Rule val temporaryFolder = TemporaryFolder() - private val paths by lazy { HeapDumpPaths(temporaryFolder.newFolder("heap-dump-paths")) } + private val directory by lazy { temporaryFolder.newFolder("heap-dump-paths") } - @Test fun `a link naming a heap dump gets the path it was opened from`() { - paths.record(WINDOW_ID, File("/dumps/leak.hprof")) + private val paths by lazy { HeapDumpPaths(directory) } - val resolved = paths.resolve(DeepLink.parse("shark://leak.hprof/starred")) + @Test fun `a heap dump that has been opened here is on record under its name`() { + paths.record(File("/dumps/leak.hprof")) - assertThat(resolved.heapDumpPath).isEqualTo(File("/dumps/leak.hprof")) - // The rest of the link is what it was: this fills in the one thing a link doesn't say. - assertThat(resolved.place).isEqualTo(Place.Starred) - assertThat(resolved.heapDumpName).isEqualTo("leak.hprof") + assertThat(paths.pathsNamed("leak.hprof")).containsExactly(File("/dumps/leak.hprof")) } /** @@ -33,80 +30,55 @@ class HeapDumpPathsTest { * has been closed, and where its heap dump was is on disk rather than in the link. */ @Test fun `a link outlives the run that copied it`() { - val fromAWindow = DeepLink(File("/dumps/leak.hprof"), Place.Leaks(), windowId = WINDOW_ID) - paths.record(WINDOW_ID, File("/dumps/leak.hprof")) + val fromAWindow = DeepLink(File("/dumps/leak.hprof"), Place.Leaks()) + paths.record(File("/dumps/leak.hprof")) - val resolved = paths.resolve(DeepLink.parse(fromAWindow.toUri())) + val link = DeepLink.parse(fromAWindow.toUri()) - assertThat(resolved.heapDumpPath).isEqualTo(File("/dumps/leak.hprof")) - } - - /** Two dumps of one name off two devices are two investigations, so which window it was copied from wins. */ - @Test fun `a link says which of two heap dumps of the same name`() { - paths.record(WINDOW_ID, File("/dumps/pixel/app.hprof")) - paths.record(OTHER_WINDOW_ID, File("/dumps/emulator/app.hprof")) - - val resolved = paths.resolve(DeepLink("app.hprof", Place.Starred, windowId = WINDOW_ID)) - - assertThat(resolved.heapDumpPath).isEqualTo(File("/dumps/pixel/app.hprof")) + assertThat(paths.pathsNamed(link.heapDumpName)).containsExactly(File("/dumps/leak.hprof")) } /** - * A link whose window is not on record — typed by hand, or copied from a run whose record has been - * forgotten — is still about a heap dump of that name, and the last one opened is the one being worked on. + * Which is what a link about a name with two heap dumps behind it has to ask about, since it says nothing + * that tells them apart. Newest first, because the one being worked on is the one opened last. */ - @Test fun `a name with no window falls back to the heap dump opened last`() { - paths.record(WINDOW_ID, File("/dumps/pixel/app.hprof")) - paths.record(OTHER_WINDOW_ID, File("/dumps/emulator/app.hprof")) + @Test fun `two heap dumps of one name are two paths, newest opened first`() { + paths.record(File("/dumps/pixel/app.hprof")) + paths.record(File("/dumps/emulator/app.hprof")) // Recorded in the same millisecond otherwise, which is not an order to read them in. - val directory = File(temporaryFolder.root, "heap-dump-paths") - File(directory, WINDOW_ID).setLastModified(FIRST_MODIFIED) - File(directory, OTHER_WINDOW_ID).setLastModified(LATER) - - val resolved = paths.resolve(DeepLink("app.hprof", Place.Starred, windowId = "qrst6789")) - - assertThat(resolved.heapDumpPath).isEqualTo(File("/dumps/emulator/app.hprof")) - } + recordOf("/dumps/pixel/app.hprof").setLastModified(FIRST_MODIFIED) + recordOf("/dumps/emulator/app.hprof").setLastModified(LATER) - /** - * `shark:///`, which is a link with nothing in it but an id. Nothing writes one now, and it - * is the shortest a link can be, so it goes on working: a window id names a heap dump too. - */ - @Test fun `a link that is only a window id finds that window's heap dump`() { - paths.record(WINDOW_ID, File("/dumps/leak.hprof")) - - val resolved = paths.resolve(DeepLink.parse("shark://$WINDOW_ID/leaks")) - - assertThat(resolved.heapDumpPath).isEqualTo(File("/dumps/leak.hprof")) + assertThat(paths.pathsNamed("app.hprof")) + .containsExactly(File("/dumps/emulator/app.hprof"), File("/dumps/pixel/app.hprof")) } - /** Which is what tells the reader to open the file, rather than a window that says nothing. */ - @Test fun `a heap dump nothing here has opened stays unresolved`() { - paths.record(WINDOW_ID, File("/dumps/leak.hprof")) + /** Which is what has the reader asked where the file is, rather than a window that says nothing. */ + @Test fun `a heap dump nothing here has opened is nowhere`() { + paths.record(File("/dumps/leak.hprof")) - val resolved = paths.resolve(DeepLink.parse("shark://another.hprof/starred")) - - assertThat(resolved.heapDumpPath).isNull() + assertThat(paths.pathsNamed("another.hprof")).isEmpty() } - @Test fun `nothing recorded at all resolves nothing`() { - assertThat(paths.resolve(DeepLink.parse("shark://leak.hprof/starred")).heapDumpPath).isNull() + @Test fun `nothing recorded at all is nowhere`() { + assertThat(paths.pathsNamed("leak.hprof")).isEmpty() } - /** Handed over by another run, or written by hand: more specific than a name, so it is left alone. */ - @Test fun `a link that already says where the dump is keeps that path`() { - paths.record(WINDOW_ID, File("/dumps/leak.hprof")) - val link = DeepLink("leak.hprof", Place.Starred, heapDumpPath = File("/elsewhere/leak.hprof")) + /** One heap dump is one record, however many times it is opened — and opening it keeps it from eviction. */ + @Test fun `the same heap dump opened twice is one path`() { + paths.record(File("/dumps/leak.hprof")) + paths.record(File("/dumps/leak.hprof")) - assertThat(paths.resolve(link).heapDumpPath).isEqualTo(File("/elsewhere/leak.hprof")) + assertThat(paths.pathsNamed("leak.hprof")).containsExactly(File("/dumps/leak.hprof")) + assertThat(directory.list()).hasSize(1) } /** Because a link is read months later, from a run started in another directory. */ @Test fun `a recorded path is absolute and has no dots in it`() { - paths.record(WINDOW_ID, File("dumps/./over/../leak.hprof")) + paths.record(File("dumps/./over/../leak.hprof")) - assertThat(paths.resolve(DeepLink.parse("shark://leak.hprof/starred")).heapDumpPath) - .isEqualTo(File(File("").absoluteFile, "dumps/leak.hprof")) + assertThat(paths.pathsNamed("leak.hprof")) + .containsExactly(File(File("").absoluteFile, "dumps/leak.hprof")) } @Test fun `only the newest heap dumps are remembered`() { @@ -114,34 +86,36 @@ class HeapDumpPathsTest { val paths = HeapDumpPaths(directory, keepCount = 2) listOf("first", "second", "third").forEachIndexed { index, name -> - paths.record(name, File("/dumps/$name.hprof")) + paths.record(File("/dumps/$name.hprof")) // Written in the same millisecond otherwise, which is not an order to evict by. - File(directory, name).setLastModified(FIRST_MODIFIED + index * MINUTE) + recordOf("/dumps/$name.hprof", directory).setLastModified(FIRST_MODIFIED + index * MINUTE) } // A directory that stops growing, which is the one thing a link loses by not carrying the path: it works // for as long as this machine remembers the file rather than for as long as the file exists. - assertThat(directory.list()).containsExactlyInAnyOrder("second", "third") + assertThat(paths.pathsNamed("first.hprof")).isEmpty() + assertThat(paths.pathsNamed("second.hprof")).containsExactly(File("/dumps/second.hprof")) + assertThat(paths.pathsNamed("third.hprof")).containsExactly(File("/dumps/third.hprof")) } @Test fun `a record nobody can read names no heap dump`() { val directory = temporaryFolder.newFolder("unreadable") - File(directory, WINDOW_ID).writeText("") + File(directory, heapDumpFileKey(File("/dumps/leak.hprof"))).writeText("") - assertThat(HeapDumpPaths(directory).resolve(DeepLink.parse("shark://$WINDOW_ID/leaks")).heapDumpPath) - .isNull() + assertThat(HeapDumpPaths(directory).pathsNamed("leak.hprof")).isEmpty() } /** A save in flight is a file in this directory too, and one nothing may read or delete. */ @Test fun `a write in flight is not a heap dump on record`() { val directory = temporaryFolder.newFolder("in-flight") val paths = HeapDumpPaths(directory, keepCount = 1) - File(directory, "$WINDOW_ID.partial").writeText("/dumps/half-written.hprof") + val inFlight = File(directory, "${heapDumpFileKey(File("/dumps/half-written.hprof"))}.partial") + inFlight.writeText("/dumps/half-written.hprof") - paths.record(OTHER_WINDOW_ID, File("/dumps/leak.hprof")) + paths.record(File("/dumps/leak.hprof")) - assertThat(File(directory, "$WINDOW_ID.partial")).exists() - assertThat(paths.resolve(DeepLink.parse("shark://half-written.hprof/starred")).heapDumpPath).isNull() + assertThat(inFlight).exists() + assertThat(paths.pathsNamed("half-written.hprof")).isEmpty() } @Test fun `remembering none of them is not something to ask for`() { @@ -149,10 +123,13 @@ class HeapDumpPathsTest { .hasMessageContaining("not 0 of them") } - companion object { - private const val WINDOW_ID = "abcd2345" - private const val OTHER_WINDOW_ID = "wxyz6789" + /** The file [HeapDumpPaths] writes about one heap dump, for a test that has to touch it directly. */ + private fun recordOf( + heapDumpPath: String, + directory: File = this.directory + ) = File(directory, heapDumpFileKey(File(heapDumpPath))) + companion object { /** Any time at all, since what is read off these is their order. */ private const val FIRST_MODIFIED = 1_600_000_000_000L private const val MINUTE = 60_000L From c0f2d2fe1acedf2289a987ac423a32ce989f8b62 Mon Sep 17 00:00:00 2001 From: Pierre-Yves Ricau Date: Fri, 28 Aug 2026 14:55:54 +0200 Subject: [PATCH 27/27] Look a link's heap dump up when the window that has it failed to open it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A window keeps the file it was given even when opening it failed, so it went on claiming every link about that name and every one of them landed on the error message. The file is usually openable from where this machine last saw it — the case that found this was a run started with a relative path, which the OS resolves against `/` for an app it launched — so a window whose heap dump failed to open is no longer a window that has it, for links and for what this run claims from its peers. And say plainly there is no file there, with the absolute path, rather than letting the parser report the path as it was given: `/shark/shark-android/…/leak_asynctask_o.hprof` is what says the working directory wasn't the checkout, where the relative path it was typed as says nothing at all. --- .../java/shark/explorer/app/ExplorerWindow.kt | 5 ++++ .../shark/explorer/app/ExplorerWindowTest.kt | 26 +++++++++++++++++++ .../main/java/shark/explorer/HeapExplorer.kt | 8 ++++++ .../java/shark/explorer/HeapExplorerTest.kt | 12 +++++++++ 4 files changed, 51 insertions(+) diff --git a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt index 41fc6eff0e..b521f4930a 100644 --- a/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt +++ b/shark/shark-explorer/shark-explorer-app/src/main/java/shark/explorer/app/ExplorerWindow.kt @@ -206,11 +206,16 @@ internal class ExplorerWindows( * By file name, which is the whole of what a link says about the heap dump — and by path for the rare link * that says where the file is, since that one is exact: two dumps called `com.squareup.hprof` off two * devices are two investigations, and a link carrying a path has already said which. + * + * A window whose heap dump **failed to open** is not a window that has it. The file it was given may well + * be openable from where this machine last saw it — a path typed wrong, a dump on a volume that wasn't + * mounted then — and landing a link on the window that says so is the one outcome that helps nobody. */ fun windowsFor(link: DeepLink): List { val path = link.heapDumpPath?.normalizedPath() return filter { window -> val heapDumpFile = window.heapDumpFile ?: return@filter false + if (window.openProblem != null) return@filter false if (path == null) heapDumpFile.name == link.heapDumpName else heapDumpFile.normalizedPath() == path } } diff --git a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt index fac46b31e3..dc39b14dca 100644 --- a/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt +++ b/shark/shark-explorer/shark-explorer-app/src/test/java/shark/explorer/app/ExplorerWindowTest.kt @@ -216,6 +216,32 @@ class ExplorerWindowTest { assertThat(windows.mapNotNull { it.linkedHeapDump }).isEmpty() } + /** + * A window whose heap dump failed to open is not a window that has it, so a link is looked up instead of + * being handed to the window that says so. + * + * Which is what a run started with a relative path leaves behind — the OS launches an app with `/` for a + * working directory, so a path off a command line typed in a checkout resolves to nothing — and the file is + * usually right there where this machine last saw it. + */ + @Test fun `a link about a heap dump a window failed to open opens the file instead`() { + val heapDumpFile = temporaryFolder.newFile(FIRST_DUMP.name) + heapDumpPaths.record(heapDumpFile) + val windows = explorerWindows(opening(FIRST_DUMP)) + val failed = windows.single() + failed.openProblem = "There is no file at ${FIRST_DUMP.absolutePath}" + // So a peer that has it open takes the link ahead of this run, for the same reason. See [DeepLinkPeers]. + assertThat(windows.windowsFor(DeepLink(FIRST_DUMP, Place.Starred))).isEmpty() + + windows.open(DeepLink(FIRST_DUMP, Place.Starred)) + + val opened = windows.last() + assertThat(windows).hasSize(2) + assertThat(opened.heapDumpFile).isEqualTo(heapDumpFile.absoluteFile) + assertThat(opened.linkedPlaces).containsExactly(Place.Starred) + assertThat(failed.linkedPlaces).isEmpty() + } + /** The one case a file name cannot answer, and the reason a link can still carry a path. */ @Test fun `a link that says where the heap dump is opens it without looking anything up`() { val heapDumpFile = temporaryFolder.newFile("fourth.hprof") diff --git a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapExplorer.kt b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapExplorer.kt index 1c9c828b18..f918b1230c 100644 --- a/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapExplorer.kt +++ b/shark/shark-explorer/shark-explorer-core/src/main/java/shark/explorer/HeapExplorer.kt @@ -2,6 +2,7 @@ package shark.explorer import java.io.Closeable import java.io.File +import java.io.FileNotFoundException import java.util.concurrent.TimeUnit.NANOSECONDS import shark.AndroidObjectSizeCalculator import shark.CancelSignal @@ -55,6 +56,13 @@ class HeapExplorer private constructor( onProgress: (String) -> Unit = {}, cancelSignal: CancelSignal = CancelSignal.NEVER ): HeapExplorer { + // Said here rather than left to the parser, which reports the path exactly as it was given. A relative + // path resolves against the working directory, and a run the OS launched from a link or from `open` has + // `/` for that, so the absolute path is the only part of the message that says which file was looked + // for — and `0 B` below would be the size of a file that isn't there. + if (!heapDumpFile.isFile) { + throw FileNotFoundException("There is no file at ${heapDumpFile.absolutePath}") + } SharkLog.d { "Opening heap dump $heapDumpFile, ${formatByteSize(heapDumpFile.length())}" } val startNanos = System.nanoTime() val steps = OpenSteps(onProgress) diff --git a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapExplorerTest.kt b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapExplorerTest.kt index 04128ddc25..7b19db928d 100644 --- a/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapExplorerTest.kt +++ b/shark/shark-explorer/shark-explorer-core/src/test/java/shark/explorer/HeapExplorerTest.kt @@ -1,5 +1,7 @@ package shark.explorer +import java.io.File +import java.io.FileNotFoundException import org.assertj.core.api.Assertions.assertThat import org.assertj.core.api.Assertions.assertThatThrownBy import org.junit.Rule @@ -545,6 +547,16 @@ class HeapExplorerTest { .isInstanceOf(Exception::class.java) } + @Test fun `opening a file that is not there says where it looked`() { + val missing = File(testFolder.root, "gone.hprof") + + assertThatThrownBy { HeapExplorer.open(missing) } + // The absolute path, since a relative one resolves against a working directory the reader can't see: + // a run the OS launched has `/` for it, so the path as given says nothing about what was looked for. + .isInstanceOf(FileNotFoundException::class.java) + .hasMessageContaining(missing.absolutePath) + } + @Test fun `a weakly reachable object nests inside the weak reference reaching it`() { HeapExplorer.open(testFolder.weaklyReachablePayloadHeapDump()).use { explorer -> val tree = explorer.tree