Skip to content

AGENTS.md

Latest commit

 

History

History
271 lines (201 loc) · 12 KB

AGENTS.md

File metadata and controls

271 lines (201 loc) · 12 KB
Raw

Instructions for Codex

To build the wazero binary, execute:

just build

To run the test suite, execute:

just test

This delegates to the Makefile which runs the full Go test suite.

To run only the trace writer tests (pure Go, no FFI):

just test-tracewriter-go

To run trace writer tests including Rust FFI:

just test-tracewriter

Codebase structure

This is a fork of wazero (WebAssembly runtime for Go) with CodeTracer tracing capabilities added. The tracing instruments WASM execution to produce Step/Call/Function/Event records that CodeTracer loads for time-travel debugging.

Key directories

  • cmd/wazero/ — CLI entry point. The run subcommand accepts --out-dir to produce a CodeTracer trace, and --stylus to load an EVM trace for Stylus replay.
  • internal/engine/interpreter/interpreter.go — Core interpreter loop with DWARF-based source-level stepping (lines 723-876). This is where Step/Call/Function trace events are generated by mapping WASM program counter offsets to source lines via DWARF.
  • internal/wasmdebug/dwarf_indexing.go — DWARF index: maps PC ranges to source lines, functions, local variables, and inlined subroutines. Built at module compile time.
  • internal/stylus/ — Stylus (Arbitrum WASM) replay harness. Creates host function stubs that replay pre-recorded EVM interaction data.
  • tracewriter/ — Trace output. Supports a pure-Go JSON writer and a Rust FFI writer (via cgo, using codetracer_trace_writer_ffi).
  • experimental/ — Upstream wazero experimental features (logging, sockets).

Trace recording architecture

Regular WASM

For regular WASM programs (targeting wasm32-wasip1), the flow is:

  1. CompileModule() decodes the WASM binary including DWARF custom sections (.debug_info, .debug_line, .debug_str, .debug_abbrev, .debug_ranges)
  2. IndexDwarfData() builds a PC-indexed lookup structure mapping WASM byte offsets to source file/line, function entries, local variables, and inlined subroutines
  3. During execution, the interpreter loop (lines 786-876) checks each instruction's PC offset against the DWARF index. When the source line changes, it emits a RegisterStep. On function entry, it emits RegisterCall + RegisterFunction.
  4. Local variable values are read from the WASM locals array and emitted as RegisterVariable events.

Stylus (Arbitrum WASM) — Replay architecture

Stylus tracing does NOT debug the actual on-chain binary. Instead, it replays the execution using a debug-built WASM binary with DWARF symbols. The pipeline is:

  1. ct arb deploy (in the codetracer repo, src/ct/stylus/deploy.nim):

    • Runs cargo build --target wasm32-unknown-unknown (debug mode, preserves DWARF)
    • Runs cargo stylus deploy to deploy the optimized on-chain binary
    • Saves the debug WASM to ~/.local/share/codetracer/contract-debug-wasm/<addr>/debug.wasm
    • This creates a mapping: on-chain contract address → debug-built WASM

  2. Transaction execution: Users send transactions to the deployed contract on a local Arbitrum devnode (nitro-testnode). The EVM records all host function calls (read_args, storage ops, msg_value, etc.) with their inputs and outputs.

  3. ct arb record (in the codetracer repo, src/ct/stylus/record.nim):

    • Looks up the contract address from the tx hash via eth_getTransactionByHash
    • Retrieves the debug WASM from the mapping at contract-debug-wasm/<addr>/debug.wasm
    • Gets the EVM trace via cargo stylus trace --use-native-tracer --tx <hash>
    • Invokes the wazero binary with --stylus <evm_trace.json> and the debug WASM

  4. Wazero Stylus replay (this repo, cmd/wazero/wazero.go):

    • Loads the debug WASM through the same CompileModule() path as regular WASM, which parses DWARF sections and builds the PC index
    • stylus.Instantiate() (internal/stylus/stylus.go) creates a "vm_hooks" host module with Go-implemented stubs for all Stylus host functions
    • Each stub replays pre-recorded data from the EVM trace: when the WASM calls read_args, the stub writes the recorded calldata into WASM memory; when it calls storage_load_bytes32, the stub returns the recorded storage value; etc.
    • The WASM's user_entrypoint function is called with the entrypoint arg from the EVM trace, and its return value is validated against the recorded result
    • Since the debug WASM executes through the same interpreter, DWARF-based stepping SHOULD produce Step/Call/Function events alongside the EVM Event records

  5. CodeTracer loads the trace: The db-backend parses the trace containing both source-level stepping events (from DWARF) and EVM interaction events (from the Stylus stubs). The EVM events appear in the Event Log panel.

EVM trace format

The EVM trace is a JSON array consumed by internal/stylus/stylus_trace.go:

[
  {"name": "user_entrypoint", "startInk": 100, "endInk": 90, "args": "0x...", "outs": "0x00000000"},
  {"name": "read_args", "startInk": 90, "endInk": 88, "args": "0x", "outs": "0xca1d209d..."},
  {"name": "storage_load_bytes32", "startInk": 88, "endInk": 85, "args": "0x...", "outs": "0x..."},
  ...
  {"name": "user_returned", "startInk": 10, "endInk": 8, "args": "0x", "outs": "0x00000000"}
]

Each event has: name (host function), startInk/endInk (gas metering), args (hex-encoded input bytes), outs (hex-encoded output bytes to inject into WASM memory).

DWARF stepping details

File filter (interpreter.go lines 753-757)

The DWARF stepping code only generates trace events for source files matching:

  • Filename ends with .rs
  • Does NOT start with /rustc (standard library internals)
  • Does NOT contain .rustup, .cargo, or /rust/deps

This means only user-authored Rust code generates stepping events. Library code from crates.io or the Rust standard library is stepped through silently.

Line-level stepping (interpreter.go line 820)

There is a second filter at line 820 with a false || prefix that currently disables line-level tracking even when function-level tracking is active. This appears to be a debug leftover that needs investigation — changing it to true || or removing the false || would enable full line-level stepping.

Inline subroutine tracking

The interpreter tracks inlined function calls using DWARF DW_TAG_inlined_subroutine entries. When the PC enters an inlined region, a synthetic call/function event is generated; when it exits, a return event is generated.

Known issues

CRITICAL: --trace-dir vs --out-dir flag mismatch

The codetracer repo's db_backend_record.nim:109 passes --trace-dir to the wazero binary, but wazero's CLI (cmd/wazero/wazero.go:227) expects --out-dir. This means ct arb record invocations produce NO trace output — the recorder is nil, so no Step/Call/Function/Event records are written. This must be fixed in either the Nim caller or the Go CLI (or both, to accept the old flag as an alias).

false || at interpreter.go line 820

The condition if false || (strings.HasSuffix(...)) disables line-level stepping. This appears to be a debugging leftover. Should be investigated and either removed (to enable full stepping) or documented with a clear rationale.

Stylus traces currently contain only Events

Because of the --trace-dir/--out-dir mismatch (when invoked via ct arb record) and/or because integration tests build with --release (stripping DWARF), current Stylus traces in practice contain only EVM Event entries, not Step/Call/Function entries. The db-backend has been hardened to handle this event-only format without panicking, but the goal is to have full source-level stepping alongside EVM events.

Integration test uses release build

stylus_flow_integration.rs:105 builds with cargo build --release --target wasm32-unknown-unknown, which strips DWARF debug info. For testing DWARF-based stepping with Stylus, the build should either use debug mode or add [profile.release] debug = true to the contract's Cargo.toml.

Cross-repo test infrastructure

Tier 1: Recording pipeline (codetracer repo)

Tests the end-to-end Stylus recording pipeline: build WASM, deploy, send tx, get EVM trace, record with wazero. Requires a running Arbitrum devnode.

cd ../codetracer && just test-stylus-flow       # Quick smoke test
cd ../codetracer && just test-stylus-flow-full   # Full content verification

Tier 2: Headless DAP analysis (codetracer repo)

Tests that the db-backend DAP server handles Stylus traces correctly. Uses pre-recorded fixtures — no devnode needed.

cd ../codetracer/src/db-backend && cargo test test_stylus_dap_event_only_trace

Tier 3: VS Code UI tests (codetracer-vscode-extension repo)

WebdriverIO tests that launch VS Code Insiders with the extension and verify UI behavior.

cd ../codetracer-vscode-extension && npm run test:wdio

You don't have access to the internet

During development, certain commands will fail because you don't have access to the internet.

The script .agents/download_internet_resources.sh is executed before your development session starts while your computer is still connected to the internet.

You can examine this script to see what kind of internet resources have been downloaded for offline use. If it's difficult for you to achieve a task without access to additional internet resources, you can always propose a PR that modifies the download.sh script instead of completing your main task.

Downloading development dependencies may also fail due to the lack of internet connectivity. We are trying to maintain the script .agents/codex-setup that is also executed before your development session starts while your computer is still connected to the internet. It tries to run all build commands that need development dependencies in order to cache the dependencies for offline use. Please propose changes to this script when you introduce new build targets with dependencies.

When you need to consult the documentation or source code modules for a particular dependency, always try to find where this dependency have been downloaded and try to access the necessary files through the file system (i.e. depending on the programming language, the operating system and the package manager being used, they should be in their standard location).

Keeping notes

In the .agents/codebase-insights.txt file, we try to maintain useful tips that may help you in your development tasks. When you discover something important or surprising about the codebase, add a remark in a comment near the relevant code or in the codebase-insights file. ALWAYS remove older remarks if they are no longer true.

You can consult this file before starting your coding tasks.

Code quality guidelines

  • ALWAYS strive to achieve high code quality.
  • ALWAYS write secure code.
  • ALWAYS make sure the code is well tested and edge cases are covered. Design the code for testability and be extremely thorough.
  • ALWAYS write defensive code and make sure all potential errors are handled.
  • ALWAYS strive to write highly reusable code with routines that have high fan in and low fan out.
  • ALWAYS keep the code DRY.
  • Aim for low coupling and high cohesion. Encapsulate and hide implementation details.
  • Go code should pass go vet and gofmt without issues.
  • Nix files are formatted with nixfmt.

Code commenting guidelines

  • Document public APIs and complex modules using standard code documentation conventions.
  • Comment the intention behind your code extensively. Omit comments only for very obvious facts that almost any developer would know.
  • Maintain the comments together with the code to keep them meaningful and current.
  • When the code is based on specific formats, standards or well-specified behavior of other software, always make sure to include relevant links (URLs) that provide the necessary technical details.

Writing git commit messages

  • You MUST use multiline git commit messages.
  • Use the conventional commits style for the first line of the commit message.
  • Use the summary section of your final response as the remaining lines in the commit message.