Skip to content

feat(codegen): add solar-codegen crate with MIR and EVM codegen#693

Draft
gakonst wants to merge 128 commits into
mainfrom
feat/codegen-mir
Draft

feat(codegen): add solar-codegen crate with MIR and EVM codegen#693
gakonst wants to merge 128 commits into
mainfrom
feat/codegen-mir

Conversation

@gakonst
Copy link
Copy Markdown
Member

@gakonst gakonst commented Jan 14, 2026

Summary

This adds the solar-codegen crate which provides MIR (Mid-level Intermediate Representation) and EVM bytecode generation for Solar.

Test Results

95 tests pass across 6 parallel test suites comparing Solar vs solc:

Metric Solar solc Difference
Bytecode size 105-1856B 370-4340B 61-72% smaller
Gas usage varies varies 2-48% cheaper
Stack depth ✓ compiles ✗ "stack too deep" Solar wins

Per-Project Results

Project Tests Contracts
arithmetic 28 Counter, Arithmetic
control-flow 16 ControlFlow (loops, conditionals, break)
storage 37 StorageInit, DynamicArray, Mappings (3 levels), Payable
events 2 Events (LOG0-LOG4)
calls 5 ExternalCall (direct, chained, with value)
stack-deep 7 Showcase, StackDeep (16+ params, 20+ locals)

Sample Output

📦 Compilation + Test Time:
   Solar:   1.02s | solc:   1.67s | -39%

📏 Bytecode Sizes (deployed):
   Arithmetic           Solar:  1413B | solc:  4122B | +65% smaller
   Counter              Solar:   105B | solc:   370B | +71% smaller

⛽ Gas Usage (per test):
   ✓ test_AddBasic()    Solar: 5735 | solc: 11182 | -48.7%
   ✓ test_MulBasic()    Solar: 5941 | solc: 11369 | -47.7%

Architecture

HIR → Lowering → MIR → Stack Scheduling → EVM Bytecode

MIR Structure

  • Core Types: ValueId, InstId, BlockId, FunctionId index types
  • MIR Types: UInt(u16), Int(u16), Address, MemPtr, StoragePtr, CalldataPtr, Function, Bool, FixedBytes
  • Value: SSA values (Inst, Arg, Immediate, Phi, Undef) with Immediate constants
  • InstKind: Comprehensive instruction kinds (Arithmetic, Bitwise, Comparison, Memory, Storage, Environment, Calls, Control Flow, SSA Phi/Select)
  • BasicBlock & Terminator: Jump, Branch, Switch, Return, Revert, Stop, SelfDestruct, Invalid
  • Function: SSA value/instruction storage, entry block, attributes (visibility, state mutability, constructor/fallback/receive flags)
  • Module: Top-level container (functions, data segments, storage layout)
  • FunctionBuilder: Ergonomic API for constructing MIR

Lowering (HIR → MIR)

  • Lowerer context with storage slot allocation and function lowering
  • Expression lowering: literals, identifiers, binary/unary ops, calls, ternary, arrays, tuples
  • Statement lowering: variable declarations, blocks, loops, if-statements, returns

Code Generation (MIR → EVM)

  • EvmCodegen struct for bytecode generation
  • Function dispatcher generation (calldata routing by 4-byte selector)
  • Instruction emission with proper stack management
  • Terminator handling (JUMP/JUMPI, RETURN, REVERT, STOP, SELFDESTRUCT)

Completed Features

  1. Basic arithmetic, storage, conditionals, loops, break
  2. External calls with call options ({value: X})
  3. Storage initialization
  4. Event emission (LOG0-LOG4)
  5. Stack-deep functions (spilling to memory when stack > 16)
  6. Pre/post increment/decrement, compound assignments
  7. Single-level and nested mappings (up to 3 levels)
  8. Payable functions + msg.value + call value checks
  9. Dynamic arrays (push/pop/length)

Optimization Passes (implemented, not yet integrated)

  • Constant folding (transform/constant_fold.rs)
  • Dead code elimination (transform/dce.rs)
  • Peephole optimizer (codegen/peephole.rs) - 15+ bytecode patterns

Known Issues (tests skipped, tracked in task list)

  • Signed arithmetic (SDIV, SLT, SGT)
  • Ternary operator edge cases
  • Continue statement in loops
  • Storage pre/post increment
  • Bitwise NOT

Remaining Work

  • Task 291: Struct support (storage layout, field access)
  • Task 293: Constructor arguments (ABI-encode, append to initcode)

This adds the solar-codegen crate which provides:

**MIR (Mid-level IR) Structure:**
- Core MIR index types (ValueId, InstId, BlockId, FunctionId)
- MIR types (UInt(u16), Address, MemPtr, StoragePtr)
- Value (SSA values: Inst, Arg, Immediate, Phi, Undef) and Immediate constants
- Comprehensive InstKind enum (Arithmetic, Bitwise, Comparison, Memory, Storage, Environment, Calls, Control Flow, SSA Phi/Select)
- BasicBlock and Terminator (Jump, Branch, Switch, Return, Revert, Stop, SelfDestruct)
- Function structure with SSA value/instruction storage, entry block, attributes
- Module as top-level container (functions, data segments, storage layout)
- FunctionBuilder API for constructing MIR

**Lowering (HIR → MIR):**
- Main Lowerer context (contract iteration, storage slot allocation, function setup)
- Expression lowering for literals, identifiers, binary/unary ops, calls, ternary
- Statement lowering for variable declarations, blocks, loops, if-statements

**Code Generation (MIR → EVM):**
- Opcode enum and EvmCodegen struct
- generate_dispatcher: Check calldata, dispatch to function by 4-byte selector
- generate_inst: Push operands, emit EVM opcode based on InstKind
- Stack management: push_value for PUSH opcodes, generate_terminator for JUMP/REVERT/RETURN
gakonst and others added 8 commits January 14, 2026 19:46
Implements --standard-json input mode to enable Foundry compatibility:
- Parse standard JSON input from stdin
- Handle import remappings by configuring FileResolver
- Correct contract-to-source file mapping in output
- Write sources to temp directory with proper base_path setup

Amp-Thread-ID: https://ampcode.com/threads/T-019bbe07-a540-7123-b1de-e9cd40659193
Co-authored-by: Amp <amp@ampcode.com>
- Add LoopContext stack to track break/continue targets in nested loops
- Implement proper function argument loading from calldata with CALLDATALOAD
- Store mutable local variables in memory (offset 0x80+) using MLOAD/MSTORE
- Function parameters still handled as SSA values for efficiency

Amp-Thread-ID: https://ampcode.com/threads/T-019bbe07-a540-7123-b1de-e9cd40659193
Co-authored-by: Amp <amp@ampcode.com>
- Add StackScheduler with LoadArg operation for calldata argument loading
- Implement memory load/store operations for mutable local variables
- Add assembler module for bytecode emission
- Improve stack model tracking and manipulation

Amp-Thread-ID: https://ampcode.com/threads/T-019bbe07-a540-7123-b1de-e9cd40659193
Co-authored-by: Amp <amp@ampcode.com>
- Add MemoryLoad, MemoryStore instructions for local variable storage
- Add CalldataLoad instruction for function argument access
- Extend MIR builder with memory allocation helpers

Amp-Thread-ID: https://ampcode.com/threads/T-019bbe07-a540-7123-b1de-e9cd40659193
Co-authored-by: Amp <amp@ampcode.com>
- Implement dataflow-based liveness analysis for register allocation
- Add phi node elimination with parallel copy support
- Fix compiler warnings with #[allow(dead_code)] annotations

Amp-Thread-ID: https://ampcode.com/threads/T-019bbe07-a540-7123-b1de-e9cd40659193
Co-authored-by: Amp <amp@ampcode.com>
- Add tempfile, alloy-json-abi dependencies for standard-json mode
- Add --standard-json CLI flag to opts
- Update version reporting for Foundry compatibility

Amp-Thread-ID: https://ampcode.com/threads/T-019bbe07-a540-7123-b1de-e9cd40659193
Co-authored-by: Amp <amp@ampcode.com>
- Add example programs for codegen testing
- Add integration tests validating EVM bytecode output
- Add function_to_dot() and module_to_dot() for graphviz output
- Format instructions and terminators with operand values
- Color-coded edges for branch conditions (green=true, red=false)
- Add --dot flag to compile example for easy CFG generation
@DaniPopes DaniPopes marked this pull request as draft January 14, 2026 21:22
Tempo Agent and others added 18 commits January 14, 2026 21:59
- Convert while-with-break to if statement in stack scheduler
- Use format string variables directly
- Replace extend with append for vector ranges
- Replace manual div_ceil with method call
- Collapse nested if statements
- Update help CLI test expectations for --standard-json flag
Record shape syntax was causing parsing issues with graphviz.
Simple box nodes work correctly with left-aligned labels.
- Fix uninlined_format_args in display.rs
- Fix manual_div_ceil in liveness.rs
- Fix collapsible_if patterns across multiple files
- Fix drain_collect and extend_with_drain issues
- Fix field_reassign_with_default in standard_json.rs
- Add type aliases to reduce type complexity
- Fix never_loop warning in spill_excess_values
- Add allow attributes for test harness modules
- Fix unused variable in build.rs
- Add unused_crate_dependencies allows for test targets

Amp-Thread-ID: https://ampcode.com/threads/T-019bbe1b-6309-775e-9653-f407558bb00b
Co-authored-by: Amp <amp@ampcode.com>
- Use stderr emitter so compilation errors are visible
- Fix panic on emitted_errors check
- Use box shape instead of record to fix graphviz parsing
…nsfer

- Add keccak256(key, slot) computation for mapping storage access
- Handle compound assignments (+=, -=, etc.) by reading current value first
- Implement address.transfer() and address.send() with 2300 gas stipend
- Fix CALL/STATICCALL/DELEGATECALL to track result value in scheduler
- Create fresh ValueIds for CALL arguments to avoid stack reuse issues

Fixes MemoryLimitOOG when running Advanced.sol tests with mappings and
external calls (transfer).
…ution

- Add can_emit_value() to check if a value is available for emission
- Add instruction_executed_untracked() for values that become stale in loops
- Fix builtin member resolution to preserve Builtin reference
- Handle type conversion calls (ICallee(addr), uint256(x)) that were returning 0
- Fix compute_member_selector to resolve type cast calls for external contracts
- Add nested mapping slot computation for multi-level mappings (m[a][b])
- Support dynamic array storage access and .length member
- Implement pre/post increment/decrement operators
- Add builtin module member access (msg.sender, block.timestamp)
- Add generate_synthetic_constructor() to create constructor for contracts without one
- Constructor emits SSTORE for each state variable with an initializer
- Modify generate_deployment_bytecode() to run constructor code before CODECOPY/RETURN
- Add generate_constructor_code() helper that strips trailing STOP from constructor bytecode

State variables like `uint256 public value = 42` are now properly initialized.
- Implement lower_emit() in stmt.rs that computes event signature hash for topic0
- Handle indexed parameters as additional topics, non-indexed as ABI-encoded data
- Add compute_event_signature() and type_to_abi_string() helpers
- Add log0-log4 builder methods in builder.rs

emit statements were previously no-ops, now properly emit LOG0-LOG4.
- Fix typo: UEUse -> upward-exposed uses and defs
- Escape doc comments with brackets to prevent broken intra-doc links
- Mark README code block as text to avoid arrow parsing errors
- Add #[ignore] to foundry tests that require anvil/solc
…suite

- Add testdata/foundry-tests with 6 test contracts and 21 test cases
- Test contracts: Counter, Events, ExternalCall, StorageInit, Showcase, StackDeep
- Rewrite test harness to use FOUNDRY_SOLC=solar and forge test
- Use per-test output directories (--out, --cache-path) for parallel execution
- Parses forge output and validates all tests pass
- Add payable check in function dispatcher: non-payable/view/pure
  functions revert if called with ETH (CALLVALUE != 0)
- Support {value: X} call options for external calls
- Add extract_call_value helper to parse call options
- Add Payable.sol test contract and Payable.t.sol tests

The payable check is emitted in emit_payable_check() after the function
JUMPDEST, before generating the function body. For payable functions,
no check is emitted.

External calls now correctly pass the value from {value: X} options
to the CALL opcode instead of hardcoding 0.
Optimization passes for the Solar codegen:

1. Constant Folding (HIR-level):
   - Evaluates constant expressions at compile time
   - Handles binary ops, unary ops, ternary expressions
   - 15 unit tests covering arithmetic, bitwise, comparison ops

2. Dead Code Elimination (MIR-level):
   - Removes instructions whose results are never used
   - Preserves side-effect instructions (SSTORE, CALL, LOG, etc.)
   - Uses value use analysis to identify dead code

Also adds InstKind::has_side_effects() helper for DCE correctness.

Amp-Thread-ID: https://ampcode.com/threads/T-019bbfc4-4637-71ce-a483-63911a6290f5
Co-authored-by: Amp <amp@ampcode.com>
1. Peephole Optimizer (not yet integrated):
   - 15+ optimization patterns (PUSH0 ADD→nop, SWAP1 SWAP1→nop, etc.)
   - 17 unit tests passing
   - Not integrated into pipeline yet (breaks jump targets)
   - TODO: integrate at assembler level before label resolution

2. Dynamic Arrays - fix pop() bug:
   - Fixed StackUnderflow in pop() caused by reusing slot_val
   - Reorder operations to avoid consuming values twice
   - Added DynamicArray.sol and DynamicArray.t.sol (6 tests pass)

Amp-Thread-ID: https://ampcode.com/threads/T-019bbfec-3633-7296-8139-120e925d8fb3
Co-authored-by: Amp <amp@ampcode.com>
- Split monolithic test into 6 parallel projects by category:
  arithmetic, control-flow, storage, events, calls, stack-deep
- Each project runs both Solar and solc for comparison
- Reports compilation time, bytecode sizes, and per-test gas usage
- Separate out-{compiler}/ directories for artifacts

Test results (95 tests):
- Bytecode: Solar 61-72% smaller than solc
- Gas: Solar 2-48% cheaper on most operations
- Stack-deep: Solar compiles what solc cannot

Added edge case tests exposing 5 bugs (tests skipped, tasks created):
- Signed arithmetic (SDIV, SLT, SGT)
- Ternary operator
- Continue statement
- Storage pre/post increment
- Bitwise NOT
- Test contracts (.t.sol) always compiled with solc for reliable test logic
- Contract-under-test bytecode injected via SOLAR_<NAME>_BYTECODE env vars
- Tests deploy Solar bytecode when env var present, fallback to solc
- Updated .gitignore to exclude out/ and cache/ directories
- stack-deep tests marked #[ignore] (solc can't compile, Solar has bugs)
The scheduler's StackModel was drifting from the actual EVM stack during
complex codegen sequences, causing incorrect DUP operations.

Changes:
- Add StackEffect struct to describe pops/pushes for each opcode
- Add StackPush enum to specify tracked/untracked stack entries
- Add emit_stack_op() helper for DUP/SWAP/POP with automatic model updates
- Add emit_op_with_effect() helper for opcodes with known stack effects
- Fix InstKind::Select to use per-operation stack tracking
- Update CALL/STATICCALL/DELEGATECALL to use emit_op_with_effect

The root cause was Select emitting 6 stack-mutating opcodes but only
updating the scheduler once at the end, causing cumulative drift.
…onment opcodes

- Fix Select instruction (ternary) stack manipulation: use DUP3,DUP3 instead of
  DUP2,DUP4 and SWAP1 instead of SWAP2 to correctly compute f + cond*(t-f)
- Add handling for environment opcodes in emit_value_fresh: CallValue, Caller,
  Origin, CalldataSize, Timestamp, BlockNumber - these can be safely re-emitted
- Move spill slot base from 0x100 to 0x1000 to avoid conflicts with dynamic
  memory allocations (structs, arrays)
Tests that calling between overloaded library functions works correctly
(e.g., find(key) calling find(key, true)).
Tests for functions that take multiple struct parameters, which
currently have issues with memory layout.
Implement solc-style backward layout analysis for better stack scheduling:

- Add StackShuffler: converts source to target layout with minimal ops
  - Phase 1: Ensure multiplicities (DUP values needing copies)
  - Phase 2: Arrange positions (SWAP to correct slots)
  - Phase 3: Pop excess values

- Add LayoutAnalysis: backward analysis through instructions
  - analyze_backward() computes ideal entry layout from exit
  - compute_entry_for_instruction() for single instruction

- Add helper functions for ideal operand layouts
  - ideal_binary_op_entry/ideal_unary_op_entry
  - is_freely_generable() for literals/labels

- Integration with StackScheduler
  - shuffle_to_layout(), prepare_binary_op(), prepare_unary_op()

All 8 shuffler unit tests pass.
Add CFG simplification passes:

1. Block Merging (CfgSimplifier):
   - Merge A→B when A has single successor B and B has single predecessor A
   - Saves 8 gas per eliminated JUMP

2. Empty Block Elimination:
   - Remove blocks with only unconditional jump
   - Redirect predecessors to final target

3. Dead Function Elimination (DeadFunctionEliminator):
   - Build reachability from entry points (public/external/constructor)
   - Remove unreachable functions

4. Call Graph Analysis (CallGraphAnalyzer):
   - Build call graph for module
   - Detect recursive functions via DFS

Includes CfgSimplifyStats for tracking optimization metrics.
All 8 unit tests pass.
Add infrastructure for stack layout merging at control flow merge points:

- BlockStackLayout: represents stack layouts with SmallVec slots
- combine_stack_layouts(): computes common layout from predecessors
- estimate_shuffle_cost(): estimates DUP/SWAP/POP operations needed

StackScheduler block layout methods:
- set/get_block_entry_layout: manage target layouts per block
- record/get_block_exit_layout: track actual layouts
- compute_merge_layout: compute entry from predecessor exits
- shuffle_to_block_entry: shuffle current stack to target
- init_from_block_entry_layout: initialize stack from expected

12 new unit tests for layout merging functionality.
All 117 tests pass.
Partial implementation of nested struct handling:

- calculate_memory_words_for_type: compute flattened memory layout
- get_struct_field_memory_offset: get byte offset for nested fields
- compute_nested_memory_struct_info: handle chained member access
- copy_struct_storage_to_memory/memory_to_storage: recursive copy helpers
- Fix variable declaration to allocate memory for uninitialized structs

testNestedMemoryValue passes - nested memory struct access works.
Cross-location copying needs further work.
Implement storage-to-memory and memory-to-storage copying for nested structs:

- copy_storage_to_memory(): recursive copy from storage to memory
- copy_memory_to_storage(): recursive copy from memory to storage
- compute_nested_storage_slot_with_type(): recursive storage slot calculation
- compute_nested_memory_struct_info_with_type(): recursive memory offset calc

Supports arbitrarily deep nesting (3+ levels tested).

Test files:
- DeepNested.sol/t.sol: 3-level nested struct storage-memory round trip
- DeepNestedSimple.sol/t.sol: 1/2/3-level storage access

All 117 unit tests and 33 struct foundry tests pass.
gakonst and others added 2 commits January 24, 2026 13:37
These instruction kinds can be re-emitted when their results are needed
as CALL operands but aren't on the stack. This fixes ICE when compiling
contracts that use keccak256 results in external calls (e.g., unifap-v2).

6/8 unifap-v2 tests now pass. Remaining 2 failures are runtime errors
in setUp() - needs further investigation.
- Fix ExprKind::Ternary lowering to use proper branching instead of
  select() which only works for single values. Tuple ternaries now
  write to scratch memory and merge block reads values back.

- Fix abi.encodePacked to properly pack values based on their types
  instead of 32-byte padding. Returns proper bytes memory format
  (length prefix + tightly packed data).

- Add get_packed_size_from_expr/get_packed_size_from_hir_type for
  type-based size inference (address=20, bool=1, bytesN=N, etc.)

- Update lower_return to handle ternary expressions returning tuples
  by reading all values from scratch memory.

All 34 unifap-v2 tests now pass.

Amp-Thread-ID: https://ampcode.com/threads/T-019bf223-95e1-7439-87e4-4ea6fff69ab2
Co-authored-by: Amp <amp@ampcode.com>
…structure (#749)

Improve current the codegen (MIR) branch by adding: pass infrastructure,
a textual MIR format (printer + parser), the `solar-mir-opt` tool, and
an `SCCP` implementation.

Builds on top of the existing codegen work in
#693 (`feature/codegen-mir`).

By adding a way to serialise and deserialise (and to verify) the MIR
module, it will make compiler more modular and easier to test individual
analysis and passes. In addition to it, by having a Pass Manager, it
will be much easier to use some analysis information between individual
passes.

**Pass manager**

`AnalysisPass` and `TransformPass` traits with `AnalysisManager` for
caching results by type. All transforms in `evm.rs` (`jump-threading`,
`cfg-simplify`, `dce`) now go through the `PassManager` instead of being
called directly.

**MIR text format**

Printer (`--emit=mir`) and hand-written recursive-descent parser
covering all instruction kinds. Parser errors show source-line snippets
with a caret. Round-trip property tests verify accuracy on every
fixture.

**solar-mir-opt**

Standalone tool for running individual passes. Accepts `.sol` and `.mir`
input, supports `--passes a,b,c`, `--print-after-each`, and
`--pipeline-default`.

**New passes**

- `SCCP` (Sparse Conditional Constant Propagation) — Propagates
constants through the CFG, folds branches with known conditions, marks
unreachable blocks. Resolves
#702.
- `MIR` validator — checks SSA invariants (`defined-before-use`,
pred/succ consistency, phi coverage, etc.)

**Liveness / DCE minor fixes**

- Precomputed InstId -> ValueId maps replacing O(n) scans in both
liveness and DCE
  - Phi-aware live_out equation for `Value::Phi` nodes
  - Fixed `sema/emit.rs` panicking on `--emit=bin` via todo!()

**Tests**

- `.mir` LIT tests under `tests/ui/codegen/mir/` (DCE, CSE, SCCP, jump
threading, cfg simplify, phi, switch, pipeline)
- `.sol` UI tests under `tests/ui/codegen/` (arithmetic, storage,
branches, loops, mappings, ternary, function calls, recursion, etc.)
- Round-trip integration tests + validator integration tests on all
fixtures
- `Mode::Mir` added to the test runner so `cargo uitest` covers both
`.sol` and `.mir`
@djolertrk
Copy link
Copy Markdown
Contributor

Hi @gakonst. I am more than happy to help with bringing this to the level we can merge it with main.

Can we please trigger the CI here?

@djolertrk
Copy link
Copy Markdown
Contributor

Hi @gakonst. I am more than happy to help with bringing this to the level we can merge it with main.

Can we please trigger the CI here?

Ping :)

# Conflicts:
#	Cargo.lock
#	crates/config/build.rs
#	crates/sema/src/ty/ty.rs
#	crates/sema/src/typeck/checker.rs
#	crates/sema/src/typeck/mod.rs
#	tests/ui/typeck/function_calls/event_error_context.sol
#	tests/ui/typeck/function_calls/event_error_context.stderr
#	tests/ui/typeck/function_implementation_checks.sol
#	tests/ui/typeck/function_implementation_checks.stderr
@djolertrk
Copy link
Copy Markdown
Contributor

Thanks @DaniPopes :) I will be working on fixing the CI.

This is the first part: it had some stale github module references, so
it couldn't run the tests.
@djolertrk
Copy link
Copy Markdown
Contributor

The next important step is to introduce infra for codegen and runtime comparison with solc. I will work on that.

@djolertrk
Copy link
Copy Markdown
Contributor

The next important step is to introduce infra for codegen and runtime comparison with solc. I will work on that.

This is kind of done: #760. I identified some codegen issues while compiling the contracts, and the fixes are included into #760 (along with sema fix #761). Furthermore, the CI for #760 is still red, since there are some mismatches detected during runtime of these contracts (compared to solc) -- so it is the main goal to fix those.

djolertrk added 2 commits May 13, 2026 16:18
Adds `codegen` CI using
[solidity-compiler-benchmarks](https://github.com/walnuthq/solidity-compiler-benchmarks)

The workflow builds `solar`, compiles the benchmark corpus with both
`solc` and `solar`, then checks:
  - bytecode size
  - gas usage on Anvil
  - runtime output equality for the same calls/inputs

While trying to build the contracts from the set of contracts I shared,
some `codegen` issues occurred. In this PR, I fixed a `codegen` bug:
cross-block MIR values were not always spilled before leaving a block,
so successor blocks could lose values after the stack model was cleared.
The fix improves liveness propagation and preallocates/spills live
cross-block values in EVM `codegen`.
Besides that, there is `fix(sema): canonicalize bare integer types`, and
it should be a separate PR; it fixes `uint/uint256` alias handling,
detected in `maple-erc20`.
Continue with fixing `codegen` to avoid mismatches in runtime comparing
to `solc`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants