refactor: rework adress definitions & introduce region reservation by nagisa · Pull Request #7 · anza-xyz/agave-runtime

nagisa · 2026-06-11T14:32:30Z

In order to introduce sysvar regions, it is becoming necessary to make our address space allocation somewhat more future proof. One of such future proofing example is allocating enough reserved space for account payloads if we were to increase the number of accounts that can be specified in a transaction.

This, however, introduces a concept of a "gap" between regions which did not exist before and complicates the conversions between region indices and guest addresses.

After a few experiments I found that adding a structure and some methods to describe the expected memory layout is the least ugly option. As we only have one gap currently, the conversion in either direction is still relatively straightforward.

Having host addresses represented by VmSlice seems not right? I switched those functions to return a regular raw pointer. Which in turn simplified the memory mapping creation somewhat.

@nagisa

* Create tests * @nagisa's review

* Write function to update account permissions * Add unit test * Add integration test * Use all accounts as mutable

Co-authored-by: Lucas <lucas.tnagel@gmail.com>

@nagisa

* Fix account mapping * Add test and fix edge case * Verify account owner for permissions * Fix rebase problems * Remove optimization * Change condition * @nagisa's review * Update program-runtime/src/memory_context.rs Co-authored-by: Simonas Kazlauskas <github@kazlauskas.me> * Format file --------- Co-authored-by: Simonas Kazlauskas <github@kazlauskas.me>

@nagisa

* Implement syscall * Write a test * Amend function with writability change * Finish implementation * Specify index in transction * @nagisa's review

@nagisa

* Implement syscall * Write integration test * @nagisa's review * Change name

In order to introduce sysvar regions, it is becoming necessary to make our address space allocation somewhat more future proof. One of such future proofing example is allocating enough reserved space for account payloads if we were to increase the number of accounts that can be specified in a transaction. This, however, introduces a concept of a "gap" between regions which did not exist before and complicates the conversions between region indices and guest addresses. After a few experiments I found that adding a structure and some methods to describe the expected memory layout is the least ugly option.

Copilot

Pull request overview

This PR refactors ABIv2 guest memory address definitions by introducing a GuestMemorySection abstraction, and extends the memory layout to reserve space for future “sysvar regions” via an address-space gap.

Changes:

Introduce GuestMemorySection to describe guest memory layout in a structured way and compute per-section region indices/guest address ranges.
Add a reserved “gap” to support sysvar regions while keeping region indices contiguous.
Update transaction context, syscalls, and memory mapping code to use section-based address/range calculations instead of hard-coded base constants.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
transaction-context/src/vm_addresses.rs	Adds `GuestMemorySection`, defines the new ABIv2 section layout, introduces sysvar reservation, and computes `NUM_REGIONS`.
transaction-context/src/transaction.rs	Updates scratchpad and region computations to use `GuestMemorySection`; adjusts region resize dispatch.
transaction-context/src/transaction_accounts.rs	Switches account payload region addressing to section-based ranges.
transaction-context/src/instruction.rs	Updates instruction frame VM slice addressing to section-based ranges.
transaction-context/src/instruction_accounts.rs	Updates resize payload VM address computation to section-based ranges.
syscalls/src/lib.rs	Updates region index/address handling and adjusts tests to iterate the new section layout.
program-runtime/src/memory_context.rs	Reworks ABIv2 region construction using the new section model (but currently contains debug leftovers).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+            return_data_scratchpad: VmSlice::new(
+                vm_addresses::RETURN_DATA_SCRATCHPAD_SECTION
+                    .guest_address_range()
+                    .start,
+                0,
+            ),
            cpi_scratchpad: VmSlice::new(
-                GUEST_INSTRUCTION_DATA_BASE_ADDRESS.saturating_add(
-                    GUEST_REGION_SIZE.saturating_mul(number_of_top_level_instructions as u64),
-                ),
+                vm_addresses::INSTRUCTION_DATA_SECTION
+                    .guest_address_range()
+                    .start,
                0,
            ),


        if let Some(caller_index) = caller_index {
            self.transaction_frame.total_number_of_instructions_in_trace = self
                .transaction_frame
                .total_number_of_instructions_in_trace
                .saturating_add(1);
            instruction.index_of_caller_instruction = caller_index;
-            let next_ptr = self
-                .transaction_frame
-                .cpi_scratchpad
-                .ptr()
-                .saturating_add(GUEST_REGION_SIZE);
-            self.transaction_frame.cpi_scratchpad = VmSlice::new(next_ptr, 0);
+            let section = vm_addresses::INSTRUCTION_DATA_SECTION;
+            let scratchpad_addrs =
+                section.guest_address_range_for(caller_index.saturating_add(1) as usize);
+            self.transaction_frame.cpi_scratchpad = VmSlice::new(scratchpad_addrs.start, 0);
        }


+            a if vm_addresses::INSTRUCTION_DATA_SECTION.contains_guest_ptr(a) => {
                if !old_region.writable {
                    return Err(InstructionError::InvalidArgument);
                }
-                let ix_address = vm_address
-                    .checked_sub(GUEST_INSTRUCTION_DATA_BASE_ADDRESS)
+                let ix_idx = vm_addresses::INSTRUCTION_ACCOUNTS_SECTION
+                    .region_index_containing(a)
                    .ok_or(InstructionError::InvalidArgument)?;
-                let ix_idx = abiv2_region_index_from_vm_address(ix_address);
                let ix = self.instruction_data.get_mut(ix_idx);


        assert_eq!(
            transaction_context.transaction_frame.cpi_scratchpad.ptr(),
-            GUEST_INSTRUCTION_DATA_BASE_ADDRESS.saturating_add(GUEST_REGION_SIZE.saturating_mul(3))
+            vm_addresses::INSTRUCTION_ACCOUNTS_SECTION
+                .guest_address_range_for(3)
+                .start
        );


+    let section = INSTRUCTION_ACCOUNTS_SECTION;
+    let regions = v2_regions
+        .get_mut(dbg!(section.region_index_range()))
+        .unwrap();
    transaction_context.instruction_accounts_regions(regions);


+pub const INSTRUCTION_DATA_SECTION: GuestMemorySection =
+    GuestMemorySection::new_following(SYSVAR_ACCOUNT_SECTION, MAX_INSTRUCTION_TRACE_LENGTH);
+pub const INSTRUCTION_ACCOUNTS_SECTION: GuestMemorySection =
+    GuestMemorySection::new_following(INSTRUCTION_DATA_SECTION, MAX_INSTRUCTION_TRACE_LENGTH);
+


nagisa · 2026-06-11T15:05:37Z

Looks like I'll need to make modifications to sbpf before this can land – AlignedMemoryMapping has code internally to create all the intermediate mappings that are now making things go wrong, or maybe this idea is bust actually... will find out.

Lichtso · 2026-06-11T15:35:44Z

What things go wrong?

How is this different from what we had before when AlignedMemoryMapping skipped the first region where the text section / bytecode resides (before SBPFv3)?

nagisa · 2026-06-11T16:09:46Z

What things go wrong?

How is this different from what we had before when AlignedMemoryMapping skipped the first region where the text section / bytecode resides (before SBPFv3)?

Well, I guess its not AlignedMemoryMapping that's going wrong, but there's definitely a mismatch between what this PR is implementing and what AlignedMemoryMapping is implementing. To start with, AlignedMemoryMapping still creates all the 0x10??? mappings to fill the reserved gap I allocated for the accounts. If we continue including the filler regions with AlignedMemoryMapping, there's little point to this PR, other than it being a very slightly nicer to read code.

And then we still have some code that directly deal with get_regions_mut() but now that has all those introduced fake regions that the new code on agave size disagrees with.

There's point to be made that we might be fine with those filler regions being present, as it how we achieve efficient address translation. When we start reusing the MemoryMapping between transactions this mapping setup becomes a simple 2MB allocation per processing thread and is fully amortized at startup.

But either way I need to rewrite some code 😓

The merge-base changed after approval.

@nagisa

* Fix account mapping * Add test and fix edge case * Verify account owner for permissions * Fix rebase problems * Remove optimization * Change condition * @nagisa's review * Update program-runtime/src/memory_context.rs Co-authored-by: Simonas Kazlauskas <github@kazlauskas.me> * Format file --------- Co-authored-by: Simonas Kazlauskas <github@kazlauskas.me>

yihau and others added 25 commits June 8, 2026 23:00

[agave-runtime]: ci update (#1)

2bea5e2

Split v1 and v2 in memory contexts

4f99cd8

Create v2 regions

8b32018

Create a VM test

50e3f73

Fix panic

c440bc7

rebase fixups

b4bc6dc

Having host addresses represented by VmSlice seems not right? I switched those functions to return a regular raw pointer. Which in turn simplified the memory mapping creation somewhat.

Fix example

a7b942c

Fix the other test

fdf8fef

Fix rebase dependency

eb09990

Tests for region creation (#2)

25f0f2d

* Create tests * @nagisa's review

refactor: no manual shifting in create_abiv2_regions (#3)

e1e3d19

[ABIv2] Update account permissions at instruction edge (anza-xyz#4)

ba9a8a6

* Write function to update account permissions * Add unit test * Add integration test * Use all accounts as mutable

Add unsafe to no mangle

b8cd1aa

Create ABIv2 experimental feature (anza-xyz#5)

6bbda0c

Co-authored-by: Lucas <lucas.tnagel@gmail.com>

Adjust feature gate for deployment (anza-xyz#6)

895acf0

Implement the SetBufferLength syscall (#1)

45b673f

Update lock file after reabse

92f42ab

return scratchpad must be resized before writing (anza-xyz#8)

4c61658

Change owner syscall (anza-xyz#9)

6150207

* Implement syscall * Write a test * Amend function with writability change * Finish implementation * Specify index in transction * @nagisa's review

Fix resize of readonly region (anza-xyz#11)

705f360

Create transfer lamports syscall (anza-xyz#10)

932a2df

* Implement syscall * Write integration test * @nagisa's review * Change name

Fixes for CI to work

ccc8dea

Run ABIv2 integration tests in CI

c0d5480

nagisa requested a review from Copilot June 11, 2026 14:32

nagisa requested a review from a team as a code owner June 11, 2026 14:32

Copilot started reviewing on behalf of nagisa June 11, 2026 14:32 View session

Lichtso previously approved these changes Jun 11, 2026

View reviewed changes

remove stray dbg!s

7f12f50

nagisa dismissed Lichtso’s stale review via 7f12f50 June 11, 2026 14:37

Lichtso previously approved these changes Jun 11, 2026

View reviewed changes

Copilot AI reviewed Jun 11, 2026

View reviewed changes

nagisa marked this pull request as draft June 11, 2026 15:13

LucasSte force-pushed the master branch from d2ac304 to c14939a Compare June 23, 2026 21:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor: rework adress definitions & introduce region reservation#7

refactor: rework adress definitions & introduce region reservation#7
nagisa wants to merge 26 commits into
anza-xyz:masterfrom
nagisa:abiv2-address-definition-refactor

nagisa commented Jun 11, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

nagisa commented Jun 11, 2026 •

edited

Loading

Uh oh!

Lichtso commented Jun 11, 2026

Uh oh!

nagisa commented Jun 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

nagisa commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

nagisa commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Lichtso commented Jun 11, 2026

Uh oh!

nagisa commented Jun 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

nagisa commented Jun 11, 2026 •

edited

Loading

nagisa commented Jun 11, 2026 •

edited

Loading