Encode cageids to enable address translations: (arg, cageid) tuple independence from calling cages.#913
Open
stupendoussuperpowers wants to merge 6 commits intofix-targetcageidfrom
Open
Encode cageids to enable address translations: (arg, cageid) tuple independence from calling cages.#913stupendoussuperpowers wants to merge 6 commits intofix-targetcageidfrom
(arg, cageid) tuple independence from calling cages.#913stupendoussuperpowers wants to merge 6 commits intofix-targetcageidfrom
Conversation
Member
|
Isn't the alternative just to do the translation yourself before the call?
Why overload the meaning of the 64 bit cageid?
…On Mon, Mar 9, 2026 at 10:02 PM Sanchit Sahay ***@***.***> wrote:
Building on the discussion here:
We now allow:
1. Grates/cages to indicate to threei that the argument they are
passing needs translation.
2. Internally translate arguments on call to make_threei_call.
3. Ensure support for existing ways to use make_threei_call both in
glibc and cages are still supported.
How this works:
1. The MSB of cageid for each argument in make_threei_call, when set
to 1, translate the address in the argument to be host-coded.
2. Example can be seen in the tests attached, when a grate wants to
translate an address, it's indicated by modifying the cageid MSB.
3. make_threei_call changes the cageid back to a a regular cageid so
that existing behaviour is not broken.
------------------------------
Tests added try to pass a grate-local address to the make_threei_call
invocation made on behalf of child, and ensure that copy_data_between_cages
hasn't regressed:
***@***.***:~/lind-wasm$ sudo lind-boot diff-cage-args_grate.cwasm diff-cage-args.cwasm
[Grate|interpose-exec] Handling function ptr: 2 from cage: 1
[Grate|interpose-exec] In exec_grate 1 handler for cage: 1
[rawposix|open] path="/tmp/redirected.txt" oflag=0 mode=0
Hello world. FD=3
[Grate|interpose-exec] Handling function ptr: 3 from cage: 1
Goodbye world! ret=4321 buf=helloworld
[Grate|interpose-exec] PASS
------------------------------
You can view, comment on, or merge this pull request online at:
#913
Commit Summary
- dceeff1
<dceeff1>
Encode arg cageid translation flag
- 3dd5bc7
<3dd5bc7>
make it look not insane
- 6949282
<6949282>
Changes post test runs.
File Changes
(4 files <https://github.com/Lind-Project/lind-wasm/pull/913/files>)
- *M* src/glibc/lind_syscall/addr_translation.h
<https://github.com/Lind-Project/lind-wasm/pull/913/files#diff-d45618d070aafae16608503f61eac6808cf6800305c2ec7b48a9d084b62c3e9a>
(19)
- *M* src/glibc/lind_syscall/lind_syscall.c
<https://github.com/Lind-Project/lind-wasm/pull/913/files#diff-31a1f23f15e47b7dfd7a496502e4a1a9e8628b76ef3cb7f10edba127752d83b0>
(17)
- *A* tests/grate-tests/diff-cage-args.c
<https://github.com/Lind-Project/lind-wasm/pull/913/files#diff-0cb7e4f7e561aaecdc7550af75aa1fbe653086c156c54330f51b2a532830c15e>
(13)
- *A* tests/grate-tests/diff-cage-args_grate.c
<https://github.com/Lind-Project/lind-wasm/pull/913/files#diff-1a9872dfed20861bff116ca10fc86f820c3cb72de013132c674fdbec93e95494>
(136)
Patch Links:
- https://github.com/Lind-Project/lind-wasm/pull/913.patch
- https://github.com/Lind-Project/lind-wasm/pull/913.diff
—
Reply to this email directly, view it on GitHub
<#913>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGROD2VKEI34K7YDFCFYPL4P5ZTPAVCNFSM6AAAAACWMQWPTKVHI2DSMVQWIX3LMV43ASLTON2WKOZUGA2DQOJQGMYTMNQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Contributor
Author
|
My thoughts (as briefly discussed in today's meeting): In the current model, the grate has no way to do this address translation, it has no information about its global address. I am not sure whether exposing global address to cages would need some fresh evaluations on memory-safety vectors. In the current system all address translations are done only in glibc, so this tries to stick with that model. The change to cageid is only observed in within the make_threei_call function when called from a cage, and the original value is preserved for every other interface. My thinking is: 1. (arg, argcageid) were explicitly designed to be useful for address translations, so this tries to limit the scope of translations to this tuple, and 2. the current MAX_CAGEID (1024) is an i32, but the cageids we pass around are u64 (for compatibility reasons) so this should not affect the maximum range for cageids. |
Member
|
Memory safety isn't an issue here.
One thing I'm not sure about what you are saying is how a situation with a
cageid + address is handled. Let me try to think aloud a moment here
because I think this might be the easiest way to understand what / why.
I think 3i needs to do the translation of addresses anyways to at least
check the upper bits are allowed. In other words, 3i needs to make sure
that a call accesses correct memory ranges for things like copying memory,
etc. So, given a 32 bit address there are three places this could be
done. First, this could be done in glibc / userspace in the program making
the call. This would work, but notice that 3i also has to check this, so
redoes this work. Second, this could be done / checked at every 3i
syscall. This would work, but seems moderately heavyweight. Thind, this
can be done only at the time a call to copy memory (or RawPOSIX) is
performed. I think this actually works because other pointers aren't read
throughout the process so it doesn't really matter if you've fixed the
addresses or not. So, maybe the third option is best assuming there aren't
data structures with nested pointers, etc. (I'm not sure this is allowed
in syscalls.)
However, I feel like I may be missing something. Please chime in and loop
in others. I feel like we may have discussed this before but didn't come
to the same point.
…On Mon, Mar 9, 2026 at 11:17 PM Sanchit Sahay ***@***.***> wrote:
*stupendoussuperpowers* left a comment (Lind-Project/lind-wasm#913)
<#913 (comment)>
My thoughts (as briefly discussed in today's meeting):
In the current model, the grate has no way to do this address translation,
it has no information about its global address.
I am not sure whether exposing global address to cages would need some
fresh evaluations on memory-safety vectors.
In the current system all address translations are done only in glibc, so
this tries to stick with that model.
The change to cageid is only observed in within the make_threei_call
function when called from a cage, and the original value is preserved for
every other interface. My thinking is: 1. (arg, argcageid) were explicitly
designed to be useful for address translations, so this tries to limit the
scope of translations to this tuple, and 2. the current MAX_CAGEID (1024)
is an i32, but the cageids we pass around are u64 (for compatibility
reasons) so this should not affect the maximum range for cageids.
—
Reply to this email directly, view it on GitHub
<#913 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGRODZGWQGRUUHMHL2LRG34P6CMDAVCNFSM6AAAAACWMQWPTKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DAMRYGMZDMNZVHA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Contributor
Author
|
The solutions you've pointed out all make sense to me and would be implementable albeit some might be more complicated than others given how things are currently written. I've had a bit of a think about this and went back to the original address translation issues/PR to dig deeper. Just so that we all have a clearer picture of the scope and reason for this PR, I want to ensure we all start with the same set of assumptions about the system. I might be rehashing some old points but it should be helpful to ground the discussion. First, copying of memory happens in two ways, and the way checks for these happen also differ.
In both the above cases, our Rust codebase (lind, rawposix, threei, etc.) assumes no responsibility for address translations. At most they can detect an error but there is no attempt to "correct" the errors. (More on this later). Coming to where address translations happen, Post #469 which was addressing #435, for all regular syscalls addresses are translated to host-space in glibc before it reaches Rust. Post #582, The only remaining wrinkle left to iron-out in address-conversions is the
Let me know if I misunderstood some of the points you were making. Tagging @Yaxuan-w and @rennergade to double-check my assumptions about the current state of the codebase. |
Contributor
|
The memory base already is in userspace so exposing it in an easier way shouldnt be a problem. |
Member
|
option 3 is the best for security since we should be doing those checks
anyways. Is there any other reason it would be a problem to do them in 3?
One downside which is perhaps why we did it like this is that in theory we
could have a 64 bit grate, etc. and doing the translation once at call time
avoids us needing to do it again and again for different grates.
…On Tue, Mar 10, 2026 at 10:35 AM Nicholas Renner ***@***.***> wrote:
*rennergade* left a comment (Lind-Project/lind-wasm#913)
<#913 (comment)>
The memory base already is in userspace so exposing it in an easier way
shouldnt be a problem.
—
Reply to this email directly, view it on GitHub
<#913 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGROD4XSDM4G2E2VCH4DM34QAR2ZAVCNFSM6AAAAACWMQWPTKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DAMZRHA4TANBXGI>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Contributor
Author
We could run checks at 3 through the various typemap helpers rawposix already uses but are currently no-ops. This should not be that big of a hurdle but might need some new negative unit tests. But should the scope of these also be to try and convert the address to be host-coded? In which case, address translations are unnecessary at any stage.
Yeah I think this point was brought up in the original issue as well. |
Member
|
If I understand the previous discussion correctly, option 3 does not necessarily conflict with the current glibc-based translation approach. It can support both fixed and non-fixed addresses. In practice, it just adds an additional option where grate developers do not need to perform address translation in userspace before invoking the syscall. Regarding performance, one possible approach could be to add a conditional compile flag. If a user prioritizes performance, we could disable the RawPOSIX-side checks and assume that all addresses have already been fixed before entering RawPOSIX. From my perspective, based on what I understand about this PR, the work is important for supporting the chroot grate. For the current stage, the lowest-cost and fastest path to getting this working might be to perform the address translation in glibc so that it matches the existing RawPOSIX address checks. At the same time, we could document the design discussion and potential improvements in an issue and revisit it later (for example after the paper submission), when we have more time to refine the design. As for where exactly the address translation should live if we keep it in glibc, I don't have a strong preference. From the discussion so far, one possible option would also be to let grate developers perform the translation before invoking the syscall. This might also help when testing new RawPOSIX functionality, since it makes the address assumptions explicit. |
Contributor
Author
|
For now, we can do either of these, and then pick something more robust later:
Both these are low effort changes (a few lines at best), and would enable chroot and similar grates to work properly. Let me know what the best approach is, and I can submit a PR for that. For the longer run: We need to decide a strict boundary on who is responsible for address translations, who does what checks, and why. Both the above stated approaches are inconsistent at best (grates must translate/indicate an address for make_threei_call, but mustn't for copy_data), and confusing for grate authors at worst (rawposix will crash on invalid address instead of reporting a proper error). My suggestion is that rawposix must receive only one kind of address, either user or host, having to support both at the same time will lead to a lot of unnecessary address-coercion codepaths. |
Member
|
Why does a grate need to translate an address? Can't RawPOSIX / the 3i
memory copying code do it? If we add it those two places, is that all we
need?
…On Tue, Mar 10, 2026 at 11:46 PM Sanchit Sahay ***@***.***> wrote:
*stupendoussuperpowers* left a comment (Lind-Project/lind-wasm#913)
<#913 (comment)>
For now, we can do either of these, and then pick something more robust
later:
1. Expose lind_get_memory_base to userspace, let grates convert their
own addresses.
2. This PR which lets glibc handle it, once indicated.
Both these are low effort changes (a few lines at best), and would enable
chroot and similar grates to work properly. Let me know what the best
approach is, and I can submit a PR for that.
For the longer run:
We need to decide a strict boundary on who is responsible for address
translations, who does what checks, and why. Both the above stated
approaches are inconsistent at best (grates *must* translate/indicate an
address for make_threei_call, but *mustn't* for copy_data), and confusing
for grate authors at worst (rawposix will crash on invalid address instead
of reporting a proper error). My suggestion is that rawposix must receive
only one kind of address, either user or host, having to support both at
the same time will lead to a lot of unnecessary address-coercion codepaths.
—
Reply to this email directly, view it on GitHub
<#913 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGROD722FP2RR6SDPC36NL4QDOSRAVCNFSM6AAAAACWMQWPTKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DAMZWGA2DEMRSGU>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Contributor
End-to-End Test ReportTest PreviewUnified Test Report grate harness
Cases
wasm harnessTest ReportDeterministic TestsSummary
Test Results by Category
Fail TestsSummary
Test Results by Category
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Contributor
End-to-End Test ReportTest PreviewUnified Test Report grate harness
Cases
wasm harnessTest ReportDeterministic TestsSummary
Test Results by Category
Fail TestsSummary
Test Results by Category
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Yaxuan-w
approved these changes
Mar 13, 2026
Member
Yaxuan-w
left a comment
Yaxuan-w
left a comment
There was a problem hiding this comment.
Approved to unblock current grate implementation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Building on the discussion here: #896 (comment)
We now allow:
How this works:
Tests added try to pass a grate-local address to the make_threei_call invocation made on behalf of child, and ensure that copy_data_between_cages hasn't regressed: