Skip to content

Commit fd67023

Browse files
jaekwonpiux2moulltzmaxwell
authored
feat(gnovm): interrealm spec; cross/crossing; readonly; defer/recover/panic fixes; blank identifier fixes.... (#4060)
Implements complete interrealm spec. This uses heapitemvalues more such that a closure or pointer doesn't need to refer to a block. It resolves #3693. Also includes various defer/panic/recover fixes from #4188. Excerpt from the new document 'Gno Memory Model' in this PR: -------- ```go func Example(arg int) (res *int) { var x int = arg + 1 return &x } ``` The above code when executed will first produce the following block: ``` BlockValue{ ... Source: <*FuncDecl node>, Values: [ {T: nil, V: nil}, // 'arg' parameter {T: nil, V: nil}, // 'res' result {T: HeapItemType{}, V: &HeapItemValue{{T: nil, V: nil}}, // 'x' variable ], ... } ``` In the above example the third slot for `x` is not initialized to the zero value of a typed value slot, but rather it is prefilled with a heap item. Variables declared in a closure or passed by reference are first discovered and marked as such from the preprocessor, and NewBlock() will prepopulate these slots with `*HeapItemValues`. When a `*HeapItemValue` is present in a block slot it is not written over but instead the value is written into the heap item's slot. When the example code executes `return &x` instead of returning a `PointerValue` with `.Base` set to the `BlockValue` and `.Index` of 2, it sets `.Base` to the `*HeapItemValue` with `.Index` of 0 since a heap item only contains one slot. The pointer's `.TV` is set to the single slot of of the heap item. This way the when the pointer is used later in another transaction there is no need to load the whole original block value, but rather a single heap item object. If `Example()` returned only `x` rather than a pointer `&x` it would not be initialized with a heap item for the slot. ```go func Example2(arg int) (res func()) { var x int = arg + 1 return func() { println(x) } } ``` The above example illustrates another use for heap items. Here we don't reference `x`, but it is captured by the anonymous function literal (closure). At runtime the closure `*FuncValue` captures the heap item object such that the closure does not depend on the block at all. Variables declared at the package (global) level may also be referred to by pointer in anonymous functions. In the future we will allow limited upgrading features for mutable realm packages (e.g. the ability to add new functions or replace or "swizzle" existing ones), so all package level declared variables are wrapped in heap item objects. Since all global package values referenced closures can be captured as heap objects, the execution and persistence of a closure function value does not depend on any parent blocks. (Omitted here is how references to package level declared functions and methods are replaced by a selector expression on the package itself; otherwise closures would still in general depend on their parent blocks). ## Loopvars Go1.22 introduced loopvars to reduce programmer errors. Gno uses heap items to implement loopvars. ```go for _, v := range values { saveClosure(func() { fmt.Println(v) }) } ``` The Gno VM does something special for loopvars. Instead of assigning the new value `v` to the same slot, or even the same heap item object's slot, it replaces the existing heap item object with a new one. This allows the closure to capture a new heap item object with every iteration. This is called a heap definition. The behavior is applied for implicit loops with `goto` statements. The preprocessor first identifies all such variable definitions whether explicit in range statements or implicit via `goto` statements that are captured by closures or passed by pointer reference, and directs the VM to execute the define statement by replacing the existing heap item object with a new one. --------- Signed-off-by: moul <[email protected]> Co-authored-by: piux2 <[email protected]> Co-authored-by: moul <[email protected]> Co-authored-by: ltzMaxwell <[email protected]>
1 parent a3bffb2 commit fd67023

File tree

293 files changed

+6357
-3205
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

293 files changed

+6357
-3205
lines changed

docs/resources/gno-interrealm.md

+108
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
# Interrealm Specification
2+
3+
Gno extends Go's type system with a interrealm rules. These rules can be
4+
checked during the static type-checking phase (but at the moment they are
5+
partially dependent on runtime checks).
6+
7+
All functions in Gno execute under a realm context as determined by the call
8+
stack. Objects that reside in a realm can only be modified if the realm context
9+
matches.
10+
11+
A function declared in p packages when called:
12+
13+
* inherits the last realm for package declared functions and closures.
14+
* inherits the last realm when a method is called on unreal receiver.
15+
* implicitly crosses to the receiver's resident realm when a method of the
16+
receiver is called. The receiver's realm is also called the "borrow realm".
17+
18+
A function declared in a realm package when called:
19+
20+
* explicitly crosses to the realm in which the function is declared if the
21+
function begins with a `crossing()` statement. The new realm is called the
22+
"current realm".
23+
* otherwise follows the same rules as for p packages.
24+
25+
The `crossing()` statement must be the first statement of a function's body.
26+
It is illegal to use anywhere else, and cannot be used in p packages. Functions
27+
that begin with the `crossing()` statement are called "crossing" functions".
28+
29+
A crossing function declared in a realm different than the last explicitly
30+
crossed realm *must* be called like `cross(fn)(...)`. That is, functions of
31+
calls that result in explicit realm crossings must be wrapped with `cross()`.
32+
33+
`std.CurrentRealm()` returns the current realm last explicitly crossed to.
34+
35+
`std.PreviousRealm()` returns the realm explicitly crossed to before that.
36+
37+
A crossing function declared in the same realm package as the callee may be
38+
called normally OR like `cross(fn)(...)`. When called normally there will be no
39+
realm crossing, but when called like `cross(fn)(...)` there is technically a
40+
realm crossing and the current realm and previous realm returned are the same.
41+
42+
The current realm and previous realm do not depend on any implicit crossing to
43+
the receiver's borrowed/storage realm even if the borrowed realm is the last
44+
realm of the call stack equal to `m.Realm`. In other words `std.CurrentRealm()`
45+
may be different than `m.Realm` (the borrow realm) when a receiver is called on
46+
a foreign object.
47+
48+
Calls of methods on receivers residing in realms different than the current
49+
realm must not be called like `cross(fn)(...)` if the method is not a
50+
crossing function itself, and vice versa. Or it could be said that implicit
51+
crossing is not real realm crossing. (When you sign a document with someone
52+
else's pen it is still your signature; signature:pen :: current:borrowed.
53+
54+
A crossing method declared in a realm cannot modify the receiver if the object
55+
resides in a different realm. However not all methods are required to be
56+
crossing methods, and crossing methods may still read the state of the
57+
receiver (and in general anything reacheable is readible).
58+
59+
New unreal objects reachable from the borrowed realm (or current realm if there
60+
was no method call that borrowed) become persisted in the borrowed realm (or
61+
current realm) upon finalization of the foreign object's method (or function).
62+
(When you put an unlabeled photo in someone else's scrap book the photo now
63+
belongs to the other person). In the future we will introduce an `attach()`
64+
function to prevent a new unreal object from being taken.
65+
66+
MsgCall can only call (realm) crossing functions.
67+
68+
MsgRun will run a file's `main()` function in the user's realm and may call
69+
both crossing functions and non-crossing functions.
70+
71+
A realm package's initialization (including init() calls) execute with current
72+
realm of itself, and it `std.PreviousRealm()` will panic unless the call stack
73+
includes a crossing function called like `cross(fn)(...)`.
74+
75+
### Justifications
76+
77+
P package code should behave the same even when copied verbatim in a realm
78+
package.
79+
80+
Realm crossing with respect to `std.CurrentRealm()` and `std.PreviousRealm()`
81+
is important enough to be explicit and warrants type-checking.
82+
83+
A crossing function of a realm should be able to call another crossing function
84+
of the same realm without necessarily explicitly crossing realms.
85+
86+
Sometimes the previous realm and current realm must be the same realm, such as
87+
when a realm consumes a service that it offers to external realms and users.
88+
89+
A method should be able to modify the receiver and associated objects of the
90+
same borrowed realm.
91+
92+
A method should be able to create new objects that reside in the same realm by
93+
default in order to maintain storage realm consistency and encapsulation and
94+
reduce fragmentation.
95+
96+
In the future an object may be migrated from one realm to another when it loses
97+
all references in one realm and gains references in another. The behavior of
98+
the object should not change after migration because this type of migration is
99+
implicit and generally not obvious without more language features.
100+
101+
Code declared in p packages (or declared in "immutable" realm packages) can
102+
help different realms enforce contracts trustlessly, even those that involve
103+
the caller's current realm. Otherwise two mutable (upgreadeable) realms cannot
104+
export trust unto the chain because functions declared in those two realms can
105+
be upgraded.
106+
107+
Both `crossing()` and `cross(fn)(...)` statements may become special syntax in
108+
future Gno versions.

docs/resources/gno-memory-model.md

+218
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
# Gno Memory Model
2+
3+
## The Typed Value
4+
5+
```go
6+
type TypedValue struct {
7+
T Type
8+
V Value
9+
N [8]byte
10+
}
11+
```
12+
13+
Both `Type` and `Value` are Go interface values. Go does not support union
14+
types, so primitive values like bools and ints are stored in the N field for
15+
performance.
16+
17+
All values in Gno are stored represented as (type, value) tuples. Vars in
18+
scope blocks, fields in structs, elements in arrays, keys and values of maps
19+
are all respresented by the same `TypedValue` struct.
20+
21+
This tuple representation lets the Gno VM implementation logic be simpler with
22+
less code. Reading and writing values are the same whether the static type of
23+
the value is an interface or something concrete with no special logic for
24+
memory optimizations which is less relevant in a massive multi-user
25+
transactional operating system where most of the data resides in disk anyways.
26+
27+
Another benefit is that it promotes the development of new types of client
28+
interfaces that can make use of the type information for object display and
29+
interaction. The original vision of Tim Burners Lee's HTML DOM based World Wide
30+
Web with restful HTTP requests like GET and POST has been steamrolled over by
31+
continuous developments in HTML, CSS, Javascript, and the browser. The internet
32+
is alive but the World Wide Web is dead. Gno is a reboot of the original vision
33+
of the Web but better because everything is integrated based on a singular
34+
well designed object-oriented language. Instead of POST requests Gno has
35+
typed method calls. All values are annotated with their types making the
36+
environment REST-ful to the core.
37+
38+
> REST (Representational State Transfer) is a software architectural style that
39+
> was created to describe the design and guide the development of the
40+
> architecture for the World Wide Web. REST defines a set of constraints for
41+
> how the architecture of a distributed, Internet-scale hypermedia system, such
42+
> as the Web, should behave. - wikipedia
43+
44+
While the Gno VM implements the memory model described here in the most
45+
straightforward way, future alternative implementations may represent values
46+
differently in machine memory even while conforming to the spec as implemented
47+
by the Gno VM.
48+
49+
50+
## Objects and Values
51+
52+
There are many types of Values, and some of these value types are also Objects.
53+
The types below are all values and those that are bolded are also objects.
54+
55+
* Primitive // bool, uint, int, uint8, ... int64
56+
* StringValue
57+
* BigintValue // only used for constant expressions
58+
* BigdecValue // only used for constant expressions
59+
* DataByteValue // invisible type for byte array optimization
60+
* PointerValue // base is always an object
61+
* **ArrayValue**
62+
* SliceValue
63+
* **StructValue**
64+
* **FuncValue**
65+
* **MapValue**
66+
* **BoundMethodValue** // func & receiver
67+
* TypeValue
68+
* PackageValue
69+
* **BlockValue** // for package, file, if, range, switch, func
70+
* RefValue // reference to an object stored in disk
71+
* **HeapItemValue** // invisible type for loopvars and closure captures
72+
73+
74+
## Pointers
75+
76+
```go
77+
type PointerValue struct {
78+
TV *TypedValue // escape val if pointer to var.
79+
Base Value // array/struct/block, or heapitem.
80+
Index int // list/fields/values index, or -1 or -2 (see below).
81+
}
82+
```
83+
84+
The pointer is a reference to a typed value slot in an array, struct, block,
85+
or heap item. It is also used internally in the VM for assigning to slots.
86+
Even internally the Base is used to tell the realm finalizer when the base
87+
has been updated.
88+
89+
## Blocks and Heap Items
90+
91+
All statements enclosed in {} parentheses will allocate a new block
92+
and push it onto the block stack of the VM. The size of the block
93+
is determined by the number of variables declared in the block statement.
94+
95+
```
96+
type BlockValue struct {
97+
ObjectInfo
98+
Source BlockNode
99+
Values []TypedValue
100+
Parent Value // Parent block if any, or RefValue{} to one.
101+
Blank TypedValue // Captures "_" underscore names.
102+
bodyStmt bodyStmt // Holds a pointer to the current statement.
103+
}
104+
```
105+
106+
The following Gno AST nodes when executed will create a new block:
107+
108+
* FuncLitStmt
109+
* BlockStmt // a list of statements wrapped in {}
110+
* ForStmt
111+
* IfCaseStmt
112+
* RangeStmt
113+
* SwitchCaseStmt
114+
* FuncDecl
115+
* FileNode
116+
* PackageNode
117+
118+
`IfStmt`s and `SwitchStmt`s also produce faux blocks that get merged onto the
119+
following `IfCaseStmt` and `SwitchCaseStmt` respectively, but this is an
120+
invisible implementation detail and the behavior may change.
121+
122+
Heap items are only used in blocks. Conceptually they are an object container
123+
around a singleton typed value slot. It is not visible to the gno developer
124+
but it is important to understand how they work when inspecting the block
125+
space.
126+
127+
```go
128+
func Example(arg int) (res *int) {
129+
var x int = arg + 1
130+
return &x
131+
}
132+
```
133+
134+
The above code when executed will first produce the following block:
135+
136+
```
137+
BlockValue{
138+
...
139+
Source: <*FuncDecl node>,
140+
Values: [
141+
{T: nil, V: nil}, // 'arg' parameter
142+
{T: nil, V: nil}, // 'res' result
143+
{T: HeapItemType{},
144+
V: &HeapItemValue{{T: nil, V: nil}}, // 'x' variable
145+
],
146+
...
147+
}
148+
```
149+
150+
In the above example the third slot for `x` is not initialized to the zero
151+
value of a typed value slot, but rather it is prefilled with a heap item.
152+
153+
Variables declared in a closure or passed by reference are first discovered and
154+
marked as such from the preprocessor, and NewBlock() will prepopulate these
155+
slots with `*HeapItemValues`. When a `*HeapItemValue` is present in a block
156+
slot it is not written over but instead the value is written into the heap
157+
item's slot.
158+
159+
When the example code executes `return &x` instead of returning a
160+
`PointerValue` with `.Base` set to the `BlockValue` and `.Index` of 2, it sets
161+
`.Base` to the `*HeapItemValue` with `.Index` of 0 since a heap item only
162+
contains one slot. The pointer's `.TV` is set to the single slot of of the heap
163+
item. This way the when the pointer is used later in another transaction there
164+
is no need to load the whole original block value, but rather a single heap
165+
item object. If `Example()` returned only `x` rather than a pointer `&x` it
166+
would not be initialized with a heap item for the slot.
167+
168+
```go
169+
func Example2(arg int) (res func()) {
170+
var x int = arg + 1
171+
return func() {
172+
println(x)
173+
}
174+
}
175+
```
176+
177+
The above example illustrates another use for heap items. Here we don't
178+
reference `x`, but it is captured by the anonymous function literal (closure).
179+
At runtime the closure `*FuncValue` captures the heap item object such that the
180+
closure does not depend on the block at all.
181+
182+
Variables declared at the package (global) level may also be referred to by
183+
pointer in anonymous functions. In the future we will allow limited upgrading
184+
features for mutable realm packages (e.g. the ability to add new functions or
185+
replace or "swizzle" existing ones), so all package level declared variables
186+
are wrapped in heap item objects.
187+
188+
Since all global package values referenced closures can be captured as heap
189+
objects, the execution and persistence of a closure function value does not
190+
depend on any parent blocks. (Omitted here is how references to package
191+
level declared functions and methods are replaced by a selector expression
192+
on the package itself; otherwise closures would still in general depend
193+
on their parent blocks).
194+
195+
## Loopvars
196+
197+
Go1.22 introduced loopvars to reduce programmer errors. Gno uses
198+
heap items to implement loopvars.
199+
200+
```go
201+
for _, v := range values {
202+
saveClosure(func() {
203+
fmt.Println(v)
204+
})
205+
}
206+
```
207+
208+
The Gno VM does something special for loopvars. Instead of assigning the new
209+
value `v` to the same slot, or even the same heap item object's slot, it
210+
replaces the existing heap item object with a new one. This allows the closure
211+
to capture a new heap item object with every iteration. This is called
212+
a heap definition.
213+
214+
The behavior is applied for implicit loops with `goto` statements. The
215+
preprocessor first identifies all such variable definitions whether explicit in
216+
range statements or implicit via `goto` statements that are captured by
217+
closures or passed by pointer reference, and directs the VM to execute the
218+
define statement by replacing the existing heap item object with a new one.

0 commit comments

Comments
 (0)