feat(gnovm): garbage collector #4194

ltzmaxwell · 2025-04-21T05:07:19Z

rebase to master. some issue to be resolved.

... Readonly is a property of TypedValue, rather than ReadonlyValue wrapper fix tests Machine.IsReadonly() also checks realm mismatch nil interface is undefined fix IsReadonly to use IsImmutable; half of convert check done enable second opconvert check ... ... first commit of interrealm_withswitch fix zrealm_crossrealm11.gno test intermediate fix tests... fix gnovm tests ... heapitems_for_3693 update tests fix heap return issues ... cleanup and fixes heap item all package vars update golden tests a funcvalue is an object; a func closure has no parent; find heap captures fix update golden tests fix one example test update golden tests fix script tests add gno-memory-model replace IsUndefined(); remove defaultValue() for safety Add interrealm spec add note on closures update gno-interrealm.gno replace callerat with current/previous realm() borrrow realm -> switchrealm() test borrow test; fix tests fix gnolang#4019 fix stdlibs/std tests initial implementation of GC make it work save debug fixup save add todo package value size metric shallowly naive gas comsumption gc benchmark init fixup fixup fixup simplify fixup fixup txtar gas consumption fixup fmt fixup save save fixup revert benchmark change remove unused fix test add comment;update const;fix test fmt fixup revert clean fixup fixup fixup fixup string value based revert to array base clean fixup fixup correct string allocation fixup fix comment clear unused store gc callback revert string value fix after rebase optimize add comment

Gno2D2 · 2025-04-21T05:08:01Z

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):

IGNORE the bot requirements for this PR (force green CI check)

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)

☑️ Contributor Actions:

Fix any issues flagged by automated checks.
Follow the Contributor Checklist to ensure your PR is ready for review.
- Add new tests, or document why they are unnecessary.
- Provide clear examples/screenshots, if necessary.
- Update documentation, if required.
- Ensure no breaking changes, or include BREAKING CHANGE notes.
- Link related issues/PRs, where applicable.

☑️ Reviewer Actions:

Complete manual checks for the PR, including the guidelines and additional checks if applicable.

📚 Resources:

Debug

Automated Checks
Maintainers must be able to edit this pull request (more info)
If
🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 The pull request was created from a fork (head branch repo: ltzmaxwell/gno)
Then
🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request
Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)
If
🟢 Condition met
└── 🟢 On every pull request
Can be checked by

Any user with comment edit permission

…unmarshal allocation

jaekwon · 2025-04-23T20:20:51Z

Linking the previous PR for comments: #3789

codecov · 2025-04-24T13:54:44Z

Codecov Report

Attention: Patch coverage is 55.49451% with 162 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
gnovm/pkg/gnolang/garbage_collector.go	47.08%	124 Missing and 21 partials ⚠️
gnovm/pkg/gnolang/alloc.go	76.81%	12 Missing and 4 partials ⚠️
gnovm/pkg/gnolang/realm.go	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

ltzmaxwell · 2025-04-27T16:17:51Z

should also count m.Values?

piux2 · 2025-04-28T15:18:59Z

gnovm/pkg/gnolang/alloc.go

+// -----------------------------------------------
+
+func (pv *PackageValue) GetShallowSize() int64 {
+	return allocPackage


type PackageValue struct { ObjectInfo FBlocks []Value // Should we count this in the shallow size as well }

mmm technically yes, FNames and FBlocks, and their string sizes... we could do this here with _allocValue and _allocName, or do it in a separate PR. The strings we can re-count each time we see them.

added here: 553a5e0

piux2 · 2025-04-28T16:16:01Z

gnovm/pkg/gnolang/alloc.go

+
+// Only count for closures.
+func (fv *FuncValue) GetShallowSize() int64 {
+	return allocFunc


type FuncValue struct { ObjectInfo Captures []TypedValue `json:",omitempty"` // Should we count this in the shallow size as well }

Good catch—thank you for pointing that out. I agree it should be counted, but in the current implementation, these objects aren't allocated by the allocator (similar to m.Values).

We might optimize this later, to ensures the GC only recalculates previously allocated objects. otherwise the results would become inconsistent.

It's true that we don't account for these during doOpFuncLit(), but that's a separate issue.
We should still count these, and even assume that each one of these have inside a HeapItemValue for simplicity.

yeah definitely. the reason I planned doing this (and the one above) after this PR is that, there potentially can be some weird behavior, that the GC calculation process exceeds the maxBytes itself, because it counts some extra objects sizes which are not allocated previously. of course, this is quite an unusual case.

it's still ok to do it here since luckily we already have the logic to handle this.

also agreed with the idea described below to ensure strict correctness.

jaekwon · 2025-04-28T18:53:50Z

I wish we could write something that collects GC'd objects with ref zero from realm finalization, e.g. into Alloc or some struct, and basically run GC to calculate sizes without actually removing anything; The purpose is to see whether the allocation bytes from runtime is actually identical to what is calculated during GC by adding both non-GC remaining objects and GC'd objects together. Whenever there is a difference in the numbers, means we are missing some allocation step, or the GC is not iterating on something.

And then, we can also add tests to ensure that objects once put into the GC pile (those appended to Alloc for testing purposes as described in the previous paragraph), their GCCycle never changes, because the GC should never see them again.

This should be its own Issue to work on after this gets merged; it could be something to work on after the beta launch.

ltzmaxwell · 2025-04-29T00:44:55Z

I wish we could write something that collects GC'd objects with ref zero from realm finalization, e.g. into Alloc or some struct, and basically run GC to calculate sizes without actually removing anything; The purpose is to see whether the allocation bytes from runtime is actually identical to what is calculated during GC by adding both non-GC remaining objects and GC'd objects together. Whenever there is a difference in the numbers, means we are missing some allocation step, or the GC is not iterating on something.

Yeah, assert that sum == remained + GC'd.

And then, we can also add tests to ensure that objects once put into the GC pile (those appended to Alloc for testing purposes as described in the previous paragraph), their GCCycle never changes, because the GC should never see them again.

👍

This should be its own Issue to work on after this gets merged; it could be something to work on after the beta launch.

OK.

rebase to master. some issue to be resolved. --------- Co-authored-by: jaekwon <[email protected]>

jaekwon and others added 2 commits April 21, 2025 12:45

fixup

feb587d

github-project-automation bot added this to 🧙‍♂️Gno.land development and 💪 Bounties & Worx Apr 21, 2025

github-project-automation bot moved this to Triage in 🧙‍♂️Gno.land development Apr 21, 2025

ltzmaxwell marked this pull request as draft April 21, 2025 05:07

github-actions bot added 📦 🤖 gnovm Issues or PRs gnovm related 📦 ⛰️ gno.land Issues or PRs gno.land package related labels Apr 21, 2025

github-actions bot assigned ltzmaxwell Apr 21, 2025

ltzmaxwell changed the title ~~feat(gnovm): garbage collector~~ WIP feat(gnovm): garbage collector Apr 21, 2025

Gno2D2 requested review from a team April 21, 2025 05:07

ltzmaxwell added 5 commits April 22, 2025 23:59

alloc N;alloc TypedValue arrays;alloc exception, frame;correct amino …

dab58f4

…unmarshal allocation

collect cache; only count values

0e00421

fixup

56da268

fixup

3f71d23

revert string allocation const

f641f03

ltzmaxwell added 9 commits April 24, 2025 10:35

ajust alloc const; replace amino allocation

e8b850b

rename; optimize

54f590d

add comment

3e1789a

rm comment

5d0fe29

merge master

ac85d65

fixup

a295e04

fixup

5e90473

cleanup

7d5cd9c

rename func

625da23

ltzmaxwell marked this pull request as ready for review April 24, 2025 13:30

ltzmaxwell changed the title ~~WIP feat(gnovm): garbage collector~~ feat(gnovm): garbage collector Apr 24, 2025

fixup

a1fddde

piux2 reviewed Apr 28, 2025

View reviewed changes

jaekwon and others added 2 commits April 28, 2025 18:48

add captures to funcvalue

8781173

correct shallow size for package value

553a5e0

jaekwon merged commit d37bc65 into gnolang:master Apr 29, 2025
64 of 70 checks passed

github-project-automation bot moved this to Done in 💪 Bounties & Worx Apr 29, 2025

github-project-automation bot moved this from Triage to Done in 🧙‍♂️Gno.land development Apr 29, 2025

jaekwon added a commit that referenced this pull request May 1, 2025

feat(gnovm): garbage collector (#4194)

61744bf

rebase to master. some issue to be resolved. --------- Co-authored-by: jaekwon <[email protected]>

This was referenced May 8, 2025

Garbage Collector - simple counting #3779

Closed

[gno] synchronous garbage collection #266

Closed

stefann-01 pushed a commit to stefann-01/gno that referenced this pull request Jul 14, 2025

feat(gnovm): garbage collector (gnolang#4194)

53fe3ca

rebase to master. some issue to be resolved. --------- Co-authored-by: jaekwon <[email protected]>

piux2 mentioned this pull request Sep 2, 2025

fix(gnovm): proper memory allocation and garbage collection #4302

Merged

Davphla mentioned this pull request Nov 21, 2025

fix(gnovm): include missing field in shallow size calculation + add overflow protection #4892

Open

feat(gnovm): garbage collector #4194

feat(gnovm): garbage collector #4194

Uh oh!

Conversation

ltzmaxwell commented Apr 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Gno2D2 commented Apr 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛠 PR Checks Summary

Manual Checks (for Reviewers):

✅ Automated Checks (for Contributors):

☑️ Contributor Actions:

☑️ Reviewer Actions:

📚 Resources:

If

Then

If

Can be checked by

Uh oh!

jaekwon commented Apr 23, 2025

Uh oh!

codecov bot commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ltzmaxwell commented Apr 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

piux2 Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

jaekwon Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

ltzmaxwell Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

piux2 Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

ltzmaxwell Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

jaekwon Apr 28, 2025

Choose a reason for hiding this comment

Uh oh!

ltzmaxwell Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

jaekwon commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ltzmaxwell commented Apr 29, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ltzmaxwell commented Apr 21, 2025 •

edited

Loading

Gno2D2 commented Apr 21, 2025 •

edited

Loading

codecov bot commented Apr 24, 2025 •

edited

Loading

ltzmaxwell commented Apr 27, 2025 •

edited

Loading

jaekwon commented Apr 28, 2025 •

edited

Loading