Skip to content

Memory management extremely error prone #415

Open
@miretskiy

Description

Recently, I've authored #414
to fix some of the memory issues; However, more issues remain.
Indeed, if you simply run go test -test.count 10 -- chances are the package test
will fail. It so happens that the failures will happen inside TestSumLoop test (oftentimes)
which was added by the above PR.

However, it's unlikely that that test is to blame -- rerunning that test 10k times didn't yield
a single failure. So, what gives? After adding more debug information, it became apparent that the
panic is experienced in the TestSumLoop test, but it is caused by another test that finished running, namely
TestImportTypeForTableType

The Finalizer set by

runtime.SetFinalizer(valueType, func(valueType *ValueType) {
causes the go test to panic
when running some other test.

The failure itself can be reproduced by running TestImportTypeForTableType by itself (with something like -test.count=50).
This leads me to think that the memory management model used in wasmer-go is somewhat unworkable in its current state.

While I appreciate the desire to have finalizers to free the memory, doing so is problematic for few reasons:

  1. SetFinalizer documentation indicates that:
// There is no guarantee that finalizers will run before a program exits,
// so typically they are useful only for releasing non-memory resources
// associated with an object during a long-running program.

So, it kinda says: don't rely on it for memory management.

  1. There are way too many objects that are created -- engine,, module, store, function, imports, exports, types, etc -- to know
    and understand which ones may be missing a KeepAlive call.

  2. Requiring callers to close resources is not that bad -- that's very much idiomatic Go.

Wasmtime also relies on finalizers; however they try a bit harder to make sure things work okay:
https://github.com/bytecodealliance/wasmtime-go/blob/66086d3cc5430a0026da19f98542946acd908006/ffi.go#L19
Perhaps something like this needs to be implement in wasmer-go as well.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    🐞 bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions