How to deal with memory?

[GeopJr]

One of the things stopping me from using Crystal (with gi-crystal) for complex GTK apps is dealing with memory. It feels like it's completely out of my control, opaque and sometimes random.

I understand that reference counting doesn't translate well to bdwgc so I'm starting this discussion to generally gather any info around this.

For example, for Collision, some time ago I converted everything from a modular structure to one big builder file specifically to avoid memory issues (that were very random due to multi-threading, so not exactly fixable here). Yet, I still see a big memory leak:

Screencast.from.2024-03-15.21-17-10.webm

Others have disabled the GC to workaround it:
AHK_X11 calls the gc manually https://github.com/search?q=repo%3Aphil294%2FAHK_X11+gc+language%3ACrystal&type=code&l=Crystal

Haki disabled the gc completely https://github.com/grkek/haki/blob/58917ad07fd9559e07f2f1f8aca18f34c5d736af/example/application.cr#L3

So, what can be done? I've been avoiding dipping my toes into gi and complex crystal c bindings, but should definitely start contributing to gi-crystal!

Researching how Go, D, Nim deal with it might be of worth

edit: gtk3 bindings have an interesting insight too https://github.com/phil294/gtk3.cr?tab=readme-ov-file#gc-related-problems

[hugopl]

There are some well know things that cause memory leaks:

• Using widget templates (easily fixable)
• Using signals and not disconnecting them (fixable but annoying with current API)

Using Fibers in general with or without -Dpreview_mt is not a good idea, since it can block the thread, i.e. block the GTK main loop and freeze the whole application.

I need to study how to integrate GTK main loop with Crystal scheduler, all this takes time that I don't have... so currently I try to write my apps with these small memory leaks and then I try to fix the most annoying leaks or blocking (to my apps) issues.

Nowadays is also not possible to create crystal gobjects from C, @BlobCodes implemented it, but I didn't like the resulting user API 😅 .

I mean, a good start would be to document these leaks and if possible have tests for them, later we think how to fix them.

[GeopJr]

Thanks for the info!

Using Fibers in general with or without -Dpreview_mt is not a good idea, since it can block the thread, i.e. block the GTK main loop and freeze the whole application.

I need to study how to integrate GTK main loop with Crystal scheduler, all this takes time that I don't have... so currently I try to write my apps with these small memory leaks and then I try to fix the most annoying leaks or blocking (to my apps) issues.

FWIW, Collision spawns separate fibers for every hash calculation it has to do (1 for md5, 1 for sha1...), with preview_mt. The main problems I encountered (a long time ago/this has been battle tested at this point) were:

1. sometimes fibers would spawn in the main thread which is being used by gtk's main loop
2. memory issues with bindings

To fix (1), I wrote https://github.com/GeopJr/non-blocking-spawn which is a carbon copy of Crystal's spawn method but will never use the main thread.
To fix (2), I just avoided using anything bindings related in any thread but the main one but instead wait for the mt stuff to finish and then use the results in an Idle function.

I believe there's a Crystal approved RFC that reworks how mt will function, which might fix all our issues!