Read current transaction across different Fibers

[tombruijn]

The Ruby gem runs into an issue with storing and accessing the current transaction from the Thread.current[:appsignal_transaction] store.
We had a customer report where their app uses a gem dependency that uses Dry::Effects, which changes the Fiber in which our EventHandler's on_finish method is called, which breaks it trying to call Thread.current[:appsignal_transaction] and the transaction is never closed.

I made a quick fix in Ruby gem 4.5.9, in PR #1402.
But this doesn't clear the current transaction in Thread.current[:appsignal_transaction], which may be leaky for as long as the original Fiber remains running and until the GC does a run.

I previously tried to fix this by using thread_variable_set and thread_variable_get, but this breaks for servers like Falcon, which handle requests in Fibers.
Related branch: https://github.com/appsignal/appsignal-ruby/tree/thread-variables

One small thing we could do to make it work slightly better is:

• Add a flag on the Ruby Transaction class when complete is called.
• In Transaction.create if a transaction is read from Thread.current[:appsignal_transaction], check if the complete flag is set and discard the transaction if it was completed.

But to make our Ruby gem actually compatible with Fibers, if at all possible, requires some more thought on how to support it. The way I understand it, the Transaction protocol doesn't support async, which Fibers are, and it may be time to switch to the Spans protocol for the Ruby gem.

[tombruijn]

Intercom conversation (private link).

[ehannes]

@tombruijn any updates on this? 😃 Also, how does fact that the Appsignal gem "may be leaky" affect apps running that gem, could you elaborate on that?

[tombruijn]

Hi @ehannes, no updates on this yet.
I think we've accepted that the protocol currently used by the AppSignal Ruby gem isn't really good for any async work. Small changes we make can improve how it works with the current protocol, but we can't fix all the issues with it. We'd need to switch to a newer version of our protocol (which already exists), which would involve an extensive overhaul of the gem and its helpers. Not impossible, but not on our immediate roadmap either.

The original issue was resolved with the fix I linked. Do you experience any other issues like it?

Also, how does fact that the Appsignal gem "may be leaky" affect apps running that gem, could you elaborate on that?

What I meant by this is that the Thread.current[:appsignal_transaction] thread variable, does not get cleared for the parent thread if the Fiber switches. So even though the transaction is closed, the Thread.current[:appsignal_transaction] does still contain the reference to the closed transaction. It will keep something in memory it shouldn't have to.

[tombruijn]

I made a small fix for not reusing completed transactions when creating a new transactions. See PR #1415

It has been released in Ruby gem 4.5.14 🚀

[ehannes]

@tombruijn

The original issue was resolved with the fix I linked. Do you experience any other issues like it?

No, we have not experienced any further issues, looks like the fix works really well 👍

Thanks for the clarification regarding the memory leak. We haven't had any issues with it, at least not yet 🤞

I made a small fix for not reusing completed transactions when creating a new transactions

Great, I'll have a look at that, much appreciated!