[Profiler] Add support for ARM64 back to the .NET profiler or allow enabling experimental feature.

[angelo-neves]

Are you requesting automatic instrumentation for a framework or library? Please describe.

• Framework or library name : [.NET Core / 5 / 6]

Is your feature request related to a problem? Please describe.
The 2.13.0 release of the DD APM for .NET has disabled support for ARM64 (#2994). We use AWS Fargate ARM for most of our services and would like to continue using the profiler as soon as possible.

Describe the solution you'd like
Allow users to enable experimental support for ARM64 instead of blocking it outright or address the underlying issues with that arch and re-enable it by default.

Describe alternatives you've considered
Rolling back to 2.12.0 is a temporary fix but I'm concerned we won't be able to keep up with other improvements and fixes in future versions if we are stuck in the last build that supported ARM64

Additional context
Also raised to our sales rep but no support ticket filed since it's not exactly a bug

[gleocadie]

Hi @angelo-neves
sorry for the late replies.

We recently disabled the profiler on ARM64 because we noticed that the callstacks were not good and we are focusing on other features.

Running on ARM64 is our next milestone. In the meantime, I can propose an experimental flag (environment variable) you would add to activate the profiler on ARM64 despite its current state.

Tell me if this temporary solution would satisfy you.

[mowemcfc]

Hi @gleocadie,

Sorry for the (very long) delay!

I'm a colleague of @angelo-neves and yes we are still interested in the described temporary solution.

Additionally, any updates on the status/roadmap of ARM64 support would be much appreciated.

Thank you.

[sciutand]

Running ecs -> ec2 clustre on M6G instances.

Is this still an issue 1 and half years later?
I've spent the best part of the day trying to get profile working only to then find this old thread mentioning it doesn't work.
The worse part is that this documentation page even gives you file paths for arm. Perhaps update this?
The fact that it's actually not supported was the last thing on my mind.

👎

[rue-vmom]

Any update on this?

[lucas-magalhaes-BTG]

Up! Still not working...

[gleocadie]

Hi @mowemcfc , @sciutand , @rue-vmom , @lucas-magalhaes-BTG , sorry for the delayed answer.
Yep still not supported yet
Reason: libunwind provides incorrect callstacks. I recently pushed fixes in libunwind to make it (one of them being for aarch64).
The past months, we were focus on other pressing features.

I can still make a flag/env var so you can try it out. This can be done quickly.
But note that, there is no guarantee. The CI does not run for Profiler on aarch64.
I plan to look at libunwind this quarter and see how are the callstacks. I'll also check with MS engineer to see if they do fancy things with the JIT.

Again sorry for the delayed answer.

[lucas-magalhaes-BTG]

Hi @mowemcfc , @sciutand , @rue-vmom , @lucas-magalhaes-BTG , sorry for the delayed answer. Sim, ainda não há suporte Motivo: libunwind fornece pilhas de chamadas incorretas. Recentemente, fiz correções no libunwind para torná-lo (uma delas é para aarch64). Nos últimos meses, estávamos focados em outros recursos urgentes.

Eu ainda posso fazer um sinalizador / env var para que você possa experimentá-lo. Isso pode ser feito rapidamente. Mas observe que não há garantia. O CI não é executado para o Profiler em aarch64. Pretendo olhar para o libunwind neste trimestre e ver como estão as pilhas de chamadas. Também vou verificar com o engenheiro da MS para ver se eles fazem coisas extravagantes com o JIT.

Mais uma vez, desculpe pela resposta atrasada.

Thank you for your attention, any news?

[edgarkz]

+1, looking forward for dotnet 8/9 profiling support on arm64 arch

@gleocadie it looks like arm64 supported was added only for alpine image?
https://github.com/DataDog/dd-trace-dotnet/releases/tag/v3.2.0

New Features
Support for alpine images on ARM64. alpine images with version 3.18 and above, running on ARM64 images are now supported on .NET 6+.

do you think the same can be enabled for glibc based distros?

[rokeeffe-thp]

@gleocadie thanks for the explanation and details. Would it be possible to get the flag/env var added so that we can test? We recently ran into some performance issues, and of course are running AArch64/ARM

[edgarkz]

Team, any updates?
The original issue is approaching three years old.
ARM64 is a modern and widely adopted architecture, especially in production environments.

[gleocadie]

Oh @rokeeffe-thp and @edgarkz so sorry I miss the notification.
ok so yes I can add the flag, but below I will explain why it won't be of any help.

The original issue is approaching three years old.
ARM64 is a modern and widely adopted architecture, especially in production environments.

You are right. We had urgent features to move forward and to be completely honest, we did not have customers hardly (not even pushing) to have that.

Ok that said. I spend a couple weeks digging and trying to understand why the callstacks are wrong:
On Arm64, when information needed to construction the callstack (dwarf information) are missing, libunwind tries to get the assembly code of the function and analyzes it to infer the caller. For non-Jitted code, that works but for Jitted code it does not work.
That's the reason why it does not work with libunwind ALONE.
During that couple of weeks, I designed a solution/fix for it. I was able to get accurate callstacks for simple example. I wrote a design doc (internal for now) but the gist is:
Having libunwind to expose an API which we could register unwind information for each managed function (this is use to compute the callstack). The profiler will have to listen to the Jit events (or other events) and registers those info into libunwind.
We have it for Q3 but as a low priority for now.

If you are a datadog customer, do not hesitate to push for that by contacting the support.

[janlonsetteig]

Hi,
Any updates on this? Would love to see this fixed