LuaJIT support, designed to support versions of openresty from 1.15+ #14
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gnurizen
force-pushed
the
lua
branch
4 times, most recently
from
6e6a16e to
770959f
Compare
|
Curious what others think about the dependency explosion from testcontainers. I'm thinking we could switch the integration tests to otel corefile machinery and use command line docker for the offset testing purposes and get rid of it. I'll explore those options. |
gnurizen
force-pushed
the
lua
branch
2 times, most recently
from
0c0701b to
6b01ad8
Compare
umanwizard
reviewed
Oct 24, 2024
There was a problem hiding this comment.
I'm most of the way through with this, not all the way, but leaving my current batch of comments now.
They are mostly just nits, questions, or requests for further comments/docs. The overall approach seems great to me! I think we're very close to being able to finally land this.
umanwizard
approved these changes
Oct 28, 2024
gnurizen
force-pushed
the
lua
branch
6 times, most recently
from
85995e7 to
23147fc
Compare
gnurizen
force-pushed
the
lua
branch
3 times, most recently
from
c9d1322 to
d133ec7
Compare
Supports stripped versions of LuaJIT which is common and supports amd64 and arm64. LuaJIT compressed 32 bit pointer mode is not supported. Offsets are pulled from native code analysis using Go's builtin x86asm and arm64asm packages. Zydis wasn't needed as the offsets are mostly extracted from from the beginning of functions so none of x86asm limitations were hit. Docker is used to extensively test the offset extraction and do some simple integration testing (sudo required). Assumptions are baked into the Go code and to a minimal extent the unwinder code about the layout of LuaJIT structs, the codebase is old and stable so making things fully generic was not necessary. The unwinder supports interpreted and JIT'd execution, the fidelity of JIT'd profiles is somewhat limited as the bytecode "PC" is not tracked in JIT execution. However the call stack is maintained so even though a JIT'd LuaJIT trace can wind through any number of functions a full stack trace can be constructed. Tested on openresty docker images from 1.15 through 1.25 (latest). Most of the changes are in line with other native unwinders with a couple exceptions: 1) The LoaderInfo is now given access to the stack deltas. This allows the stack deltas to be consulted to determine the bounds of the LuaJIT interpreter "function" (see comments for extractInterpreterBounds). 2) The way the JIT traces are mapped is a little odd. First we establish the entire anonyous/executable section with a pid mapping and then as we learn about and walk the GCtrace objects we fill in specific information about how to walk over each JIT trace. The stack adjustments are specific to each trace and we store the LuaJIT global G pointer in the pid mapping table to save the unwinder from having to remember context and rely on best effort techniques to find it. This should have the advantage of making the unwinder work in untested contexts like multi-threaded openresty or applications where multiple LuaJIT instances are used fom the same process/threads. update tracers Remove ReportFallbackSymbol Fix synchronize mappings for realloc case and add tests remove unused code Revert FileIDMapper changes remove lj_bc headers and manually code in go Store all information needed to symbolize a frame in one frame instead of using previousFrame hack cleanup test cleanup
gnurizen
force-pushed
the
lua
branch
2 times, most recently
from
3732626 to
1037c96
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Supports stripped versions of LuaJIT which is common and supports amd64
and arm64. LuaJIT compressed 32 bit pointer mode is not supported.
Offsets are pulled from native code analysis using Go's builtin
x86asm and arm64asm packages. Zydis wasn't needed as the offsets are
mostly extracted from from the beginning of functions so none of x86asm
limitations were hit. Docker is used to extensively test the offset
extraction and do some simple integration testing (sudo required).
Assumptions are baked into the Go code and to a minimal extent the
unwinder code about the layout of LuaJIT structs, the codebase is old
and stable so making things fully generic was not necessary.
The unwinder supports interpreted and JIT'd execution, the fidelity of
JIT'd profiles is somewhat limited as the bytecode "PC" is not tracked
in JIT execution. However the call stack is maintained so even though
a JIT'd LuaJIT trace can wind through any number of functions a full
stack trace can be constructed.
Tested on openresty docker images from 1.15 through 1.25 (latest).
Most of the changes are in line with other native unwinders with a
couple exceptions:
The LoaderInfo is now given access to the stack deltas. This allows
the stack deltas to be consulted to determine the bounds of the
LuaJIT interpreter "function" (see comments for
extractInterpreterBounds).
The way the JIT traces are mapped is a little odd. First we establish
the entire anonyous/executable section with a pid mapping and then
as we learn about and walk the GCtrace objects we fill in specific
information about how to walk over each JIT trace. The stack adjustments
are specific to each trace and we store the LuaJIT global G pointer
in the pid mapping table to save the unwinder from having to remember
context and rely on best effort techniques to find it. This should
have the advantage of making the unwinder work in untested contexts like
multi-threaded openresty or applications where multiple LuaJIT instances
are used fom the same process/threads.