-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
PCRE2: optimize memory allocations #15395
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
ysbaddaden
wants to merge
17
commits into
crystal-lang:master
from
ysbaddaden:refactor/pcre2-memory-allocations
Closed
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
f24a948
Regex::PCRE2: set jit stack and match data as ThreadLocal
ysbaddaden 93e141f
LibC: add bindings for pthread local storage
ysbaddaden 4aeeb47
WIP: Thread::Local(T) struct
ysbaddaden b6ebabc
Regex::PCRE2: use Thread::Local(T) instead of ThreadLocal annotation
ysbaddaden 621fe5f
WIP: extract Thread::Local(T) implementations
ysbaddaden a14032e
LibC: add bindings for windows FLS API
ysbaddaden f5da032
fixup! WIP: extract Thread::Local(T) implementations
ysbaddaden c225013
WIP: try FlsAlloc on windows
ysbaddaden 4b0accf
Add destructor to Thread::Local(T)
ysbaddaden 64458fb
Fix: register destructor to free jit stack and match data on thread t…
ysbaddaden 19f631b
fixup! Add destructor to Thread::Local(T)
ysbaddaden 7487cb5
Windows: go back to TlsAlloc with custom destructor array
ysbaddaden 0cbd5e8
Fix: initializer on wasi
ysbaddaden 9fe3a62
Fix: crystal tool format
ysbaddaden 72961ca
Try FLS on windows again?
ysbaddaden 52f908d
Apply review suggestions
ysbaddaden 8688b4b
cleanup
ysbaddaden File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
class Thread | ||
struct Local(T) | ||
# def initialize | ||
# {% raise "T must be a Reference or Pointer" unless T < Reference || T < Pointer %} | ||
# end | ||
|
||
# def initialize(&destructor : Proc(T, Nil)) | ||
# {% raise "T must be a Reference or Pointer" unless T < Reference || T < Pointer %} | ||
# end | ||
|
||
def get(& : -> T) : T | ||
get? || set(yield) | ||
end | ||
|
||
# def get? : T? | ||
# def set(value : T) : T | ||
end | ||
end | ||
|
||
{% if flag?(:wasi) %} | ||
require "./wasi/thread_local" | ||
{% elsif flag?(:unix) %} | ||
require "./unix/thread_local" | ||
{% elsif flag?(:win32) %} | ||
require "./win32/thread_local" | ||
{% else %} | ||
{% raise "Thread not supported" %} | ||
{% end %} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,30 @@ | ||||||||||||||||||||||||||||||||||||||||||||||||||
require "c/pthread" | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||
class Thread | ||||||||||||||||||||||||||||||||||||||||||||||||||
struct Local(T) | ||||||||||||||||||||||||||||||||||||||||||||||||||
@key : LibC::PthreadKeyT | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||
def initialize | ||||||||||||||||||||||||||||||||||||||||||||||||||
{% raise "T must be a Reference or Pointer" unless T < Reference || T < Pointer %} | ||||||||||||||||||||||||||||||||||||||||||||||||||
err = LibC.pthread_key_create(out @key, nil) | ||||||||||||||||||||||||||||||||||||||||||||||||||
raise RuntimeError.from_os_error("pthread_key_create", Errno.new(err)) unless err == 0 | ||||||||||||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||
def initialize(&destructor : Proc(T, Nil)) | ||||||||||||||||||||||||||||||||||||||||||||||||||
{% raise "T must be a Reference or Pointer" unless T < Reference || T < Pointer %} | ||||||||||||||||||||||||||||||||||||||||||||||||||
err = LibC.pthread_key_create(out @key, destructor.unsafe_as(Proc(Void*, Nil))) | ||||||||||||||||||||||||||||||||||||||||||||||||||
raise RuntimeError.from_os_error("pthread_key_create", Errno.new(err)) unless err == 0 | ||||||||||||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||||||||||||
Comment on lines
+7
to
+17
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. suggestion: Extract the common code into a helper to remove duplication.
Suggested change
ditto for win32 |
||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||
def get? : T? | ||||||||||||||||||||||||||||||||||||||||||||||||||
pointer = LibC.pthread_getspecific(@key) | ||||||||||||||||||||||||||||||||||||||||||||||||||
pointer.as(T) if pointer | ||||||||||||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||
def set(value : T) : T | ||||||||||||||||||||||||||||||||||||||||||||||||||
err = LibC.pthread_setspecific(@key, value.as(Void*)) | ||||||||||||||||||||||||||||||||||||||||||||||||||
raise RuntimeError.from_os_error("pthread_setspecific", Errno.new(err)) unless err == 0 | ||||||||||||||||||||||||||||||||||||||||||||||||||
value | ||||||||||||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||||||||||||
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
class Thread | ||
struct Local(T) | ||
@value : T? | ||
|
||
def initialize | ||
{% raise "T must be a Reference or Pointer" unless T < Reference || T < Pointer %} | ||
end | ||
|
||
def initialize(&destructor : T ->) | ||
{% raise "T must be a Reference or Pointer" unless T < Reference || T < Pointer %} | ||
end | ||
|
||
def get? : T? | ||
@value | ||
end | ||
|
||
def set(value : T) : T | ||
@value = value | ||
end | ||
end | ||
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,30 @@ | ||||||||||||||||||||||||||||||||||||||||||||||||||
require "c/fibersapi" | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||
class Thread | ||||||||||||||||||||||||||||||||||||||||||||||||||
struct Local(T) | ||||||||||||||||||||||||||||||||||||||||||||||||||
@key : LibC::DWORD | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||
def initialize | ||||||||||||||||||||||||||||||||||||||||||||||||||
{% raise "T must be a Reference or Pointer" unless T < Reference || T < Pointer %} | ||||||||||||||||||||||||||||||||||||||||||||||||||
@key = LibC.FlsAlloc | ||||||||||||||||||||||||||||||||||||||||||||||||||
raise RuntimeError.from_winerror("FlsAlloc: out of indexes") if @key == LibC::FLS_OUT_OF_INDEXES | ||||||||||||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||
def initialize(&destructor : Proc(T, Nil)) | ||||||||||||||||||||||||||||||||||||||||||||||||||
{% raise "T must be a Reference or Pointer" unless T < Reference || T < Pointer %} | ||||||||||||||||||||||||||||||||||||||||||||||||||
@key = LibC.FlsAlloc(destructor.unsafe_as(LibC::FLS_CALLBACK_FUNCTION)) | ||||||||||||||||||||||||||||||||||||||||||||||||||
raise RuntimeError.from_winerror("FlsAlloc: out of indexes") if @key == LibC::FLS_OUT_OF_INDEXES | ||||||||||||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||||||||||||
Comment on lines
+7
to
+17
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||
def get? : T? | ||||||||||||||||||||||||||||||||||||||||||||||||||
pointer = LibC.FlsGetValue(@key) | ||||||||||||||||||||||||||||||||||||||||||||||||||
pointer.as(T) if pointer | ||||||||||||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||
def set(value : T) : T | ||||||||||||||||||||||||||||||||||||||||||||||||||
ret = LibC.FlsSetValue(@key, value.as(Void*)) | ||||||||||||||||||||||||||||||||||||||||||||||||||
raise RuntimeError.from_winerror("FlsSetValue") if ret == 0 | ||||||||||||||||||||||||||||||||||||||||||||||||||
value | ||||||||||||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||||||||||||
end | ||||||||||||||||||||||||||||||||||||||||||||||||||
end |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps these could be uncommented and then called via
super
in descendant classes to avoid duplicating the code?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It avoids some repetition, but I'm not sure it would improve readability?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm wondering if we could use
Box
for this? Seems to be exactly the use case. Technically we don't need support for boxing structs onto the heap though. So this draws in additional complexity which we might want to avoid here.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, you lost me @straight-shoota 😕
Do you mean toOh, toBox
the destructor proc?Box
the value in case it's not a Pointer/Reference! 😅There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's fine to not support non pointer values. If we ever need to, we'll figure it out then.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea for using
Box
would be that it might simplify the code forThread::Local
. All explicit casts would becomeBox(T).box
andBox(T).unbox
and no need for checkingT
type in the macro.I suppose it doesn't simplify that much though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But
Box
always allocates into the GC heap, and we'd put the pointer to the box in the local storage that we'd have to dereference on everyget
😕There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Errata: Box doesn't allocate into the GC heap when T is Nil or Reference but we want to store pointers and that will allocate.
Maybe it's an overlook? 🤔