fix: Reduce the chance of CurlEventPublisher
exhausting file descriptors on high throughput PHP-FPM servers
#216
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Requirements
Related issues
Describe the solution you've provided
TLDR:
--connect-timeout
vs--max-time
(docs)Consider the following code:
+
However long the request takes is how long file descriptors hang open for.
Web apps like Laravel frequently use PHP-FPM as part of their server infrastructure where each request is a new process. As a result, on high throughput servers, EventProcessor.php::__destruct gets invoked quite frequently. Having many child worker processes seems to also mean new sub-processes consuming many file descriptors. While the Relay Proxy is great at allowing the PHP SDK to quickly read from data stores like Redis, the impressions event processing system ends up quickly bottle-necking if there's any slow-down in publishing events.
Describe alternatives you've considered
I think ideally:
CurlEventPublisher
should use the actualcurl
client or something likeGuzzle
for making async requests instead of usingshell_exec
.CurlEventPublisher
should possibly explore batch requesting strategies that won't choke out technologies like PHP-FPM (e.g. using the file system + a separate worker thread). With more consideration for making sure these processes terminate - like logging warnings when it happens.I ultimately stuck with a simple change to minimize the chance of it being a breaking change.
Additional context
This is related to an issue I filed a bit ago: https://support.launchdarkly.com/hc/en-us/requests/87721