ai/live: Reuse HTTP connections for trickle requests #3470
Conversation
Initially we were re-creating connections to avoid being stuck in a degraded state over time, but it seems that we may have trouble even establishing a connection sometimes. Attempt to re-use established connections to mitigate this problem. Both scenarios should be caught by the slow orchestrator checker, but hopefully connection reuse makes the checker trip less often.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3470 +/- ##
===================================================
+ Coverage 30.35863% 30.36115% +0.00252%
===================================================
Files 151 151
Lines 44531 44524 -7
===================================================
- Hits 13519 13518 -1
+ Misses 30222 30216 -6
Partials 790 790
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
| // ignore orch certs for now | ||
| TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, | ||
| }}).Do(req) | ||
| resp, err := httpClient().Do(req) |
There was a problem hiding this comment.
just checking it was intentional to create a new client on this one
There was a problem hiding this comment.
yep - this method is not on the hot path and actually only gets invoked if there is a problem sending the segment. So it seems best to have a fresh client here to mitigate any issues with the existing connection, similar to why we reset the client when we retry (here)
There was a problem hiding this comment.
Added some comments clarifying this, thanks - c3330b0
Initially we were re-creating connections to avoid being stuck in a degraded state over time, but it seems that we may have trouble even establishing a connection sometimes. To mitigate this problem, try to re-use established connections.
Both scenarios should be caught by the slow orchestrator checker, but hopefully connection reuse makes the checker trip less often.