Fix MDC connector context not propagating to async querier thread#1613
Draft
Sangeet Mishra (sangeet259) wants to merge 1 commit into10.8.xfrom
Draft
Fix MDC connector context not propagating to async querier thread#1613Sangeet Mishra (sangeet259) wants to merge 1 commit into10.8.xfrom
Sangeet Mishra (sangeet259) wants to merge 1 commit into10.8.xfrom
Conversation
|
🎉 All Contributor License Agreements have been signed. Ready to merge. |
Since v10.8.5, logs from the async query path (TableQuerierProcessor, TableQuerier, etc.) lost the [connector-name|task-N] prefix because Kafka Connect's MDC is thread-local and was not propagated to the background executor thread introduced by the RecordQueue architecture. Capture MDC.getCopyOfContextMap() on the calling (poll) thread before submitting to the executor, then restore it on the executor thread before ConnectLogContext is created. This allows ConnectLogContext to find and extend the connector.context MDC entry as intended. MDC is cleared in a finally block to prevent leaking to pooled threads.
airlock-confluentinc
bot
force-pushed
the
fix-mdc-context-propagation-async-querier
branch
from
fe4863d to
27a0b79
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
[connector-name|task-N]prefixVerified with kafka-docker-playground
Ran a JDBC MySQL source connector with the fix applied. Logs from the async query thread now include the connector context:
After fix (v10.8.6-SNAPSHOT):
Before fix (v10.8.5):
Test plan
ConnectLogContextTest— MDC append, prefix fallback, nested contextsRecordQueueMdcTest— MDC propagation to executor, cleanup between executions, no-MDC fallbackJdbcSourceTaskLifecycleTestpasses