Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dl/translation: some minor qol fixes #25398

Draft
wants to merge 6 commits into
base: dev
Choose a base branch
from
Draft

dl/translation: some minor qol fixes #25398

wants to merge 6 commits into from

Conversation

bharathv
Copy link
Contributor

@bharathv bharathv commented Mar 16, 2025 •
edited
Loading

  • improves logging
  • refactors the main translation logic a bit for readability
  • deflakes datalake_upgrade_test.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v24.3.x
  • v24.2.x
  • v24.1.x

Release Notes

  • none

bharathv added 6 commits March 16, 2025 09:42
@bharathv
dl/translation/debugging: improve logging for future debugging needs
5a8ca6d
@bharathv
dl/translation: propagate translation task context to translator
4465ca4 
With these changes we now propagate the translate finish errors into
the translator so it can make an informed decision about the next
iteration of the loop.

Particularly we are interested in the case where there is no data to
translate and avoid warnings/backoff. Currently that case is treated
like any other error, a warning is logged along with additional sleep.
@bharathv
dl/translator: avoid log spew if there is no data to translate
8bb44fb 
Currently it is treated like any other error and wan is logged.
@bharathv
dl/translator: refactor partition translator main loop
ffd7121 
no functional change, just factors out the main loop logic
to make it easy to read
@bharathv
dl/translation: cleanup property handling
16c9116
@bharathv
dl/tests: deflake upgrade test
c001abb
@bharathv
Copy link
Contributor Author

/dt

@vbotbuildovich
Copy link
Collaborator

CI test results

test results on build#63205
test_id test_kind job_url test_status passed
rptest.tests.log_segment_ms_test.SegmentMsTest.test_segment_rolling.use_alter_cfg=True.server_cfg=None.topic_cfg=300000 ducktape https://buildkite.com/redpanda/redpanda/builds/63205#0195a03b-8861-4716-8380-522935f86389 FLAKY 1/2
rptest.tests.node_pool_migration_test.NodePoolMigrationTest.test_migrating_redpanda_nodes_to_new_pool.balancing_mode=off.test_mode=TestMode.FAST_MOVES.cleanup_policy=compact ducktape https://buildkite.com/redpanda/redpanda/builds/63205#0195a03e-09bf-42a3-80c9-a26f4ce05a71 FLAKY 1/2
rptest.tests.scaling_up_test.ScalingUpTest.test_scaling_up_with_recovered_topic ducktape https://buildkite.com/redpanda/redpanda/builds/63205#0195a03e-09c0-44b4-b520-f3a2932a4a3e FLAKY 1/3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area/redpanda
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bharathv @vbotbuildovich