Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: add more log context to continuous_batch_parser::consume_records #25395

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

WillemKauf
Copy link
Contributor

@WillemKauf WillemKauf commented Mar 15, 2025

The ERROR log line we see appearing in the continuous_batch_parser would be more helpful with added information about the batch consumer type and record batch header.

Add a new ERROR log line to output this additional context in the cold path.

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v24.3.x
  • v24.2.x
  • v24.1.x

Release Notes

  • none

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Mar 15, 2025

CI test results

test results on build#63192
test_id test_kind job_url test_status passed
cloud_storage_rpfixture.cloud_storage_rpfixture unit https://buildkite.com/redpanda/redpanda/builds/63192#0195988e-9613-41a8-8403-65043aac9ef1 FAIL 0/2
partition_balancer_simulator_test_rpunit.partition_balancer_simulator_test_rpunit unit https://buildkite.com/redpanda/redpanda/builds/63192#0195988e-9613-41a8-8403-65043aac9ef1 FLAKY 1/2
rptest.tests.consumer_group_test.ConsumerGroupTest.test_group_lag_metrics ducktape https://buildkite.com/redpanda/redpanda/builds/63192#019598d8-f30f-4e6a-8cd1-cce545aac06a FLAKY 1/2
rptest.tests.data_migrations_api_test.DataMigrationsApiTest.test_migrated_topic_data_integrity.transfer_leadership=True.params=.cancellation.dir.in.stage.preparing.use_alias.False ducktape https://buildkite.com/redpanda/redpanda/builds/63192#019598e6-84fa-4a1f-83b4-d37ad2020dcc FLAKY 1/2
rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=False.mixed_versions=False.with_tiered_storage=True.with_iceberg=False.with_chunked_compaction=True.cloud_storage_type=CloudStorageType.S3 ducktape https://buildkite.com/redpanda/redpanda/builds/63192#019598d8-f30f-4e6a-8cd1-cce545aac06a FLAKY 1/2
rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=True.mixed_versions=False.with_tiered_storage=True.with_iceberg=True.with_chunked_compaction=False.cloud_storage_type=CloudStorageType.S3 ducktape https://buildkite.com/redpanda/redpanda/builds/63192#019598d8-f30e-4e20-92f2-f85c21a25b58 FLAKY 1/2
test results on build#63196
test_id test_kind job_url test_status passed
rptest.tests.compaction_recovery_test.CompactionRecoveryUpgradeTest.test_index_recovery_after_upgrade ducktape https://buildkite.com/redpanda/redpanda/builds/63196#01959a61-e39c-404f-ae52-9d0a90af1e60 FLAKY 1/2
rptest.tests.data_migrations_api_test.DataMigrationsApiTest.test_migrated_topic_data_integrity.transfer_leadership=True.params=.cancellation.dir.in.stage.preparing.use_alias.False ducktape https://buildkite.com/redpanda/redpanda/builds/63196#01959a47-cc7b-4856-856a-2a4767f588d1 FLAKY 1/2
rptest.tests.datalake.datalake_e2e_test.DatalakeE2ETests.test_topic_lifecycle.cloud_storage_type=CloudStorageType.S3.catalog_type=CatalogType.NESSIE ducktape https://buildkite.com/redpanda/redpanda/builds/63196#01959a47-cc7a-4990-9acc-ab6c805b2d07 FLAKY 1/2
rptest.tests.datalake.datalake_upgrade_test.DatalakeUpgradeTest.test_upload_through_upgrade.cloud_storage_type=CloudStorageType.S3.query_engine=QueryEngineType.SPARK ducktape https://buildkite.com/redpanda/redpanda/builds/63196#01959a61-e39d-4c78-ac0b-b6626b887978 FAIL 0/20
rptest.tests.scaling_up_test.ScalingUpTest.test_scaling_up_with_recovered_topic ducktape https://buildkite.com/redpanda/redpanda/builds/63196#01959a61-e39b-411d-9726-dd7954415a9d FLAKY 1/2

This was being implicitly converted to `std::error_code`.
…records`

This `ERROR` log line would be more helpful with added information about the
batch consumer type and record batch header.

Add a new `ERROR` log line to output these in the cold path.
@vbotbuildovich
Copy link
Collaborator

Retry command for Build#63196

please wait until all jobs are finished before running the slash command

/ci-repeat 1
tests/rptest/tests/datalake/datalake_upgrade_test.py::DatalakeUpgradeTest.test_upload_through_upgrade@{"cloud_storage_type":1,"query_engine":"spark"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants