Skip to content

Fix 4 failing CI integration tests: empty broker list crash, TOPIC_NOT_EXIST consumer deadlock, test ordering race - #116

Merged
nnhy merged 4 commits into
masterfrom
copilot/fix-build-test-failure
Jul 7, 2026
Merged

Fix 4 failing CI integration tests: empty broker list crash, TOPIC_NOT_EXIST consumer deadlock, test ordering race#116
nnhy merged 4 commits into
masterfrom
copilot/fix-build-test-failure

Conversation

Copilot AI commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Three distinct bugs caused 4 CI integration tests to fail against RocketMQ 5.3.1 with auto-topic-creation enabled.

Fix 1 — MqBase.cs: crash on empty broker list (RequestSyncTest)

.First() on a LINQ sequence throws InvalidOperationException when RocketMQ returns no writable brokers for a brand-new topic.

// Before — throws if no writable broker exists
var writeQueueNums = rs.Where(...).Select(e => e.WriteQueueNums).First();

// After — safe default
var writeQueueNums = rs.Where(...).Select(e => e.WriteQueueNums).FirstOrDefault();
if (writeQueueNums > 0)
    DefaultTopicQueueNums = Math.Min(DefaultTopicQueueNums, writeQueueNums);

Fix 2 — Consumer.cs: consumer permanently stuck on new topic (MultiTopic, ProducerTracerTests)

InitOffsetAsync called UpdateOffset() which throws ResponseException(TOPIC_NOT_EXIST) for topics not yet registered on the broker. The exception escaped Rebalance() after _Queues was already assigned but before return true, so DoSchedule() was never called — the consumer deadlocked with no pull threads, and all subsequent rebalance checks found _Queues unchanged and short-circuited.

Two sub-fixes:

  • QueryMaxOffset/QueryMinOffset return -1 for unknown topics (via ignoreError=true); clamp to 0 to start from the beginning rather than storing a sentinel.
  • Catch ResponseException(TOPIC_NOT_EXIST) from UpdateOffset so the rebalance can complete; the offset is committed after the first successful consumption.

Fix 3 — ConsumerRetryDLQIntegrationTests.cs: message published before consumer ready (ConsumerRetry_DisabledRetry)

With FromLastOffset=true, publishing before the consumer starts means the consumer begins at MaxOffset=1, permanently skipping the message at offset 0. Reordered to: start consumer → wait for rebalance → start producer and publish.

Copilot AI added 3 commits July 7, 2026 10:13
…consumer init, and test ordering

- MqBase.cs: Replace .First() with .FirstOrDefault() to avoid InvalidOperationException when no writable broker found for a brand new topic (fixes RequestSyncTest)
- Consumer.cs: In InitOffsetAsync, default offset from -1 to 0 when QueryMaxOffset/QueryMinOffset returns -1 for non-existent topic; catch ResponseException(TOPIC_NOT_EXIST) from UpdateOffset so Rebalance completes and DoPull threads start (fixes MultiTopic and ProducerTracerTests)
- ConsumerRetryDLQIntegrationTests.cs: Start consumer before producer and wait 3s for rebalance before publishing, ensuring FromLastOffset=true consumer sees the message (fixes ConsumerRetry_DisabledRetry_MessageNotRedelivered)
Copilot AI changed the title [WIP] Fix failing GitHub Actions job 'build-test' Fix 4 failing CI integration tests: empty broker list crash, TOPIC_NOT_EXIST consumer deadlock, test ordering race Jul 7, 2026
Copilot AI requested a review from nnhy July 7, 2026 10:19
Copilot finished work on behalf of nnhy July 7, 2026 10:19
@nnhy
nnhy marked this pull request as ready for review July 7, 2026 10:44
Copilot AI review requested due to automatic review settings July 7, 2026 10:44
@nnhy
nnhy merged commit 75128a0 into master Jul 7, 2026
1 check passed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

该 PR 针对 RocketMQ 5.3.1(开启自动建 Topic)场景下的 4 个 CI 集成测试失败问题,分别从 Broker 路由初始化健壮性、消费者初始偏移量初始化对“Topic 尚未创建”的容错,以及测试用例启动顺序/竞态三方面进行修复,从而避免崩溃、死锁与消息被永久跳过。

Changes:

  • MqBase.OnStart():当名称服务器返回“无可写 Broker”时不再因 First()InvalidOperationException 崩溃。
  • Consumer.InitOffsetAsync():对 QueryMaxOffset/QueryMinOffset 返回 -1(Topic 未创建)进行归零处理,并捕获 UpdateOffsetTOPIC_NOT_EXIST 使 Rebalance 能继续完成。
  • ConsumerRetry_DisabledRetry 集成测试:调整为“先启动消费者并等待 Rebalance → 再发送消息”,避免 FromLastOffset=true 下的竞态导致跳过消息。

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
XUnitTestRocketMQ/Consumers/ConsumerRetryDLQIntegrationTests.cs 调整测试启动顺序,确保消费者就绪后再发消息,避免偏移量竞态导致用例不稳定。
NewLife.RocketMQ/MqBase.cs 获取路由信息时对“空可写 Broker 列表”做保护,避免启动期崩溃。
NewLife.RocketMQ/Consumer.cs 初始 offset 初始化对 Topic 未创建场景增加容错,避免 Rebalance 中断导致消费线程不启动。

Comment on lines +112 to +113
// 提前生成唯一标识,避免 lambda 闭包引用顺序问题
var stamp = Guid.NewGuid().ToString("N")[..8];
return true; // 第二次起放行,偏移推进
};
consumer.Start();
Thread.Sleep(3000); // 等待消费者重新平衡完成
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants