Skip to content

HBASE-29291: Add a command to refresh/sync hbase:meta table #7058

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: HBASE-29081
Choose a base branch
from

Conversation

Kota-SH
Copy link
Contributor

@Kota-SH Kota-SH commented Jun 2, 2025

@Kota-SH Kota-SH marked this pull request as draft June 2, 2025 20:05
@Kota-SH Kota-SH marked this pull request as ready for review June 2, 2025 20:08
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 37s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 buf 0m 0s buf was not available.
+0 🆗 buf 0m 0s buf was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
_ HBASE-29081 Compile Tests _
+0 🆗 mvndep 0m 11s Maven dependency ordering for branch
+1 💚 mvninstall 3m 21s HBASE-29081 passed
+1 💚 compile 6m 0s HBASE-29081 passed
+1 💚 checkstyle 1m 35s HBASE-29081 passed
+1 💚 spotbugs 6m 34s HBASE-29081 passed
+1 💚 spotless 1m 6s branch has no errors when running spotless:check.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 12s Maven dependency ordering for patch
+1 💚 mvninstall 3m 51s the patch passed
+1 💚 compile 6m 59s the patch passed
+1 💚 cc 6m 59s the patch passed
+1 💚 javac 6m 59s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 22s /results-checkstyle-hbase-client.txt hbase-client: The patch generated 1 new + 3 unchanged - 0 fixed = 4 total (was 3)
-0 ⚠️ checkstyle 0m 18s /buildtool-patch-checkstyle-hbase-server.txt The patch fails to run checkstyle in hbase-server
+1 💚 spotbugs 11m 51s the patch passed
+1 💚 hadoopcheck 17m 59s Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚 hbaseprotoc 11m 42s the patch passed
-1 ❌ spotless 0m 20s patch has 38 errors when running spotless:check, run spotless:apply to fix.
_ Other Tests _
+1 💚 asflicense 0m 35s The patch does not generate ASF License warnings.
88m 53s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7058/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #7058
JIRA Issue HBASE-29291
Optional Tests dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless cc buflint bufcompat hbaseprotoc
uname Linux 8a006e1723b6 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision HBASE-29081 / 6e01319
Default Java Eclipse Adoptium-17.0.11+9
spotless https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7058/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count 85 (vs. ulimit of 30000)
modules C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7058/1/console
versions git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 32s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ HBASE-29081 Compile Tests _
+0 🆗 mvndep 0m 11s Maven dependency ordering for branch
+1 💚 mvninstall 3m 12s HBASE-29081 passed
+1 💚 compile 2m 20s HBASE-29081 passed
+1 💚 javadoc 1m 27s HBASE-29081 passed
+1 💚 shadedjars 6m 1s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 13s Maven dependency ordering for patch
+1 💚 mvninstall 3m 7s the patch passed
+1 💚 compile 2m 19s the patch passed
+1 💚 javac 2m 19s the patch passed
+1 💚 javadoc 1m 29s the patch passed
+1 💚 shadedjars 6m 1s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 0m 34s hbase-protocol-shaded in the patch passed.
+1 💚 unit 1m 36s hbase-client in the patch passed.
-1 ❌ unit 230m 22s /patch-unit-hbase-server.txt hbase-server in the patch failed.
+1 💚 unit 6m 48s hbase-thrift in the patch passed.
271m 30s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7058/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #7058
JIRA Issue HBASE-29291
Optional Tests javac javadoc unit compile shadedjars
uname Linux 43360259a509 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision HBASE-29081 / 6e01319
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7058/1/testReport/
Max. process+thread count 5566 (vs. ulimit of 30000)
modules C: hbase-protocol-shaded hbase-client hbase-server hbase-thrift U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7058/1/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@taklwu taklwu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM and have few minor comments, let me try to use the copilot to have another round of verification

@@ -742,7 +742,7 @@ private static void deleteFromMetaTable(final Connection connection, final Delet
* @param connection connection we're using
* @param deletes Deletes to add to hbase:meta This list should support #remove.
*/
private static void deleteFromMetaTable(final Connection connection, final List<Delete> deletes)
public static void deleteFromMetaTable(final Connection connection, final List<Delete> deletes)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this change for testing purpose?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, I added in case we need to delete regions from the meta table during refresh.

Comment on lines +375 to +377
private boolean isSystemDirectory(String dirName) {
return dirName.startsWith(".") || dirName.startsWith("-") || dirName.startsWith("_");
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: do we still have system directory start with _ ? is it _logs ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I wasn't sure about that. So I just added it just in case.

Set<RegionInfo> latestSet = new HashSet<>(latest);

// Check for size or set differences
if (currentSet.size() != latestSet.size() || !currentSet.equals(latestSet)) {
Copy link
Contributor

@taklwu taklwu Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Set's equals should have compare the size before calling the containsAll

Suggested change
if (currentSet.size() != latestSet.size() || !currentSet.equals(latestSet)) {
if (!currentSet.equals(latestSet)) {

Comment on lines +286 to +299
// Check for boundary changes
for (RegionInfo cr : current) {
RegionInfo lr = latest.stream()
.filter(r -> r.getRegionNameAsString().equals(cr.getRegionNameAsString()))
.findFirst()
.orElse(null);

if (lr == null || hasBoundaryChanged(cr, lr)) {
LOG.info("Region mismatch or boundary changed for {}", cr.getRegionNameAsString());
return true;
}
}
return false;
}
Copy link
Contributor

@taklwu taklwu Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: just in case of large amount of regions need to be synced, let's optimize this code for better looping from worst case of O(N^2) to O(N).

Suggested change
// Check for boundary changes
for (RegionInfo cr : current) {
RegionInfo lr = latest.stream()
.filter(r -> r.getRegionNameAsString().equals(cr.getRegionNameAsString()))
.findFirst()
.orElse(null);
if (lr == null || hasBoundaryChanged(cr, lr)) {
LOG.info("Region mismatch or boundary changed for {}", cr.getRegionNameAsString());
return true;
}
}
return false;
}
// Check for boundary changes
Map<String, RegionInfo> latestMap = latest.stream()
.collect(Collectors.toMap(RegionInfo::getRegionNameAsString, Function.identity()));
for (RegionInfo cr : current) {
RegionInfo lr = latestMap.get(cr.getRegionNameAsString());
if (lr == null || hasBoundaryChanged(cr, lr)) {
LOG.info("Region mismatch or boundary changed for {}", cr.getRegionNameAsString());
return true;
}
}
return false;
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point. And afaik there will not be duplicate regions. I will make this change.

Comment on lines +112 to +114
if (c.getEnvironment().getRegionInfo().getTable().isSystemTable()) {
return;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this and below changes in ReadOnlyController be out of this PR? or is this blocking the refreshMeta ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is blocking the refresh_meta. We need to be able to put and delete in the system table (hbase:meta). So I made these changes in the ReadOnlyController.

Comment on lines +98 to +100
currentRegions = getCurrentRegions(env.getMasterServices().getConnection());
LOG.info("Found {} current regions in meta table",
currentRegions != null ? currentRegions.size() : 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: getCurrentRegions by calling MetaTableAccessor.getAllRegions will return empty set instead of null, is this null check required?

Suggested change
currentRegions = getCurrentRegions(env.getMasterServices().getConnection());
LOG.info("Found {} current regions in meta table",
currentRegions != null ? currentRegions.size() : 0);
currentRegions = getCurrentRegions(env.getMasterServices().getConnection());
LOG.info("Found {} current regions in meta table", currentRegions.size());

@taklwu taklwu requested a review from Copilot June 3, 2025 19:10
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request implements a new command to refresh/sync the hbase:meta table, addressing Jira ticket HBASE-29291. The changes include adding a refreshMeta method across various Admin interfaces and RPC endpoints, integration and unit tests for the new procedure, and corresponding protocol buffer definitions and updates.

  • Added refreshMeta methods in HBase admin and client classes.
  • Introduced new tests and integration procedures for the meta table refresh workflow.
  • Updated RPC and protobuf definitions to support the refreshMeta command.

Reviewed Changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated no comments.

Show a summary per file
File Description
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/client/ThriftAdmin.java Added a refreshMeta stub throwing NotImplementedException
hbase-server/src/test/java/org/apache/hadoop/hbase/rsgroup/VerifyingRSGroupAdmin.java Delegated refreshMeta call to underlying admin
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRefreshMetaProcedureIntegration.java Integrated end-to-end test for meta refresh behavior
hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRefreshMetaProcedure.java Added various unit tests for RefreshMetaProcedure functionality
hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/ReadOnlyController.java Allowed deletes on system tables during read-only mode
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java Implemented RPC endpoint for refreshMeta
hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java Added refreshMeta implementation using RefreshMetaProcedure
hbase-server/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java Changed deleteFromMetaTable method visibility to public
hbase-protocol-shaded/src/main/protobuf/server/master/MasterProcedure.proto Added RefreshMeta state definitions
hbase-protocol-shaded/src/main/protobuf/server/master/Master.proto Added RefreshMeta request/response messages
hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java Added async refreshMeta call
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncHBaseAdmin.java Exposed async refreshMeta in client API
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncAdmin.java Added refreshMeta to AsyncAdmin interface
hbase-client/src/main/java/org/apache/hadoop/hbase/client/AdminOverAsyncAdmin.java Exposed refreshMeta over AsyncAdmin
hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java Added refreshMeta method in the Admin API
Comments suppressed due to low confidence (2)

hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestRefreshMetaProcedureIntegration.java:168

  • [nitpick] The fixed wait time of 3000 ms might be insufficient on slower environments, potentially leading to flaky tests. Consider increasing the timeout or making it configurable.
TEST_UTIL.waitFor(3000, () -> {

hbase-server/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java:745

  • The visibility of deleteFromMetaTable has been changed from private to public. Please confirm that exposing this method aligns with the overall API design and does not introduce unintended access.
public static void deleteFromMetaTable(final Connection connection, final List<Delete> deletes)

Copy link
Contributor

@taklwu taklwu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: one more thought, we don't need to address in this PR, but assuming one cluster with 100k regions, would this refresh_meta for all tables still work?

@Kota-SH
Copy link
Contributor Author

Kota-SH commented Jun 3, 2025

@taklwu - Thanks for the review!

nit: one more thought, we don't need to address in this PR, but assuming one cluster with 100k regions, would this refresh_meta for all tables still work?

Well, it becomes a heavy operation, and that's something we should test for.
There is another idea about periodically persisting a region to hfiles mapping in the storage, and being able to apply edits on demand based on that mapping. For now, it is just an idea that was discussed a long time ago upstream.

I'm thinking this refresh_meta implementation will be a fallback operation, in case we need to refresh_meta manually without depending on the active cluster persisting updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants