Skip to content

feat(cache): add updated_at columns and auth mapper operations for version-validated caching#10793

Open
yuqi1129 wants to merge 7 commits intoapache:branch-cache-improvementfrom
yuqi1129:feat/cache-db-schema-mapper
Open

feat(cache): add updated_at columns and auth mapper operations for version-validated caching#10793
yuqi1129 wants to merge 7 commits intoapache:branch-cache-improvementfrom
yuqi1129:feat/cache-db-schema-mapper

Conversation

@yuqi1129
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

  • Adds updated_at BIGINT NOT NULL DEFAULT 0 to role_meta, user_meta, group_meta, owner_meta tables for version-validated strong consistency
  • Creates entity_change_log table for HA cross-node targeted invalidation of name→id cache
  • Adds covering indexes for auth read predicates on the above tables
  • Adds mapper methods:
    • touchUpdatedAt / batchGetUpdatedAt on RoleMetaMapper
    • touchUpdatedAt / getUserInfo on UserMetaMapper
    • touchUpdatedAt / getGroupInfoByUserId on GroupMetaMapper
    • selectOwnerByMetadataObjectId / selectChangedOwners on OwnerMetaMapper
  • Creates EntityChangeLogMapper for entity structural change tracking (insert/select/prune)
  • Adds result record types in po/auth: UserAuthInfo, GroupAuthInfo, OwnerInfo, RoleUpdatedAt, ChangedOwnerInfo, EntityChangeRecord
  • Provides MySQL upgrade script upgrade-1.2.0-to-1.3.0-mysql.sql

Part of the Gravitino auth cache improvement design: Phase 1.2 + Phase 2 DB schema work.

Why are the changes needed?

The current JcasbinAuthorizer reloads all role policies on every request when the cache misses. The updated_at version sentinels enable strong-consistency version checks so that only stale entries are reloaded, and the entity_change_log table enables targeted cross-node cache invalidation in HA deployments without full cache flushes.

Does this PR introduce any user-facing changes?

No. All changes are internal DB schema and mapper additions; no public API changes.

How was this patch tested?

  • ./gradlew :core:test -PskipITs
  • Manual: apply upgrade SQL against a 1.2.0 schema, verify columns and indexes exist

🤖 Generated with Claude Code

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces DB-side version sentinels (updated_at) and an append-only change log (entity_change_log) to support version-validated, strong-consistency authorization caching and targeted cross-node cache invalidation in HA deployments.

Changes:

  • Adds a MySQL upgrade script to introduce updated_at columns on auth metadata tables, new covering indexes, and the new entity_change_log table.
  • Adds new MyBatis mapper/provider methods to update/read updated_at, query owner changes, and insert/select/prune entity change log rows.
  • Adds new po/auth record types to represent lightweight auth/cache query results.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
scripts/mysql/upgrade-1.2.0-to-1.3.0-mysql.sql MySQL schema upgrade: add updated_at columns, indexes, backfill, and entity_change_log table.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/UserAuthInfo.java Record for user id + staleness sentinel.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/GroupAuthInfo.java Record for group id + staleness sentinel.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/OwnerInfo.java Record for owner identity result.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/RoleUpdatedAt.java Record for role id + updated_at batch query results.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/ChangedOwnerInfo.java Record for owner change poller results.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/EntityChangeRecord.java Record for entity change poller results.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/UserMetaBaseSQLProvider.java Adds SQL for touching user updated_at and fetching UserAuthInfo.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/RoleMetaBaseSQLProvider.java Adds SQL for touching role updated_at and batch-getting role updated_at.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/GroupMetaBaseSQLProvider.java Adds SQL for touching group updated_at and fetching group auth info for a user.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/OwnerMetaBaseSQLProvider.java Adds SQL for selecting owner by object id and scanning changed owners.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/EntityChangeLogBaseSQLProvider.java Base SQL for selecting/inserting/pruning entity change log entries.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/DefaultMapperPackageProvider.java Registers the new EntityChangeLogMapper.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/UserMetaSQLProviderFactory.java Exposes new user SQL provider methods.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/UserMetaMapper.java Adds mapper methods for touching user updated_at and selecting UserAuthInfo.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/RoleMetaSQLProviderFactory.java Exposes new role SQL provider methods.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/RoleMetaMapper.java Adds mapper methods for touching role updated_at and batch-getting RoleUpdatedAt.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/GroupMetaSQLProviderFactory.java Exposes new group SQL provider methods.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/GroupMetaMapper.java Adds mapper methods for touching group updated_at and selecting GroupAuthInfo.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/OwnerMetaSQLProviderFactory.java Exposes new owner SQL provider methods.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/OwnerMetaMapper.java Adds mapper methods returning OwnerInfo and ChangedOwnerInfo.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/EntityChangeLogSQLProviderFactory.java Provider factory for entity change log operations across backends.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/EntityChangeLogMapper.java New MyBatis mapper for entity_change_log.

Comment thread scripts/mysql/upgrade-1.2.0-to-1.3.0-mysql.sql
…er operations

- Add updated_at BIGINT NOT NULL DEFAULT 0 to role_meta, user_meta, group_meta, owner_meta
- Create entity_change_log table for HA cross-node targeted metadataIdCache invalidation
- Add covering indexes for auth read-path version checks
- Add new mapper methods: touchUpdatedAt/batchGetUpdatedAt (RoleMetaMapper),
  touchUpdatedAt/getUserInfo (UserMetaMapper), touchUpdatedAt/getGroupInfoByUserId
  (GroupMetaMapper), selectOwnerByMetadataObjectId/selectChangedOwners (OwnerMetaMapper)
- Create EntityChangeLogMapper with selectChanges/insertChange/pruneOldEntries
- Add result types UserAuthInfo, GroupAuthInfo, OwnerInfo, RoleUpdatedAt,
  ChangedOwnerInfo, EntityChangeRecord (plain Java classes, Java 11 compatible)
- Add schema-1.3.0 and upgrade-1.2.0-to-1.3.0 scripts for MySQL, H2, PostgreSQL
- Add unit tests TestAuthMappers covering all new mapper methods

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@yuqi1129 yuqi1129 force-pushed the feat/cache-db-schema-mapper branch from 75bd9f5 to dd6c855 Compare April 16, 2026 01:40
@yuqi1129 yuqi1129 marked this pull request as draft April 16, 2026 01:53
@yuqi1129 yuqi1129 marked this pull request as draft April 16, 2026 01:53
yuqi1129 and others added 4 commits April 16, 2026 23:10
…ervice write paths

- RoleMetaService: touchUpdatedAt on privilege grant/revoke (same transaction)
- UserMetaService: touchUpdatedAt on role assign/revoke (same transaction)
- GroupMetaService: touchUpdatedAt on role assign/revoke (same transaction)
- OwnerRelPO: add updatedAt field; insertOwnerRel SQL includes updated_at column
- POConverters: set updatedAt=currentTimeMillis in initializeOwnerRelPOsWithVersion
- CatalogMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- SchemaMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- TableMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- FilesetMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- TopicMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- ViewMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- ModelMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- MetalakeMetaService: INSERT entity_change_log on rename (ALTER) and drop (DROP)
- All entity_change_log INSERTs are in the same DB transaction as the data change

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hange_log pruning

PostgreSQL does not support DELETE ... LIMIT syntax. Override
pruneOldEntityChanges in EntityChangeLogPostgreSQLProvider to use
DELETE ... WHERE id IN (SELECT id ... LIMIT 1000) instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 17, 2026

Code Coverage Report

Overall Project 65.33% +0.72% 🟢
Files changed 87.15% 🟢

Module Coverage
aliyun 1.73% 🔴
api 47.27% 🟢
authorization-common 85.96% 🟢
aws 1.1% 🔴
azure 2.6% 🔴
catalog-common 10.2% 🔴
catalog-fileset 80.02% 🟢
catalog-glue 75.36% 🟢
catalog-hive 81.83% 🟢
catalog-jdbc-clickhouse 79.06% 🟢
catalog-jdbc-common 43.93% 🟢
catalog-jdbc-doris 80.28% 🟢
catalog-jdbc-hologres 54.03% 🟢
catalog-jdbc-mysql 79.23% 🟢
catalog-jdbc-oceanbase 78.38% 🟢
catalog-jdbc-postgresql 82.05% 🟢
catalog-jdbc-starrocks 78.27% 🟢
catalog-kafka 77.01% 🟢
catalog-lakehouse-generic 45.07% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 87.16% 🟢
catalog-lakehouse-paimon 77.71% 🟢
catalog-model 77.72% 🟢
cli 44.51% 🟢
client-java 77.63% 🟢
common 48.67% 🟢
core 81.53% +0.66% 🟢
filesystem-hadoop3 76.97% 🟢
flink 40.55% 🟢
flink-runtime 0.0% 🔴
gcp 14.2% 🔴
hadoop-common 10.39% 🔴
hive-metastore-common 46.14% 🟢
iceberg-common 55.2% 🟢
iceberg-rest-server 67.15% 🟢
integration-test-common 0.0% 🔴
jobs 66.17% 🟢
lance-common 23.88% 🔴
lance-rest-server 57.84% 🟢
lineage 53.02% 🟢
optimizer 82.95% 🟢
optimizer-api 21.95% 🔴
server 85.62% 🟢
server-common 69.76% 🟢
spark 32.79% 🔴
spark-common 39.09% 🔴
trino-connector 34.27% 🔴
Files
Module File Coverage
common ConfigConstants.java 0.0% 🔴
core DefaultMapperPackageProvider.java 100.0% 🟢
GroupMetaBaseSQLProvider.java 100.0% 🟢
OwnerMetaBaseSQLProvider.java 100.0% 🟢
RoleMetaBaseSQLProvider.java 100.0% 🟢
UserMetaBaseSQLProvider.java 100.0% 🟢
OwnerRelPO.java 100.0% 🟢
MetalakeMetaService.java 100.0% 🟢
CatalogMetaService.java 98.89% 🟢
FilesetMetaService.java 96.83% 🟢
OwnerMetaSQLProviderFactory.java 96.3% 🟢
RoleMetaSQLProviderFactory.java 96.0% 🟢
GroupMetaSQLProviderFactory.java 95.65% 🟢
UserMetaSQLProviderFactory.java 95.65% 🟢
EntityChangeLogSQLProviderFactory.java 94.12% 🟢
GroupMetaService.java 94.03% 🟢
UserMetaService.java 94.03% 🟢
RoleMetaService.java 93.22% 🟢
POConverters.java 88.11% 🟢
TableMetaService.java 78.52% 🟢
SchemaMetaService.java 78.1% 🟢
ViewMetaService.java 76.15% 🟢
ModelMetaService.java 75.89% 🟢
EntityChangeLogBaseSQLProvider.java 75.0% 🟢
TopicMetaService.java 72.97% 🟢
EntityChangeRecord.java 69.57% 🟢
ChangedOwnerInfo.java 63.64% 🟢
OwnerInfo.java 63.64% 🟢
UserAuthInfo.java 63.64% 🟢
GroupAuthInfo.java 54.55% 🔴
RoleUpdatedAt.java 47.37% 🔴
EntityChangeLogMapper.java 0.0% 🔴
GroupMetaMapper.java 0.0% 🔴
OwnerMetaMapper.java 0.0% 🔴
RoleMetaMapper.java 0.0% 🔴
UserMetaMapper.java 0.0% 🔴

@yuqi1129 yuqi1129 changed the base branch from main to cache_improvement April 23, 2026 08:36
@yuqi1129 yuqi1129 requested a review from Copilot April 23, 2026 08:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds database-level version sentinels (updated_at) and an append-only entity_change_log to enable version-validated auth caching and targeted cross-node cache invalidation in HA deployments, and wires these into the relational store via new MyBatis mappers and service-layer write hooks.

Changes:

  • Adds updated_at columns + covering indexes to auth tables, and introduces entity_change_log + group_user_rel in schema/upgrade scripts (MySQL/H2/PostgreSQL).
  • Adds new MyBatis mapper operations/record types for reading/touching updated_at and for change-log insert/select/prune.
  • Updates entity services to write change-log entries on rename/drop and touches auth updated_at on role/group/user changes; adds mapper tests and bumps script version to 1.3.0.

Reviewed changes

Copilot reviewed 43 out of 43 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
scripts/postgresql/upgrade-1.2.0-to-1.3.0-postgresql.sql PostgreSQL upgrade: adds updated_at, indexes, backfill, and new tables (incl. entity_change_log).
scripts/postgresql/schema-1.3.0-postgresql.sql PostgreSQL 1.3.0 baseline schema including new auth/cache tracking columns and change-log tables.
scripts/mysql/upgrade-1.2.0-to-1.3.0-mysql.sql MySQL upgrade: adds updated_at, indexes, backfill, and new tables (incl. entity_change_log).
scripts/mysql/schema-1.3.0-mysql.sql MySQL 1.3.0 baseline schema including new auth/cache tracking columns and change-log tables.
scripts/h2/upgrade-1.2.0-to-1.3.0-h2.sql H2 upgrade: adds updated_at, indexes, backfill, and new tables (incl. entity_change_log).
scripts/h2/schema-1.3.0-h2.sql H2 1.3.0 baseline schema including new auth/cache tracking columns and change-log tables.
core/src/test/java/org/apache/gravitino/storage/relational/mapper/provider/base/TestAuthMappers.java Adds tests validating new mapper operations (touchUpdatedAt, auth info reads, change-log ops).
core/src/main/java/org/apache/gravitino/storage/relational/utils/POConverters.java Initializes OwnerRelPO.updatedAt on creation.
core/src/main/java/org/apache/gravitino/storage/relational/service/ViewMetaService.java Writes entity_change_log entries on view rename/drop.
core/src/main/java/org/apache/gravitino/storage/relational/service/UserMetaService.java Touches user_meta.updated_at when user-role mappings change.
core/src/main/java/org/apache/gravitino/storage/relational/service/TopicMetaService.java Writes entity_change_log entries on topic rename/drop.
core/src/main/java/org/apache/gravitino/storage/relational/service/TableMetaService.java Writes entity_change_log entries on table rename/drop.
core/src/main/java/org/apache/gravitino/storage/relational/service/SchemaMetaService.java Writes entity_change_log entries on schema rename/drop.
core/src/main/java/org/apache/gravitino/storage/relational/service/RoleMetaService.java Touches role_meta.updated_at when role privileges change.
core/src/main/java/org/apache/gravitino/storage/relational/service/ModelMetaService.java Writes entity_change_log entries on model rename/drop.
core/src/main/java/org/apache/gravitino/storage/relational/service/MetalakeMetaService.java Writes entity_change_log entries on metalake rename/drop.
core/src/main/java/org/apache/gravitino/storage/relational/service/GroupMetaService.java Touches group_meta.updated_at when group-role mappings change.
core/src/main/java/org/apache/gravitino/storage/relational/service/FilesetMetaService.java Writes entity_change_log entries on fileset rename/drop.
core/src/main/java/org/apache/gravitino/storage/relational/service/CatalogMetaService.java Writes entity_change_log entries on catalog rename/drop.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/UserAuthInfo.java Adds a result type for user auth lookup (userId, updatedAt).
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/RoleUpdatedAt.java Adds a result type for batch role updated_at reads.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/OwnerInfo.java Adds a result type for owner lookup.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/GroupAuthInfo.java Adds a result type for group membership auth lookup.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/EntityChangeRecord.java Adds a result type for entity_change_log polling.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/ChangedOwnerInfo.java Adds a result type for polling changed owners by updated_at.
core/src/main/java/org/apache/gravitino/storage/relational/po/OwnerRelPO.java Adds updatedAt field + builder setter for owner relations.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/UserMetaBaseSQLProvider.java Adds SQL for touching user updated_at and fetching user auth info.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/RoleMetaBaseSQLProvider.java Adds SQL for touching role updated_at and batch reading role updated_at.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/OwnerMetaBaseSQLProvider.java Adds updated_at to owner insert and adds owner/change polling queries.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/GroupMetaBaseSQLProvider.java Adds SQL for touching group updated_at and fetching group auth info by user.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/EntityChangeLogBaseSQLProvider.java Adds SQL for entity_change_log select/insert/prune.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/DefaultMapperPackageProvider.java Registers EntityChangeLogMapper for MyBatis initialization.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/UserMetaSQLProviderFactory.java Exposes new user updated-at/auth-info SQL methods.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/UserMetaMapper.java Adds mapper methods touchUpdatedAt and getUserInfo.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/RoleMetaSQLProviderFactory.java Exposes new role updated-at SQL methods.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/RoleMetaMapper.java Adds mapper methods touchUpdatedAt and batchGetUpdatedAt.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/OwnerMetaSQLProviderFactory.java Exposes new owner polling SQL methods.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/OwnerMetaMapper.java Adds mapper methods for owner lookup and changed-owner polling.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/GroupMetaSQLProviderFactory.java Exposes new group updated-at/auth-info SQL methods.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/GroupMetaMapper.java Adds mapper methods touchUpdatedAt and getGroupInfoByUserId.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/EntityChangeLogSQLProviderFactory.java Adds backend-specific SQL provider logic for change-log pruning.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/EntityChangeLogMapper.java Adds mapper interface for entity_change_log operations.
common/src/main/java/org/apache/gravitino/config/ConfigConstants.java Bumps current script version constant to 1.3.0.

Comment on lines +3 to +4
-- or more contributor license agreements. See the NOTICE file--
-- distributed with this work for additional information
Comment thread scripts/mysql/upgrade-1.2.0-to-1.3.0-mysql.sql
Comment thread scripts/postgresql/upgrade-1.2.0-to-1.3.0-postgresql.sql Outdated
@yuqi1129 yuqi1129 self-assigned this Apr 23, 2026
@yuqi1129 yuqi1129 marked this pull request as ready for review April 23, 2026 09:10
@yuqi1129 yuqi1129 requested review from Copilot and diqiu50 April 23, 2026 09:10
@yuqi1129 yuqi1129 changed the base branch from cache_improvement to branch-cache-improvement April 23, 2026 09:17
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR advances Gravitino’s relational-auth caching work by introducing DB-level version sentinels (updated_at) and an append-only entity_change_log to support targeted, cross-node cache invalidation in HA setups.

Changes:

  • Adds updated_at columns to auth-related meta tables and introduces entity_change_log + group_user_rel in upgrade/schema SQL across MySQL/PostgreSQL/H2.
  • Adds MyBatis mapper/provider methods and lightweight POJOs for auth/cache polling queries (touch/query updated_at; insert/select/prune change log; owner lookup/change polling).
  • Updates multiple relational *MetaService paths to emit entity_change_log entries on DROP and on rename-driven ALTER, and adds an H2-backed mapper test suite.

Reviewed changes

Copilot reviewed 43 out of 43 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
scripts/postgresql/upgrade-1.2.0-to-1.3.0-postgresql.sql PostgreSQL upgrade DDL for updated_at, indexes, group_user_rel, and entity_change_log.
scripts/postgresql/schema-1.3.0-postgresql.sql New PostgreSQL 1.3.0 baseline schema including new columns/tables.
scripts/mysql/upgrade-1.2.0-to-1.3.0-mysql.sql MySQL upgrade DDL for updated_at, covering indexes, and new tables.
scripts/mysql/schema-1.3.0-mysql.sql New MySQL 1.3.0 baseline schema including new columns/tables.
scripts/h2/upgrade-1.2.0-to-1.3.0-h2.sql H2 upgrade DDL (MySQL-mode) for new columns/indexes/tables.
scripts/h2/schema-1.3.0-h2.sql New H2 1.3.0 baseline schema including new columns/tables.
core/src/test/java/org/apache/gravitino/storage/relational/mapper/provider/base/TestAuthMappers.java Adds mapper-level tests for updated_at touch/query, owner queries, and change-log operations.
core/src/main/java/org/apache/gravitino/storage/relational/utils/POConverters.java Ensures newly created owner relations set updated_at.
core/src/main/java/org/apache/gravitino/storage/relational/service/ViewMetaService.java Emits entity_change_log entries on view rename (ALTER) and delete (DROP).
core/src/main/java/org/apache/gravitino/storage/relational/service/UserMetaService.java Touches user_meta.updated_at when user-role relations are updated.
core/src/main/java/org/apache/gravitino/storage/relational/service/TopicMetaService.java Emits entity_change_log entries on topic rename/delete.
core/src/main/java/org/apache/gravitino/storage/relational/service/TableMetaService.java Emits entity_change_log entries on table rename/delete.
core/src/main/java/org/apache/gravitino/storage/relational/service/SchemaMetaService.java Emits entity_change_log entries on schema rename/delete (cascade and non-cascade).
core/src/main/java/org/apache/gravitino/storage/relational/service/RoleMetaService.java Touches role_meta.updated_at when role privileges change.
core/src/main/java/org/apache/gravitino/storage/relational/service/ModelMetaService.java Emits entity_change_log entries on model rename/delete.
core/src/main/java/org/apache/gravitino/storage/relational/service/MetalakeMetaService.java Emits entity_change_log entries on metalake rename/delete.
core/src/main/java/org/apache/gravitino/storage/relational/service/GroupMetaService.java Touches group_meta.updated_at when group-role relations change.
core/src/main/java/org/apache/gravitino/storage/relational/service/FilesetMetaService.java Emits entity_change_log entries on fileset rename/delete.
core/src/main/java/org/apache/gravitino/storage/relational/service/CatalogMetaService.java Emits entity_change_log entries on catalog rename/delete.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/UserAuthInfo.java Adds result PO for user auth/version sentinel reads.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/RoleUpdatedAt.java Adds result PO for role updated_at batch reads.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/OwnerInfo.java Adds result PO for owner lookup results.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/GroupAuthInfo.java Adds result PO for group membership/version sentinel reads.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/EntityChangeRecord.java Adds result PO for entity change log polling.
core/src/main/java/org/apache/gravitino/storage/relational/po/auth/ChangedOwnerInfo.java Adds result PO for owner change polling results.
core/src/main/java/org/apache/gravitino/storage/relational/po/OwnerRelPO.java Adds updatedAt field + builder validation for owner relations.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/UserMetaBaseSQLProvider.java Adds SQL for touching user updated_at and fetching user auth info.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/RoleMetaBaseSQLProvider.java Adds SQL for touching role updated_at and batch-getting role updated_at.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/OwnerMetaBaseSQLProvider.java Extends owner insert to include updated_at; adds owner lookup and change-poll SQL.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/GroupMetaBaseSQLProvider.java Adds SQL for touching group updated_at and querying groups by user membership.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/EntityChangeLogBaseSQLProvider.java Introduces base SQL for selecting/inserting/pruning entity change log rows.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/DefaultMapperPackageProvider.java Registers the new EntityChangeLogMapper.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/UserMetaSQLProviderFactory.java Wires new user SQL-provider methods into the factory.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/UserMetaMapper.java Adds mapper APIs for touching updated_at and retrieving UserAuthInfo.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/RoleMetaSQLProviderFactory.java Wires new role SQL-provider methods into the factory.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/RoleMetaMapper.java Adds mapper APIs for touching updated_at and batch querying role updated_at.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/OwnerMetaSQLProviderFactory.java Wires new owner SQL-provider methods into the factory.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/OwnerMetaMapper.java Adds mapper APIs for owner lookup and changed-owner polling.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/GroupMetaSQLProviderFactory.java Wires new group SQL-provider methods into the factory.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/GroupMetaMapper.java Adds mapper APIs for touching updated_at and querying groups by user.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/EntityChangeLogSQLProviderFactory.java Adds provider factory + PostgreSQL-specific prune override.
core/src/main/java/org/apache/gravitino/storage/relational/mapper/EntityChangeLogMapper.java Adds MyBatis mapper for entity change log insert/select/prune.
common/src/main/java/org/apache/gravitino/config/ConfigConstants.java Bumps CURRENT_SCRIPT_VERSION to 1.3.0 so new schemas are used.

Comment thread scripts/h2/schema-1.3.0-h2.sql
Comment on lines +21 to +28
/** Entity change poller result -- one row per entity_change_log entry. */
public class EntityChangeRecord {
private String metalakeName;
private String entityType;
private String fullName;
private String operateType;
private long createdAt;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need so far

Comment thread scripts/postgresql/schema-1.3.0-postgresql.sql
Comment thread scripts/mysql/schema-1.3.0-mysql.sql
Comment thread scripts/h2/upgrade-1.2.0-to-1.3.0-h2.sql
Comment thread scripts/h2/schema-1.3.0-h2.sql
Comment thread scripts/mysql/schema-1.3.0-mysql.sql
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants