Skip to content

refactor: align column name: catalogKey -> catalogName #65

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nryanov merged 6 commits into from
Aug 10, 2026
Merged

refactor: align column name: catalogKey -> catalogName #65

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
bd9061b
refactor: align column name: catalogKey -> catalogName
nryanov Aug 9, 2026
f8e790b
test fix
nryanov Aug 9, 2026
5ff4364
update
nryanov Aug 9, 2026
e38cf46
remove unneccessary comments
nryanov Aug 9, 2026
5fa72af
update
nryanov Aug 10, 2026
4ecaea8
update
nryanov Aug 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/catalogs/lance/kasanari.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@

Set `kasanari.repository.backend` to `postgres` (default) or `yugabyte`.

On YugabyteDB, Lance INTERNAL rows are isolated by `catalog_key` (the management `catalogId`, injected automatically) and hash-sharded so one catalog stays on one tablet. See [YugabyteDB backend](../../operations/yugabyte.md).
On YugabyteDB, Lance INTERNAL rows are hash-sharded on `catalog_name`. See [YugabyteDB backend](../../operations/yugabyte.md).

## PROXY catalog setup

Expand Down
70 changes: 57 additions & 13 deletions docs/catalogs/paimon/jdbc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ Not applicable for this implementation. For Kasanari-owned metadata, use [Paimon

## PROXY catalog setup

`catalog-key` defaults to `jdbc`. When several PROXY JDBC catalogs share one JDBC metastore, set a distinct `catalog-key` per catalog.

### Minimal runnable config

```json
Expand All @@ -24,7 +26,8 @@ Not applicable for this implementation. For Kasanari-owned metadata, use [Paimon
"warehouse": "s3a://warehouse",
"jdbc-url": "jdbc:postgresql://localhost:5432/postgres",
"jdbc-user": "postgres",
"jdbc-password": "postgres"
"jdbc-password": "postgres",
"catalog-key": "paimon_proxy_jdbc"
}
}
}
Expand Down Expand Up @@ -52,22 +55,63 @@ Not applicable for this implementation. For Kasanari-owned metadata, use [Paimon
"jdbc-user": "postgres",
"jdbc-password": "postgres",
"jdbc-driver": "org.postgresql.Driver",
"jdbc-table-prefix": "paimon_"
"jdbc-table-prefix": "paimon_",
"catalog-key": "paimon_proxy_jdbc"
}
}
}
```

## Additional properties
### Shared JDBC metastore

```json
{
"catalogId": "paimon_proxy_jdbc_a",
"catalogType": "PAIMON",
"mode": "PROXY",
"spec": {
"fileIoProperties": {},
"catalogProperties": {
"type": "jdbc",
"warehouse": "s3a://warehouse-a",
"jdbc-url": "jdbc:postgresql://localhost:5432/postgres",
"jdbc-user": "postgres",
"jdbc-password": "postgres",
"catalog-key": "catalog_a"
}
}
}
```

```json
{
"catalogId": "paimon_proxy_jdbc_b",
"catalogType": "PAIMON",
"mode": "PROXY",
"spec": {
"fileIoProperties": {},
"catalogProperties": {
"type": "jdbc",
"warehouse": "s3a://warehouse-b",
"jdbc-url": "jdbc:postgresql://localhost:5432/postgres",
"jdbc-user": "postgres",
"jdbc-password": "postgres",
"catalog-key": "catalog_b"
}
}
}
```

| config | default value | meaning |
|------------------------------------|--------------------------------|--------------------------------------------------------------------|
| `type` | none | Paimon proxy catalog type. Must be `jdbc` for this page. |
| `warehouse` | none | Paimon warehouse path. Required. |
| `jdbc-url` | none | JDBC endpoint for external Paimon JDBC catalog metadata. Required. |
| `jdbc-user` | none | JDBC username. |
| `jdbc-password` | none | JDBC password. |
| `jdbc-driver` | JDBC driver default resolution | Explicit JDBC driver class name. |
| `jdbc-table-prefix` | implementation-specific | Prefix for metadata tables in JDBC backend. |
| `fs.s3a.*` (in `fileIoProperties`) | none | Hadoop S3A options used by Paimon storage access. |
## Additional properties

| config | default value | meaning |
|------------------------------------|--------------------------------|--------------------------------------------------------------------------------|
| `type` | none | Paimon proxy catalog type. Must be `jdbc` for this page. |
| `warehouse` | none | Paimon warehouse path. Required. |
| `jdbc-url` | none | JDBC endpoint for external Paimon JDBC catalog metadata. Required. |
| `jdbc-user` | none | JDBC username. |
| `jdbc-password` | none | JDBC password. |
| `jdbc-driver` | JDBC driver default resolution | Explicit JDBC driver class name. |
| `jdbc-table-prefix` | implementation-specific | Prefix for metadata tables in JDBC backend. |
| `catalog-key` | `jdbc` | Isolation key in the JDBC metastore. Distinct per catalog when sharing one DB. |
| `fs.s3a.*` (in `fileIoProperties`) | none | Hadoop S3A options used by Paimon storage access. |
4 changes: 1 addition & 3 deletions docs/catalogs/paimon/kasanari.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,7 @@
"kasanari.jdbc.connection-pool.initial-size": "2",
"kasanari.jdbc.connection-pool.min-size": "1",
"kasanari.jdbc.connection-pool.max-size": "5",
"kasanari.jdbc.connection-pool.max-lifetime.millis": "0",
"kasanari.catalog.key": "paimon_internal"
"kasanari.jdbc.connection-pool.max-lifetime.millis": "0"
}
}
}
Expand All @@ -72,7 +71,6 @@ Not applicable for this implementation. Use one of the `PROXY` pages:
| `uri` | none | JDBC URL for Kasanari metadata repositories. Required. |
| `kasanari.jdbc.user` | none | JDBC username for metadata DB. Required. |
| `kasanari.jdbc.password` | none | JDBC password for metadata DB. Required. |
| `kasanari.catalog.key` | `default` | Logical key used to isolate metadata records in JDBC repositories. |
| `kasanari.jdbc.connection-pool.initial-size` | `2` | Initial JDBC pool size. |
| `kasanari.jdbc.connection-pool.min-size` | `1` | Minimum JDBC pool size. |
| `kasanari.jdbc.connection-pool.max-size` | `5` | Maximum JDBC pool size. |
Expand Down
26 changes: 12 additions & 14 deletions docs/operations/yugabyte.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ jdbc:yugabytedb://yb-tserver:5433/kasanari?load-balance=true

Credentials use the same keys as PostgreSQL:

| Key | Purpose |
|-----|---------|
| `uri` | JDBC URL |
| `kasanari.jdbc.user` | user |
| `kasanari.jdbc.password` | password |
| Key | Purpose |
|-------------------------------|------------|
| `uri` | JDBC URL |
| `kasanari.jdbc.user` | user |
| `kasanari.jdbc.password` | password |
| `kasanari.repository.backend` | `yugabyte` |

### Management
Expand All @@ -51,7 +51,7 @@ kasanari.authorization.casbin.repository.backend=yugabyte

### INTERNAL catalogs

Set the same properties inside each catalog’s `catalogProperties` when registering via the Management API. For Lance INTERNAL, Kasanari also injects `kasanari.catalog.key=<management catalog id>` so rows are isolated per catalog.
Set the same properties inside each catalog’s `catalogProperties` when registering via the Management API. For Lance INTERNAL, Kasanari also injects `kasanari.catalog.name=<management catalog id>` when unset.

## Database layout

Expand All @@ -61,14 +61,12 @@ Create the application database as **colocated** (small global tables):
CREATE DATABASE kasanari WITH COLOCATION = true;
```

| Plane | Distribution |
|-------|----------------|
| `kasanari_catalogs`, Casbin bindings | Colocated (`WITH (colocation = true)`) — list/refresh across catalogs |
| Iceberg / Paimon / Lance INTERNAL | Hash-sharded on `catalog_key`, `WITH (colocation = false)` — all rows for one catalog share a tablet |
| Plane | Distribution |
|--------------------------------------|-------------------------------------------------------------------------------------------------------|
| `kasanari_catalogs`, Casbin bindings | Colocated (`WITH (colocation = true)`) — list/refresh across catalogs |
| Iceberg / Paimon / Lance INTERNAL | Hash-sharded on `catalog_name`, `WITH (colocation = false)` — all rows for one catalog share a tablet |

Cross-catalog distributed transactions are not used. Catalog operations always target a single `catalog_key`. The only cross-catalog read is Casbin listing bindings for a role (colocated tables).
## Naming: `catalog_name`

## Naming: `catalog_key`

Yugabyte modules use a unified `catalog_key` column for Iceberg, Paimon, and Lance.
INTERNAL JDBC repositories use a unified `catalog_name` column for Iceberg, Paimon, and Lance.

1 change: 0 additions & 1 deletion docs/quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,6 @@ curl -sS -X POST "http://localhost:9090/management/v1/catalogs" \
"uri": "jdbc:postgresql://catalog-storage:5432/postgres",
"kasanari.jdbc.user": "postgres",
"kasanari.jdbc.password": "postgres",
"kasanari.catalog.key": catalog_id,
"s3.access-key": "admin",
"s3.secret-key": "password",
"s3.path.style.access": "true",
Expand Down
4 changes: 1 addition & 3 deletions examples/spark/notebooks/paimon-kasanari.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"output_type": "stream",
"text": [
"201\n",
"{\"catalogId\":\"paimon_spark_internal\",\"catalogType\":\"PAIMON\",\"mode\":\"INTERNAL\",\"spec\":{\"fileIoProperties\":{},\"catalogProperties\":{\"warehouse\":\"s3://warehouse\",\"uri\":\"jdbc:postgresql://catalog-storage:5432/postgres\",\"kasanari.jdbc.user\":\"postgres\",\"kasanari.jdbc.password\":\"postgres\",\"kasanari.catalog.key\":\"paimon_spark_internal\",\"s3.access-key\":\"admin\",\"s3.secret-key\":\"password\",\"s3.path.style.access\":\"true\",\"s3.endpoint\":\"http://minio:9000\",\"s3.ssl.enabled\":\"false\"}},\"version\":1}\n"
"{\"catalogId\":\"paimon_spark_internal\",\"catalogType\":\"PAIMON\",\"mode\":\"INTERNAL\",\"spec\":{\"fileIoProperties\":{},\"catalogProperties\":{\"warehouse\":\"s3://warehouse\",\"uri\":\"jdbc:postgresql://catalog-storage:5432/postgres\",\"kasanari.jdbc.user\":\"postgres\",\"kasanari.jdbc.password\":\"postgres\",\"s3.access-key\":\"admin\",\"s3.secret-key\":\"password\",\"s3.path.style.access\":\"true\",\"s3.endpoint\":\"http://minio:9000\",\"s3.ssl.enabled\":\"false\"}},\"version\":1}\n"
]
}
],
Expand All @@ -43,7 +43,6 @@
" \"uri\": \"jdbc:postgresql://catalog-storage:5432/postgres\",\n",
" \"kasanari.jdbc.user\": \"postgres\",\n",
" \"kasanari.jdbc.password\": \"postgres\",\n",
" \"kasanari.catalog.key\": catalog_id,\n",
" \"s3.access-key\": \"admin\",\n",
" \"s3.secret-key\": \"password\",\n",
" \"s3.path.style.access\": \"true\",\n",
Expand Down Expand Up @@ -80,7 +79,6 @@
" \"uri\": \"jdbc:postgresql://catalog-storage:5432/postgres\",\n",
" \"kasanari.jdbc.user\": \"postgres\",\n",
" \"kasanari.jdbc.password\": \"postgres\",\n",
" \"kasanari.catalog.key\": \"paimon_spark_internal\",\n",
" \"s3.access-key\": \"admin\",\n",
" \"s3.secret-key\": \"password\",\n",
" \"s3.path.style.access\": \"true\",\n",
Expand Down
7 changes: 5 additions & 2 deletions modules/catalog/catalog-lance/src/main/java/kasanari/catalog/lance/KasanariLanceCatalog.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,11 +91,14 @@ public void initialize(Map<String, String> properties, BufferAllocator allocator
this.allocator = allocator;
this.dataSource = new KasanariDataSource(properties);
this.transactionManager = new JdbcTransactionManager(dataSource);
var catalogKey = properties.getOrDefault(KasanariLanceProperties.CATALOG_KEY, "");
var catalogName = properties.get(KasanariLanceProperties.CATALOG_NAME);
if (catalogName == null || catalogName.isBlank()) {
throw new IllegalArgumentException("Required key `" + KasanariLanceProperties.CATALOG_NAME + "` is not set");
}
var bundle = BackendFactoryLoader.load(
LanceRepositoryBundleFactory.class,
dataSource.repositoryBackend()
).create(catalogKey, transactionManager);
).create(catalogName, transactionManager);
bundle.schemaInitializer().run();
this.namespaceRepository = bundle.namespaceRepository();
this.tableRepository = bundle.tableRepository();
Expand Down
3 changes: 1 addition & 2 deletions ...s/catalog/catalog-lance/src/main/java/kasanari/catalog/lance/KasanariLanceProperties.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,5 @@
public abstract class KasanariLanceProperties {
public static final String LOCATION = "lance.warehouse.location";
public static final String STORAGE_PROPERTIES_PREFIX = "lance.storage.";
/** Management catalog id used for Yugabyte row isolation / hash sharding. */
public static final String CATALOG_KEY = "kasanari.catalog.key";
public static final String CATALOG_NAME = "kasanari.catalog.name";
}
1 change: 1 addition & 0 deletions .../catalog/catalog-lance/src/test/java/kasanari/catalog/lance/KasanariLanceCatalogTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ protected LanceCatalogAdapter setupCatalogAdapter() throws Exception {
properties.put(KasanariDataSourceConfiguration.URI, postgres.jdbcUrl());
properties.put(KasanariDataSourceConfiguration.USER, postgres.username());
properties.put(KasanariDataSourceConfiguration.PASSWORD, postgres.password());
properties.put(KasanariLanceProperties.CATALOG_NAME, "kasanari-test");
properties.put(KasanariLanceProperties.LOCATION, "s3://warehouse");
properties.put(KasanariLanceProperties.STORAGE_PROPERTIES_PREFIX + "aws_region", "us-east-1");
properties.put(KasanariLanceProperties.STORAGE_PROPERTIES_PREFIX + "aws_access_key_id", s3Container.username());
Expand Down
8 changes: 4 additions & 4 deletions ...s/catalog/catalog-paimon/src/main/java/kasanari/catalog/paimon/KasanariCatalogLoader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,19 @@

public class KasanariCatalogLoader implements CatalogLoader {
private final FileIO fileIO;
private final String catalogKey;
private final String catalogName;
private final CatalogContext context;
private final String warehouse;

public KasanariCatalogLoader(FileIO fileIO, String catalogKey, CatalogContext context, String warehouse) {
public KasanariCatalogLoader(FileIO fileIO, String catalogName, CatalogContext context, String warehouse) {
this.fileIO = fileIO;
this.catalogKey = catalogKey;
this.catalogName = catalogName;
this.context = context;
this.warehouse = warehouse;
}

@Override
public Catalog load() {
return new KasanariPaimonCatalog(fileIO, catalogKey, context, warehouse);
return new KasanariPaimonCatalog(fileIO, catalogName, context, warehouse);
}
}
10 changes: 5 additions & 5 deletions ...s/catalog/catalog-paimon/src/main/java/kasanari/catalog/paimon/KasanariPaimonCatalog.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,10 @@ public class KasanariPaimonCatalog extends AbstractCatalog {
private final java.util.function.Function<Handle, CatalogLock> catalogLockFactory;

private final FileIO fileIO;
private final String catalogKey;
private final String catalogName;
private final String warehouse;

public KasanariPaimonCatalog(FileIO fileIO, String catalogKey, CatalogContext context, String warehouse) {
public KasanariPaimonCatalog(FileIO fileIO, String catalogName, CatalogContext context, String warehouse) {
super(fileIO, context);

var options = context.options().toMap();
Expand All @@ -110,7 +110,7 @@ public KasanariPaimonCatalog(FileIO fileIO, String catalogKey, CatalogContext co
var bundle = BackendFactoryLoader.load(
PaimonRepositoryBundleFactory.class,
dataSource.repositoryBackend()
).create(catalogKey, dataSource);
).create(catalogName, dataSource);
this.databaseRepository = bundle.databaseRepository();
this.tableRepository = bundle.tableRepository();
this.viewRepository = bundle.viewRepository();
Expand All @@ -122,7 +122,7 @@ public KasanariPaimonCatalog(FileIO fileIO, String catalogKey, CatalogContext co
bundle.schemaInitializer().run();

this.fileIO = fileIO;
this.catalogKey = catalogKey;
this.catalogName = catalogName;
this.warehouse = warehouse;
}

Expand Down Expand Up @@ -1287,7 +1287,7 @@ public PagedList<Snapshot> listSnapshotsPaged(Identifier identifier, @Nullable I

@Override
public CatalogLoader catalogLoader() {
return new KasanariCatalogLoader(fileIO, catalogKey, context, warehouse);
return new KasanariCatalogLoader(fileIO, catalogName, context, warehouse);
}

@Override
Expand Down
12 changes: 6 additions & 6 deletions ...og/catalog-paimon/src/main/java/kasanari/catalog/paimon/KasanariPaimonCatalogFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,12 @@
import java.util.Map;

public class KasanariPaimonCatalogFactory implements PaimonCatalogFactory {
private static final String CATALOG_KEY = "kasanari.catalog.key";
private static final String DEFAULT_CATALOG_KEY = "default";

@Override
public PaimonCatalogAdapter create(Map<String, String> fileIoProperties, Map<String, String> properties) {
public PaimonCatalogAdapter create(String name, Map<String, String> fileIoProperties, Map<String, String> properties) {
if (name == null || name.isBlank()) {
throw new IllegalArgumentException("catalog name is required for the INTERNAL Paimon catalog");
}

var configuration = new Configuration();
fileIoProperties.forEach(configuration::set);

Expand All @@ -30,12 +31,11 @@ public PaimonCatalogAdapter create(Map<String, String> fileIoProperties, Map<Str
}

var warehousePath = new Path(warehouse);
var catalogKey = properties.getOrDefault(CATALOG_KEY, DEFAULT_CATALOG_KEY);

try {
FileIO fileIO = FileIO.get(warehousePath, catalogContext);
fileIO.checkOrMkdirs(warehousePath);
var catalog = new KasanariPaimonCatalog(fileIO, catalogKey, catalogContext, warehouse);
var catalog = new KasanariPaimonCatalog(fileIO, name, catalogContext, warehouse);
return new DefaultPaimonCatalogAdapter(catalog);
} catch (IOException e) {
throw new UncheckedIOException(e);
Expand Down
2 changes: 1 addition & 1 deletion ...es/catalog/catalog-paimon/src/main/java/kasanari/catalog/paimon/PaimonCatalogFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
import java.util.Map;

public interface PaimonCatalogFactory {
PaimonCatalogAdapter create(Map<String, String> fileIoProperties, Map<String, String> properties);
PaimonCatalogAdapter create(String name, Map<String, String> fileIoProperties, Map<String, String> properties);
}
2 changes: 1 addition & 1 deletion ...talog/catalog-paimon/src/main/java/kasanari/catalog/paimon/ProxyPaimonCatalogFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

public class ProxyPaimonCatalogFactory implements PaimonCatalogFactory {
@Override
public PaimonCatalogAdapter create(Map<String, String> fileIoProperties, Map<String, String> properties) {
public PaimonCatalogAdapter create(String name, Map<String, String> fileIoProperties, Map<String, String> properties) {
var config = new Configuration();
fileIoProperties.forEach(config::set);

Expand Down
2 changes: 1 addition & 1 deletion ...log/catalog-paimon/src/test/java/kasanari/catalog/paimon/FileSystemPaimonCatalogTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ protected PaimonCatalogAdapter setupCatalogAdapter() {
options.put("warehouse", "s3a://warehouse");

var factory = new ProxyPaimonCatalogFactory();
return factory.create(config, options);
return factory.create("test-catalog", config, options);
}

@Override
Expand Down
2 changes: 1 addition & 1 deletion ...s/catalog/catalog-paimon/src/test/java/kasanari/catalog/paimon/HivePaimonCatalogTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ protected PaimonCatalogAdapter setupCatalogAdapter() {
options.put("uri", hive.thriftUri());

var factory = new ProxyPaimonCatalogFactory();
return factory.create(config, options);
return factory.create("test-catalog", config, options);
}

@Override
Expand Down
2 changes: 1 addition & 1 deletion ...s/catalog/catalog-paimon/src/test/java/kasanari/catalog/paimon/JdbcPaimonCatalogTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ protected PaimonCatalogAdapter setupCatalogAdapter() {
options.put("jdbc-table-prefix", "paimon_");

var factory = new ProxyPaimonCatalogFactory();
return factory.create(config, options);
return factory.create("test-catalog", config, options);
}

@Override
Expand Down
Loading
Loading