Skip to content

Commit b235303

Browse files
authored
[docs] Add document for LIST/ADD/DROP PARTITION (#517)
1 parent 6c38e4e commit b235303

File tree

4 files changed

+94
-7
lines changed

4 files changed

+94
-7
lines changed

fluss-connectors/fluss-connector-flink/src/main/java/com/alibaba/fluss/connector/flink/catalog/FlinkCatalog.java

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -427,6 +427,8 @@ public List<CatalogPartitionSpec> listPartitions(
427427
ObjectPath objectPath, CatalogPartitionSpec catalogPartitionSpec)
428428
throws TableNotExistException, TableNotPartitionedException,
429429
PartitionSpecInvalidException, CatalogException {
430+
// TODO, list partitions by catalogPartitionSpec. Trace by
431+
// https://github.com/alibaba/fluss/issues/514
430432
throw new UnsupportedOperationException();
431433
}
432434

fluss-connectors/fluss-connector-flink/src/test/java/com/alibaba/fluss/connector/flink/catalog/FlinkCatalogITCase.java

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,15 @@ void testPartitionedTable() throws Exception {
213213
expectedShowPartitionsResult = Arrays.asList("+I[b=2]", "+I[b=3]");
214214
showPartitionIterator = tEnv.executeSql("show partitions test_partitioned_table").collect();
215215
assertResultsIgnoreOrder(showPartitionIterator, expectedShowPartitionsResult, true);
216+
217+
// 4. show partitions with spec.
218+
assertThatThrownBy(
219+
() ->
220+
tEnv.executeSql(
221+
"show partitions test_partitioned_table partition (b=2)")
222+
.collect())
223+
.rootCause()
224+
.isInstanceOf(UnsupportedOperationException.class);
216225
}
217226

218227
@Test

website/docs/engine-flink/ddl.md

Lines changed: 79 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -88,13 +88,13 @@ CREATE TABLE my_log_table (
8888

8989
### Partitioned (PrimaryKey/Log) Table
9090

91-
The following SQL statement creates a Partitioned PrimaryKey Table in Fluss. Note that the partitioned field (`dt` in this case) must be a subset of the primary key (`dt, shop_id, user_id` in this case).
92-
Currently, Fluss only supports one partitioned field with `STRING` type.
93-
9491
:::note
95-
Currently, partitioned table must enable auto partition and set auto partition time unit.
92+
1. Currently, Fluss only supports one partitioned field with `STRING` type
93+
2. For the Partitioned PrimaryKey Table, the partitioned field (`dt` in this case) must be a subset of the primary key (`dt, shop_id, user_id` in this case)
9694
:::
9795

96+
The following SQL statement creates a Partitioned PrimaryKey Table in Fluss.
97+
9898
```sql title="Flink SQL"
9999
CREATE TABLE my_part_pk_table (
100100
dt STRING,
@@ -103,17 +103,48 @@ CREATE TABLE my_part_pk_table (
103103
num_orders INT,
104104
total_amount INT,
105105
PRIMARY KEY (dt, shop_id, user_id) NOT ENFORCED
106+
) PARTITIONED BY (dt);
107+
```
108+
109+
The following SQL statement creates a Partitioned Log Table in Fluss.
110+
111+
```sql title="Flink SQL"
112+
CREATE TABLE my_part_log_table (
113+
order_id BIGINT,
114+
item_id BIGINT,
115+
amount INT,
116+
address STRING,
117+
dt STRING
118+
) PARTITIONED BY (dt);
119+
```
120+
:::note
121+
After the Partitioned (PrimaryKey/Log) Table is created, you need first manually create the corresponding partition using the [Add Partition](/docs/engine-flink/ddl.md#add-partition) statement
122+
before you write/read data into this partition.
123+
:::
124+
125+
#### Auto partitioned (PrimaryKey/Log) table
126+
127+
Fluss also support creat Auto Partitioned (PrimaryKey/Log) Table. The following SQL statement creates an Auto Partitioned PrimaryKey Table in Fluss.
128+
129+
```sql title="Flink SQL"
130+
CREATE TABLE my_auto_part_pk_table (
131+
dt STRING,
132+
shop_id BIGINT,
133+
user_id BIGINT,
134+
num_orders INT,
135+
total_amount INT,
136+
PRIMARY KEY (dt, shop_id, user_id) NOT ENFORCED
106137
) PARTITIONED BY (dt) WITH (
107138
'bucket.num' = '4',
108139
'table.auto-partition.enabled' = 'true',
109140
'table.auto-partition.time-unit' = 'day'
110141
);
111142
```
112143

113-
The following SQL statement creates a Partitioned Log Table in Fluss.
144+
The following SQL statement creates an Auto Partitioned Log Table in Fluss.
114145

115146
```sql title="Flink SQL"
116-
CREATE TABLE my_part_log_table (
147+
CREATE TABLE my_auto_part_log_table (
117148
order_id BIGINT,
118149
item_id BIGINT,
119150
amount INT,
@@ -126,6 +157,9 @@ CREATE TABLE my_part_log_table (
126157
);
127158
```
128159

160+
For more details about Auto Partitioned (PrimaryKey/Log) Table, refer to [Auto Partitioning Options](/docs/table-design/data-distribution/partitioning/#auto-partitioning-options).
161+
162+
### Options
129163

130164
The supported option in "with" parameters when creating a table are as follows:
131165

@@ -172,4 +206,43 @@ DROP TABLE my_table;
172206

173207
This will entirely remove all the data of the table in the Fluss cluster.
174208

209+
## Show Partitions
210+
211+
To show all the partitions of a partitioned table, run:
212+
```sql title="Flink SQL"
213+
SHOW PARTITIONS my_part_pk_table;
214+
```
215+
216+
For more details, refer to the [Flink SHOW PARTITIONS](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/table/sql/show/#show-partitions) documentation.
217+
218+
:::note
219+
Currently, we only support show all partitions of a partitioned table, but not support show partitions with the given partition spec.
220+
:::
221+
222+
## Add Partition
223+
224+
Fluss support manually add partitions to an exists partitioned table by Fluss Catalog. If the specified partition
225+
not exists, Fluss will create the partition. If the specified partition already exists, Fluss will ignore the request
226+
or throw an exception.
227+
228+
To add partitions, run:
229+
```sql title="Flink SQL"
230+
ALTER TABLE my_part_pk_table ADD PARTITION (dt = '2025-03-05');
231+
```
232+
233+
For more details, refer to the [Flink ALTER TABLE(ADD)](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/table/sql/alter/#add) documentation.
234+
235+
## Drop Partition
236+
237+
Fluss also support manually drop partitions from an exists partitioned table by Fluss Catalog. If the specified partition
238+
not exists, Fluss will ignore the request or throw an exception.
239+
240+
241+
To drop partitions, run:
242+
```sql title="Flink SQL"
243+
ALTER TABLE my_part_pk_table DROP PARTITION (dt = '2025-03-05');
244+
```
245+
246+
For more details, refer to the [Flink ALTER TABLE(DROP)](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/table/sql/alter/#drop) documentation.
247+
175248

website/docs/table-design/data-distribution/partitioning.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,10 @@ sidebar_position: 2
77
## Partitioned Tables
88
In Fluss, a **Partitioned Table** organizes data based on one or more partition keys, providing a way to improve query performance and manageability for large datasets. Partitions allow the system to divide data into distinct segments, each corresponding to specific values of the partition keys.
99

10-
For partitioned tables, Fluss supports auto partitioning creation. Partitions can be automatically created based on the auto partitioning rules configured at the time of table creation, and expired partitions are automatically removed, ensuring data not expanding unlimited.
10+
For partitioned tables, Fluss not only supports manage partitions by users, like create/drop partitions, but also supports automatic manage partitions.
11+
- For manually managing partitions, user can create new partitions or drop exists partitions. Learn how to create or drop partitions please refer to [Add Partition](/docs/engine-flink/ddl.md#add-partition) and [Drop Partition](/docs/engine-flink/ddl.md#drop-partition).
12+
- For automatically managing partitions, the partitions will be created based on the auto partitioning rules configured at the time of table creation, and expired partitions are automatically removed, ensuring data not expanding unlimited. See [Auto Partitioning Options](/docs/table-design/data-distribution/partitioning.md#auto-partitioning-options).
13+
- Manual management and automated management are orthogonal and can coexist on the same table
1114

1215
### Key Benefits of Partitioned Tables
1316
- **Improved Query Performance:** By narrowing down the query scope to specific partitions, the system reads fewer data, reducing query execution time.

0 commit comments

Comments
 (0)