Skip to content

Commit 7321d0d

Browse files
linghengqianlinghengqian
authored
Add doc for Presto optional plugin (#35505)
1 parent a9ae3b8 commit 7321d0d

File tree

5 files changed

+368
-89
lines changed

5 files changed

+368
-89
lines changed

‎docs/document/content/user-manual/shardingsphere-jdbc/optional-plugins/presto/_index.cn.md

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
+++
2+
title = "Presto"
3+
weight = 6
4+
+++
5+
6+
## 背景信息
7+
8+
ShardingSphere 默认情况下不提供对 `com.facebook.presto.jdbc.PrestoDriver` 的 `driverClassName` 的支持。
9+
ShardingSphere 对 Presto JDBC Driver 的支持位于可选模块中。
10+
11+
## 前提条件
12+
13+
要在 ShardingSphere 的配置文件为数据节点使用类似 `jdbc:presto://localhost:8080/iceberg/demo_ds_0` 的 `jdbcUrl`，
14+
可能的 Maven 依赖关系如下，
15+
16+
```xml
17+
<dependencies>
18+
<dependency>
19+
<groupId>org.apache.shardingsphere</groupId>
20+
<artifactId>shardingsphere-jdbc</artifactId>
21+
<version>${shardingsphere.version}</version>
22+
</dependency>
23+
<dependency>
24+
<groupId>org.apache.shardingsphere</groupId>
25+
<artifactId>shardingsphere-parser-sql-presto</artifactId>
26+
<version>${shardingsphere.version}</version>
27+
</dependency>
28+
<dependency>
29+
<groupId>com.facebook.presto</groupId>
30+
<artifactId>presto-jdbc</artifactId>
31+
<version>0.292</version>
32+
</dependency>
33+
</dependencies>
34+
```
35+
36+
## 配置示例
37+
38+
### 启动 Presto
39+
40+
编写 Docker Compose 文件来启动 Presto。这将启动一个既为协调器又为工作节点的 Presto 节点，并为该节点配置 Iceberg 连接器。
41+
此外，此 Iceberg 连接器将使用本地文件系统目录启动 Hive Metastore Server。
42+
43+
```yaml
44+
services:
45+
presto:
46+
image: prestodb/presto:0.292
47+
ports:
48+
- "8080:8080"
49+
volumes:
50+
- ./iceberg.properties:/opt/presto-server/etc/catalog/iceberg.properties
51+
```
52+
53+
同级文件夹包含文件 `iceberg.properties`，内容如下，
54+
55+
```properties
56+
connector.name=iceberg
57+
iceberg.catalog.type=hive
58+
hive.metastore=file
59+
hive.metastore.catalog.dir=file:/home/iceberg_data
60+
```
61+
62+
### 创建业务相关的 schema，库和表
63+
64+
通过第三方工具在 Presto 内创建业务相关的 schema，库和表。
65+
以 DBeaver Community 为例，若使用 Ubuntu 22.04.5，可通过 Snapcraft 快速安装，
66+
67+
```shell
68+
sudo apt update && sudo apt upgrade -y
69+
sudo snap install dbeaver-ce
70+
snap run dbeaver-ce
71+
```
72+
73+
在 DBeaver Community 内，使用 `jdbc:presto://localhost:8080/iceberg` 的 `jdbcUrl`，`test` 的`username` 连接至 Presto，
74+
`password` 留空。
75+
执行如下 SQL，
76+
77+
```sql
78+
-- noinspection SqlNoDataSourceInspectionForFile
79+
CREATE SCHEMA iceberg.demo_ds_0;
80+
CREATE SCHEMA iceberg.demo_ds_1;
81+
CREATE SCHEMA iceberg.demo_ds_2;
82+
```
83+
84+
分别使用 `jdbc:presto://localhost:8080/iceberg/demo_ds_0` ，
85+
`jdbc:presto://localhost:8080/iceberg/demo_ds_1` 和 `jdbc:presto://localhost:8080/iceberg/demo_ds_2` 的 `jdbcUrl` 连接至 Presto 来执行如下 SQL，
86+
87+
```sql
88+
-- noinspection SqlNoDataSourceInspectionForFile
89+
CREATE TABLE IF NOT EXISTS t_order (
90+
order_id BIGINT NOT NULL,
91+
order_type INTEGER,
92+
user_id INTEGER NOT NULL,
93+
address_id BIGINT NOT NULL,
94+
status VARCHAR(50)
95+
);
96+
truncate table t_order;
97+
```
98+
99+
### 在业务项目创建 ShardingSphere 数据源
100+
101+
在业务项目引入`前提条件`涉及的依赖后，在业务项目的 classpath 上编写 ShardingSphere 数据源的配置文件`demo.yaml`，
102+
103+
```yaml
104+
dataSources:
105+
ds_0:
106+
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
107+
driverClassName: com.facebook.presto.jdbc.PrestoDriver
108+
jdbcUrl: jdbc:presto://localhost:8080/iceberg/demo_ds_0
109+
username: test
110+
ds_1:
111+
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
112+
driverClassName: com.facebook.presto.jdbc.PrestoDriver
113+
jdbcUrl: jdbc:presto://localhost:8080/iceberg/demo_ds_1
114+
username: test
115+
ds_2:
116+
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
117+
driverClassName: com.facebook.presto.jdbc.PrestoDriver
118+
jdbcUrl: jdbc:presto://localhost:8080/iceberg/demo_ds_2
119+
username: test
120+
rules:
121+
- !SHARDING
122+
tables:
123+
t_order:
124+
actualDataNodes: <LITERAL>ds_0.t_order, ds_1.t_order, ds_2.t_order
125+
keyGenerateStrategy:
126+
column: order_id
127+
keyGeneratorName: snowflake
128+
defaultDatabaseStrategy:
129+
standard:
130+
shardingColumn: user_id
131+
shardingAlgorithmName: inline
132+
shardingAlgorithms:
133+
inline:
134+
type: INLINE
135+
props:
136+
algorithm-expression: ds_${user_id % 2}
137+
keyGenerators:
138+
snowflake:
139+
type: SNOWFLAKE
140+
```
141+
142+
### 享受集成
143+
144+
创建 ShardingSphere 的数据源以享受集成，
145+
146+
```java
147+
import com.zaxxer.hikari.HikariConfig;
148+
import com.zaxxer.hikari.HikariDataSource;
149+
import java.sql.Connection;
150+
import java.sql.SQLException;
151+
import java.sql.Statement;
152+
public class ExampleUtils {
153+
void test() throws SQLException {
154+
HikariConfig config = new HikariConfig();
155+
config.setJdbcUrl("jdbc:shardingsphere:classpath:demo.yaml");
156+
config.setDriverClassName("org.apache.shardingsphere.driver.ShardingSphereDriver");
157+
try (HikariDataSource dataSource = new HikariDataSource(config);
158+
Connection connection = dataSource.getConnection();
159+
Statement statement = connection.createStatement()) {
160+
statement.execute("INSERT INTO t_order (user_id, order_type, address_id, status) VALUES (1, 1, 1, 'INSERT_TEST')");
161+
statement.executeQuery("SELECT * FROM t_order");
162+
statement.execute("DELETE FROM t_order WHERE user_id=1");
163+
statement.execute("DROP TABLE IF EXISTS t_order");
164+
}
165+
}
166+
}
167+
```
168+
169+
## 使用限制
170+
171+
### SQL 限制
172+
173+
ShardingSphere JDBC DataSource 尚不支持执行 Presto 的 `create table` 和 `truncate table` 语句。
174+
175+
### 事务限制
176+
177+
Presto 不支持 ShardingSphere 集成级别的本地事务，XA 事务或 Seata 的 AT 模式事务。
178+
对于 Presto 在 ShardingSphere 集成级别的本地事务，在 ShardingSphere 一侧存在已知问题。
179+
180+
### 连接器限制
181+
182+
受 https://github.com/prestodb/presto/issues/23226 影响，Presto Memory 连接器的健康检查存在已知问题，
183+
不应在 ShardingSphere 的配置文件内连接至 Presto Memory 连接器。

‎docs/document/content/user-manual/shardingsphere-jdbc/optional-plugins/presto/_index.en.md

Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,185 @@
1+
+++
2+
title = "Presto"
3+
weight = 6
4+
+++
5+
6+
## Background Information
7+
8+
ShardingSphere does not provide support for `driverClassName` of `com.facebook.presto.jdbc.PrestoDriver` by default.
9+
ShardingSphere's support for Presto JDBC Driver is in an optional module.
10+
11+
## Prerequisites
12+
13+
To use a `jdbcUrl` like `jdbc:presto://localhost:8080/iceberg/demo_ds_0` for the data node in the ShardingSphere configuration file,
14+
Possible Maven dependencies are as follows,
15+
16+
```xml
17+
<dependencies>
18+
<dependency>
19+
<groupId>org.apache.shardingsphere</groupId>
20+
<artifactId>shardingsphere-jdbc</artifactId>
21+
<version>${shardingsphere.version}</version>
22+
</dependency>
23+
<dependency>
24+
<groupId>org.apache.shardingsphere</groupId>
25+
<artifactId>shardingsphere-parser-sql-presto</artifactId>
26+
<version>${shardingsphere.version}</version>
27+
</dependency>
28+
<dependency>
29+
<groupId>com.facebook.presto</groupId>
30+
<artifactId>presto-jdbc</artifactId>
31+
<version>0.292</version>
32+
</dependency>
33+
</dependencies>
34+
```
35+
36+
## Configuration Example
37+
38+
### Start Presto
39+
40+
Write a Docker Compose file to start Presto.
41+
This will start a Presto node that is both a coordinator and a worker node, and configure the Iceberg Connector for the node.
42+
In addition, this Iceberg Connector will start a Hive Metastore Server using a local file system directory.
43+
44+
```yaml
45+
services:
46+
presto:
47+
image: prestodb/presto:0.292
48+
ports:
49+
- "8080:8080"
50+
volumes:
51+
- ./iceberg.properties:/opt/presto-server/etc/catalog/iceberg.properties
52+
```
53+
54+
The same folder contains the file `iceberg.properties`, the contents are as follows,
55+
56+
```properties
57+
connector.name=iceberg
58+
iceberg.catalog.type=hive
59+
hive.metastore=file
60+
hive.metastore.catalog.dir=file:/home/iceberg_data
61+
```
62+
63+
### Create business-related schemas, databases, and tables
64+
65+
Use third-party tools to create business-related schemas, databases, and tables in Presto.
66+
Taking DBeaver Community as an example, if you use Ubuntu 24.04, you can quickly install it through Snapcraft.
67+
68+
```shell
69+
sudo apt update && sudo apt upgrade -y
70+
sudo snap install dbeaver-ce
71+
snap run dbeaver-ce
72+
```
73+
74+
In DBeaver Community, use `jdbcUrl` of `jdbc:presto://localhost:8080/iceberg`, `username` of `test` to connect to Presto, and leave `password` blank.
75+
Execute the following SQL,
76+
77+
```sql
78+
-- noinspection SqlNoDataSourceInspectionForFile
79+
CREATE SCHEMA iceberg.demo_ds_0;
80+
CREATE SCHEMA iceberg.demo_ds_1;
81+
CREATE SCHEMA iceberg.demo_ds_2;
82+
```
83+
84+
Use the `jdbcUrl` of `jdbc:presto://localhost:8080/iceberg/demo_ds_0`,
85+
`jdbc:presto://localhost:8080/iceberg/demo_ds_1` and `jdbc:presto://localhost:8080/iceberg/demo_ds_2` to connect to Presto and execute the following SQL,
86+
87+
```sql
88+
-- noinspection SqlNoDataSourceInspectionForFile
89+
CREATE TABLE IF NOT EXISTS t_order (
90+
order_id BIGINT NOT NULL,
91+
order_type INTEGER,
92+
user_id INTEGER NOT NULL,
93+
address_id BIGINT NOT NULL,
94+
status VARCHAR(50)
95+
);
96+
truncate table t_order;
97+
```
98+
99+
### Create ShardingSphere data source in business project
100+
101+
After the business project introduces the dependencies involved in `Prerequisites`,
102+
write the ShardingSphere data source configuration file `demo.yaml` on the classpath of the business project.
103+
104+
```yaml
105+
dataSources:
106+
ds_0:
107+
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
108+
driverClassName: com.facebook.presto.jdbc.PrestoDriver
109+
jdbcUrl: jdbc:presto://localhost:8080/iceberg/demo_ds_0
110+
username: test
111+
ds_1:
112+
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
113+
driverClassName: com.facebook.presto.jdbc.PrestoDriver
114+
jdbcUrl: jdbc:presto://localhost:8080/iceberg/demo_ds_1
115+
username: test
116+
ds_2:
117+
dataSourceClassName: com.zaxxer.hikari.HikariDataSource
118+
driverClassName: com.facebook.presto.jdbc.PrestoDriver
119+
jdbcUrl: jdbc:presto://localhost:8080/iceberg/demo_ds_2
120+
username: test
121+
rules:
122+
- !SHARDING
123+
tables:
124+
t_order:
125+
actualDataNodes: <LITERAL>ds_0.t_order, ds_1.t_order, ds_2.t_order
126+
keyGenerateStrategy:
127+
column: order_id
128+
keyGeneratorName: snowflake
129+
defaultDatabaseStrategy:
130+
standard:
131+
shardingColumn: user_id
132+
shardingAlgorithmName: inline
133+
shardingAlgorithms:
134+
inline:
135+
type: INLINE
136+
props:
137+
algorithm-expression: ds_${user_id % 2}
138+
keyGenerators:
139+
snowflake:
140+
type: SNOWFLAKE
141+
```
142+
143+
### Enjoy integration
144+
145+
Create a ShardingSphere data source to enjoy integration,
146+
147+
```java
148+
import com.zaxxer.hikari.HikariConfig;
149+
import com.zaxxer.hikari.HikariDataSource;
150+
import java.sql.Connection;
151+
import java.sql.SQLException;
152+
import java.sql.Statement;
153+
public class ExampleUtils {
154+
void test() throws SQLException {
155+
HikariConfig config = new HikariConfig();
156+
config.setJdbcUrl("jdbc:shardingsphere:classpath:demo.yaml");
157+
config.setDriverClassName("org.apache.shardingsphere.driver.ShardingSphereDriver");
158+
try (HikariDataSource dataSource = new HikariDataSource(config);
159+
Connection connection = dataSource.getConnection();
160+
Statement statement = connection.createStatement()) {
161+
statement.execute("INSERT INTO t_order (user_id, order_type, address_id, status) VALUES (1, 1, 1, 'INSERT_TEST')");
162+
statement.executeQuery("SELECT * FROM t_order");
163+
statement.execute("DELETE FROM t_order WHERE user_id=1");
164+
statement.execute("DROP TABLE IF EXISTS t_order");
165+
}
166+
}
167+
}
168+
```
169+
170+
## Usage Limitations
171+
172+
### SQL Limitations
173+
174+
ShardingSphere JDBC DataSource does not yet support the execution of Presto's `create table` and `truncate table` statements.
175+
176+
### Transaction Limitations
177+
178+
Presto does not support local transactions, XA transactions, or Seata's AT mode transactions at the ShardingSphere integration level.
179+
180+
There are known issues on the ShardingSphere side for Presto's local transactions at the ShardingSphere integration level.
181+
182+
### Connector Limitations
183+
184+
Affected by https://github.com/prestodb/presto/issues/23226 , there are known issues with the health check of the Presto Memory connector,
185+
developers should not connect to the Presto Memory connector in the ShardingSphere configuration file.

‎infra/database/type/presto/src/main/java/org/apache/shardingsphere/infra/database/presto/metadata/database/PrestoDatabaseMetaData.java

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,8 @@
2121
import org.apache.shardingsphere.infra.database.core.metadata.database.enums.QuoteCharacter;
2222
import org.apache.shardingsphere.infra.database.core.metadata.database.metadata.DialectDatabaseMetaData;
2323
import org.apache.shardingsphere.infra.database.core.metadata.database.metadata.option.IdentifierPatternType;
24-
import org.apache.shardingsphere.infra.database.core.metadata.database.metadata.option.datatype.DialectDataTypeOption;
2524
import org.apache.shardingsphere.infra.database.core.metadata.database.metadata.option.schema.DefaultSchemaOption;
2625
import org.apache.shardingsphere.infra.database.core.metadata.database.metadata.option.schema.DialectSchemaOption;
27-
import org.apache.shardingsphere.infra.database.presto.metadata.database.option.PrestoDataTypeOption;
2826

2927
/**
3028
* Database meta data of Presto.
@@ -51,11 +49,6 @@ public DialectSchemaOption getSchemaOption() {
5149
return new DefaultSchemaOption(false, "default");
5250
}
5351

54-
@Override
55-
public DialectDataTypeOption getDataTypeOption() {
56-
return new PrestoDataTypeOption();
57-
}
58-
5952
@Override
6053
public String getDatabaseType() {
6154
return "Presto";

0 commit comments

Comments
 (0)