Skip to content

Commit 5687cb5

Browse files
MeggielqkMeggielqk
authored
Merge pull request #2685 from emqx/20241010-update-mssql-doc
doc: update ms sql server doc
2 parents 8f39ec5 + 67a2554 commit 5687cb5

File tree

2 files changed

+116
-147
lines changed

2 files changed

+116
-147
lines changed

‎en_US/data-integration/data-bridge-sqlserver.md‎

Lines changed: 54 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -55,11 +55,11 @@ This section describes how to start Microsoft SQL Server 2019 on Linux/MacOS usi
5555

5656
1. Install Microsoft SQL Server via Docker, and then start the docker image with the command below. Use `mqtt_public1` as the password. For the password policy of Microsoft SQL Server, see [Password Complexity](https://learn.microsoft.com/en-us/sql/relational-databases/security/password-policy?view=sql-server-ver16#password-complexity).
5757

58-
Note: By starting a Docker container with the environment variable `ACCEPT_EULA=Y` you agree to the terms of Microsoft EULA, see also [MICROSOFT SOFTWARE LICENSE TERMS MICROSOFT SQL SERVER 2019 STANDARD(EN_US)](https://www.microsoft.com/en-us/Useterms/Retail/SQLServerStandard/2019/Useterms_Retail_SQLServerStandard_2019_English.htm).
58+
Note: By starting a Docker container with the environment variable `ACCEPT_EULA=Y` you agree to the terms of Microsoft EULA, see also [End-User Licensing Agreement](https://go.microsoft.com/fwlink/?linkid=857698).
5959

6060
```bash
6161
# To start the Microsoft SQL Server docker image and set the password as `mqtt_public1`
62-
$ docker run --name sqlserver -p 1433:1433 -e ACCEPT_EULA=Y -e MSSQL_SA_PASSWORD=mqtt_public1 -d mcr.microsoft.com/mssql/server:2019-CU19-ubuntu-20.04
62+
$ docker run --name sqlserver -p 1433:1433 -e ACCEPT_EULA=Y -e MSSQL_SA_PASSWORD=mqtt_public1 -d mcr.microsoft.com/mssql/server:2022-CU15-ubuntu-22.04
6363
```
6464

6565
2. Access the container.
@@ -71,116 +71,100 @@ This section describes how to start Microsoft SQL Server 2019 on Linux/MacOS usi
7171
3. Enter the preset password to connect to the server in the container. The characters are not echoed when entering the password. Click `Enter` directly after entering the password.
7272

7373
```bash
74-
$ /opt/mssql-tools/bin/sqlcmd -S 127.0.0.1 -U sa
75-
$ Password:
74+
$ /opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P mqtt_public1 -N -C
7675
1>
7776
```
7877

7978
::: tip
8079

81-
The `mssql-tools` have been installed in the Microsoft SQL Server container provided by Microsoft, but the executable file is not in `$PATH`. Therefore, you need to specify the executable file path for `mssql-tools` before proceeding. As for the Docker deployment in this example, the file path should be `opt`.
80+
The `mssql-tools18` package have been installed in the Microsoft SQL Server container provided by Microsoft, but the executable file is not in `$PATH`. Therefore, you need to specify the executable file path for `sqlcmd` before proceeding. As for the Docker deployment in this example, the file path should be `/opt`.
8281

83-
For more information on how to use `mssql-tools`, see [sqlcmd-utility](https://learn.microsoft.com/en-us/sql/tools/sqlcmd/sqlcmd-utility?view=sql-server-ver16).
82+
For more information on how to use `mssql-tools18`, see [sqlcmd-utility](https://learn.microsoft.com/en-us/sql/tools/sqlcmd/sqlcmd-utility?view=sql-server-ver16).
8483

8584
:::
8685

87-
So far, the Microsoft SQL Server 2019 instance has been deployed and can be connected.
86+
So far, the Microsoft SQL Server 2022 instance has been deployed and can be connected.
8887

8988
### Create Database and Data Tables
9089

91-
This section describes how to create a database and data table in Microsoft SQL Server.
90+
Use the connection created from the previous section and the following SQL statements to create data tables.
9291

93-
1. Create a database `mqtt` in Microsoft SQL Server using the connection created from the previous section.
92+
- Create the following data table for storing the MQTT message, including the message ID, topic, QoS, payload, and publish time of each message.
9493

95-
```bash
96-
...
97-
Password:
98-
1> USE master
99-
2> GO
100-
Changed database context to 'master'.
101-
1> IF NOT EXISTS(SELECT name FROM sys.databases WHERE name = 'mqtt') BEGIN CREATE DATABASE mqtt END
102-
2> GO
103-
```
104-
105-
2. Use the following SQL statements to create a data table.
106-
107-
- Create the following data table for storing the MQTT message, including the message ID, topic, QoS, payload, and publish time of each message.
94+
```sql
95+
CREATE TABLE dbo.t_mqtt_msg (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL,
96+
msgid VARCHAR(64) NULL,
97+
topic VARCHAR(100) NULL,
98+
qos tinyint NOT NULL DEFAULT 0,
99+
payload VARCHAR(100) NULL,
100+
arrived DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP);
101+
GO
102+
```
108103

109-
```sql
110-
CREATE TABLE mqtt.dbo.t_mqtt_msg (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL,
111-
msgid VARCHAR(64) NULL,
112-
topic VARCHAR(100) NULL,
113-
qos tinyint NOT NULL DEFAULT 0,
114-
payload VARCHAR(100) NULL,
115-
arrived DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP);
116-
GO
117-
```
104+
- Create the following data table for recording the online/offline status of clients.
118105

119-
- Create the following data table for recording the online/offline status of clients.
120-
121-
```sql
122-
CREATE TABLE mqtt.dbo.t_mqtt_events (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL,
123-
clientid VARCHAR(255) NULL,
124-
event_type VARCHAR(255) NULL,
125-
event_time DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP);
126-
GO
127-
```
106+
```sql
107+
CREATE TABLE dbo.t_mqtt_events (id int PRIMARY KEY IDENTITY(1000000001,1) NOT NULL,
108+
clientid VARCHAR(255) NULL,
109+
event_type VARCHAR(255) NULL,
110+
event_time DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP);
111+
GO
112+
```
128113

129114
### Install and Configure ODBC Driver
130115

131-
You need to configure the ODBC driver to be able to access the Microsoft SQL Server database. You can use either FreeTDS or the msodbcsql17 driver provided by Microsoft as the ODBC driver (The connection properties for msodbcsql18 have not been adapted yet).
116+
You need to configure the ODBC driver to be able to access the Microsoft SQL Server database. You can use either FreeTDS or the msodbcsql18 driver provided by Microsoft as the ODBC driver.
132117

133118
EMQX uses the DSN Name specified in the `odbcinst.ini` configuration to determine the path to the driver dynamic library. In the examples below, the DSN Name is `ms-sql`. For more information, refer to [Connection Properties](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/connection-string-keywords-and-data-source-names-dsns?view=sql-server-ver16#connection-properties).
134119

135-
::: tip Note:
120+
::: tip Note
136121

137122
You can choose your own DSN name according to your preference, but it is recommended to use only English letters. Additionally, the DSN Name is case-sensitive.
138123

139124
:::
140125

141-
#### Install and Configure msodbcsql17 Driver as ODBC Driver
126+
#### Install and Configure msodbcsql18 Driver as ODBC Driver
142127

143128
<!-- TODO: update tag version in command and dockerfile -->
144129

145-
If you need to use msodbcsql17 driver as the ODBC driver, refer to the Microsoft instructions:
130+
If you need to use msodbcsql18 driver as the ODBC driver, refer to the Microsoft instructions:
146131

147132
- [Install the Microsoft ODBC driver for SQL Server (Linux)](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver16&tabs=alpine18-install%2Calpine17-install%2Cdebian8-install%2Credhat7-13-install%2Crhel7-offline)
148133
- [Install the Microsoft ODBC driver for SQL Server (macOS)](https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/install-microsoft-odbc-driver-sql-server-macos?view=sql-server-ver16)
149134

150-
Restricted by [Microsoft EULA terms](https://www.microsoft.com/en-us/Useterms/Retail/SQLServerStandard/2019/Useterms_Retail_SQLServerStandard_2019_English.htm), the Docker image provided by EMQX does not include the msodbcsql17 driver. To use it in Docker or Kubernetes, you need to create a new image installed with ODBC driver based on the image provided by [EMQX-Enterprise](https://hub.docker.com/r/emqx/emqx-enterprise) to access the Microsoft SQL Server database. Using the new image means that you agree to the [Microsoft SQL Server EULA](https://www.microsoft.com/en-us/Useterms/Retail/SQLServerStandard/2019/Useterms_Retail_SQLServerStandard_2019_English.htm).
135+
Restricted by [Microsoft EULA terms](https://go.microsoft.com/fwlink/?linkid=857698), the Docker image provided by EMQX does not include the msodbcsql18 driver. To use it in Docker or Kubernetes, you need to create a new image installed with ODBC driver based on the image provided by [EMQX Enterprise](https://hub.docker.com/r/emqx/emqx-enterprise) to access the Microsoft SQL Server database. Using the new image means that you agree to the [Microsoft SQL Server EULA](https://go.microsoft.com/fwlink/?linkid=857698).
151136

152137
Follow the instructions below to build a new image:
153138

154-
1. Get the corresponding EMQX [Dockerfile](https://github.com/emqx/emqx/blob/master/deploy/docker/Dockerfile.msodbc). You can save the file locally.
139+
1. Use the following Dockerfile to build a new image.
155140

156-
The image version in this example is `emqx/emqx-enterprise:5.0.3-alpha.2`. You can build the image based on the EMQX-Enterprise version you need, or use the latest version image `emqx/emqx-enterprise:latest`.
141+
The base image version in this example is `emqx/emqx-enterprise:5.8.1`. You can build the image based on the EMQX Enterprise version you need, or use the latest version image `emqx/emqx-enterprise:latest`.
157142

158-
```docker
159-
# FROM emqx/emqx-enterprise:latest
160-
FROM emqx/emqx-enterprise:5.0.3-alpha.2
143+
```dockerfile
144+
FROM emqx/emqx-enterprise:5.8.1
161145

162-
USER root
146+
USER root
163147

164-
RUN apt-get update \
165-
&& apt-get install -y gnupg2 curl apt-utils \
166-
&& curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - \
167-
&& curl https://packages.microsoft.com/config/debian/11/prod.list > /etc/apt/sources.list.d/mssql-release.list \
168-
&& apt-get update \
169-
&& ACCEPT_EULA=Y apt-get install -y msodbcsql17 unixodbc-dev \
170-
&& sed -i 's/ODBC Driver 17 for SQL Server/ms-sql/g' /etc/odbcinst.ini \
171-
&& apt-get clean \
172-
&& rm -rf /var/lib/apt/lists/*
148+
RUN apt-get -qq update && apt-get install -yqq curl gpg && \
149+
. /etc/os-release && \
150+
curl -fsSL https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor -o /usr/share/keyrings/microsoft-prod.gpg && \
151+
curl -fsSL "https://packages.microsoft.com/config/${ID}/${VERSION_ID}/prod.list" > /etc/apt/sources.list.d/mssql-release.list && \
152+
apt-get -qq update && \
153+
ACCEPT_EULA=Y apt-get install -yqq msodbcsql18 unixodbc-dev && \
154+
sed -i 's/ODBC Driver 18 for SQL Server/ms-sql/g' /etc/odbcinst.ini && \
155+
apt-get clean && \
156+
rm -rf /var/lib/apt/lists/*
173157

174-
USER emqx
175-
```
158+
USER emqx
159+
```
176160

177-
2. Build a new image using the command `docker build -f=Dockerfile.msodbc -t emqx-enterprise-with-msodbc:5.0.3-alpha.2 .`
161+
2. Build a new image using the command `docker build -t emqx/emqx-enterprise:5.8.1-msodbc`.
178162

179163
3. After building, you can use `docker image ls` to obtain a list of local images. You can also upload or save the image for later use.
180164

181-
::: tip
165+
::: tip Note
182166

183-
Check that the DSN Name in `odbcinst.ini` should be `ms-sql` if you install the msodbcsql17 driver using this example. You can change the DSN Name according to your needs.
167+
Check that the DSN Name in `odbcinst.ini` should be `ms-sql` if you install the msodbcsql18 driver using this example. You can change the DSN Name according to your needs.
184168

185169
:::
186170

@@ -238,7 +222,7 @@ The following steps assume that you run both EMQX and Microsoft SQL Server on th
238222
4. In the **Configuration** step, configure the following information:
239223
- **Connector name**: Enter a name for the connector, which should be a combination of upper and lower-case letters and numbers, for example: `my_sqlserver`.
240224
- **Server Host**: Enter `127.0.0.1:1433`, or the URL if the Microsoft SQL Server is running remotely.
241-
- **Database Name**: Enter `mqtt`.
225+
- **Database Name**: Enter `master`.
242226
- **Username**: Enter `sa`.
243227
- **Password**: Enter the preset password `mqtt_public1`, or use the actual password.
244228
- **SQL Server Driver Name**: Enter `ms-sql`, as the DSN Name configured in `odbcinst.ini`
@@ -332,10 +316,10 @@ mqttx pub -i emqx_c -t t/1 -m '{ "msg": "hello SQL Server" }'
332316

333317
Check the running statistics of the Microsoft SQL Server Sink.
334318

335-
- For the Sink used to store messages, there should be 1 new matching and 1 new outgoing message. Check whether the data is written into the `mqtt.dbo.t_mqtt_msg` data table.
319+
- For the Sink used to store messages, there should be 1 new matching and 1 new outgoing message. Check whether the data is written into the `t_mqtt_msg` data table.
336320

337321
```bash
338-
1> SELECT * from mqtt.dbo.t_mqtt_msg
322+
1> SELECT * from dbo.t_mqtt_msg
339323
2> GO
340324
id msgid topic qos payload arrived
341325
----------- ---------------------------------------------------------------- ---------------------------------------------------------------------------------------------------- --- ---------------------------------------------------------------------------------------------------- -----------------------
@@ -345,10 +329,10 @@ id msgid top
345329
1>
346330
```
347331
348-
- For the Sink used to record online/offline status, there should be 2 new events recorded: client connected and client disconnected. Check whether the status recording is written into the `mqtt.dbo.t_mqtt_events` data table.
332+
- For the Sink used to record online/offline status, there should be 2 new events recorded: client connected and client disconnected. Check whether the status recording is written into the `t_mqtt_events` data table.
349333
350334
```bash
351-
1> SELECT * from mqtt.dbo.t_mqtt_events
335+
1> SELECT * from dbo.t_mqtt_events
352336
2> GO
353337
id clientid event_type event_time
354338
----------- ---------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -----------------------

0 commit comments

Comments
 (0)