Skip to content

Commit 92aa071

Browse files
vcalescuvcalescu
authored
Merge pull request #334 from sshanks-kx/refactor
refactor
2 parents 86d74b9 + f0d0165 commit 92aa071

File tree

6 files changed

+264
-219
lines changed

6 files changed

+264
-219
lines changed

‎README.md‎

Lines changed: 1 addition & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,7 @@
11
Documentation for kdb+ and q
22
============================
33

4-
5-
6-
Version 2.0 of the documentation site at [code.kx.com/q](https://code.kx.com/q).
7-
8-
4+
Source for [code.kx.com/q](https://code.kx.com/q).
95

106
Platform
117
--------
@@ -14,18 +10,3 @@ This site is generated with the [MkDocs](https://mkdocs.org/) static-site genera
1410

1511
See [`install.md`](CONTRIBUTING/install.md) for how to compile the site.
1612

17-
18-
Not upwardly compatible
19-
-----------------------
20-
21-
Version 2.0 substantially reorganizes the URL structure of the site.
22-
Most keywords and operators now have short URLs, e.g. code.kx.com/q/ref/aj and code.kx.com/q/ref/fill.
23-
24-
This marks a break from [Version 1.0](https://github.com/kxsystems/docs-v1/).
25-
There is no Git continuity between the two repos.
26-
27-
The terminology revision begun in Version 1.0 is complete: _adverbs_ are now _iterators_. The Reference material for them has been completely rewritten and the 2013 whitepaper “Efficient use of adverbs” re-issued as [“Iterators”](https://code.kx.com/q/wp/iterators/). The concept of _rank_ is extended from functions to all applicable values. _Variadic_ replaces _ambivalent_ for values (e.g. derived functions and matrixes) that may be applied as either unary or binary.
28-
29-
The Cookbook articles are now in the [Knowledge Base](https://code.kx.com/q/kb/).
30-
31-

‎docs/basics/errors.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -620,7 +620,7 @@ On launch
620620

621621
error | explanation
622622
------|------------
623-
{timestamp} couldn't connect to license daemon | Could not connect to KX license server ([kdb+ On Demand](../learn/licensing.md#licensing-server-for-kdb-on-demand))
623+
{timestamp} couldn't connect to license daemon | Could not connect to KX license server ([kdb+ On Demand](../learn/licensing.md#licensing-server))
624624
[](){#cores}cores | The license is for [fewer cores than available](../learn/licensing.md#core-restrictions)
625625
[](){#cpu}cpu | The license is for fewer CPUs than available
626626
[](){#exp}exp | License expiry date is prior to system date

‎docs/interfaces/q-server-for-odbc3.md‎

Lines changed: 146 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,7 @@ description: The ODBC3 server allows applications to query kdb+ via the ODBC int
55
# :fontawesome-solid-database: kdb+ server for ODBC3
66

77

8-
9-
The ODBC3 server allows applications to query kdb+ via the ODBC interface.
8+
The ODBC3 driver allows applications to query kdb+ through the ODBC interface.
109
:fontawesome-brands-github: [KxSystems/kdb/c/qodbc3.zip](https://github.com/KxSystems/kdb/blob/master/c/qodbc3.zip)
1110

1211
Currently the applications may run on the following platforms: w64, w32, l64, l32. Primary compatibility target has been [Tableau](https://www.tableau.com/), although other uses are welcome.
@@ -15,25 +14,52 @@ Requirements: V3.2 or later.
1514

1615
!!! tip "Reporting problems"
1716

18-
When reporting a problem (e.g. SQL error, wrong results, slowness, segfault etc.) make sure to include steps to reproduce along with your ODBC trace.
17+
When reporting a problem (e.g. SQL error, wrong results, slowness, segfault etc.) make sure to include steps to reproduce along with your [ODBC trace](#debugging).
1918

2019
:fontawesome-regular-map: [**Data visualization with kdb+ using ODBC**: a Tableau case study](../wp/data-visualization/index.md "White paper")
2120

22-
2321
## Installation
2422

25-
### Windows
23+
### kdb+ configuration
24+
25+
The kdb+ process should be [listening on port](../basics/listening-port.md) which relates to the port choosen and defined in the odbc configuration.
26+
27+
Unzip qodbc3.zip, and copy `ps.k` to `$QHOME`. Ensure you have `ps.k` loaded into the kdb+ process:
28+
29+
```q
30+
q)\l ps.k
31+
```
32+
33+
Run `.s.ver` to check that ps.k has been loaded and for the current version e.g.
34+
```q
35+
q).s.ver
36+
1.15
37+
```
38+
39+
The following is an example of starting kdb+ with ps.k listening on port 5000
40+
41+
```bash
42+
q ps.k -p 5000
43+
```
44+
45+
### ODBC driver and DSN configuration
46+
47+
Software that uses ODBC connections refer to individual connections using a `DSN`(Data Source Name).
48+
The DSN details the ODBC driver (e.g. qodbc3) and connection details specific to the data source (e.g. hostname, username, etc).
49+
50+
The following details adding a new DSN to connect to kdb+:
51+
52+
#### Windows
53+
54+
The following details installing the ODBC driver and configuring a user or system DSN as an administrator.
2655

2756
1. Close Tableau or anything that uses ODBC
2857
2. Extract qodbc3.zip to temporary location. Go to the directory corresponding to your OS architecture (w64 or w32)
2958
3. Run `d0.exe` as Administrator. This will copy `qodbc3.dll` to the correct location – you don’t need to do that yourself.
3059
4. You will now be able to add new kdb+ DSNs (data sources) in the `ODBC Data Source Administrator (64-bit)` (or `32-bit` if on 32-bit OS). Make sure to select `kdb+(odbc3)` in the list of drivers. You will be prompted for DSN name, hostname, port, username and password.
31-
5. In the ODBC data source administrator, click _Start Tracing_ on the _Tracing_ tab.
32-
6. Copy `q.tdc` to My Documents\My Tableau Repository\Datasources
33-
7. Copy `ps.k` to `$QHOME`
34-
60+
5. Use `Test Connection` within the DSN configuration screen to check that the connection to kdb+ is working.
3561

36-
### Linux
62+
#### Linux
3763

3864
Requirements: [unixODBC](http://www.unixodbc.org) 2.3.4, [Binutils](https://www.gnu.org/software/binutils/) (ld)
3965

@@ -44,36 +70,68 @@ $ cd qodbc3/l64
4470
$ ld -o qodbc3.so -shared qodbc3.o c.o -lodbc -lodbcinst -lm
4571
```
4672

47-
Add a DSN entry to your `~/.odbc.ini` file:
73+
The ODBC driver should be installed in the corresponding system `ini` file and a DSN entry created in either the system data sources (for all users) or user data source (current user).
74+
The linux `odbcinst` tool can be used to find these locations if not known, for example:
75+
76+
```bash
77+
$ odbcinst -j
78+
unixODBC 2.3.7
79+
DRIVERS............: /etc/odbcinst.ini
80+
SYSTEM DATA SOURCES: /etc/odbc.ini
81+
FILE DATA SOURCES..: /etc/ODBCDataSources
82+
USER DATA SOURCES..: /root/.odbc.ini
83+
SQLULEN Size.......: 8
84+
SQLLEN Size........: 8
85+
SQLSETPOSIROW Size.: 8
86+
```
87+
88+
Add the driver to your list of drivers by editing the driver `ini` file ( `/etc/odbcinst.ini`). An example of the definition follows (note: the driver location should be altered to the location chosen on your system):
89+
90+
```ini
91+
[kdb+(odbc3)]
92+
Description = ODBC for kdb+
93+
Driver = /src/qodbc3/l64/qodbc3.so
94+
```
95+
96+
After adding, this can be verified by listing the system ODBC drivers:
97+
98+
```bash
99+
$ odbcinst -q -d
100+
[PostgreSQL]
101+
[MySQL]
102+
[FreeTDS]
103+
[MariaDB]
104+
[kdb+(odbc3)]
105+
```
106+
107+
Add a DSN entry to your system data source file (`/etc/odbc.ini`) or user data source file (`~/.odbc.ini`), with the appropriate settings to communicate with the kdb+ server. For example:
48108

49109
```ini
50110
[your_dsn_name]
51111
Description=kdb+
52-
Driver=/path/to/qodbc3.so
53-
HOST=your.host:port
112+
Driver=kdb+(odbc3)
113+
HOST=localhost:5000
54114
UID=username
55115
PWD=password
56116
```
57117

58-
You should now be able to connect to your DSN with `isql`:
118+
You should now be able to connect to kdb+ represented by the DSN with `isql`:
59119

60120
```bash
61121
$ isql -3 -v -k 'DSN=your_dsn_name;'
62122
```
63123

124+
### Tableau configuration
64125

65-
## Running
126+
If there is a requirement to connect Tableau with kdb+, copy `q.tdc` to the appropriate directory for your application as detailed [here](https://help.tableau.com/current/pro/desktop/en-us/connect_customize.htm#installing-tdc-and-properties-files).
66127

67-
Ensure you have `ps.k` loaded into the kdb+ process specified in your DSN:
68-
69-
```q
70-
q)\l ps.k
71-
```
128+
Follow the Tableau directions for restarting Tableau Desktop/Server after the file is copied to the appropriate location.
72129

73-
The kdb+ process should also be [listening on port](../basics/listening-port.md) which relates to the port choosen and defined in the odbc configuration.
130+
The `q.tdc` file is a `Tableau Datasource Customization` (TDC) file to customize Tableau-specific settings for the kdb+ ODBC connection. The driver name within the file must match the name of the kdb+ driver installed (`kdb+(odbc3)`).
74131

132+
!!! note "The destination directory can be different depending on whether Tableau Desktop or Tableau Server is being used"
75133

76-
## Notes
134+
## Tableau Notes
77135

78136
To use q from Tableau’s Custom SQL use the `q()` function, e.g.:
79137

@@ -83,9 +141,6 @@ To use q from Tableau’s Custom SQL use the `q()` function, e.g.:
83141

84142
Parameters can be supplied by Tableau. Note that Tableau’s _string_ type corresponds to q’s symbol and _datetime_ corresponds to timestamp.
85143

86-
`test.q` provides additional examples of SQL usage, including the create/insert/update/delete statement syntax.
87-
88-
89144
## Compatibility
90145

91146
The driver translates SQL expressions into q and inherits q’s data model. This gives rise to the following SQL compatibility issues:
@@ -97,3 +152,69 @@ The driver translates SQL expressions into q and inherits q’s data model. This
97152

98153
Also, SQL selects from partitioned tables are not supported – one should pre-select from a partitioned table using the `q()` function instead.
99154

155+
`test.q` provides additional examples of SQL usage, including the create/insert/update/delete statement syntax.
156+
157+
## Debugging
158+
159+
ODBC implementations provide a tracing capability to log interactions with an ODBC driver. This can aid in diagnosing any issues. Tracing can have a detremental
160+
effect to ODBC performance.
161+
162+
This is a feature of the ODBC system, and not a feature unique to the kdb+ driver. Associated documentation, features, and troubleshooting can be found online for your OS.
163+
164+
!!! warn "Connection details can be logged, therefore the log file should be stored in a private location"
165+
166+
### Windows
167+
168+
In the ODBC data source administrator, click _Start Tracing_ on the _Tracing_ tab.
169+
170+
See below an example of the data recorded to the `SQL.LOG`:
171+
172+
```txt
173+
powershell da8-9a0 ENTER SQLExecDirectW
174+
HSTMT 0x00000256C37A6920
175+
WCHAR * 0x00000256AA98112C [ -3] "SELECT * FROM t\ 0"
176+
SDWORD -3
177+
...
178+
powershell da8-9a0 EXIT SQLGetData with return code 0 (SQL_SUCCESS)
179+
HSTMT 0x00000256C37A6920
180+
UWORD 2
181+
SWORD -8 <SQL_C_WCHAR>
182+
PTR 0x00000256A79BB1D0 [ 2] "1"
183+
SQLLEN 4092
184+
SQLLEN * 0x000000DAA198E160 (2)
185+
```
186+
187+
Please ensure tracing is disabled after debugging complete.
188+
189+
### Linux
190+
191+
In the `[ODBC]` section of the driver `ini` file (`/etc/odbcinst.ini`), set `Trace` to `1` to enable tracing and `TraceFile` to the location of the log file to create.
192+
Additional configuration options are available. For example:
193+
194+
```txt
195+
[ODBC]
196+
Trace = 1
197+
TraceFile = /tmp/odbc.log
198+
TraceOptions = 3
199+
ODBCTraceFlush = 1
200+
```
201+
202+
An example of the data recorded:
203+
```txt
204+
[ODBC][194][1729684200.251390][SQLPrepare.c][196]
205+
Entry:
206+
Statement = 0x55ba1f4cdb30
207+
SQL = [select * from t][length = 15]
208+
...
209+
[ODBC][194][1729684200.252562][SQLGetData.c][237]
210+
Entry:
211+
Statement = 0x55ba1f4cdb30
212+
Column Number = 1
213+
Target Type = 1 SQL_CHAR
214+
Buffer Length = 301
215+
Target Value = 0x55ba1f4cfd10
216+
StrLen Or Ind = 0x7ffe4093ef20
217+
```
218+
219+
Please ensure tracing is disabled after debugging complete.
220+

‎docs/learn/licensing.md‎

Lines changed: 25 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -17,29 +17,6 @@ Everyone. All use of kdb+ is governed by a license.
1717

1818
<!-- :fontawesome-regular-hand-point-right: [Licenses](https://kx.com/connect-with-us/licenses/) at kx.com -->
1919

20-
21-
=== "Non-commercial use"
22-
23-
Free 64-bit kdb+ On-Demand Personal Edition is for personal, non-commercial use.
24-
Currently, it may be used on up to 2 computers, and up to a maximum of 16 cores per computer, but is not licensed for use on any cloud – only on personal computers.
25-
It may not be used for any commercial purposes.
26-
See the [full terms and conditions](https://kx.com/download-kdb/).
27-
28-
It requires a `kc.lic` license key file and an always-on internet connection to operate.
29-
30-
31-
=== "Commercial use"
32-
33-
Use of commercial kdb+ is covered by your license agreement with KX.
34-
35-
Your copy of kdb+ will need access to a valid license key file.
36-
37-
If you wish to begin using kdb+ commercially, please contact sales@kx.com.
38-
39-
40-
## License key files
41-
42-
All 64-bit versions of kdb+ need a valid license key file to run.
4320
Without one, kdb+ signals an [error](../basics/errors.md#license-errors) `'k4.lic` and aborts.
4421

4522
```txt
@@ -56,13 +33,35 @@ If both are found, the `kc.lic` file is used.
5633

5734
## Obtain a license key file
5835

36+
A license file can be a commercial license or an on-demand person license (for non-commercial use).
37+
5938
### On-Demand
6039

61-
Follow the [download instructions](https://kx.com/kdb-personal-edition-download/) to get your kc.lic.
40+
Free 64-bit kdb+ On-Demand Personal Edition is for personal, non-commercial use.
41+
Currently, it may be used on up to 2 computers, and up to a maximum of 16 cores per computer, but is not licensed for use on any cloud – only on personal computers.
42+
It may not be used for any commercial purposes.
43+
See the [full terms and conditions](https://kx.com/download-kdb/). Follow the [download instructions](https://kx.com/kdb-personal-edition-download/) to get your kc.lic.
44+
45+
It requires a `kc.lic` license key file and an always-on internet connection to operate.
46+
47+
#### Licensing server
48+
49+
If kdb+ with an on-demand license cannot contact the KX license server it will abort with a timestamped message.
50+
51+
```q
52+
'2018.03.28T11:20:03.831 couldn't connect to license daemon -- exiting
53+
```
54+
55+
If an HTTP proxy is required, the environment variables `http_proxy` or `HTTP_PROXY` define the URL of the HTTP proxy to use.
56+
Since 4.1t 2022.11.01,4.0 2022.10.26,4.0ce 2022.09.16 the on-demand system honours the NO_PROXY/no_proxy environment variables, with the lowercase version taking precedence.
6257

6358
### Commercial
6459

65-
To begin using kdb+ commercially, contact sales@kx.com.
60+
Use of commercial kdb+ is covered by your license agreement with KX.
61+
62+
Your copy of kdb+ will need access to a valid license key file.
63+
64+
If you wish to begin using kdb+ commercially, please contact sales@kx.com.
6665

6766
## Install the license key file
6867

@@ -88,7 +87,7 @@ If you are sharing use of a commercial license, you will probably want to set th
8887
A list of possible errors can be found [here](../basics/errors.md#license-errors).
8988

9089

91-
## Keeping the license key file elsewhere
90+
### Keeping the license key file elsewhere
9291

9392
The default location for the license key file is the `QHOME` folder. You do not have to keep the license key file there. You can use the environment variable `QLIC` to specify a different filepath.
9493

@@ -100,22 +99,6 @@ The default location for the license key file is the `QHOME` folder. You do not
10099
QLIC='/Users/simon/q'
101100
```
102101

103-
104-
## Licensing server for kdb+ On Demand
105-
106-
As well as a license key file, kdb+ On Demand also requires frequent contact with the KX licensing server.
107-
For this you need an always-on Net connection.
108-
109-
If kdb+ cannot contact the KX server it will abort with a timestamped message.
110-
111-
```q
112-
'2018.03.28T11:20:03.831 couldn't connect to license daemon -- exiting
113-
```
114-
115-
If an HTTP proxy is required, the environment variables `http_proxy` or `HTTP_PROXY` define the URL of the HTTP proxy to use.
116-
Since 4.1t 2022.11.01,4.0 2022.10.26,4.0ce 2022.09.16 the on-demand system honours the NO_PROXY/no_proxy environment variables, with the lowercase version taking precedence.
117-
118-
119102
## Core restrictions
120103

121104
If the license is for fewer cores than the total number on the machine, the number of cores available to kdb+ must be [restricted with OS programs](../kb/cpu-affinity.md), or kdb+ will signal `'cores` and abort.
@@ -150,7 +133,6 @@ The number of licensed cores is always 16 for the on-demand license.
150133
scutil --set HostName "mymbp.local"
151134
```
152135

153-
154136
## License questions
155137

156138
Designated Contacts should send license questions to licadmin@kx.com.

0 commit comments

Comments
 (0)