Skip to content

Commit 7592a50

Browse files
authored
Merge pull request #327 from sshanks-kx/refactor
refactor
2 parents cfbebb0 + d6385d9 commit 7592a50

File tree

50 files changed

+777
-911
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+777
-911
lines changed

docs/architecture/index.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,9 @@ keywords: hdb, kdb+, q, rdb, tick, tickerplant, streaming
77

88
A kdb+ tick based archecture can be used to capture, process and analyse vasts amount of real-time and historical data.
99

10-
The following diagram illustrates the components that are often found in a vanilla kdb+ tick setup:
10+
The following diagram illustrates the components that are often found in a vanilla kdb+ tick setup:<br>
1111

12-
![architecture](../img/architecture.png)
12+
![architecture](../img/tick_arch.svg)
1313

1414
## Components
1515

docs/architecture/rq.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@ If known/common queries can be designed, the RDB can load additional scripts to
3030

3131
### End-of-day
3232

33+
![RDB_end-of-day](../img/rdb_end_of_day.svg)
34+
3335
The end-of-day event is governed by the tickerplant process. The tickerplant calls the RDB [`.u.end`](#uend) function when this event occurs.
3436
The main end-of-day event for an RDB is to save todays data from memory to disk, clear its tables and uses IPC to instruct the HDB to be aware of a new days dataset for it to access.
3537

@@ -42,6 +44,10 @@ The function [`.u.rep`](#urep) is then used to [replay the log](../kb/logging.md
4244

4345
!!! Note "The RDB should be able to access the tickerplant log from a directory on the same machine. The RDB/tickerplant can be changed to reside on different hosts but this increases the resources needed to transmit the log file contents over the network."
4446

47+
The following diagram shows the steps taken by an RDB to recover from a TP log:
48+
49+
![RDB_end-of-day](../img/TP_log_recovery.svg)
50+
4551
## Usage
4652

4753
```bash
@@ -84,12 +90,12 @@ Where
8490

8591
### .u.end
8692

87-
Perform end-of-day actions of saving tables to disk, clearing tables and running reload on HDB instance to make it aware of new day of data.
93+
Perform [end-of-day actions](end-of-day) of saving tables to disk, clearing tables and running reload on HDB instance to make it aware of new day of data.
8894

8995
```q
9096
.u.end[x]
9197
```
92-
Where x is the date that has ended.
98+
Where x is the date that has ended, as a date atom type.
9399

94100
Actions performed:
95101

@@ -103,7 +109,7 @@ Actions performed:
103109

104110
### .u.rep
105111

106-
Initialise RDB by creating tables, which is then populated with any existing tickerplant log. Will set the HDB directory to use at end-of-day.
112+
Initialise RDB by creating tables, which is then [populated with any existing tickerplant log](#recovery). Sets the HDB directory to be used at end-of-day.
107113

108114
```q
109115
.u.rep[x;y]

docs/architecture/uq.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -134,13 +134,13 @@ Where
134134
* `y` is list of syms used to filter table data, with empty sym representing for all table data
135135

136136
If `x` is empty symbol, client is subscribed to all known tables using `y` criteria. This is achieved by calling .u.sub for each table in [`.u.t`](#variables).
137-
It then returns a list of all the return values provided by .u.sub (i.e. a list of pairs comprising of table name and table definition).
138-
139-
An error is returned if the table does not exist.
140-
141137
For the subscribing client, any previous registered in the given tables are removed prior to reinstating new criteria provided i.e. calls [`.u.del`](#udel).
138+
Calls [`.u.add`](#uadd) to record the client subscription.
142139

143-
Calls [`.u.add`](#uadd) to record the client subscription and passes the returned values to client.
140+
Returns
141+
* a two item list if x is an indivial table name. First item is the table name subscribed to as a symbol. Second item is an empty table (table schema).
142+
* a list of two item lists as described above for each individual table, if x is an empty symbol (i.e. subscribe to all tables)
143+
* an error if the table does not exist.
144144

145145
### .u.end
146146

docs/basics/funsql.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,12 @@ keywords: delete, exec, functional, kdb+, q, select, sql, update
1010

1111
The functional forms of [`delete`](../ref/delete.md), [`exec`](../ref/exec.md), [`select`](../ref/select.md) and [`update`](../ref/update.md) are particularly useful for programmatically-generated queries, such as when column names are dynamically produced.
1212

13+
Functional form is an alternative to using a [qSQL template](qsql.md) to construct a query. For example, the folowing are equivalent:
14+
```q
15+
q)select n from t
16+
q)?[t;();0b;(enlist `n)!enlist `n]
17+
```
18+
1319
!!! info "Performance"
1420

1521
The q interpreter parses `delete`, `exec`, `select`, and `update` into their equivalent functional forms, so there is no performance difference.

docs/basics/peach.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@ Each secondary thread has its own heap, a minimum of 64MB.
138138

139139
Since V2.7 2011.09.21, [`.Q.gc[]`](../ref/dotq.md#gc-garbage-collect) in the main thread collects garbage in the secondary threads too.
140140

141-
Automatic garbage collection within each thread (triggered by a wsfull, or hitting the artificial heap limit as specified with [`-w`](cmdline.md#-w-workspace) on the command line) is executed only for that particular thread, not across all threads.
141+
Automatic garbage collection within each thread (triggered by a [`wsfull`](../basics/errors.md#wsfull)), or hitting the artificial heap limit as specified with [`-w`](cmdline.md#-w-workspace) on the command line) is executed only for that particular thread, not across all threads.
142142

143143
Symbols are internalized from a single memory area common to all threads.
144144

docs/basics/qsql.md

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,6 @@ keywords: exec, delete, kdb+, q, query, select, sql, update, upsert
55
---
66
# QSQL query templates
77

8-
9-
108
<div markdown="1" class="typewriter">
119
[delete](../ref/delete.md) delete rows or columns from a table
1210
[exec](../ref/exec.md) return columns from a table, possibly with new columns
@@ -17,9 +15,6 @@ keywords: exec, delete, kdb+, q, query, select, sql, update, upsert
1715
The query templates of qSQL share a query syntax that varies from the [syntax of q](syntax.md) and closely resembles [conventional SQL](https://www.w3schools.com/sql/).
1816
For many use cases involving ordered data it is significantly more expressive.
1917

20-
:fontawesome-brands-wikipedia-w:
21-
[Structured Query Language](https://en.wikipedia.org/wiki/SQL "Wikipedia")
22-
2318

2419
## Template syntax
2520

docs/basics/syscmds.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -342,10 +342,10 @@ Executing [`.Q.gc[]`](../ref/dotq.md#qgc-garbage-collect) additionally attempts
342342
Since V3.3 2015.08.23 (Linux only) unused pages in the heap are dropped from RSS during `.Q.gc[]`.
343343

344344
When q is denied additional address space from the OS, it invokes `.Q.gc[]` and retries the request to the OS.
345-
Should that fail, it will exit with `'wsfull`.
345+
If the subsequent attempt fail, the request exits with [`'wsfull`](../basics/errors.md#wsfull).
346346

347-
When secondary threads are configured and `.Q.gc[]` is invoked in the main thread it will automatically invoke `.Q.gc[]` in each secondary thread.
348-
If the call is instigated in a secondary thread – i.e., not the main thread – it will affect that thread’s local heap only.
347+
When secondary threads are configured and `.Q.gc[]` is invoked in the main thread, `.Q.gc[]` is automatically invoked in each secondary thread.
348+
If the call is instigated in a secondary thread, it affects that thread’s local heap only.
349349

350350
!!! detail "Notes on the allocator"
351351

docs/img/TP_log_recovery.svg

Lines changed: 3 additions & 0 deletions
Loading

docs/img/architecture.png

-62.9 KB
Binary file not shown.

docs/img/rdb_end_of_day.svg

Lines changed: 3 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)