Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DEVELOPING.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ Additional Pythonic practices exist concerning the Python type system. Some of t
- When type checking is required, check against the standard [abstract base classes](https://docs.python.org/3/library/collections.abc.html) where applicable. These implement what's known as "virtual subclasses" by hooking into the mechanism behind `issubclass` to make classes which do not inherit them still match against them. For example, for any class `x` which implements `__contains__`, `__iter__`, and `__len__`, `issubclass(x, collections.abc.Collection) is True`.
- When delegating the methods (particularly the `__init__` method) to the super-class, have the method accept and pass on arbitrary arguments to ensure future changes to the signature of the method do not break the delegation chain. This can be accomplished by having `*args` and `**kwargs` in the parameter list, and then calling the super-class method with `*args` and `**kwargs`.

Other miscellaneous Pythonic tips include:
Other miscellaneous Pythonic tips include:
- Prefer `try` over `if` blocks where reasonable. See [EAFP](https://docs.python.org/3/glossary.html#term-eafp). For example, instead of checking if a key in a dictionary is present with an `if` statement prior to accessing it, one should access the dictionary within a `try` block, and handle the `KeyError` should it arise.
- Avoid extracting a sequence from an iterator without good reason. Iterators can produce infinite sequences (which will freeze the process). Additionally it can be a waste of memory to extract a sequence, as it's common for less than the entire output of an iterator to be consumed.
- Do not rely on CPython-specific behavior, such as the builtin function `id` returning the address of the object in memory.
Expand Down
11 changes: 0 additions & 11 deletions custom_theme/partials/header.html.tpl
Original file line number Diff line number Diff line change
@@ -1,17 +1,6 @@
{#-
This file was automatically generated - do not edit
-#}
<!-- Announcement begins-->
{% block announce %}
<div class="md-grid">
<h2>New Documentation Site!</h2>
<p style="font-size: 14px;">
We are excited to announce the launch of our enhanced product documentation site for <a href="https://docs.kx.com/3.1/PyKX/home.htm" style="color: var(--md-typeset-a-color);">PyKX</a> at <a href="https://docs.kx.com/home/index.htm" style="color: var(--md-typeset-a-color);">docs.kx.com</a>.
It offers improved search capabilities, organized navigation, and developer-focused content. Please, take a moment to explore the site and share your feedback with us.
</p>
</div>
{% endblock %}
<!-- ends -->
{% set class = "md-header" %}
{% if "navigation.tabs.sticky" in features %}
{% set class = class ~ " md-header--lifted" %}
Expand Down
6 changes: 3 additions & 3 deletions docs/api/pykx-q-data/type_conversions.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,13 @@ A breakdown of each of the `pykx.K` types and their analogous `Python`, `NumPy`,
| [Timespan](#pykxtimespanatom) | timedelta | timedelta64[ns] | DurationArray |
| [Minute](#pykxminuteatom) | timedelta | timedelta64[m] | Not Supported |
| [Second](#pykxsecondatom) | timedelta | timedelta64[s] | DurationArray |
| [Time](#TimeAtom) | timedelta | timedelta64[ms] | DurationArray |
| [Time](#pykxtimeatom) | timedelta | timedelta64[ms] | DurationArray |
| [Dictionary](#pykxdictionary) | dict | Not Supported | Not Supported |
| [Table](#pykxtable) | dict | records | Table |

??? "Cheat Sheet: `Pandas 1.*`, `Pandas 2.*`, `Pandas 2.* PyArrow backed`"

**Note:** Creating PyArrow backed Pandas objects uses `as_arrow=True` using NumPy arrays as an intermediate data format.
**Note:** Creating PyArrow backed Pandas objects uses `as_arrow=True` using NumPy arrays as an intermediate data format.

| PyKX type | Pandas 1.\* dtype | Pandas 2.\* dtype | Pandas 2.\* as_arrow=True dtype |
| ------------------------------- | ----------------- | ----------------- | ------------------------------- |
Expand All @@ -50,7 +50,7 @@ A breakdown of each of the `pykx.K` types and their analogous `Python`, `NumPy`,
| [Timespan](#pykxtimespanatom) | timedelta64[ns] | timedelta64[ns] | duration[ns][pyarrow] |
| [Minute](#pykxminuteatom) | timedelta64[ns] | timedelta64[s] | duration[s][pyarrow] |
| [Second](#pykxsecondatom) | timedelta64[ns] | timedelta64[s] | duration[s][pyarrow] |
| [Time](#TimeAtom) | timedelta64[ns] | timedelta64[ms] | duration[ms][pyarrow] |
| [Time](#pykxtimeatom) | timedelta64[ns] | timedelta64[ms] | duration[ms][pyarrow] |
| [Dictionary](#pykxdictionary) | Not Supported | Not Supported | Not Supported |
| [Table](#pykxtable) | DataFrame | DataFrame | DataFrame |

Expand Down
12 changes: 0 additions & 12 deletions docs/api/tick.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,3 @@ tags: tick, rdb, hdb, idb, streaming, stream
# Streaming tickerplant

::: pykx.tick
rendering:
show_root_heading: false
options:
show_root_heading: false
members_order: source
members:
- STREAMING
- BASIC
- TICK
- RTP
- HDB
- GATEWAY
Binary file added docs/examples/AsynchronousQueries/archive.zip
Binary file not shown.
40 changes: 40 additions & 0 deletions docs/examples/AsynchronousQueries/async_query.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import asyncio
import time

import pykx as kx


class ConnectionManager:
connections = {}

async def open(self, port):
self.connections[port] = await kx.AsyncQConnection(port=port)

async def __aenter__(self):
return self

async def __aexit__(self, exc_type, exc_val, exc_tb):
for v in self.connections.values():
await v.close()

def __call__(self, port, query, *args):
return self.connections[port](query, *args)


async def main():
async with ConnectionManager() as cm:
await cm.open(5050)
await cm.open(5051)
start = time.monotonic_ns()
queries = [
cm(5050, '{system"sleep 10"; til x + y}', 6, 7),
cm(5051, '{system"sleep 5"; til 10}[]')
]
queries = [await x for x in queries]
end = time.monotonic_ns()
[print(x) for x in queries]
print(f'took {(end - start) / 1_000_000_000} seconds')


if __name__ == '__main__':
asyncio.run(main())
51 changes: 51 additions & 0 deletions docs/examples/AsynchronousQueries/async_querying.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
title: Asynchronous Querying Example
date: July 2025
author: KX Systems, Inc.
tags: PyKX, q, asyncio, IPC, asynchronous
---

# PyKX Calling into multiple q servers without blocking

_This example provides a quick start for setting up a Python process using `PyKX` to call into
multiple q servers without blocking each other._

To follow along, feel free to download this <a href="./archive.zip" download>zip archive</a> that
contains a copy of the python scripts and this writeup.

## Quickstart

This example creates a python process that sends 2 queries meant to simulate long running queries to
two separate q servers to show how to query q servers without blocking using `PyKX`.

### Run the Example

The example uses 2 servers opened up on ports 5050 and 5051, these servers can be opened with the
commands `$ q -p 5050` and `$ q -p 5051` respectively.
The script `async_query.py` can then be run to send the two queries simultaneously with
`$ python async_query.py`.

### Outcome

The script will send the two queries and print their results followed by the total time taken
by the script.

The first query takes 10 seconds and the second takes 5 seconds to complete showing that both
queries were processed without blocking eachother.

```bash
0 1 2 3 4 5 6 7 8 9 10 11 12
0 1 2 3 4 5 6 7 8 9
took 10.001731808 seconds
```

### Important notes on usage of `QConnections`

While the `#!python with` syntax for `QConnection` objects is useful for sending one shot requests it
should be avoided where possible when repeatedly querying the same server. This is because
connecting to q servers is blocking and the closing of `QConnection` objects is also blocking which
will cause other queries to be delayed in certain cases. There is a simple class called
`ConnectionManager` provided in this example to handle opening connections to servers and allow
querying them by supplying a port alongside the query and any arguments. This class will also clean
up the stored `QConnection` objects when its `#!python with` block ends, much like the normal
`QConnection` objects do themselves.
2 changes: 1 addition & 1 deletion docs/examples/db-management.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@
"id": "2e91160e",
"metadata": {},
"source": [
"Database interactions are facilitated through use of the `pykx.DB` class. All methods/attributes used in this notebook are contained within this class. \n",
"Database interactions are facilitated through use of the `pykx.DB` class. All methods/attributes used in this notebook are contained within this class. Only one `DB` object can exist at a time within a process.\n",
"\n",
"Initialise the `DB` class to start. The expected input is the file path where you intend to save the partitioned database and its associated tables. In this case we're going to use the temporary directory we just created. "
]
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/interface-overview.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -644,7 +644,7 @@
"metadata": {},
"outputs": [],
"source": [
"conn.qsql.select('tab', where=['col1=`a', 'col2<0.3'])"
"conn.qsql.select('tab', where=((kx.Column('col1')=='a') & (kx.Column('col2')>0.3)))"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/server/server.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,4 +91,4 @@ set the `#!python conn_gc_time` to `#!python 10.0` then this clean-up happens ev

!!! Note

[reval](../../api/pykx-execution/q.md#reval) will not impose read only exection on a PyKX server as Python manages the sockets rather than `q`.
[reval](../../api/pykx-execution/q.md#reval) will not impose read only execution on a PyKX server as Python manages the sockets rather than `q`.
36 changes: 0 additions & 36 deletions docs/extras/comparisons.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/extras/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,4 +86,4 @@ The `#!python unique` attribute ensures that all items in the `#!python Vector`/
In the context of databases, upsert is an operation that combines both updating and inserting data into a table. When you perform an upsert, the database checks whether a record with a specific key already exists in the table. If a record with that key exists, the database updates the existing record with new values. If no record with that key exists, the database inserts a new record with the provided data. Learn more about [upsert](../api/pykx-execution/q.md#upsert).

## Vector
A vector is a mathematical concept used to represent quantities that have both magnitude (size) and direction. In other words, vectors are arrays of numerical values that represent points in multidimensional space. A vector is typically represented as an arrow in space, pointing from one point to another. Learn more about [using in-built methods on PyKX vectors](../examples/interface-overview.ipynb#using-in-built-methods-on-pykx-vectors) and [adding values to PyKX vectors/lists](../user-guide/fundamentals/indexing.md#assigning-and-adding-values-to-vectorslists).
A vector is a mathematical concept used to represent quantities that have both magnitude (size) and direction. In other words, vectors are arrays of numerical values that represent points in multidimensional space. A vector is typically represented as an arrow in space, pointing from one point to another. Learn more about [using in-built methods on PyKX vectors](../examples/interface-overview.ipynb#using-in-built-methods-on-pykx-vectors) and [adding values to PyKX vectors/lists](../user-guide/fundamentals/indexing.md#b-assigning-and-adding-values-to-vectorslists).
4 changes: 2 additions & 2 deletions docs/getting-started/installing.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ We provide assistance to user-built installations of PyKX only on a best-effort

You can install PyKX from three sources:

!!! Note "Installing in air-capped environments"
!!! Note "Installing in air-gapped environments"

If you are installing in a location without internet connection you may find [this section](#installing-in-an-air-gapped-environment) useful.

Expand Down Expand Up @@ -87,7 +87,7 @@ You can install PyKX from three sources:

```

At this point you have [partial access to PyKX](../user-guide/advanced/modes.md#operating-in-the-absence-of-a-kx-license). To gain access to all PyKX features, follow the steps in the next section, otherwise go straight to [3. Verify PyKX Installation](#3-verify-pykx-installation).
At this point you have [partial access to PyKX](../user-guide/advanced/modes.md#1a-running-in-unlicensed-mode). To gain access to all PyKX features, follow the steps in the next section, otherwise go straight to [3. Verify PyKX Installation](#3-verify-pykx-installation).

## 2. Install a kdb Insights license

Expand Down
4 changes: 2 additions & 2 deletions docs/getting-started/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ You can generate PyKX objects in three ways. Click on the tabs below to follow t

## 3. Interact with PyKX objects

You can interact with PyKX objects in a variety of ways, for example, through [indexing using Pythonic syntax](../user-guide/fundamentals/indexing.md), passing [PyKX objects to q/NumPy](../user-guide/fundamentals/creating.md#converting-pykx-objects-to-pythonic-types) functions, [querying via Python/SQL/qSQL](..//user-guide/fundamentals/query/index.md) syntax or by [using the q functionality](../user-guide/advanced/context_interface.md) via the context interface. Each way is described in more depth under the the User guide > Fundamentals section. For now, we recommend a few examples:
You can interact with PyKX objects in a variety of ways, for example, through [indexing using Pythonic syntax](../user-guide/fundamentals/indexing.md), passing [PyKX objects to q/NumPy](../user-guide/fundamentals/creating.md#2-convert-pykx-objects-to-pythonic-types) functions, [querying via Python/SQL/qSQL](..//user-guide/fundamentals/query/index.md) syntax or by [using the q functionality](../user-guide/advanced/context_interface.md) via the context interface. Each way is described in more depth under the the User guide > Fundamentals section. For now, we recommend a few examples:

* Create a PyKX list and interact with it using indexing and slices:

Expand Down Expand Up @@ -268,7 +268,7 @@ You can interact with PyKX objects in a variety of ways, for example, through [i
a 0.6212161 3.97236
..
'))
>>> kx.q.qsql.select(qtable, where = 'x=`a')
>>> qtable.select(where=kx.Column('col1')=='a')
pykx.Table(pykx.q('
x x1 x2
---------------------
Expand Down
12 changes: 12 additions & 0 deletions docs/help/issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,18 @@ pykx.LongAtom(pykx.q('2'))
>>> kx.q('func', kx.q('::'))
pykx.LongAtom(pykx.q('2'))
```
* If any issues occur when converting objects using `PYKX_ALLOCATOR=True` then the `no_allocator=True` keyword argument can be used to selectively turn off the use of the allocator.

```python
>>> df = pd.read_parquet('nested_arrs.parquet')
>>> kx.toq(df, no_allocator=True)
```

Or for PyKX under q you can use.
```q
q) df: .pykx.pyeval"pd.read_parquet('nested_arrs.parquet')";
q) .pykx.toq .pykx.noalloc df;
```

### Limitations
Embedding q in a Python process imposes some restrictions on functionality. The embedded q process does not run the main loop that it would when running natively, hence it is limited in usage of q IPC and q timers.
Expand Down
3 changes: 2 additions & 1 deletion docs/pykx-under-q/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -1191,17 +1191,18 @@ q).pykx.toq0 b
2
```

```q
// Convert a Python string to q symbol or string

q).pykx.toq0[.pykx.eval"\"test\""]
`test

q).pykx.toq0[.pykx.eval"\"test\"";1b]
"test"
```

## `.pykx.toraw`


_Tag a q object to be indicate a raw conversion when called in Python_

```q
Expand Down
Loading