python-sdk

icon	python

Python SDK

The Python SDK spicepy queries Spice.ai from Python. It uses Apache Arrow Flight to stream results, returning Apache Arrow records that convert directly into pandas dataframes.

Requirements

• Python 3.10 or later

The following packages are installed automatically:

• pyarrow
• pandas
• certifi
• requests

Installation

Install from the GitHub repository, pinned to a release tag:

pip install git+https://github.com/spiceai/spicepy@v3.1.0

{% hint style="danger" %} Do not run pip install spicepy. The spicepy name on PyPI belongs to an unrelated third-party project, not to this SDK. Install from the GitHub URL above. {% endhint %}

To use parameterized queries, two additional packages are required:

pip install adbc-driver-flightsql adbc-driver-manager

Usage

Create a Client, then call query():

from spicepy import Client

client = Client(
    api_key='API_KEY',
    flight_url='grpc+tls://flight.spiceai.io',
)

data = client.query('SELECT trip_distance, total_amount FROM taxi_trips ORDER BY trip_distance DESC LIMIT 10;', timeout=5*60)
pd = data.read_pandas()

Client has the following arguments, all optional:

• api_key (string): App API key, used to authenticate with Spice.ai Cloud. Falls back to the SPICE_API_KEY environment variable.
• flight_url (string): Arrow Flight endpoint (default: grpc://localhost:50051). Use grpc+tls:// for TLS and grpc:// for plaintext.
• http_url (string): HTTP endpoint, used for dataset refreshes (default: https://data.spiceai.io).
• tls_root_cert (Path or string): Path to the TLS certificate to use for the secure connection (omit for automatic detection).
• user_agent (string): Overrides the reported user agent.

{% hint style="warning" %} The SPICE_API_KEY environment variable authenticates HTTP requests only — it is not applied to Arrow Flight. Pass api_key to the constructor when querying Spice.ai Cloud. {% endhint %}

Once a Client is obtained, queries can be made using the query() function, which returns a pyarrow.flight.FlightStreamReader. It has the following arguments:

• query (string, required): The SQL query.
• timeout (int, optional): The timeout in seconds.

If no timeout is specified, it will default to a 10 min timeout then cancel the query, and a TimeoutError exception will be raised.

Call read_pandas() to read the whole result into a dataframe, or read it incrementally — see Streaming.

Usage with local Spice runtime

Follow the quickstart guide to install and run spice locally. flight_url already defaults to the local runtime:

from spicepy import Client

client = Client(http_url='http://localhost:8090')
data = client.query('SELECT trip_distance, total_amount FROM taxi_trips ORDER BY trip_distance DESC LIMIT 10;', timeout=5*60)
pd = data.read_pandas()

{% hint style="info" %} flight_url defaults to the local runtime, but http_url defaults to Spice.ai Cloud. When working entirely locally, set http_url as above so dataset refreshes are sent to the local runtime. {% endhint %}

Parameterized queries

query_with_params(sql, params) binds positional $1, $2 placeholders and returns a pyarrow.RecordBatchReader. It requires the two ADBC packages listed under Installation.

reader = client.query_with_params(
    'SELECT trip_distance, fare_amount FROM taxi_trips WHERE trip_distance > $1 LIMIT 10',
    [5.0],
)

for batch in reader:
    print(batch.to_pandas())

Parameter values may be plain Python values, whose Arrow type is inferred, or (value, pyarrow_type) tuples to set the type explicitly. Pass [] for a query with no parameters; None raises ValueError.

Refreshing a dataset

refresh_dataset(dataset, refresh_opts=None) triggers a refresh of an accelerated dataset over the HTTP endpoint. RefreshOpts accepts refresh_sql, refresh_mode, and refresh_jitter_max.

from spicepy import Client, RefreshOpts

client.refresh_dataset('taxi_trips', RefreshOpts(refresh_mode='full'))

Contributing

Contribute to or file an issue with the spicepy library at: https://github.com/spiceai/spicepy