Skip to content

Commit 8fb1b4d

Browse files
authored
Adopt orjson for JSON performance optimization (#556)
# Adopt orjson for JSON Performance Optimization ## Problem The Pinecone Python client uses Python's standard library `json` module for serializing and deserializing JSON in REST API requests and responses. This can be a performance bottleneck, especially for applications making many API calls or handling large payloads. ## Solution Replace the standard library `json` module with `orjson`, a fast JSON library written in Rust. `orjson` provides significant performance improvements for both serialization (`dumps`) and deserialization (`loads`) operations. ## Changes ### Dependency Addition - Added `orjson>=3.0.0` to `pyproject.toml` dependencies with a loose version constraint to avoid conflicts with other applications ### Code Updates - **Synchronous REST client** (`rest_urllib3.py`): Replaced `json.dumps()` with `orjson.dumps()` for request serialization - **Asynchronous REST client** (`rest_aiohttp.py`): Replaced `json.dumps()` with `orjson.dumps()` and pre-serialize requests (using `data` parameter instead of `json=` parameter) for better performance - **Response deserializer** (`deserializer.py`): Replaced `json.loads()` with `orjson.loads()` for response parsing - **Multipart encoding** (`api_client_utils.py`, `asyncio_api_client.py`): Replaced `json.dumps()` with `orjson.dumps()` for multipart form data - **Query response parsing** (`vector.py`, `vector_asyncio.py`): Replaced `json.loads()` with `orjson.loads()` for parsing query responses ### Test Updates - Updated `test_bulk_import.py` to compare parsed JSON dicts instead of JSON strings, since orjson produces more compact JSON (no spaces after colons/commas) ## Performance Improvements Benchmark results show significant performance improvements across all tested scenarios: ### Serialization (dumps) - **Small payloads (10 vectors, 128 dim)**: ~14-23x faster - **Medium payloads (100 vectors, 128 dim)**: ~10-12x faster - **Large payloads (100 vectors, 512 dim)**: ~20x faster - **Query responses (1000 matches)**: ~11x faster ### Deserialization (loads) - **Small payloads (10 vectors, 128 dim)**: ~6-7x faster - **Medium payloads (100 vectors, 128 dim)**: ~5-6x faster - **Large payloads (100 vectors, 512 dim)**: ~6x faster - **Query responses (1000 matches)**: ~4-5x faster ### Round-trip (dumps + loads) - **Small payloads**: ~8x faster - **Medium payloads**: ~8-9x faster These improvements are especially beneficial for: - High-throughput applications making many API calls - Applications handling large vector payloads - Real-time applications where latency matters ## Usage Example No changes required for users - the API remains the same: ```python from pinecone import Pinecone pc = Pinecone(api_key="your-api-key") index = pc.Index("my-index") # These operations now benefit from orjson performance improvements index.upsert(vectors=[...]) # Faster serialization results = index.query(vector=[...]) # Faster deserialization ``` ## Testing - All existing unit tests pass (316+ tests) - Performance tests added in `tests/perf/test_orjson_performance.py` to measure improvements - Test suite updated to handle orjson's compact JSON output format ## Breaking Changes None. This is a transparent performance improvement with no API changes.
1 parent 27e751c commit 8fb1b4d

File tree

11 files changed

+213
-35
lines changed

11 files changed

+213
-35
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,3 +161,4 @@ dmypy.json
161161
*~
162162

163163
tests/integration/proxy_config/logs
164+
benchmark_results.json

pinecone/db_data/resources/asyncio/vector_asyncio.py

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,10 @@
33
from pinecone.utils.tqdm import tqdm
44
import logging
55
import asyncio
6-
import json
76
from typing import List, Any, Literal, AsyncIterator
87

8+
import orjson
9+
910
from pinecone.core.openapi.db_data.api.vector_operations_api import AsyncioVectorOperationsApi
1011
from pinecone.core.openapi.db_data.models import (
1112
QueryResponse as OpenAPIQueryResponse,
@@ -571,11 +572,12 @@ async def query_namespaces(
571572
from pinecone.openapi_support.rest_utils import RESTResponse
572573

573574
if isinstance(raw_result, RESTResponse):
574-
response = json.loads(raw_result.data.decode("utf-8"))
575+
response = orjson.loads(raw_result.data)
575576
aggregator.add_results(response)
576577
else:
577-
# Fallback: if somehow we got an OpenAPIQueryResponse, parse it
578-
response = json.loads(raw_result.to_dict())
578+
# Fallback: if somehow we got an OpenAPIQueryResponse, use dict directly
579+
# to_dict() returns a dict, not JSON, so no parsing needed
580+
response = raw_result.to_dict()
579581
aggregator.add_results(response)
580582

581583
final_results = aggregator.get_results()

pinecone/db_data/resources/sync/vector.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,9 @@
22

33
from pinecone.utils.tqdm import tqdm
44
import logging
5-
import json
65
from typing import Any, Literal
6+
7+
import orjson
78
from multiprocessing.pool import ApplyResult
89
from concurrent.futures import as_completed
910

@@ -649,7 +650,7 @@ def query_namespaces(
649650
futures: list[Future[Any]] = cast(list[Future[Any]], async_futures)
650651
for result in as_completed(futures):
651652
raw_result = result.result()
652-
response = json.loads(raw_result.data.decode("utf-8"))
653+
response = orjson.loads(raw_result.data)
653654
aggregator.add_results(response)
654655

655656
final_results = aggregator.get_results()

pinecone/openapi_support/api_client_utils.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
1-
import json
2-
import mimetypes
31
import io
2+
import mimetypes
43
import os
5-
from urllib3.fields import RequestField
64
from urllib.parse import quote
5+
from urllib3.fields import RequestField
76

7+
import orjson
88
from typing import Any
99
from .serializer import Serializer
1010
from .exceptions import PineconeApiValueError
@@ -116,7 +116,8 @@ def parameters_to_multipart(params, collection_types):
116116
if isinstance(
117117
v, collection_types
118118
): # v is instance of collection_type, formatting as application/json
119-
v = json.dumps(v, ensure_ascii=False).encode("utf-8")
119+
# orjson.dumps() returns bytes, no need to encode
120+
v = orjson.dumps(v)
120121
field = RequestField(k, v)
121122
field.make_multipart(content_type="application/json; charset=utf-8")
122123
new_params.append(field)

pinecone/openapi_support/asyncio_api_client.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
import json
21
import io
3-
from urllib3.fields import RequestField
42
import logging
3+
from urllib3.fields import RequestField
54

5+
import orjson
66
from typing import Any
77

88

@@ -203,7 +203,8 @@ def parameters_to_multipart(self, params, collection_types):
203203
if isinstance(
204204
v, collection_types
205205
): # v is instance of collection_type, formatting as application/json
206-
v = json.dumps(v, ensure_ascii=False).encode("utf-8")
206+
# orjson.dumps() returns bytes, no need to encode
207+
v = orjson.dumps(v)
207208
field = RequestField(k, v)
208209
field.make_multipart(content_type="application/json; charset=utf-8")
209210
new_params.append(field)

pinecone/openapi_support/deserializer.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
1-
import json
21
import re
32
from typing import TypeVar, Type, Any
43

4+
import orjson
5+
56
from .model_utils import deserialize_file, file_type, validate_and_convert_types
67

78
T = TypeVar("T")
@@ -53,7 +54,7 @@ def deserialize(
5354

5455
# fetch data from response object
5556
try:
56-
received_data = json.loads(response.data)
57+
received_data = orjson.loads(response.data)
5758
except ValueError:
5859
received_data = response.data
5960

pinecone/openapi_support/rest_aiohttp.py

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import ssl
22
import certifi
3-
import json
3+
4+
import orjson
45
from .rest_utils import RestClientInterface, RESTResponse, raise_exceptions_or_return
56
from ..config.openapi_configuration import Configuration
67

@@ -61,7 +62,7 @@ async def request(
6162
headers["Content-Type"] = "application/json"
6263

6364
if "application/x-ndjson" in headers.get("Content-Type", "").lower():
64-
ndjson_data = "\n".join(json.dumps(record) for record in body)
65+
ndjson_data = "\n".join(orjson.dumps(record).decode("utf-8") for record in body)
6566

6667
async with self._retry_client.request(
6768
method, url, params=query_params, headers=headers, data=ndjson_data
@@ -72,8 +73,11 @@ async def request(
7273
)
7374

7475
else:
76+
# Pre-serialize with orjson for better performance than aiohttp's json parameter
77+
# which uses standard library json
78+
body_data = orjson.dumps(body) if body is not None else None
7579
async with self._retry_client.request(
76-
method, url, params=query_params, headers=headers, json=body
80+
method, url, params=query_params, headers=headers, data=body_data
7781
) as resp:
7882
content = await resp.read()
7983
return raise_exceptions_or_return(

pinecone/openapi_support/rest_urllib3.py

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
1-
import json
21
import logging
3-
import ssl
42
import os
3+
import ssl
54
from urllib.parse import urlencode, quote
5+
6+
import orjson
67
from ..config.openapi_configuration import Configuration
78
from .rest_utils import raise_exceptions_or_return, RESTResponse, RestClientInterface
89

@@ -141,7 +142,7 @@ def request(
141142
+ bcolors.ENDC
142143
)
143144
else:
144-
formatted_body = json.dumps(body)
145+
formatted_body = orjson.dumps(body).decode("utf-8")
145146
print(
146147
bcolors.OKBLUE
147148
+ "curl -X {method} '{url}' {formatted_headers} -d '{data}'".format(
@@ -184,9 +185,11 @@ def request(
184185
if content_type == "application/x-ndjson":
185186
# for x-ndjson requests, we are expecting an array of elements
186187
# that need to be converted to a newline separated string
187-
request_body = "\n".join(json.dumps(element) for element in body)
188+
request_body = "\n".join(
189+
orjson.dumps(element).decode("utf-8") for element in body
190+
)
188191
else: # content_type == "application/json":
189-
request_body = json.dumps(body)
192+
request_body = orjson.dumps(body).decode("utf-8")
190193
r = self.pool_manager.request(
191194
method,
192195
url,

pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ classifiers = [
3030
dependencies = [
3131
"typing-extensions>=3.7.4",
3232
"certifi>=2019.11.17",
33+
"orjson>=3.0.0",
3334
"pinecone-plugin-interface>=0.0.7,<0.1.0",
3435
"python-dateutil>=2.5.3",
3536
"pinecone-plugin-assistant==3.0.0",
Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
"""Performance tests comparing orjson vs standard json library.
2+
3+
These tests measure the performance improvements from using orjson
4+
for JSON serialization and deserialization in REST API requests/responses.
5+
"""
6+
7+
import json
8+
import random
9+
10+
import orjson
11+
import pytest
12+
13+
14+
def create_vector_payload(num_vectors: int, dimension: int) -> list[dict]:
15+
"""Create a typical upsert payload with vectors."""
16+
vectors = []
17+
for i in range(num_vectors):
18+
vector = {
19+
"id": f"vec_{i}",
20+
"values": [random.random() for _ in range(dimension)],
21+
"metadata": {
22+
"category": f"cat_{i % 10}",
23+
"score": random.randint(0, 100),
24+
"tags": [f"tag_{j}" for j in range(3)],
25+
},
26+
}
27+
vectors.append(vector)
28+
return vectors
29+
30+
31+
def create_query_response(num_matches: int, dimension: int, include_values: bool = True) -> dict:
32+
"""Create a typical query response payload."""
33+
matches = []
34+
for i in range(num_matches):
35+
match = {
36+
"id": f"vec_{i}",
37+
"score": random.random(),
38+
"metadata": {"category": f"cat_{i % 10}", "score": random.randint(0, 100)},
39+
}
40+
if include_values:
41+
match["values"] = [random.random() for _ in range(dimension)]
42+
matches.append(match)
43+
return {"matches": matches}
44+
45+
46+
class TestOrjsonSerialization:
47+
"""Benchmark orjson.dumps() vs json.dumps()."""
48+
49+
@pytest.mark.parametrize("num_vectors,dimension", [(10, 128), (100, 128), (100, 512)])
50+
def test_json_dumps_vectors(self, benchmark, num_vectors, dimension):
51+
"""Benchmark json.dumps() for vector payloads."""
52+
payload = create_vector_payload(num_vectors, dimension)
53+
result = benchmark(json.dumps, payload)
54+
assert isinstance(result, str)
55+
assert len(result) > 0
56+
57+
@pytest.mark.parametrize("num_vectors,dimension", [(10, 128), (100, 128), (100, 512)])
58+
def test_orjson_dumps_vectors(self, benchmark, num_vectors, dimension):
59+
"""Benchmark orjson.dumps() for vector payloads."""
60+
payload = create_vector_payload(num_vectors, dimension)
61+
result = benchmark(orjson.dumps, payload)
62+
assert isinstance(result, bytes)
63+
assert len(result) > 0
64+
65+
@pytest.mark.parametrize("num_matches,dimension", [(10, 128), (100, 128), (1000, 128)])
66+
def test_json_dumps_query_response(self, benchmark, num_matches, dimension):
67+
"""Benchmark json.dumps() for query responses."""
68+
payload = create_query_response(num_matches, dimension)
69+
result = benchmark(json.dumps, payload)
70+
assert isinstance(result, str)
71+
assert len(result) > 0
72+
73+
@pytest.mark.parametrize("num_matches,dimension", [(10, 128), (100, 128), (1000, 128)])
74+
def test_orjson_dumps_query_response(self, benchmark, num_matches, dimension):
75+
"""Benchmark orjson.dumps() for query responses."""
76+
payload = create_query_response(num_matches, dimension)
77+
result = benchmark(orjson.dumps, payload)
78+
assert isinstance(result, bytes)
79+
assert len(result) > 0
80+
81+
82+
class TestOrjsonDeserialization:
83+
"""Benchmark orjson.loads() vs json.loads()."""
84+
85+
@pytest.mark.parametrize("num_vectors,dimension", [(10, 128), (100, 128), (100, 512)])
86+
def test_json_loads_vectors(self, benchmark, num_vectors, dimension):
87+
"""Benchmark json.loads() for vector payloads."""
88+
payload = create_vector_payload(num_vectors, dimension)
89+
json_str = json.dumps(payload)
90+
result = benchmark(json.loads, json_str)
91+
assert isinstance(result, list)
92+
assert len(result) == num_vectors
93+
94+
@pytest.mark.parametrize("num_vectors,dimension", [(10, 128), (100, 128), (100, 512)])
95+
def test_orjson_loads_vectors(self, benchmark, num_vectors, dimension):
96+
"""Benchmark orjson.loads() for vector payloads."""
97+
payload = create_vector_payload(num_vectors, dimension)
98+
json_bytes = json.dumps(payload).encode("utf-8")
99+
result = benchmark(orjson.loads, json_bytes)
100+
assert isinstance(result, list)
101+
assert len(result) == num_vectors
102+
103+
@pytest.mark.parametrize("num_matches,dimension", [(10, 128), (100, 128), (1000, 128)])
104+
def test_json_loads_query_response(self, benchmark, num_matches, dimension):
105+
"""Benchmark json.loads() for query responses."""
106+
payload = create_query_response(num_matches, dimension)
107+
json_str = json.dumps(payload)
108+
result = benchmark(json.loads, json_str)
109+
assert isinstance(result, dict)
110+
assert len(result["matches"]) == num_matches
111+
112+
@pytest.mark.parametrize("num_matches,dimension", [(10, 128), (100, 128), (1000, 128)])
113+
def test_orjson_loads_query_response(self, benchmark, num_matches, dimension):
114+
"""Benchmark orjson.loads() for query responses."""
115+
payload = create_query_response(num_matches, dimension)
116+
json_bytes = json.dumps(payload).encode("utf-8")
117+
result = benchmark(orjson.loads, json_bytes)
118+
assert isinstance(result, dict)
119+
assert len(result["matches"]) == num_matches
120+
121+
@pytest.mark.parametrize("num_matches,dimension", [(10, 128), (100, 128), (1000, 128)])
122+
def test_orjson_loads_from_string(self, benchmark, num_matches, dimension):
123+
"""Benchmark orjson.loads() with string input (like from decoded response)."""
124+
payload = create_query_response(num_matches, dimension)
125+
json_str = json.dumps(payload)
126+
result = benchmark(orjson.loads, json_str)
127+
assert isinstance(result, dict)
128+
assert len(result["matches"]) == num_matches
129+
130+
131+
class TestRoundTrip:
132+
"""Benchmark complete round-trip serialization/deserialization."""
133+
134+
@pytest.mark.parametrize("num_vectors,dimension", [(10, 128), (100, 128)])
135+
def test_json_round_trip(self, benchmark, num_vectors, dimension):
136+
"""Benchmark json round-trip (dumps + loads)."""
137+
138+
def round_trip(payload):
139+
json_str = json.dumps(payload)
140+
return json.loads(json_str)
141+
142+
payload = create_vector_payload(num_vectors, dimension)
143+
result = benchmark(round_trip, payload)
144+
assert isinstance(result, list)
145+
assert len(result) == num_vectors
146+
147+
@pytest.mark.parametrize("num_vectors,dimension", [(10, 128), (100, 128)])
148+
def test_orjson_round_trip(self, benchmark, num_vectors, dimension):
149+
"""Benchmark orjson round-trip (dumps + loads)."""
150+
151+
def round_trip(payload):
152+
json_bytes = orjson.dumps(payload)
153+
return orjson.loads(json_bytes)
154+
155+
payload = create_vector_payload(num_vectors, dimension)
156+
result = benchmark(round_trip, payload)
157+
assert isinstance(result, list)
158+
assert len(result) == num_vectors

0 commit comments

Comments
 (0)