Skip to content

Commit 1f0e7bf

Browse files
committed
added more imroved export scripting
1 parent 97235bf commit 1f0e7bf

File tree

3 files changed

+340
-87
lines changed

3 files changed

+340
-87
lines changed

docs/rest/request/post-v1requestquery-clickhouse.mdx

Lines changed: 162 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -39,36 +39,64 @@ that would be visible in the request table at
3939
| -------------------------------------------------------------- | ----------------------------------- |
4040
| [Get Request by User](/guides/cookbooks/getting-user-requests) | Get all the requests made by a user |
4141

42-
### Filter
42+
### Filter Structure
4343

44-
A filter is either a FilterLeaf or a FilterBranch, and can be composed of multiple filters generating an [AST](https://en.wikipedia.org/wiki/Abstract_syntax_tree) of ANDs/ORs.
44+
<Note>
45+
**Important:** Filters use an AST (Abstract Syntax Tree) structure where **each condition must be a separate leaf node**. You cannot combine multiple conditions in a single `request_response_rmt` object.
46+
</Note>
47+
48+
A filter is either a **FilterLeaf** or a **FilterBranch**, and can be composed of multiple filters generating an [AST](https://en.wikipedia.org/wiki/Abstract_syntax_tree) of ANDs/ORs.
4549

46-
Here is how it is represented in typescript:
50+
#### TypeScript Types
4751

4852
```ts
4953
export interface FilterBranch {
5054
left: FilterNode;
51-
operator: "or" | "and"; // Can add more later
55+
operator: "or" | "and";
5256
right: FilterNode;
5357
}
5458

59+
export type FilterLeaf = {
60+
request_response_rmt: {
61+
[field: string]: {
62+
[operator: string]: any;
63+
};
64+
};
65+
};
66+
5567
export type FilterNode = FilterLeaf | FilterBranch | "all";
5668
```
5769

58-
This allows us to build complex filters like this:
70+
#### Simple Filter (Single Condition)
5971

6072
```json
6173
{
6274
"filter": {
63-
"operator": "and",
64-
"right": {
75+
"request_response_rmt": {
76+
"model": {
77+
"contains": "gpt-4"
78+
}
79+
}
80+
}
81+
}
82+
```
83+
84+
#### Complex Filter (Multiple Conditions)
85+
86+
**Each condition is a separate leaf, connected with `and`/`or` operators:**
87+
88+
```json
89+
{
90+
"filter": {
91+
"left": {
6592
"request_response_rmt": {
6693
"model": {
6794
"contains": "gpt-4"
6895
}
6996
}
7097
},
71-
"left": {
98+
"operator": "and",
99+
"right": {
72100
"request_response_rmt": {
73101
"user_id": {
74102
"equals": "abc@email.com"
@@ -79,48 +107,170 @@ This allows us to build complex filters like this:
79107
}
80108
```
81109

110+
#### Match All Requests (No Filter)
111+
112+
```json
113+
{
114+
"filter": "all"
115+
}
116+
```
117+
118+
### Filtering by Date Range
119+
120+
<Note>
121+
Date ranges use **inclusive** bounds - both `gte` (greater than or equal) and `lte` (less than or equal) include the specified timestamps.
122+
</Note>
123+
124+
**Single date filter:**
125+
126+
```json
127+
{
128+
"filter": {
129+
"request_response_rmt": {
130+
"request_created_at": {
131+
"gte": "2024-01-01T00:00:00Z"
132+
}
133+
}
134+
}
135+
}
136+
```
137+
138+
**Date range (start AND end):**
139+
140+
<Warning>
141+
**Important:** Each date condition must be a separate leaf! Don't put both `gte` and `lte` in the same object.
142+
</Warning>
143+
144+
```json
145+
{
146+
"filter": {
147+
"left": {
148+
"request_response_rmt": {
149+
"request_created_at": {
150+
"gte": "2024-01-01T00:00:00Z"
151+
}
152+
}
153+
},
154+
"operator": "and",
155+
"right": {
156+
"request_response_rmt": {
157+
"request_created_at": {
158+
"lte": "2024-12-31T23:59:59Z"
159+
}
160+
}
161+
}
162+
}
163+
}
164+
```
165+
166+
**Available date operators:**
167+
- `gte` - Greater than or equal (start date, inclusive)
168+
- `lte` - Less than or equal (end date, inclusive)
169+
- `gt` - Greater than (exclusive)
170+
- `lt` - Less than (exclusive)
171+
- `equals` - Exact timestamp match
172+
82173
### Filtering by Properties
83174

84175
<Note>
85176
**Important:** When filtering by custom properties, you must nest the `properties` filter inside a `request_response_rmt` object.
86177
</Note>
87178

179+
**Single property:**
180+
88181
```json
89182
{
90183
"filter": {
91184
"request_response_rmt": {
92185
"properties": {
93186
"environment": {
94-
"like": "%-dev"
187+
"equals": "production"
95188
}
96189
}
97190
}
98191
}
99192
}
100193
```
101194

102-
You can combine property filters with other filters using the `and`/`or` operators:
195+
**Combining property filter with other filters:**
103196

104197
```json
105198
{
106199
"filter": {
107-
"operator": "and",
108200
"left": {
109201
"request_response_rmt": {
110202
"model": {
111203
"equals": "gpt-4"
112204
}
113205
}
114206
},
207+
"operator": "and",
115208
"right": {
116209
"request_response_rmt": {
117210
"properties": {
118211
"environment": {
119-
"like": "%-dev"
212+
"equals": "production"
120213
}
121214
}
122215
}
123216
}
124217
}
125218
}
126219
```
220+
221+
### Complete Example: Date Range + Property Filter
222+
223+
This example shows how to combine a date range with a property filter:
224+
225+
```json
226+
{
227+
"filter": {
228+
"left": {
229+
"left": {
230+
"request_response_rmt": {
231+
"request_created_at": {
232+
"gte": "2024-08-01T00:00:00Z"
233+
}
234+
}
235+
},
236+
"operator": "and",
237+
"right": {
238+
"request_response_rmt": {
239+
"request_created_at": {
240+
"lte": "2024-08-31T23:59:59Z"
241+
}
242+
}
243+
}
244+
},
245+
"operator": "and",
246+
"right": {
247+
"request_response_rmt": {
248+
"properties": {
249+
"appname": {
250+
"equals": "LlamaCoder"
251+
}
252+
}
253+
}
254+
}
255+
},
256+
"limit": 100,
257+
"offset": 0
258+
}
259+
```
260+
261+
### Available Filter Operators
262+
263+
Different fields support different operators:
264+
265+
**Text fields** (`model`, `user_id`, `provider`, etc.):
266+
- `equals` / `not-equals`
267+
- `like` / `ilike` (case-insensitive)
268+
- `contains` / `not-contains`
269+
270+
**Number fields** (`status`, `latency`, `cost`, etc.):
271+
- `equals` / `not-equals`
272+
- `gte` / `lte` / `gt` / `lt`
273+
274+
**Timestamp fields** (`request_created_at`, `response_created_at`):
275+
- `equals`
276+
- `gte` / `lte` / `gt` / `lt`
Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
node_modules
2-
output.jsonl
2+
output.jsonl
3+
.helicone-export-state.json

0 commit comments

Comments
 (0)