Skip to content

Commit 958be08

Browse files
authored
feat: add delete-with-limit support for Collection.Delete (#476)
* docs(14): research phase domain * docs(14): create phase plan for delete-with-limit * feat(14-01): add Limit field to CollectionDeleteOp and ApplyToDelete to limitOption - Add Limit *int32 field with json omitempty to CollectionDeleteOp - Add validation in PrepareAndValidate: rejects limit <= 0 and limit without filter - Add ApplyToDelete method to limitOption - Update option compatibility matrix and WithLimit godoc for Delete support * feat(14-01): wire limit through embedded delete path - Convert int32 limit to uint32 for EmbeddedDeleteRecordsRequest - Pass limit to DeleteRecords call (safe: PrepareAndValidate ensures > 0) * docs(14-01): complete delete-with-limit API implementation plan - Add 14-01-SUMMARY.md with execution results - Update STATE.md position and metrics - Update ROADMAP.md with plan progress - Mark DEL-01 through DEL-04 complete in REQUIREMENTS.md * chore: merge plan 01 changes from feat/delete-with-limit * test(14-02): add delete-with-limit unit tests for option, validation, and JSON - WithLimit ApplyToDelete sets limit pointer correctly - Rejects zero and negative limit values with ErrInvalidLimit - PrepareAndValidate rejects limit without where/where_document filter - PrepareAndValidate accepts limit with where or where_document - JSON omits limit when nil, includes when set * test(14-02): add HTTP serialization test for delete with limit - New 'with where and limit' test case in TestCollectionDelete - Verifies limit field serializes in JSON body sent to server - Fix gci lint alignment in CollectionDeleteOp struct * docs(14-02): complete delete-with-limit tests plan - SUMMARY.md with test results and metrics - STATE.md advanced to plan 02 complete - ROADMAP.md updated with phase 14 progress - REQUIREMENTS.md DEL-05 marked complete * docs(phase-14): complete phase execution * docs(phase-14): evolve PROJECT.md after phase completion * docs(14): add phase context and validation artifacts * test(14): complete UAT - 6 passed, 0 issues Also fix gci struct alignment in CollectionDeleteOp. * fix(delete): add overflow guard, sentinel error, and safe conversion for limit - Add math.MaxInt32 overflow check in ApplyToDelete before int→int32 cast - Use ErrInvalidLimit sentinel in PrepareAndValidate for consistent errors.Is - Use intToUint32 helper in embedded path instead of raw uint32 cast - Update limitOption and DeleteOption doc comments to include Delete - Add end-to-end, overflow, sentinel, embedded, and HTTP wire tests * refactor(delete): use ErrLimitOverflow sentinel for overflow case Extract overflow error into a dedicated sentinel so callers can use errors.Is(err, ErrLimitOverflow) for programmatic error handling. * fix: break infinite recursion in UnmarshalJSON on all collection ops All five *Op types (Get, Query, Add, Update, Delete) had UnmarshalJSON methods that called json.Unmarshal(b, c) without a type alias, causing infinite recursion. Use the same Alias pattern as MarshalJSON.
1 parent 124830a commit 958be08

19 files changed

Lines changed: 1557 additions & 30 deletions

.planning/PROJECT.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,4 +71,4 @@ Go applications can use Chroma and embedding providers through a stable, portabl
7171
| Pivot Phase 7 from vLLM/Nemotron to VoyageAI | vLLM lacks NVOmniEmbedModel support; VoyageAI multimodal validates portability with text/image/video | ✓ Good |
7272

7373
---
74-
*Last updated: 2026-03-28 — Phase 13 (Collection ForkCount) complete: added ForkCount method to Collection interface with HTTP and embedded implementations, unit tests, documentation, and runnable example.*
74+
*Last updated: 2026-03-29 — Phase 14 (delete-with-limit) complete: added Limit parameter to Collection.Delete with validation, embedded path wiring, and comprehensive test coverage.*

.planning/REQUIREMENTS.md

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,14 @@
7676
- [x] **FC-05**: Forking docs page includes ForkCount section with Go and Python examples and API reference row
7777
- [x] **FC-06**: Runnable Fork + ForkCount example exists under `examples/v2/`
7878

79+
### Delete with Limit
80+
81+
- [x] **DEL-01**: `WithLimit(n)` applies to `Collection.Delete` via `ApplyToDelete` method on `limitOption`, reusing the existing option function
82+
- [x] **DEL-02**: `CollectionDeleteOp` has a `Limit *int32` field with `json:"limit,omitempty"` tag
83+
- [x] **DEL-03**: `PrepareAndValidate` rejects limit without where/where_document filter and limit <= 0 with exact upstream error messages
84+
- [x] **DEL-04**: Embedded path converts `*int32` limit to `*uint32` and passes to `EmbeddedDeleteRecordsRequest.Limit`
85+
- [x] **DEL-05**: Tests cover option application, validation edge cases, and HTTP serialization round-trip
86+
7987
## v2 Requirements
8088

8189
### Provider Adoption
@@ -142,12 +150,17 @@
142150
| FC-04 | Phase 13 | Planned |
143151
| FC-05 | Phase 13 | Planned |
144152
| FC-06 | Phase 13 | Planned |
153+
| DEL-01 | Phase 14 | Complete |
154+
| DEL-02 | Phase 14 | Complete |
155+
| DEL-03 | Phase 14 | Complete |
156+
| DEL-04 | Phase 14 | Complete |
157+
| DEL-05 | Phase 14 | Planned |
145158

146159
**Coverage:**
147-
- v1 requirements: 41 total
148-
- Mapped to phases: 41
160+
- v1 requirements: 46 total
161+
- Mapped to phases: 46
149162
- Unmapped: 0
150163

151164
---
152165
*Requirements defined: 2026-03-18*
153-
*Last updated: 2026-03-28 -- added FC-01/02/03/04/05/06 for phase 13*
166+
*Last updated: 2026-03-29 -- added DEL-01/02/03/04/05 for phase 14*

.planning/ROADMAP.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ This roadmap initializes GSD planning for the current brownfield milestone focus
2727
- [x] **Phase 11: Fork Double-Close Bug** - Fix EF pointer sharing in Fork() that causes double-close on client.Close(). (issue #454) (completed 2026-03-26)
2828
- [x] **Phase 12: SDK Auto-Wiring Research** - Trace contentEmbeddingFunction auto-wiring behavior in official Chroma SDKs. (issue #455) (completed 2026-03-28)
2929
- [x] **Phase 13: Collection.ForkCount** - Add ForkCount endpoint support for upstream /fork_count API. (issue #460) (completed 2026-03-28)
30-
- [ ] **Phase 14: Delete with Limit** - Add delete-with-limit support for upstream limit parameter. (issue #439)
30+
- [x] **Phase 14: Delete with Limit** - Add delete-with-limit support for upstream limit parameter. (issue #439) [1/2 plans complete] (completed 2026-03-29)
3131
- [ ] **Phase 15: OpenRouter Embeddings Compatibility** - Add first-class OpenRouter support via provider preferences and encoding_format. (issue #438)
3232
- [ ] **Phase 16: Twelve Labs Embedding Function** - Add Twelve Labs multimodal embedding provider. (issue #190)
3333
- [ ] **Phase 17: Cloud RRF and GroupBy Test Coverage** - Add cloud integration tests for Search API RRF and GroupBy primitives. (issue #462)
@@ -178,7 +178,7 @@ Plans:
178178
| 11. Fork Double-Close Bug | 2/2 | Complete | 2026-03-26 |
179179
| 12. SDK Auto-Wiring Research | 1/1 | Complete | 2026-03-28 |
180180
| 13. Collection.ForkCount | 2/2 | Complete | 2026-03-28 |
181-
| 14. Delete with Limit | 0/0 | Not started | - |
181+
| 14. Delete with Limit | 2/2 | Complete | 2026-03-29 |
182182
| 15. OpenRouter Embeddings | 0/0 | Not started | - |
183183
| 16. Twelve Labs EF | 0/0 | Not started | - |
184184
| 17. Cloud RRF/GroupBy Tests | 0/0 | Not started | - |
@@ -266,14 +266,16 @@ Plans:
266266
**Goal:** Add limit parameter support to collection delete operations, matching upstream Chroma PRs #6573/#6582.
267267
**Depends on:** None (independent)
268268
**Issues**: #439
269+
**Requirements**: [DEL-01, DEL-02, DEL-03, DEL-04, DEL-05]
269270
**Success Criteria** (what must be TRUE):
270271
1. Delete operations accept an optional limit parameter.
271272
2. HTTP transport sends the limit when specified.
272273
3. Tests cover delete-with-limit happy path and edge cases.
273-
**Plans:** 0 plans
274+
**Plans:** 2/2 plans complete
274275

275276
Plans:
276-
- [ ] TBD (run /gsd:plan-phase 14 to break down)
277+
- [x] 14-01-PLAN.md — Add Limit field to CollectionDeleteOp, ApplyToDelete to limitOption, validation, and embedded path wiring
278+
- [x] 14-02-PLAN.md — Add unit tests for option application, validation, and HTTP serialization
277279

278280
### Phase 15: OpenRouter Embeddings Compatibility
279281
**Goal:** Extend the OpenAI embedding function to support OpenRouter-specific fields (encoding_format, input_type, provider preferences) and relax model validation for provider-prefixed IDs.

.planning/STATE.md

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,14 @@
22
gsd_state_version: 1.0
33
milestone: v0.4.1
44
milestone_name: Provider-Neutral Multimodal Foundations
5-
status: "Phase 13 shipped — PR #475"
6-
stopped_at: Completed 13-02-PLAN.md
7-
last_updated: "2026-03-29T14:41:08.046Z"
5+
status: Ready to plan
6+
stopped_at: Completed 14-02-PLAN.md
7+
last_updated: "2026-03-29T18:54:40.196Z"
88
progress:
99
total_phases: 18
10-
completed_phases: 13
11-
total_plans: 29
12-
completed_plans: 29
10+
completed_phases: 14
11+
total_plans: 31
12+
completed_plans: 31
1313
---
1414

1515
# Project State
@@ -23,7 +23,7 @@ See: .planning/PROJECT.md (updated 2026-03-18)
2323

2424
## Current Position
2525

26-
Phase: 14
26+
Phase: 15
2727
Plan: Not started
2828

2929
## Performance Metrics
@@ -72,6 +72,8 @@ Plan: Not started
7272
| Phase 11 P02 | 1min | 1 tasks | 1 files |
7373
| Phase 13 P01 | 2min | 2 tasks | 5 files |
7474
| Phase 13 P02 | 2min | 2 tasks | 2 files |
75+
| Phase 14 P01 | 4min | 2 tasks | 3 files |
76+
| Phase 14 P02 | 2min | 2 tasks | 2 files |
7577

7678
## Accumulated Context
7779

@@ -157,6 +159,6 @@ None.
157159

158160
## Session
159161

160-
**Last Date:** 2026-03-28T15:25:22.491Z
161-
**Stopped At:** Completed 13-02-PLAN.md
162+
**Last Date:** 2026-03-29T18:50:49.649Z
163+
**Stopped At:** Completed 14-02-PLAN.md
162164
**Resume File:** None
Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,256 @@
1+
---
2+
phase: 14-delete-with-limit
3+
plan: 01
4+
type: execute
5+
wave: 1
6+
depends_on: []
7+
files_modified:
8+
- pkg/api/v2/options.go
9+
- pkg/api/v2/collection.go
10+
- pkg/api/v2/client_local_embedded.go
11+
autonomous: true
12+
requirements: [DEL-01, DEL-02, DEL-03, DEL-04]
13+
14+
must_haves:
15+
truths:
16+
- "WithLimit(n) can be passed to Collection.Delete alongside a where or where_document filter"
17+
- "Delete with limit but no filter returns a clear validation error before any network call"
18+
- "Delete with limit <= 0 returns ErrInvalidLimit"
19+
- "Limit serializes in the HTTP JSON body when set, omitted when nil"
20+
- "Embedded path passes limit through to EmbeddedDeleteRecordsRequest as *uint32"
21+
artifacts:
22+
- path: "pkg/api/v2/options.go"
23+
provides: "ApplyToDelete method on limitOption"
24+
contains: "func (o *limitOption) ApplyToDelete"
25+
- path: "pkg/api/v2/collection.go"
26+
provides: "Limit field on CollectionDeleteOp and validation in PrepareAndValidate"
27+
contains: "Limit *int32"
28+
- path: "pkg/api/v2/client_local_embedded.go"
29+
provides: "Limit wiring to EmbeddedDeleteRecordsRequest"
30+
contains: "deleteObject.Limit"
31+
key_links:
32+
- from: "pkg/api/v2/options.go"
33+
to: "pkg/api/v2/collection.go"
34+
via: "ApplyToDelete sets CollectionDeleteOp.Limit"
35+
pattern: "op\\.Limit = &limit"
36+
- from: "pkg/api/v2/collection.go"
37+
to: "pkg/api/v2/client_local_embedded.go"
38+
via: "PrepareAndValidate runs before embedded call; Limit passed through"
39+
pattern: "deleteObject\\.Limit"
40+
---
41+
42+
<objective>
43+
Add delete-with-limit support to the V2 collection API.
44+
45+
Purpose: Implement the limit parameter for delete operations matching upstream Chroma PRs #6573/#6582, enabling users to cap the number of records deleted by a filter-based delete.
46+
Output: Working limitOption.ApplyToDelete, CollectionDeleteOp.Limit field with validation, and embedded path wiring.
47+
</objective>
48+
49+
<execution_context>
50+
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
51+
@$HOME/.claude/get-shit-done/templates/summary.md
52+
</execution_context>
53+
54+
<context>
55+
@.planning/PROJECT.md
56+
@.planning/ROADMAP.md
57+
@.planning/STATE.md
58+
@.planning/phases/14-delete-with-limit/14-CONTEXT.md
59+
@.planning/phases/14-delete-with-limit/14-RESEARCH.md
60+
61+
<interfaces>
62+
<!-- Key types and contracts the executor needs. -->
63+
64+
From pkg/api/v2/options.go:
65+
```go
66+
type limitOption struct {
67+
limit int
68+
}
69+
70+
func WithLimit(limit int) *limitOption {
71+
return &limitOption{limit: limit}
72+
}
73+
74+
func (o *limitOption) ApplyToGet(op *CollectionGetOp) error { ... }
75+
func (o *limitOption) ApplyToSearchRequest(req *SearchRequest) error { ... }
76+
77+
// Error sentinel:
78+
var ErrInvalidLimit = errors.New("limit must be greater than 0")
79+
```
80+
81+
From pkg/api/v2/collection.go:
82+
```go
83+
type DeleteOption interface {
84+
ApplyToDelete(*CollectionDeleteOp) error
85+
}
86+
87+
type CollectionDeleteOp struct {
88+
FilterOp // Where and WhereDocument filters
89+
FilterIDOp // ID filter
90+
}
91+
92+
func (c *CollectionDeleteOp) PrepareAndValidate() error { ... }
93+
func (c *CollectionDeleteOp) MarshalJSON() ([]byte, error) { ... }
94+
```
95+
96+
From pkg/api/v2/client_local_embedded.go:
97+
```go
98+
func (c *embeddedCollection) Delete(ctx context.Context, opts ...CollectionDeleteOption) error {
99+
// ... builds deleteObject, calls PrepareAndValidate, then:
100+
return c.client.embedded.DeleteRecords(localchroma.EmbeddedDeleteRecordsRequest{
101+
CollectionID: collectionID,
102+
IDs: documentIDsToStrings(deleteObject.Ids),
103+
Where: where,
104+
WhereDocument: whereDocument,
105+
TenantID: tenantName,
106+
DatabaseName: databaseName,
107+
})
108+
}
109+
```
110+
</interfaces>
111+
</context>
112+
113+
<tasks>
114+
115+
<task type="auto">
116+
<name>Task 1: Add Limit field to CollectionDeleteOp and ApplyToDelete to limitOption</name>
117+
<read_first>
118+
- pkg/api/v2/collection.go (lines 1090-1150 for CollectionDeleteOp struct and PrepareAndValidate)
119+
- pkg/api/v2/options.go (lines 520-575 for limitOption type and existing Apply methods; lines 15-30 for option matrix comment)
120+
</read_first>
121+
<files>pkg/api/v2/collection.go, pkg/api/v2/options.go</files>
122+
<action>
123+
1. In `pkg/api/v2/collection.go`, add `Limit *int32` field to `CollectionDeleteOp` struct (per D-02):
124+
```go
125+
type CollectionDeleteOp struct {
126+
FilterOp // Where and WhereDocument filters
127+
FilterIDOp // ID filter
128+
Limit *int32 `json:"limit,omitempty"`
129+
}
130+
```
131+
132+
2. In `CollectionDeleteOp.PrepareAndValidate()` (line 1115), add limit validation AFTER the existing filter validation block (per D-05, D-06):
133+
```go
134+
if c.Limit != nil {
135+
if *c.Limit <= 0 {
136+
return errors.New("limit must be greater than 0")
137+
}
138+
if c.Where == nil && c.WhereDocument == nil {
139+
return errors.New("limit can only be specified when a where or where_document clause is provided")
140+
}
141+
}
142+
```
143+
144+
3. In `pkg/api/v2/options.go`, add `ApplyToDelete` method to `limitOption` (per D-01), after the existing `ApplyToSearchRequest` method (line 574):
145+
```go
146+
func (o *limitOption) ApplyToDelete(op *CollectionDeleteOp) error {
147+
if o.limit <= 0 {
148+
return ErrInvalidLimit
149+
}
150+
limit := int32(o.limit)
151+
op.Limit = &limit
152+
return nil
153+
}
154+
```
155+
156+
4. Update the option compatibility matrix comment in `options.go` (line 25) to show WithLimit works with Delete:
157+
```
158+
WithLimit | ✓ | | ✓ | | | ✓
159+
```
160+
161+
5. Update the `WithLimit` godoc (around line 525-531) to mention Delete support:
162+
```go
163+
// WithLimit sets the maximum number of results to return or delete.
164+
//
165+
// Works with [Collection.Get], [Collection.Search], and [Collection.Delete].
166+
// For Delete, a where or where_document filter must also be specified.
167+
// Use with [WithOffset] for pagination (Get and Search only).
168+
// The limit must be greater than 0.
169+
//
170+
// For [Collection.Query], use [WithNResults] instead.
171+
```
172+
173+
6. Add a Delete example to the `WithLimit` godoc, after the existing Pagination Example:
174+
```go
175+
// # Delete Example
176+
//
177+
// err := collection.Delete(ctx,
178+
// WithWhere(EqString("status", "archived")),
179+
// WithLimit(100),
180+
// )
181+
```
182+
</action>
183+
<verify>
184+
<automated>cd /Users/tazarov/GolandProjects/chroma-go && go build ./pkg/api/v2/...</automated>
185+
</verify>
186+
<acceptance_criteria>
187+
- pkg/api/v2/collection.go contains `Limit *int32` inside CollectionDeleteOp struct
188+
- pkg/api/v2/collection.go PrepareAndValidate contains `"limit can only be specified when a where or where_document clause is provided"`
189+
- pkg/api/v2/collection.go PrepareAndValidate contains `"limit must be greater than 0"`
190+
- pkg/api/v2/options.go contains `func (o *limitOption) ApplyToDelete(op *CollectionDeleteOp) error`
191+
- pkg/api/v2/options.go option matrix row for WithLimit shows checkmark in Delete column
192+
- `go build ./pkg/api/v2/...` exits 0
193+
</acceptance_criteria>
194+
<done>CollectionDeleteOp has Limit field with validation, limitOption implements ApplyToDelete, option matrix and godoc updated</done>
195+
</task>
196+
197+
<task type="auto">
198+
<name>Task 2: Wire limit through embedded path</name>
199+
<read_first>
200+
- pkg/api/v2/client_local_embedded.go (lines 965-1002 for embeddedCollection.Delete method)
201+
</read_first>
202+
<files>pkg/api/v2/client_local_embedded.go</files>
203+
<action>
204+
In `embeddedCollection.Delete()` (around line 994), add limit conversion before the `DeleteRecords` call and pass it through (per D-04):
205+
206+
Replace the current `return c.client.embedded.DeleteRecords(...)` block with:
207+
```go
208+
var limit *uint32
209+
if deleteObject.Limit != nil {
210+
l := uint32(*deleteObject.Limit)
211+
limit = &l
212+
}
213+
214+
return c.client.embedded.DeleteRecords(localchroma.EmbeddedDeleteRecordsRequest{
215+
CollectionID: collectionID,
216+
IDs: documentIDsToStrings(deleteObject.Ids),
217+
Where: where,
218+
WhereDocument: whereDocument,
219+
TenantID: tenantName,
220+
DatabaseName: databaseName,
221+
Limit: limit,
222+
})
223+
```
224+
225+
The int32-to-uint32 conversion is safe because PrepareAndValidate has already validated `*Limit > 0`.
226+
</action>
227+
<verify>
228+
<automated>cd /Users/tazarov/GolandProjects/chroma-go && go build ./pkg/api/v2/...</automated>
229+
</verify>
230+
<acceptance_criteria>
231+
- pkg/api/v2/client_local_embedded.go contains `deleteObject.Limit`
232+
- pkg/api/v2/client_local_embedded.go contains `uint32(*deleteObject.Limit)`
233+
- pkg/api/v2/client_local_embedded.go EmbeddedDeleteRecordsRequest struct literal includes `Limit: limit`
234+
- `go build ./pkg/api/v2/...` exits 0
235+
</acceptance_criteria>
236+
<done>Embedded delete path converts int32 limit to uint32 and passes to EmbeddedDeleteRecordsRequest</done>
237+
</task>
238+
239+
</tasks>
240+
241+
<verification>
242+
- `go build ./pkg/api/v2/...` compiles without errors
243+
- `go vet ./pkg/api/v2/...` passes
244+
</verification>
245+
246+
<success_criteria>
247+
- CollectionDeleteOp has Limit *int32 field with JSON omitempty tag
248+
- PrepareAndValidate rejects limit without filter and limit <= 0 with exact upstream error messages
249+
- limitOption.ApplyToDelete sets Limit on CollectionDeleteOp
250+
- Embedded path converts and passes limit to EmbeddedDeleteRecordsRequest
251+
- All code compiles cleanly
252+
</success_criteria>
253+
254+
<output>
255+
After completion, create `.planning/phases/14-delete-with-limit/14-01-SUMMARY.md`
256+
</output>

0 commit comments

Comments
 (0)