Per-field population control in the Local (and REST) API #16811
MurzNN
started this conversation in
Feature Requests & Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Problem
The current
depthparameter inpayload.find()is a single integer applied uniformly to all relationship and upload fields in a document.There is no way to say "populate field A but not field B" in a single query. The only existing per-field mechanism is
maxDepthon the field config, which is a static, config-time cap — not a dynamic, per-query override.The existing
populateoption (keyed by collection slug) controls which fields to select from an already-populated document, not whether to populate a given relationship field at all.Motivating Use Case
Consider a
Classescollection with two relationship fields pointing toUsers:A list page needs to display the teacher's name and the count of students. The ideal query is:
With
depth=1, Payload populates bothteacher(desired) and every document instudents(not desired — only the count/IDs are needed). A class with 200 students triggers 200 extra database lookups and returns a large payload.The only current workarounds are:
depth=0and issue a separatepayload.findByIDfor each teacher — N+1 problem.maxDepth: 0on thestudentsfield in the collection config — permanently disables population for all queries, not just this one.depth=1and discard the student objects — wastes DB and memory.Proposed Solutions
Option A: A new
populateFieldsoption — per-field depth mapAdd a new top-level option to
find/findByID/findGlobalthat maps field paths (on the queried document) to a depth override:Or using a boolean shorthand:
Type sketch:
Where the change would live:
The population decision is made in
relationshipPopulationPromise.ts, specifically theshouldPopulatecheck:This check would also consult the
populateFieldsmap (threaded throughafterRead→promise.ts→relationshipPopulationPromise.ts) using the current field'snameor dot-notation path.The
promise.tsArgstype already carriespopulateanddepththrough the traversal:populateFieldswould be added alongsidepopulatein that args chain.Option B: Extend
selectto accept objects on relationship fieldsExtend the existing
selectoption so that a relationship field can receive an object value meaning "populate this field AND use this as the select for the populated document". Atruevalue retains existing behavior (include the field; population controlled bydepth).Why this is type-compatible today:
SelectIncludeTypeis already defined as a recursive object:An object value on a relationship field is already syntactically valid — no type changes are strictly required.
Why
sanitizeSelectis safe:sanitizeSelectalready stops recursing when it encounters arelationshiporuploadfield, so the object value is preserved as-is and not misinterpreted as a nested group/tab select at the DB level:Where the change would live:
relationshipPopulationPromisecurrently decides whether to populate based solely ondepth:And uses
populateArg?.[relatedCollection.config.slug]for the select of the populated doc:The required changes:
promise.tspasses the field-specificselectvalue (i.e.,select[field.name]) down torelationshipPopulationPromise.relationshipPopulationPromisechecks: if the field's select value is an object, forceshouldPopulate = trueand use that object as the select for the populated document (overridingpopulateArganddefaultPopulate).Semantics summary:
selectvalue for a relationship fielddepthtruedepth(existing behavior)false{ id: true, name: true }Expected Outcome
With Option B, the motivating query becomes:
Result shape:
{ "docs": [ { "id": "...", "teacher": { "id": "...", "name": "Jane Smith" }, "students": ["id1", "id2", "id3"] } ] }This eliminates unnecessary database lookups, reduces response size, and avoids N+1 patterns — all without requiring config changes or post-processing.
What do you think about the proposed options?
Beta Was this translation helpful? Give feedback.
All reactions