feat(firestore): refactor pipeline API for uniform stage options#14322
Open
bhshkh wants to merge 3 commits intogoogleapis:mainfrom
Open
feat(firestore): refactor pipeline API for uniform stage options#14322bhshkh wants to merge 3 commits intogoogleapis:mainfrom
bhshkh wants to merge 3 commits intogoogleapis:mainfrom
Conversation
Contributor
There was a problem hiding this comment.
Code Review
This pull request refactors the Firestore Pipeline API by replacing specific settings structs with a unified options pattern using map[string]any and the StageOption interface. The changes involve updating method signatures for various pipeline stages and adding ergonomic helper functions like Fields and Accumulators. Review feedback highlights concerns regarding the integration tests, specifically the skipping of failing functional tests and the presence of commented-out test cases for features like MapKeys and ArrayFilter. These issues should be addressed to ensure the refactoring does not introduce regressions or leave the codebase with undocumented gaps in feature support.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Design decision here
Overview
This PR standardizes the API surface for the Firestore Pipeline builder in the Go SDK, ensuring idiomatic conformance and aligning closely with cross-platform design paradigms (Java/Node.js). It replaces complex stage-specific configuration structs with a universal functional option pattern and introduces typed slice constants for varied expressions.
Key Features and API Changes:
Universal
RawOptionsEscape Hatch:Introduced
RawOptionsto serve as a universal pass-through for backend options that might not be natively modeled by the Go SDK (e.g.,RawOptions{"limit": 5, "distance_field": "dist"}). This deprecates intermediate parameter structs likePipelineFindNearestOptionsacross pipeline stages.Standardized Variadic Arguments with Slices:
Methods that formerly relied on raw variadic argument unpacking (e.g.
...Ordering,...any) have been updated to accept typed slices. To preserve clean query ergonomics, we introduced domain-specific helper constructors (Orders(),Fields(),Accumulators(),Selectables()).Pipeline.Sort(Orders(Ascending(FieldOf("f")), Descending(FieldOf("__name__"))))Pipeline.Aggregate(Accumulators(Count(DocumentID).As("total")))Pipeline.Select(Fields("a", "b"))Functional Stage Options Migration:
Implemented standard functional options for specialized stage modifiers:
WithFindNearestLimit(),WithFindNearestDistanceField()WithAggregateGroups()WithUnnestIndexField()WithExplainMode()Internal Code Simplification & Testing:
baseStageremoval):Removed the
baseStagestruct embedding across our internal pipeline stages. We replaced it with explicitoptions map[string]anyfields within each stage and introduced a sharedstageOptionsToProto(options)helper. This simplifies the internal struct hierarchy, removes opaque state inherited from struct embedding, and makes configuration serialization significantly more explicit and easier to trace inside each stage'stoProto()implementation.Refactored internal
pipelineStagestructures (newAggregateStage,newFindNearestStage,newSortStage, etc.) to reliably map the newRawOptionsdown to standard*pb.Valuepayloads.AlwaysUseImplicitOrderBy:Added dedicated test cases (
TestQuery_AlwaysUseImplicitOrderBy) to explicitly verify the behavior of the existingWithAlwaysUseImplicitOrderByclient toggle, ensuring it appropriately serializes implicit inequality fields and__name__into theOrderByproto requests.Fully migrated the integration suite inside
pipeline_integration_test.goandpipeline_test.goto assert against the newly stabilizedRawOptionsand slice-builder interfaces.