diff --git a/docs/api-design.md b/docs/api-design.md new file mode 100644 index 0000000..4fe177f --- /dev/null +++ b/docs/api-design.md @@ -0,0 +1,154 @@ +# API Design + +The Search service provides APIs for searching resources and managing the +resources available to search. Platform operators can quickly use this service +to search for resources they need across the platform. + +## Motivation + +The platform supports quick and easy registration of resources though Custom +Resource Definitions and API aggregation. Given this, it's important our search +service can adjust dynamically at runtime to resources being indexed and +searchable so it doesn't require software changes to index new resources. + +## Goals + +- Introduce a dynamic method of configuring which resources in the platform + should be indexed and searchable +- Enable platform operators to quickly search across resource types while also + allowing field based selection to narrow the scope of the search +- Support full-text search capabilities configurable through dynamic index + policies + + +## Proposal + +The Search API will initially offer these new API resources to end-users: + +- **ResourceIndexPolicy** configures which resources in the platform should be + indexed and which fields within the resource are applicable to indexing +- **ResourceSearchQuery** allows users to execute field filtering and full-text + searching capabilities across all indexed resources + + +> [!IMPORTANT] +> +> The system does not support AuthZ based policies to ensure users have access +> to results returned by search queries. This API should only be used for +> non-sensitive resources and for internal-use only. +> +> This service will be made multi-tenant friendly in the future. + +## Design Details + +### Resource index policies + +The `ResourceIndexPolicy` resource will be managed by platform operators to +control which resources in the platform are made available to users. The search +service will begin indexing a policies immediately after a policy is created. + +The resource index policy will allow users to define the resource applicable to +the index policy and filter resources using CEL expressions. Index policies will +also allow users to configure which fields from the resource are indexed and how +they're indexed (field filtering, full-text search, etc.) + +Here's an example index policy that would configure search service to index all +active organizations. + +```yaml +apiVersion: search.miloapis.com/v1alpha1 +kind: ResourceIndexPolicy +metadata: + name: organizations +spec: + # Identifies the resource type this policy applies to. Uses a versioned + # reference since field paths may differ between API versions. + targetResource: + group: resourcemanager.miloapis.com + version: v1alpha1 + kind: Organization + + # CEL expressions that filter which resources are indexed. Multiple + # conditions can be specified and are evaluated with OR semantics - a + # resource is indexed if it satisfies ANY condition. Use && within a + # single expression to require multiple criteria together. + # + # Each condition has: + # - name: A unique identifier for the condition, used in status reporting + # and debugging to identify which condition(s) matched a resource. + # - expression: A CEL expression that must evaluate to a boolean. The + # resource is available as the root object in the expression context. + # + # Available CEL operations: + # - Field access: spec.replicas, metadata.name, status.phase + # - Map access: metadata.labels["app"], metadata.annotations["key"] + # - Comparisons: ==, !=, <, <=, >, >= + # - Logical operators: &&, ||, ! + # - String functions: contains(), startsWith(), endsWith(), matches() + # - List functions: exists(), all(), size(), map(), filter() + # - Membership: "value" in list, "key" in map + # - Ternary: condition ? trueValue : falseValue + conditions: + # Index resources that are ready and not being deleted + - name: active-resources + expression: | + status.conditions.exists(c, c.type == 'Ready' && c.status == 'True') + && !has(metadata.deletionTimestamp) + # Also index resources in production namespaces regardless of status + - name: production-resources + expression: metadata.namespace.startsWith("prod-") + + # Defines which fields from the resource are indexed and how they behave + # in search operations. + fields: + # The JSONPath to the field value in the resource. Supports nested paths + # and map key access using bracket notation. + - path: metadata.name + # When true, the field value is included in full-text search operations. + # The value is tokenized and analyzed for relevance-based matching. + searchable: true + # When true, the field can be used in filter expressions for exact + # matching, range queries, and other structured filtering operations. + filterable: true + # When true, the search service will return aggregated counts of unique + # values for this field. Enables clients to discover available filter + # values and build faceted navigation interfaces. + facetable: true + + - path: metadata.annotations["kubernetes.io/description"] + searchable: true + filterable: true + facetable: false +``` + +The index policy will used a versioned reference to resources since the field +paths for resources may be different between versions. The system should monitor +for deprecated resource versions being referenced in index policies. + +> [!NOTE] +> +> The `filterable` and `facteable` options are forward looking options that will +> be added to the search service in a future release. The primary functionality +> we will target for launch is **full-text searching**. + +#### Condition evaluation + +Conditions provide fine-grained control over which resource instances are +indexed. When multiple conditions are specified, they are evaluated using OR +semantics - a resource is indexed if it satisfies ANY condition. This allows +defining multiple independent criteria for inclusion. + +Use `&&` within a single CEL expression when you need AND semantics: + +```yaml +conditions: + - name: ready-in-production + expression: | + status.conditions.exists(c, c.type == 'Ready' && c.status == 'True') + && metadata.namespace.startsWith("prod-") +``` + +Conditions are re-evaluated when resources change. If a resource no longer +satisfies any condition (e.g., it transitions from Ready to NotReady), it will +be removed from the search index. Similarly, resources that begin satisfying +a condition after an update will be added to the index.