docs: add docs for selector, metadata

docs/concepts/metadata.md

@@ -0,0 +1,177 @@
+# Metadata
+
+Metadata in flagd provides contextual information about flags and flag sets.
+It enables rich observability, logical separation, and debugging capabilities.
+
+## Overview
+
+Flagd supports metadata at two levels:
+
+- **Flag Set-Level Metadata**: Applied to entire flag configurations
+- **Flag-Level Metadata**: Applied to individual flags  
+
+## Metadata Inheritance
+
+Flagd uses a hierarchical metadata system where flags inherit metadata from their containing flag set, with the ability to override specific values at the flag level.
+
+### Flag Set-Level Metadata
+
+The most common pattern is defining metadata at the configuration level, where all flags inherit it:
+
+```json
+{
+  "metadata": {
+    "flagSetId": "payment-service",
+    "team": "payments",
+    "version": "v1.2.0",
+    "environment": "production"
+  },
+  "flags": {
+    "checkout-flow": {
+      "state": "ENABLED",
+      "variants": {"on": true, "off": false},
+      "defaultVariant": "on"
+      // Inherits all set-level metadata
+    },
+    "payment-gateway": {
+      "state": "DISABLED", 
+      "variants": {"on": true, "off": false},
+      "defaultVariant": "off"
+      // Also inherits all set-level metadata
+    }
+  }
+}
+```
+
+### Flag-Level Overrides
+
+Individual flags can override inherited metadata or add flag-specific metadata:
+
+```json
+{
+  "metadata": {
+    "flagSetId": "payment-service",
+    "team": "payments",
+    "version": "v1.2.0"
+  },
+  "flags": {
+    "standard-feature": {
+      "state": "ENABLED",
+      "variants": {"on": true, "off": false},
+      "defaultVariant": "on"
+      // Inherits: flagSetId="payment-service", team="payments", version="v1.2.0"
+    },
+    "experimental-feature": {
+      "metadata": {
+        "flagSetId": "experiments",     // Override: different flag set
+        "owner": "research-team",       // Addition: flag-specific metadata
+        "experimental": true            // Addition: flag-specific metadata
+        // Still inherits: team="payments", version="v1.2.0"
+      },
+      "state": "DISABLED",
+      "variants": {"on": true, "off": false},
+      "defaultVariant": "off"
+    }
+  }
+}
+```
+
+### Inheritance Behavior
+
+1. **Default Inheritance**: Flags inherit all set-level metadata
+2. **Selective Override**: Flag-level metadata overrides specific inherited values
+3. **Additive Enhancement**: Flag-level metadata can add new keys not present at set level
+4. **Preserved Inheritance**: Non-overridden set-level metadata remains inherited
+
+## Metadata Reflection
+
+Metadata reflection provides transparency by echoing selector and configuration information back in API responses. This enables debugging, auditing, and verification of flag targeting.
+
+### Selector Reflection
+
+When making requests with selectors, flagd "reflects" the parsed selector information in the "top-level" `metadata` field:
+
+**Request:**
+
+```bash
+curl -H "Flagd-Selector: flagSetId=payment-service" \
+  http://localhost:8014/ofrep/v1/evaluate/flags
+```
+
+**Response includes reflected metadata:**
+
+```json
+{
+  "flags": {
+    "checkout-flow": {
+      "key": "checkout-flow",
+      "value": true,
+      "variant": "on",
+      "metadata": {
+        "flagSetId": "payment-service",
+        "team": "payments"
+      }
+    }
+  },
+  "metadata": {
+    "flagSetId": "payment-service"  // Reflected from selector
+  }
+}
+```
+
+### Configuration Reflection
+
+Flag evaluation responses include the complete merged metadata for each flag:
+
+```json
+{
+  "key": "experimental-feature",
+  "value": false,
+  "variant": "off", 
+  "metadata": {
+    "flagSetId": "experiments",       // Overridden at flag level
+    "owner": "research-team",         // Added at flag level
+    "experimental": true,             // Added at flag level
+    "team": "payments",               // Inherited from set level
+    "version": "v1.2.0"               // Inherited from set level
+  }
+}
+```
+
+## Common Metadata Fields
+
+### Standard Fields
+
+Some metadata fields are defined in the flag-definition schema for common use-cases:
+
+- **`flagSetId`**: Logical grouping identifier for selectors
+- **`version`**: Configuration or flag version
+
+### Custom Fields
+
+You can define any custom metadata fields relevant to your use case:
+
+```json
+{
+  "metadata": {
+    "flagSetId": "user-service",
+    "version": "v34",
+    "costCenter": "engineering",
+    "compliance": "pci-dss",
+    "lastReviewed": "2024-01-15",
+    "approver": "team-lead"
+  }
+}
+```
+
+## Use Cases
+
+**Debugging**: Metadata reflection shows which selectors were used and how inheritance resolved, making it easier to troubleshoot flag targeting issues.
+
+**Governance**: Track team ownership, compliance requirements, and approval workflows through custom metadata fields.
+
+**Environment Management**: Use metadata for version tracking, environment identification, and change management across deployments.
+
+**Multi-Tenancy**: Isolate tenants through flag sets and maintain tenant-specific configurations and governance.
+
+**Observability**: Metadata attributes can be used in telemetry spans and metrics, providing operational visibility into flag usage patterns and configuration context.  

docs/concepts/selectors.md

@@ -0,0 +1,126 @@
+# Selectors
+
+Selectors are query expressions that allow you to filter flag configurations from flagd's sync service. They enable providers to request only specific subsets of flags instead of receiving all flags, making flagd more efficient and flexible for complex deployments.
+
+## Overview
+
+In flagd, **selectors** provide a way to query flags based on different criteria. This is particularly powerful because flagd decouples **flag sources** from **flag sets**, allowing for more granular control over which flags are synchronized and evaluated.
+
+### Key Concepts
+
+- **Flag Source**: Where flag configuration data comes from (file, HTTP endpoint, gRPC service, etc.)
+- **Flag Set**: A logical grouping of flags identified by a `flagSetId`
+- **Selector**: A query expression that filters flags by source, flag set, or other criteria
+- **Flag Set Metadata**: The selector information is "reflected" back in response metadata for transparency
+
+## Source vs Flag Set Decoupling
+
+### Before: Tight Coupling
+
+Historically, each source provided exactly one flag set, and providers had to target specific sources:
+
+```yaml
+# Old approach - targeting a specific source
+selector: "my-flag-source.json"
+```
+
+### After: Flexible Flag Sets
+
+Now, sources and flag sets are decoupled. A single source can contain multiple flag sets, and flag sets can span multiple sources:
+
+```yaml
+# New approach - targeting a logical flag set
+selector: "flagSetId=project-42"
+```
+
+## Flag Set Configuration
+
+Flag sets are typically configured at the top level of a flag configuration, with all flags in that configuration inheriting the same `flagSetId`. This is the recommended approach for most use cases.
+
+### Set-Level Configuration
+
+The most common pattern is to set the `flagSetId` at the configuration level, where all flags inherit it:
+
+```json
+{
+  "metadata": {
+    "flagSetId": "payment-service",
+    "version": "v1.2.0"
+  },
+  "flags": {
+    "new-checkout-flow": {
+      "state": "ENABLED",
+      "variants": {
+        "on": true,
+        "off": false
+      },
+      "defaultVariant": "on"
+    },
+    "stripe-integration": {
+      "state": "DISABLED",
+      "variants": { "on": true, "off": false },
+      "defaultVariant": "off"
+    }
+  }
+}
+```
+
+In this example, both `new-checkout-flow` and `stripe-integration` flags belong to the `payment-service` flag set.
+
+### Metadata Integration
+
+Selectors work closely with flagd's metadata system. For advanced patterns like flag-level overrides of `flagSetId` or complex metadata inheritance, see the [Metadata concepts](metadata.md) section.
+
+## Metadata Reflection
+
+When you make a request with a selector, flagd "reflects" the selector information back in the response metadata for transparency and debugging. For complete details on metadata selector reflection, inheritance, and configuration patterns, see the [Metadata concepts](metadata.md) section.
+
+## Use Cases
+
+### Multi-Tenant Applications
+
+```yaml
+# Tenant A's flags
+selector: "flagSetId=tenant-a"
+
+# Tenant B's flags
+selector: "flagSetId=tenant-b"
+```
+
+### Environment Separation
+
+```yaml
+# Development environment
+selector: "flagSetId=dev-features"
+
+# Production environment
+selector: "flagSetId=prod-features"
+```
+
+### Feature Team Isolation
+
+```yaml
+# Payment team's flags
+selector: "flagSetId=payments"
+
+# User interface team's flags
+selector: "flagSetId=ui-components"
+```
+
+### Legacy Source-Based Selection
+
+```yaml
+# Still supported for backward compatibility
+selector: "source=legacy-config.json"
+```
+
+## Best Practices
+
+1. **Use Flag Sets for Logical Grouping**: Prefer `flagSetId` over `source` for new deployments
+2. **Plan Your Flag Set Strategy**: Design flag sets around logical boundaries (teams, features, environments)
+3. **Leverage Metadata**: Use metadata for debugging and auditing
+4. **Document Your Schema**: Clearly document your flag set naming conventions for your team
+
+## Migration Considerations
+
+The selector enhancement maintains full backward compatibility. See the [migration guide](../guides/migrating-to-flag-sets.md) for detailed guidance on transitioning from source-based to flag-set-based selection patterns.