diff --git a/docs/ai-agents/embedded/api/README.md b/docs/ai-agents/embedded/api/README.md
index 40fa46f1feeb..ef150dddef7a 100644
--- a/docs/ai-agents/embedded/api/README.md
+++ b/docs/ai-agents/embedded/api/README.md
@@ -6,7 +6,7 @@ products: embedded
The Airbyte API allows you to build a fully integrated Airbyte Embedded Experience.
-## Implementation Steps
+## Quick Start
Follow these steps to implement Airbyte Embedded with the API:
@@ -23,4 +23,48 @@ For each user who wants to connect their data:
This approach separates one-time organizational setup from per-user operations, making your integration more scalable.
-The complete API reference can be found at [api.airbyte.ai/api/v1/docs](https://api.airbyte.ai/api/v1/docs).
+## Core Concepts
+
+### [Authentication](./authentication.md)
+Understand the different token types and how to securely authenticate your API calls:
+- **Access Tokens**: Organization-level administrative access
+- **Scoped Tokens**: User-level limited access for individual workspaces
+- **Widget Tokens**: Specialized tokens for the Embedded Widget
+- Security best practices and implementation examples
+
+### [Workspace Management](./workspace-management.md)
+Learn how Airbyte Embedded creates isolated environments for each customer:
+- Multi-tenant architecture and data isolation
+- Automatic workspace creation
+- External ID best practices
+- Template tag filtering for workspace-specific configurations
+
+### [Schema Discovery](./schema-discovery.md)
+Programmatically explore your customers' data structures:
+- Discover available streams (tables/collections)
+- Query field schemas and data types
+- Understand primary keys and relationships
+- Build dynamic UIs based on available data
+
+## API Guides
+
+### Setup & Configuration
+- [Connection Templates](./connection-templates.md) - Configure destinations for your users
+- [Source Templates](./source-templates.md) - Define available data connectors
+- [Configuring Sources](./configuring-sources.md) - Collect user credentials and create sources
+
+### Advanced Topics
+- [Schema Discovery](./schema-discovery.md) - Explore data structures programmatically
+- [Workspace Management](./workspace-management.md) - Manage customer isolation
+- [Authentication](./authentication.md) - Secure token management
+
+## API Reference
+
+The complete API reference with all endpoints, request/response schemas, and interactive testing is available at [api.airbyte.ai/api/v1/docs](https://api.airbyte.ai/api/v1/docs).
+
+## Need Help?
+
+- **Documentation**: Browse the guides above for detailed implementation instructions
+- **API Reference**: [api.airbyte.ai/api/v1/docs](https://api.airbyte.ai/api/v1/docs)
+- **Support**: Contact [sonar@airbyte.io](mailto:sonar@airbyte.io) for assistance
+- **Sample App**: See a complete implementation in our [embedded demo app](https://github.com/airbytehq/embedded-demoapp)
diff --git a/docs/ai-agents/embedded/api/authentication.md b/docs/ai-agents/embedded/api/authentication.md
new file mode 100644
index 000000000000..c036bed998a5
--- /dev/null
+++ b/docs/ai-agents/embedded/api/authentication.md
@@ -0,0 +1,436 @@
+---
+products: embedded
+---
+
+# Authentication
+
+Airbyte Embedded uses OAuth 2.0 Bearer tokens for API authentication. This guide explains the different token types, how to generate them, and best practices for secure implementation.
+
+## Token Types
+
+Airbyte Embedded uses three types of tokens, each with different permission levels:
+
+### 1. Access Token (Organization-Level)
+
+**Purpose:** Full administrative access to your organization
+
+**Permissions:**
+- Create and manage source templates
+- Create and manage connection templates
+- Generate scoped tokens for customers
+- Access all workspaces in your organization
+- Manage organization settings
+
+**Lifetime:** Long-lived (does not expire automatically)
+
+**When to Use:**
+- Backend server operations
+- Template management
+- Administrative tasks
+- Generating scoped tokens
+
+**Security:**
+- 🔐 **Never** expose to end-users
+- 🔐 **Never** include in frontend code
+- 🔐 **Never** commit to version control
+- ✅ **Store** securely in environment variables
+- ✅ **Use** only in trusted backend services
+
+**How to Get:**
+1. Log in to [cloud.airbyte.com](https://cloud.airbyte.com)
+2. Navigate to Settings → Applications
+3. Create or copy your API credentials
+4. Store the access token securely
+
+### 2. Scoped Token (User-Level)
+
+**Purpose:** Limited access for individual end-users
+
+**Permissions:**
+- Configure sources in a single workspace
+- View source configuration
+- Check connection status
+- Limited to one workspace (specified by external_id)
+
+**Lifetime:** Short-lived (typically 1-24 hours, configurable)
+
+**When to Use:**
+- Embedded widget authentication
+- Customer-facing operations
+- Frontend applications
+- Mobile applications
+
+**Security:**
+- ✅ **Safe** to use in frontend code
+- ✅ **Safe** to pass to end-users
+- ✅ **Automatically expires**
+- ⚠️ **Limited** to one workspace only
+
+**How to Generate:**
+
+```bash
+curl -X POST 'https://api.airbyte.ai/api/v1/embedded/scoped-token' \
+ -H 'Authorization: Bearer YOUR_ACCESS_TOKEN' \
+ -H 'Content-Type: application/json' \
+ -d '{
+ "external_id": "customer-123",
+ "selected_source_template_tags": ["production"],
+ "selected_connection_template_tags": ["production"]
+ }'
+```
+
+**Response:**
+```json
+{
+ "token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...",
+ "expires_at": "2024-10-10T19:00:00Z",
+ "workspace_id": "550e8400-e29b-41d4-a716-446655440000"
+}
+```
+
+### 3. Widget Token
+
+**Purpose:** Specialized token for the Embedded Widget UI component
+
+**Permissions:**
+- Display widget UI
+- Configure sources through widget
+- All permissions of a scoped token
+
+**Lifetime:** Short-lived (matches scoped token lifetime)
+
+**When to Use:**
+- When embedding the Airbyte Widget in your app
+- For visual source configuration UIs
+
+**How to Generate:**
+
+```bash
+curl -X POST 'https://api.airbyte.ai/api/v1/embedded/widget-token' \
+ -H 'Authorization: Bearer YOUR_ACCESS_TOKEN' \
+ -H 'Content-Type: application/json' \
+ -d '{
+ "external_id": "customer-123",
+ "selected_source_template_tags": ["production"],
+ "selected_connection_template_tags": ["production"]
+ }'
+```
+
+## Authentication Flow
+
+### Backend-to-Backend Authentication
+
+For server-to-server API calls:
+
+```
+Your Backend Airbyte API
+ | |
+ | POST /api/v1/integrations/... |
+ | Authorization: Bearer ACCESS_TOKEN |
+ |------------------------------------>|
+ | |
+ | 200 OK + Response |
+ |<------------------------------------|
+```
+
+**Example:**
+```bash
+curl -X GET 'https://api.airbyte.ai/api/v1/embedded/source-templates' \
+ -H 'Authorization: Bearer YOUR_ACCESS_TOKEN'
+```
+
+### Customer-Facing Authentication
+
+For end-user operations:
+
+```
+Your Backend Your Frontend Airbyte API
+ | | |
+ | 1. Request token | |
+ |<---------------------| |
+ | | |
+ | 2. Generate scoped | |
+ | token with | |
+ | ACCESS_TOKEN | |
+ |------------------------------------>|
+ | | |
+ | Scoped Token | |
+ |<------------------------------------|
+ | | |
+ | 3. Return token | |
+ |--------------------->| |
+ | | |
+ | | 4. API calls with |
+ | | scoped token |
+ | |--------------------->|
+ | | |
+ | | Response |
+ | |<---------------------|
+```
+
+## Implementation Examples
+
+### Backend Token Management (Node.js)
+
+```javascript
+// Store access token securely
+const AIRBYTE_ACCESS_TOKEN = process.env.AIRBYTE_ACCESS_TOKEN;
+
+// Generate scoped token for customer
+async function generateCustomerToken(customerId) {
+ const response = await fetch(
+ 'https://api.airbyte.ai/api/v1/embedded/scoped-token',
+ {
+ method: 'POST',
+ headers: {
+ 'Authorization': `Bearer ${AIRBYTE_ACCESS_TOKEN}`,
+ 'Content-Type': 'application/json',
+ },
+ body: JSON.stringify({
+ external_id: customerId,
+ selected_source_template_tags: ['production'],
+ selected_connection_template_tags: ['production'],
+ }),
+ }
+ );
+
+ const data = await response.json();
+ return {
+ token: data.token,
+ expiresAt: data.expires_at,
+ workspaceId: data.workspace_id,
+ };
+}
+```
+
+### Frontend Token Usage (React)
+
+```javascript
+// In your React component
+function SourceConfiguration({ customerId }) {
+ const [scopedToken, setScopedToken] = useState(null);
+
+ useEffect(() => {
+ // Call your backend to get scoped token
+ fetch(`/api/customer/${customerId}/airbyte-token`)
+ .then(res => res.json())
+ .then(data => setScopedToken(data.token));
+ }, [customerId]);
+
+ if (!scopedToken) return
Loading...
;
+
+ // Use scoped token for API calls
+ return (
+
+ );
+}
+```
+
+### Token Caching and Refresh
+
+```javascript
+class TokenManager {
+ constructor(accessToken) {
+ this.accessToken = accessToken;
+ this.tokenCache = new Map();
+ }
+
+ async getScopedToken(customerId) {
+ // Check cache
+ const cached = this.tokenCache.get(customerId);
+ if (cached && new Date(cached.expiresAt) > new Date()) {
+ return cached.token;
+ }
+
+ // Generate new token
+ const response = await fetch(
+ 'https://api.airbyte.ai/api/v1/embedded/scoped-token',
+ {
+ method: 'POST',
+ headers: {
+ 'Authorization': `Bearer ${this.accessToken}`,
+ 'Content-Type': 'application/json',
+ },
+ body: JSON.stringify({
+ external_id: customerId,
+ selected_source_template_tags: ['production'],
+ selected_connection_template_tags: ['production'],
+ }),
+ }
+ );
+
+ const data = await response.json();
+
+ // Cache the token
+ this.tokenCache.set(customerId, {
+ token: data.token,
+ expiresAt: data.expires_at,
+ });
+
+ return data.token;
+ }
+
+ clearExpiredTokens() {
+ const now = new Date();
+ for (const [customerId, cached] of this.tokenCache.entries()) {
+ if (new Date(cached.expiresAt) <= now) {
+ this.tokenCache.delete(customerId);
+ }
+ }
+ }
+}
+```
+
+## Security Best Practices
+
+### Token Storage
+
+**Access Tokens (Organization-Level):**
+- ✅ Store in environment variables
+- ✅ Use secrets management (AWS Secrets Manager, HashiCorp Vault)
+- ✅ Rotate periodically
+- ❌ Never commit to Git
+- ❌ Never log in plaintext
+- ❌ Never expose to frontend
+
+**Scoped Tokens (User-Level):**
+- ✅ Generate on-demand
+- ✅ Let them expire naturally
+- ✅ Safe to pass to frontend
+- ⚠️ Still avoid logging if possible
+- ⚠️ Implement rate limiting for token generation
+
+### API Call Security
+
+```javascript
+// ✅ Good: Backend generates scoped token
+app.post('/api/customer/:id/token', async (req, res) => {
+ const scopedToken = await generateScopedToken(req.params.id);
+ res.json({ token: scopedToken });
+});
+
+// ❌ Bad: Exposing access token to frontend
+app.get('/api/config', (req, res) => {
+ res.json({
+ airbyteToken: process.env.AIRBYTE_ACCESS_TOKEN // NEVER DO THIS!
+ });
+});
+```
+
+### Rate Limiting
+
+Implement rate limiting for scoped token generation:
+
+```javascript
+const rateLimit = require('express-rate-limit');
+
+const tokenLimiter = rateLimit({
+ windowMs: 15 * 60 * 1000, // 15 minutes
+ max: 100, // Limit each customer to 100 token requests per window
+ keyGenerator: (req) => req.params.customerId,
+ message: 'Too many token requests, please try again later.',
+});
+
+app.post('/api/customer/:customerId/token',
+ tokenLimiter,
+ generateTokenHandler
+);
+```
+
+### Token Validation
+
+Always validate tokens before use:
+
+```javascript
+async function validateScopedToken(token) {
+ try {
+ const response = await fetch(
+ 'https://api.airbyte.ai/api/v1/embedded/scoped-token/info',
+ {
+ headers: {
+ 'Authorization': `Bearer ${token}`,
+ },
+ }
+ );
+
+ if (!response.ok) {
+ throw new Error('Invalid token');
+ }
+
+ const info = await response.json();
+ return {
+ isValid: true,
+ workspaceId: info.workspace_id,
+ expiresAt: info.expires_at,
+ };
+ } catch (error) {
+ return { isValid: false };
+ }
+}
+```
+
+## Troubleshooting
+
+### "Unauthorized" (401) Errors
+
+**Possible causes:**
+1. Token is expired
+2. Token is invalid or malformed
+3. Token type doesn't match required permissions
+4. Token was revoked
+
+**Solutions:**
+- Check token expiration timestamp
+- Verify you're using the correct token type
+- Generate a new token
+- Ensure Bearer token format: `Authorization: Bearer `
+
+### "Forbidden" (403) Errors
+
+**Possible causes:**
+1. Scoped token trying to access different workspace
+2. Insufficient permissions for operation
+3. Resource doesn't exist in the workspace
+
+**Solutions:**
+- Verify external_id matches the target workspace
+- Check if operation is allowed with scoped token
+- Use access token for administrative operations
+- Ensure resource exists before accessing
+
+### Token Expiration Handling
+
+```javascript
+async function makeAuthenticatedRequest(url, scopedToken, customerId) {
+ let response = await fetch(url, {
+ headers: { 'Authorization': `Bearer ${scopedToken}` }
+ });
+
+ // If token expired, get new one and retry
+ if (response.status === 401) {
+ const newToken = await generateScopedToken(customerId);
+ response = await fetch(url, {
+ headers: { 'Authorization': `Bearer ${newToken}` }
+ });
+ }
+
+ return response;
+}
+```
+
+## API Reference
+
+For complete authentication API documentation:
+- [Scoped Token Generation](https://api.airbyte.ai/api/v1/docs#/Embedded/post_api_v1_embedded_scoped_token)
+- [Widget Token Generation](https://api.airbyte.ai/api/v1/docs#/Embedded/post_api_v1_embedded_widget_token)
+- [Token Info Endpoint](https://api.airbyte.ai/api/v1/docs#/Embedded/get_api_v1_embedded_scoped_token_info)
+
+## Next Steps
+
+- Implement [Workspace Management](./workspace-management.md)
+- Configure [Source Templates](./source-templates.md)
+- Set up [Connection Templates](./connection-templates.md)
+- Use the [Embedded Widget](../widget/README.md)
diff --git a/docs/ai-agents/embedded/api/schema-discovery.md b/docs/ai-agents/embedded/api/schema-discovery.md
new file mode 100644
index 000000000000..87ab8707c519
--- /dev/null
+++ b/docs/ai-agents/embedded/api/schema-discovery.md
@@ -0,0 +1,519 @@
+---
+products: embedded
+---
+
+# Schema Discovery
+
+Schema discovery allows you to programmatically explore the data structure of your customers' connected sources. This is useful for building dynamic UIs, validating data schemas, and understanding what data is available from each source.
+
+## Overview
+
+Once a customer has configured a source, you can discover:
+- **Streams**: The tables/collections/endpoints available in the source
+- **Fields**: The columns/properties within each stream
+- **Data Types**: The type of each field (string, number, date, etc.)
+- **Primary Keys**: Which fields uniquely identify records
+- **Relationships**: How streams relate to each other (for some sources)
+
+## Discovery Process
+
+### 1. Trigger Discovery
+
+Discovery is the process of connecting to the source and retrieving its schema metadata. This happens automatically when:
+- A source is first configured
+- A source is updated
+- Manual refresh is triggered
+
+To manually trigger discovery:
+
+```bash
+curl -X POST 'https://api.airbyte.ai/api/v1/integrations/sources/{source_id}/discover' \
+ -H 'Authorization: Bearer YOUR_ACCESS_TOKEN' \
+ -H 'Content-Type: application/json'
+```
+
+**Response:**
+```json
+{
+ "job_id": "discover_12345",
+ "status": "running"
+}
+```
+
+### 2. Query the Catalog
+
+Once discovery is complete, query the full catalog:
+
+```bash
+curl -X GET 'https://api.airbyte.ai/api/v1/integrations/sources/{source_id}/catalog/query' \
+ -H 'Authorization: Bearer YOUR_ACCESS_TOKEN'
+```
+
+**Response:**
+```json
+{
+ "streams": [
+ {
+ "name": "customers",
+ "json_schema": {
+ "type": "object",
+ "properties": {
+ "id": {
+ "type": "string"
+ },
+ "email": {
+ "type": "string",
+ "format": "email"
+ },
+ "created_at": {
+ "type": "string",
+ "format": "date-time"
+ },
+ "metadata": {
+ "type": "object"
+ }
+ }
+ },
+ "supported_sync_modes": ["full_refresh", "incremental"],
+ "source_defined_cursor": true,
+ "default_cursor_field": ["created_at"],
+ "source_defined_primary_key": [["id"]],
+ "namespace": "public"
+ },
+ {
+ "name": "orders",
+ "json_schema": {
+ "type": "object",
+ "properties": {
+ "id": {
+ "type": "string"
+ },
+ "customer_id": {
+ "type": "string"
+ },
+ "amount": {
+ "type": "number"
+ },
+ "status": {
+ "type": "string",
+ "enum": ["pending", "completed", "cancelled"]
+ },
+ "created_at": {
+ "type": "string",
+ "format": "date-time"
+ }
+ }
+ },
+ "supported_sync_modes": ["full_refresh", "incremental"],
+ "source_defined_cursor": true,
+ "default_cursor_field": ["created_at"],
+ "source_defined_primary_key": [["id"]]
+ }
+ ]
+}
+```
+
+## Stream Information
+
+### Available Streams
+
+Get a list of all streams without detailed schema:
+
+```bash
+curl -X GET 'https://api.airbyte.ai/api/v1/integrations/sources/{source_id}/streams' \
+ -H 'Authorization: Bearer YOUR_ACCESS_TOKEN'
+```
+
+**Response:**
+```json
+{
+ "streams": [
+ {
+ "name": "customers",
+ "namespace": "public"
+ },
+ {
+ "name": "orders",
+ "namespace": "public"
+ },
+ {
+ "name": "products",
+ "namespace": "public"
+ }
+ ]
+}
+```
+
+### Single Stream Details
+
+Get detailed information about a specific stream:
+
+```bash
+curl -X GET 'https://api.airbyte.ai/api/v1/integrations/sources/{source_id}/streams/customers' \
+ -H 'Authorization: Bearer YOUR_ACCESS_TOKEN'
+```
+
+**Response:**
+```json
+{
+ "name": "customers",
+ "namespace": "public",
+ "json_schema": {
+ "type": "object",
+ "properties": {
+ "id": { "type": "string" },
+ "email": { "type": "string" },
+ "created_at": { "type": "string", "format": "date-time" }
+ }
+ },
+ "supported_sync_modes": ["full_refresh", "incremental"],
+ "source_defined_cursor": true,
+ "default_cursor_field": ["created_at"],
+ "source_defined_primary_key": [["id"]]
+}
+```
+
+## Understanding Schema Structure
+
+### JSON Schema Format
+
+Airbyte uses [JSON Schema](https://json-schema.org/) to describe data types:
+
+| JSON Schema Type | Description | Example Values |
+|-----------------|-------------|----------------|
+| `string` | Text data | `"hello"`, `"user@example.com"` |
+| `number` | Numeric data (int or float) | `42`, `3.14` |
+| `integer` | Whole numbers only | `42`, `-7` |
+| `boolean` | True/false values | `true`, `false` |
+| `object` | Nested JSON objects | `{"key": "value"}` |
+| `array` | Lists of values | `[1, 2, 3]` |
+| `null` | Null/empty values | `null` |
+
+### Format Specifiers
+
+Some fields have additional format information:
+
+| Format | Description | Example |
+|--------|-------------|---------|
+| `date-time` | ISO 8601 timestamp | `"2024-10-09T19:00:00Z"` |
+| `date` | Date only | `"2024-10-09"` |
+| `time` | Time only | `"19:00:00"` |
+| `email` | Email address | `"user@example.com"` |
+| `uri` | URL/URI | `"https://example.com"` |
+| `uuid` | UUID identifier | `"550e8400-e29b-41d4-a716-446655440000"` |
+
+### Sync Modes
+
+Each stream supports different sync modes:
+
+**Full Refresh:**
+- Downloads all records every sync
+- Replaces previous data
+- Use for small datasets or when you need complete snapshots
+
+**Incremental:**
+- Downloads only new/changed records
+- Uses a cursor field (typically timestamp)
+- More efficient for large datasets
+
+### Primary Keys
+
+Primary keys uniquely identify records and enable:
+- Deduplication
+- Upsert operations
+- Change data capture (CDC)
+
+Primary keys are represented as an array of arrays:
+```json
+"source_defined_primary_key": [["id"]] // Single column key
+"source_defined_primary_key": [["tenant_id"], ["user_id"]] // Composite key
+```
+
+## Use Cases
+
+### Building Dynamic UI
+
+Use schema discovery to build adaptive UIs that change based on available data:
+
+```javascript
+async function buildDataExplorer(sourceId, token) {
+ // Get available streams
+ const streamsResponse = await fetch(
+ `https://api.airbyte.ai/api/v1/integrations/sources/${sourceId}/streams`,
+ { headers: { 'Authorization': `Bearer ${token}` } }
+ );
+ const { streams } = await streamsResponse.json();
+
+ // Let user select a stream
+ const selectedStream = await showStreamSelector(streams);
+
+ // Get detailed schema for selected stream
+ const schemaResponse = await fetch(
+ `https://api.airbyte.ai/api/v1/integrations/sources/${sourceId}/streams/${selectedStream.name}`,
+ { headers: { 'Authorization': `Bearer ${token}` } }
+ );
+ const schema = await schemaResponse.json();
+
+ // Build dynamic table/form based on schema
+ return buildUIFromSchema(schema);
+}
+```
+
+### Data Validation
+
+Validate that expected fields are available:
+
+```javascript
+async function validateRequiredFields(sourceId, streamName, requiredFields, token) {
+ const response = await fetch(
+ `https://api.airbyte.ai/api/v1/integrations/sources/${sourceId}/streams/${streamName}`,
+ { headers: { 'Authorization': `Bearer ${token}` } }
+ );
+ const schema = await response.json();
+
+ const availableFields = Object.keys(schema.json_schema.properties);
+ const missingFields = requiredFields.filter(
+ field => !availableFields.includes(field)
+ );
+
+ if (missingFields.length > 0) {
+ throw new Error(`Missing required fields: ${missingFields.join(', ')}`);
+ }
+
+ return true;
+}
+```
+
+### Schema Monitoring
+
+Monitor schema changes over time:
+
+```javascript
+async function detectSchemaChanges(sourceId, previousCatalog, token) {
+ const response = await fetch(
+ `https://api.airbyte.ai/api/v1/integrations/sources/${sourceId}/catalog/query`,
+ { headers: { 'Authorization': `Bearer ${token}` } }
+ );
+ const currentCatalog = await response.json();
+
+ const changes = {
+ addedStreams: [],
+ removedStreams: [],
+ modifiedStreams: [],
+ };
+
+ // Compare catalogs to detect changes
+ const previousStreamNames = new Set(previousCatalog.streams.map(s => s.name));
+ const currentStreamNames = new Set(currentCatalog.streams.map(s => s.name));
+
+ // Find added streams
+ currentCatalog.streams.forEach(stream => {
+ if (!previousStreamNames.has(stream.name)) {
+ changes.addedStreams.push(stream.name);
+ }
+ });
+
+ // Find removed streams
+ previousCatalog.streams.forEach(stream => {
+ if (!currentStreamNames.has(stream.name)) {
+ changes.removedStreams.push(stream.name);
+ }
+ });
+
+ // Find modified streams (simplified - check field count)
+ currentCatalog.streams.forEach(currentStream => {
+ const previousStream = previousCatalog.streams.find(
+ s => s.name === currentStream.name
+ );
+ if (previousStream) {
+ const currentFields = Object.keys(currentStream.json_schema.properties).length;
+ const previousFields = Object.keys(previousStream.json_schema.properties).length;
+ if (currentFields !== previousFields) {
+ changes.modifiedStreams.push(currentStream.name);
+ }
+ }
+ });
+
+ return changes;
+}
+```
+
+### AI/ML Feature Engineering
+
+Use schema information to build AI features:
+
+```javascript
+async function generateFeatureDescriptions(sourceId, streamName, token) {
+ const response = await fetch(
+ `https://api.airbyte.ai/api/v1/integrations/sources/${sourceId}/streams/${streamName}`,
+ { headers: { 'Authorization': `Bearer ${token}` } }
+ );
+ const schema = await response.json();
+
+ const features = Object.entries(schema.json_schema.properties).map(
+ ([fieldName, fieldSchema]) => ({
+ name: fieldName,
+ type: fieldSchema.type,
+ format: fieldSchema.format,
+ isNumeric: ['number', 'integer'].includes(fieldSchema.type),
+ isTimestamp: fieldSchema.format === 'date-time',
+ isPrimaryKey: schema.source_defined_primary_key?.some(
+ key => key[0] === fieldName
+ ),
+ })
+ );
+
+ return features;
+}
+```
+
+## Best Practices
+
+### Caching
+
+Schema discovery can be slow for large sources. Cache results:
+
+```javascript
+class SchemaCache {
+ constructor(ttlMinutes = 60) {
+ this.cache = new Map();
+ this.ttl = ttlMinutes * 60 * 1000;
+ }
+
+ async getSchema(sourceId, streamName, token) {
+ const cacheKey = `${sourceId}:${streamName}`;
+ const cached = this.cache.get(cacheKey);
+
+ if (cached && Date.now() - cached.timestamp < this.ttl) {
+ return cached.schema;
+ }
+
+ const response = await fetch(
+ `https://api.airbyte.ai/api/v1/integrations/sources/${sourceId}/streams/${streamName}`,
+ { headers: { 'Authorization': `Bearer ${token}` } }
+ );
+ const schema = await response.json();
+
+ this.cache.set(cacheKey, {
+ schema,
+ timestamp: Date.now(),
+ });
+
+ return schema;
+ }
+}
+```
+
+### Error Handling
+
+Discovery can fail for various reasons:
+
+```javascript
+async function discoverSourceSchema(sourceId, token) {
+ try {
+ // Trigger discovery
+ const discoverResponse = await fetch(
+ `https://api.airbyte.ai/api/v1/integrations/sources/${sourceId}/discover`,
+ {
+ method: 'POST',
+ headers: { 'Authorization': `Bearer ${token}` },
+ }
+ );
+
+ if (!discoverResponse.ok) {
+ throw new Error(`Discovery failed: ${discoverResponse.statusText}`);
+ }
+
+ // Poll for completion (simplified - use proper polling with backoff)
+ await new Promise(resolve => setTimeout(resolve, 5000));
+
+ // Get catalog
+ const catalogResponse = await fetch(
+ `https://api.airbyte.ai/api/v1/integrations/sources/${sourceId}/catalog/query`,
+ { headers: { 'Authorization': `Bearer ${token}` } }
+ );
+
+ if (!catalogResponse.ok) {
+ throw new Error(`Catalog query failed: ${catalogResponse.statusText}`);
+ }
+
+ return await catalogResponse.json();
+ } catch (error) {
+ console.error('Schema discovery failed:', error);
+ throw error;
+ }
+}
+```
+
+### Performance
+
+For sources with many streams, use pagination and filtering:
+
+```javascript
+// Get only specific streams
+async function getRelevantStreams(sourceId, relevantStreamNames, token) {
+ const catalog = await fetch(
+ `https://api.airbyte.ai/api/v1/integrations/sources/${sourceId}/catalog/query`,
+ { headers: { 'Authorization': `Bearer ${token}` } }
+ ).then(r => r.json());
+
+ return catalog.streams.filter(stream =>
+ relevantStreamNames.includes(stream.name)
+ );
+}
+```
+
+## Troubleshooting
+
+### Discovery Takes Too Long
+
+**Possible causes:**
+- Source has many tables/collections
+- Source connection is slow
+- Source requires complex authentication
+
+**Solutions:**
+- Be patient - first discovery can take several minutes
+- Cache results to avoid repeated discoveries
+- Check source connection health
+- Contact support if discovery consistently times out
+
+### Missing Streams
+
+**Possible causes:**
+- User doesn't have permissions in the source system
+- Streams are filtered by source configuration
+- Schema hasn't been discovered yet
+
+**Solutions:**
+- Verify user permissions in the source system
+- Check source configuration settings
+- Trigger manual discovery
+- Wait for initial discovery to complete
+
+### Incorrect Field Types
+
+**Possible causes:**
+- Source doesn't provide type information
+- Airbyte inferred types from sample data
+- Type information is ambiguous
+
+**Solutions:**
+- Review the source's native schema
+- Use format specifiers for additional context
+- Validate data at runtime
+- Contact support if types are consistently wrong
+
+## API Reference
+
+- [Discover Source](https://api.airbyte.ai/api/v1/docs#/Integrations/post_api_v1_integrations_sources__id__discover)
+- [Query Catalog](https://api.airbyte.ai/api/v1/docs#/Integrations/get_api_v1_integrations_sources__id__catalog_query)
+- [List Streams](https://api.airbyte.ai/api/v1/docs#/Integrations/get_api_v1_integrations_sources__id__streams)
+- [Get Stream Details](https://api.airbyte.ai/api/v1/docs#/Integrations/get_api_v1_integrations_sources__id__streams__stream_name_)
+
+## Next Steps
+
+- Configure [Source Templates](./source-templates.md)
+- Set up [Connection Templates](./connection-templates.md)
+- Manage [Workspaces](./workspace-management.md)
+- Implement [Authentication](./authentication.md)
diff --git a/docs/ai-agents/embedded/api/workspace-management.md b/docs/ai-agents/embedded/api/workspace-management.md
new file mode 100644
index 000000000000..10309e8551f5
--- /dev/null
+++ b/docs/ai-agents/embedded/api/workspace-management.md
@@ -0,0 +1,224 @@
+---
+products: embedded
+---
+
+# Workspace Management
+
+Airbyte Embedded creates isolated workspaces for each of your customers, providing secure multi-tenant data pipelines. This guide explains how workspaces work and how to manage them effectively.
+
+## What is a Workspace?
+
+A workspace is an isolated environment within Airbyte Embedded where a single customer's:
+- Data sources are configured
+- Data destinations are defined
+- Data pipelines execute
+- Connection settings are stored
+
+Each workspace is completely isolated from others through Row Level Security (RLS) in the database, ensuring that customer data never crosses boundaries.
+
+## Workspace Architecture
+
+### Multi-Tenant Isolation
+
+Airbyte Embedded uses a multi-tenant architecture where:
+
+1. **Your Organization**: The top-level entity that represents your company
+2. **Customer Workspaces**: Individual workspaces created for each of your end-users
+3. **Row Level Security**: Database-level security that enforces data isolation
+
+```
+Your Organization
+├── Customer Workspace A (email: customer-a@example.com)
+│ ├── Source: Stripe
+│ ├── Destination: Your S3 Bucket
+│ └── Connection: Stripe → S3
+├── Customer Workspace B (email: customer-b@example.com)
+│ ├── Source: Salesforce
+│ ├── Destination: Your S3 Bucket
+│ └── Connection: Salesforce → S3
+└── Customer Workspace C (email: customer-c@example.com)
+ ├── Source: PostgreSQL
+ ├── Destination: Your S3 Bucket
+ └── Connection: PostgreSQL → S3
+```
+
+## Creating Workspaces
+
+Workspaces are automatically created when you generate a scoped token for a new customer. You don't need to explicitly create workspaces - they're created on-demand.
+
+### Automatic Workspace Creation
+
+When you call the scoped token endpoint with a new `external_id`:
+
+```bash
+curl -X POST 'https://api.airbyte.ai/api/v1/embedded/scoped-token' \
+ -H 'Authorization: Bearer YOUR_ACCESS_TOKEN' \
+ -H 'Content-Type: application/json' \
+ -d '{
+ "external_id": "customer-unique-id-123",
+ "selected_source_template_tags": ["production"],
+ "selected_connection_template_tags": ["production"]
+ }'
+```
+
+This will:
+1. Create a new workspace if one doesn't exist for `external_id: customer-unique-id-123`
+2. Return a scoped token that only has access to that workspace
+3. Associate the specified template tags with that workspace
+
+### External IDs
+
+The `external_id` is your customer's unique identifier in your system. Choose an identifier that:
+
+- Is unique per customer
+- Is stable (doesn't change)
+- Is not personally identifiable (for security)
+
+**Good examples:**
+- UUID: `"550e8400-e29b-41d4-a716-446655440000"`
+- Internal customer ID: `"cust_abc123xyz"`
+- Hashed identifier: `"hash_of_customer_email"`
+
+**Bad examples:**
+- Email addresses (not stable if customer changes email)
+- Names (not unique, can change)
+- Temporary session IDs (not stable)
+
+## Managing Workspace Access
+
+### Scoped Tokens
+
+Scoped tokens provide temporary, restricted access to a specific workspace. This follows the principle of least privilege.
+
+**Token Characteristics:**
+- Expires after a configurable time period
+- Only grants access to one workspace
+- Can only perform specific operations (create/update sources, connections)
+- Cannot access other workspaces or organization settings
+
+### Organization-Level Access
+
+Your organization's access token has full access to:
+- All workspaces in your organization
+- Template management
+- Organization settings
+
+**Security Note:** Never expose your organization access token to end-users. Always generate scoped tokens for customer interactions.
+
+## Viewing Workspaces
+
+As the organization owner, you can view all customer workspaces in your Airbyte Cloud dashboard:
+
+1. Log in to [cloud.airbyte.com](https://cloud.airbyte.com)
+2. Navigate to the workspace selector
+3. See all workspaces created via Embedded
+
+Each workspace will show:
+- The external ID you assigned
+- All sources configured by that customer
+- All connections and their sync status
+- Data pipeline performance metrics
+
+
+
+## Template Tags and Workspace Filtering
+
+Template tags allow you to control which sources and destinations are available in specific workspaces.
+
+### Use Cases
+
+**Environment-Based Filtering:**
+```json
+{
+ "external_id": "prod-customer-123",
+ "selected_source_template_tags": ["production"],
+ "selected_connection_template_tags": ["production"]
+}
+```
+
+**Tier-Based Access:**
+```json
+{
+ "external_id": "premium-customer-456",
+ "selected_source_template_tags": ["premium", "enterprise"],
+ "selected_connection_template_tags": ["premium"]
+}
+```
+
+**Customer-Specific Configuration:**
+```json
+{
+ "external_id": "customer-789",
+ "selected_source_template_tags": ["customer-789-specific"],
+ "selected_connection_template_tags": ["default"]
+}
+```
+
+Learn more in the [Template Tags guide](../widget/template-tags.md).
+
+## Best Practices
+
+### Security
+
+1. **Never reuse external IDs** across different customers
+2. **Use scoped tokens** for all customer-facing operations
+3. **Rotate access tokens** regularly
+4. **Monitor workspace access** through audit logs
+
+### Scalability
+
+1. **Use UUIDs as external IDs** for best scalability
+2. **Implement token caching** to reduce API calls
+3. **Handle token expiration** gracefully with refresh logic
+4. **Monitor workspace creation rate** to detect unusual activity
+
+### Organization
+
+1. **Use template tags** to organize workspace permissions
+2. **Document your external ID scheme** for your team
+3. **Implement workspace cleanup** for deleted customers
+4. **Monitor workspace health** through Airbyte's dashboard
+
+## Troubleshooting
+
+### Workspace Not Found
+
+If you get a "workspace not found" error:
+
+1. Verify the external ID matches exactly (case-sensitive)
+2. Ensure the scoped token hasn't expired
+3. Check that the workspace was created successfully
+4. Verify you're using the correct organization
+
+### Permission Denied
+
+If operations fail with permission errors:
+
+1. Verify you're using a scoped token (not org token) for customer operations
+2. Check that the scoped token hasn't expired
+3. Ensure the token was generated for the correct external ID
+4. Verify template tags are configured correctly
+
+### Multiple Workspaces for Same Customer
+
+If you accidentally create multiple workspaces:
+
+1. Ensure your external ID generation is consistent
+2. Check for race conditions in workspace creation
+3. Implement idempotency in your token generation
+4. Contact support to merge or clean up workspaces
+
+## API Reference
+
+For complete API documentation, see:
+- [Scoped Token Generation](./configuring-sources.md#generating-scoped-tokens)
+- [Source Templates](./source-templates.md)
+- [Connection Templates](./connection-templates.md)
+- [Full API Reference](https://api.airbyte.ai/api/v1/docs)
+
+## Next Steps
+
+- Learn about [Source Templates](./source-templates.md)
+- Understand [Connection Templates](./connection-templates.md)
+- Implement [Source Configuration](./configuring-sources.md)
+- Explore the [Embedded Widget](../widget/README.md)