Skip to content

Conversation

teallarson
Copy link
Contributor

Overview

This PR addresses critical documentation drift between the Sonar implementation and the Airbyte documentation repository. Based on a comprehensive drift analysis performed on 2025-10-08, this update adds missing API documentation for HIGH PRIORITY features that are fully implemented in Sonar but completely undocumented.

Reference: docs-drift-analysis-2025-10-08.md (in sonar repo)

Changes Summary

New Documentation Files (3)

  1. workspaces.md - Workspace Management API

    • List workspaces with cursor-based pagination
    • Get, update, and delete workspaces
    • Workspace statistics and sync operations
    • Complete pagination guide and filtering examples
  2. authentication.md - Complete Authentication Guide

    • All three token types (Operator Bearer, Scoped, Widget)
    • Region selection (US/EU)
    • Template filtering via tags during token generation
    • Token lifecycle management and security best practices
    • 20+ code examples
  3. tags.md - Template Tag Management

    • Tag CRUD operations
    • Tag selection modes (ANY vs ALL)
    • Tagging source and connection templates
    • Tier-based access control examples

Updated Documentation Files (3)

  1. source-templates.md - Enhanced with stream management, PATCH endpoint, tag operations, source definition catalogs

  2. connection-templates.md - Added sync_on_create, non_breaking_changes_preference, cron validation, PATCH endpoint, tag operations

  3. configuring-sources.md - Added region selection, stream management, JMESPath querying, JSON Patch operations

Impact

  • 25+ previously undocumented API endpoints now documented
  • ~2,500 lines of documentation added
  • 100+ code examples
  • 12 HIGH PRIORITY documentation gaps addressed

Validation

All documentation validated against:

  • Sonar codebase at commit 350a75fe73
  • FastAPI route definitions
  • Pydantic schemas
  • Existing documentation style

Breaking Changes

None - Documentation only

This PR addresses critical documentation drift identified in the October 2025
drift analysis. The Sonar implementation has evolved significantly beyond the
current documentation, and this update adds missing API documentation for HIGH
PRIORITY features that are fully implemented but undocumented.

## New Documentation Files

### 1. workspaces.md - Workspace Management API
Complete documentation for workspace management including:
- List workspaces with cursor-based pagination
- Get, update, and delete workspaces
- Workspace statistics and sync operations
- Filtering and pagination examples

### 2. authentication.md - Complete Authentication Guide
Comprehensive authentication documentation covering:
- All three token types (Operator Bearer, Scoped, Widget)
- Region selection (US/EU)
- Template filtering via tags
- Token lifecycle management and security best practices
- Common integration patterns with code examples

### 3. tags.md - Template Tag Management
Central tag management documentation including:
- Tag CRUD operations
- Tag selection modes (ANY vs ALL)
- Tagging source and connection templates
- Filtering workflows and best practices
- Tier-based access control examples

## Updated Documentation Files

### 4. source-templates.md - Enhanced Source Template Documentation
Added missing functionality:
- PATCH endpoint for template updates
- Tag management operations
- Stream management (discover, query, add, update, delete)
- JMESPath catalog querying
- Source definition catalog operations
- JSON Patch examples

### 5. connection-templates.md - Complete Configuration Options
Added missing functionality:
- sync_on_create configuration option
- non_breaking_changes_preference enum values and behavior
- Cron expression validation endpoint
- PATCH endpoint for updates
- Tag management operations
- Multi-tenant and compliance patterns

### 6. configuring-sources.md - Advanced Source Configuration
Added missing functionality:
- Region selection in token generation
- Connection template tag filtering
- Stream management operations
- JMESPath catalog querying
- JSON Patch operations for streams
- Best practices and common workflows

## Impact

This update documents 25+ previously undocumented API endpoints and addresses
12 HIGH PRIORITY documentation gaps:

- Workspace Management API (6 endpoints)
- Stream Operations (5 endpoints)
- Template Tagging System (8 endpoints)
- Authentication details (3 token types, regions, filtering)
- Pagination documentation
- Source Definition Catalogs (6 endpoints)

## Validation

All documentation validated against:
- Sonar codebase at commit 350a75fe73
- FastAPI route definitions
- Pydantic schemas
- Existing documentation style

Reference: docs-drift-analysis-2025-10-08.md
Copy link
Contributor

github-actions bot commented Oct 8, 2025

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

Helpful Resources

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

  • /format-fix - Fixes most formatting issues.
  • /bump-version - Bumps connector versions.
    • You can specify a custom changelog by passing changelog. Example: /bump-version changelog="My cool update"
    • Leaving the changelog arg blank will auto-populate the changelog from the PR title.
  • /run-cat-tests - Runs legacy CAT tests (Connector Acceptance Tests)
  • /build-connector-images - Builds and publishes a pre-release docker image for the modified connector(s).
  • JVM connectors:
    • /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
      Example: /update-connector-cdk-version connector=destination-bigquery
    • /bump-bulk-cdk-version type=patch changelog='foo' - Bump the Bulk CDK's version. type can be major/minor/patch.
  • Python connectors:
    • /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
    • /poe source example lock - Alias for /poe connector source-example lock.
    • /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
    • /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.

📝 Edit this welcome message.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

markdownlint

[markdownlint] reported by reviewdog 🐶
MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- Use clear, descriptive names"]

- Use clear, descriptive names


[markdownlint] reported by reviewdog 🐶
MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- any - Template must have a..."]

- `any` - Template must have at least one of the specified tags


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Authentication"]

#### Authentication


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Request Example"]

#### Request Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Response Example"]

#### Response Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Authentication"]

#### Authentication


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Request Body"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Request Example"]

#### Request Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Response Example"]

#### Response Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Authentication"]

#### Authentication


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Path Parameters"]

#### Path Parameters


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Request Example"]

#### Request Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Response Example"]

#### Response Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Important Notes"]

#### Important Notes


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Authentication"]

#### Authentication


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Path Parameters"]

#### Path Parameters


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Request Body"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Request Example"]

#### Request Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Response Example"]

#### Response Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Authentication"]

#### Authentication


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Path Parameters"]

#### Path Parameters


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Request Example"]

#### Request Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Response Example"]

#### Response Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Authentication"]

#### Authentication


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Path Parameters"]

#### Path Parameters


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Request Body"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Request Example"]

#### Request Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Response Example"]

#### Response Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Authentication"]

#### Authentication


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Path Parameters"]

#### Path Parameters


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Request Example"]

#### Request Example


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "#### Response Example"]

#### Response Example


[markdownlint] reported by reviewdog 🐶
MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- Source templates: Must h..."]

- **Source templates**: Must have "pro-tier" OR "enterprise" tag (`any` mode)


[markdownlint] reported by reviewdog 🐶
MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```json"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Request Example"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Response Example"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Path Parameters"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Request Example"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Response Example"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Path Parameters"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Request Example"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Response Example"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Request Example"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Response Example"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Response Fields"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Endpoint"]


[markdownlint] reported by reviewdog 🐶
MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]


[markdownlint] reported by reviewdog 🐶
MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- Keep your local workspace ca..."]

- Keep your local workspace cache in sync with Airbyte Cloud


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Request Example"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Response Example"]


[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Response Fields"]

Copy link
Contributor

github-actions bot commented Oct 8, 2025

Deploy preview for airbyte-docs ready!

✅ Preview
https://airbyte-docs-m9ro3gr66-airbyte-growth.vercel.app

Built with commit 019d933.
This pull request is being automatically deployed with vercel-action

@teallarson
Copy link
Contributor Author

Updated the agents to run markdown linting and vale (writing style) linting.

Going to have them re-review what they wrote here (not re-run the full analysis).

- Add language specifiers to all code blocks
- Fix spacing around code blocks and lists
- Convert headings to sentence-case per style guide
- Fix terminology (e.g. to for example, Cors to CORS, saas to SaaS)
- Add 35 technical terms to Vale vocabulary
- Update markdownlint config to allow duplicate headings with siblings_only

Resolves 148 total linting errors (114 markdownlint + 34 vale)
@teallarson
Copy link
Contributor Author

@tgonzalezc5 we can decide what to do with this PR, but it contains all of the docs our agents wrote last week that are missing in the embedded docs! I think it's probably worth a review and ship, but not sure how it might conflict with anything you had in flight.

They seem mostly correct (I'm least sure about the schema/catalog things) but should probably be reviewed as though some pretty junior external contractor wrote them IMO.

Would love any feedback you have on how it did, too. We can tweak and improve the agents!

Copy link
Contributor

@tgonzalezc5 tgonzalezc5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great.

Copy link
Contributor

@girarda girarda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few things are off. I'll review and fix before we merge

@girarda girarda self-requested a review October 13, 2025 19:08
| `inactive_count` | integer | Number of workspaces with status `inactive` |
| `total_count` | integer | Total number of workspaces |

### Response fields
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[markdownlint] reported by reviewdog 🐶
MD024/no-duplicate-heading Multiple headings with the same content [Context: "### Response fields"]

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

vale

docs/ai-agents/embedded/api/tags.md|666 col 5| [Google.Headings] 'Workflow 3: feature staging' should use sentence-style capitalization.
docs/ai-agents/embedded/api/tags.md|735 col 1| [Google.Colons] ': A' should be in lowercase.
docs/ai-agents/embedded/api/tags.md|737 col 1| [Google.Colons] ': V' should be in lowercase.
docs/ai-agents/embedded/api/tags.md|748 col 1| [Google.Colons] ': A' should be in lowercase.
docs/ai-agents/embedded/api/tags.md|750 col 1| [Google.Colons] ': L' should be in lowercase.
docs/ai-agents/embedded/api/tags.md|767 col 1| [Google.Colons] ': T' should be in lowercase.
docs/ai-agents/embedded/api/tags.md|769 col 1| [Google.Colons] ': U' should be in lowercase.

```

Sources created from a deleted Source Template will stop showing up in the Widget.
When a source template is updated, all existing sources created from it will also be updated.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[Google.Will] Avoid using 'will'.


```

Sources created from a deleted source template will stop showing up in the widget.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[Google.Will] Avoid using 'will'.


- `workspace_name` is a unique identifier within your organization for each customer's tenant
- `region_id` (optional) is the region where the workspace should be created:
- US region: `645a183f-b12b-4c6e-8ad3-99e165603450` (default)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[Google.We] Try to avoid using first-person plural like 'US'.

For widget integration, see [Authentication - Widget Token](./authentication.md#widget-token).

# Creating a Source
## Creating a Source
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[Google.Headings] 'Creating a Source' should use sentence-style capitalization.


### Stream selection modes

There are three stream selection modes available:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚫 [vale] reported by reviewdog 🐶
[write-good.ThereIs] Don't start a sentence with 'There are'.


```

**Cause:** Invalid request parameters
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[Google.Colons] ': I' should be in lowercase.


**Cause:** Invalid request parameters

**Solution:** Check that all parameters match the expected format and values
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[Google.Colons] ': C' should be in lowercase.

@@ -0,0 +1,845 @@
# Template Tags
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[Google.Headings] 'Template Tags' should use sentence-style capitalization.


- Deleting a tag removes all associations with source and connection templates
- This operation cannot be undone
- Templates will remain but will no longer have this tag
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[Google.Will] Avoid using 'will'.


- Deleting a tag removes all associations with source and connection templates
- This operation cannot be undone
- Templates will remain but will no longer have this tag
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [vale] reported by reviewdog 🐶
[Google.Will] Avoid using 'will'.

@girarda girarda merged commit b50994c into master Oct 13, 2025
34 checks passed
@girarda girarda deleted the docs/sonar-api-documentation-update branch October 13, 2025 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants