docs: improve README files for batch 03 (quest_db → zigpoll)#8
Conversation
There was a problem hiding this comment.
Pull request overview
Standardizes a batch of connector example README.md files (quest_db → zigpoll) to better align with the repository’s Connector SDK README template, mainly by normalizing headings, notes, and adding the fivetran init --template <connector> Getting started block.
Changes:
- Updated Getting started sections to include
fivetran init --template <connector-name>usage and a consistent configuration reminder note. - Standardized formatting across README files (sentence-case headings,
> Note:blocks, bullet style, andjsonfenced blocks). - Added/updated template-aligned boilerplate sections such as Requirements file and Additional considerations.
Reviewed changes
Copilot reviewed 35 out of 35 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| quest_db/README.md | Adds fivetran init block and standardizes notes/bullets. |
| rabbitmq/README.md | Fixes H1 naming and adds fivetran init block + note formatting. |
| raven_db/README.md | Adds fivetran init block and improves note/list formatting. |
| redis/README.md | Adds fivetran init block and standardizes configuration/notes formatting. |
| redshift/large_data_volume/README.md | Adds fivetran init block and normalizes template sections/notes. |
| redshift/simple_redshift_connector/README.md | Adds fivetran init block and standardizes config fence + section casing. |
| redshift/using_unload/README.md | Adds fivetran init block and normalizes template sections/notes. |
| resend/README.md | Adds fivetran init block and standardizes template sections/notes. |
| rethink_db/README.md | Adds fivetran init block and standardizes template sections/notes. |
| s3_csv_validation/README.md | Adds fivetran init block and standardizes template sections/notes. |
| sam_gov/README.md | Adds fivetran init block; expands/normalizes configuration narrative and adds Requirements file section. |
| sap_ariba/README.md | Adds fivetran init block and standardizes config fencing + notes/spacing. |
| sap_hana_sql/README.md | Adds fivetran init block and standardizes notes/formatting. |
| sendcloud/README.md | Adds fivetran init block and standardizes references to connector.py and table header casing. |
| sensor_tower/README.md | Fixes H1 naming, adds fivetran init block, and standardizes sections/notes. |
| sensource/README.md | Adds fivetran init block and standardizes sections/notes and parameter label casing. |
| similarweb/README.md | Fixes H1 naming, adds fivetran init block, and standardizes lists/notes/config fencing. |
| smartsheets/README.md | Adds fivetran init block and standardizes sections/notes/config fencing. |
| snipeitapp/README.md | Adds fivetran init block and standardizes notes and table header casing. |
| solace/README.md | Removes non-template formatting, adds fivetran init block, and standardizes sections/notes. |
| sql_server/README.md | Adds fivetran init block and standardizes sections/notes. |
| status_cake/README.md | Adds fivetran init block and standardizes sections/notes and lists. |
| suitedash/README.md | Adds fivetran init block and removes non-template bolding in feature bullets/sections. |
| supabase/README.md | Adds fivetran init block and standardizes config fencing and notes/spacing. |
| talon_one/README.md | Adds fivetran init block and standardizes OS bullets/notes and heading casing. |
| teradata/README.md | Adds fivetran init block and standardizes notes/spacing and duplicated reference placement. |
| tidb/README.md | Adds fivetran init block, moves contributor to Accreditation, and standardizes notes/config fencing. |
| timescale_db/README.md | Adds fivetran init block and standardizes casing, notes, and minor formatting fixes. |
| toast/README.md | Restructures to match template sections and adds fivetran init block + standardized notes. |
| tulip_interfaces/README.md | Adds Requirements + fivetran init block and introduces Requirements file section + formatting fixes. |
| veeva_vault/basic_auth/README.md | Adds fivetran init block and standardizes quotes/notes/spacing. |
| veeva_vault/session_id_auth/README.md | Adds fivetran init block and standardizes quotes/notes/spacing. |
| vercel/README.md | Adds fivetran init block and standardizes sections/notes/spacing and heading levels. |
| yugabyte_db/README.md | Adds fivetran init block and standardizes config formatting and notes/spacing. |
| zigpoll/README.md | Adds fivetran init block and standardizes config fencing, notes, and spacing. |
fivetran-JenasVimal
force-pushed
the
readme-batch-03
branch
2 times, most recently
from
7a7c90c to
e742edb
Compare
fivetran-JenasVimal
force-pushed
the
readme-batch-03
branch
from
e742edb to
7ac4475
Compare
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
fivetran-JenasVimal
requested review from
Copilot,
fivetran-rishabhghosh and
fivetran-sahilkhirwal
| - Windows 10 or later | ||
| - macOS 13 (Ventura) or later |
There was a problem hiding this comment.
The Operating system bullets in ## Requirements must match the fixed template verbatim (including the Windows: 10 or later (64-bit only) and macOS architecture details). Please update these lines to match _template_connector/README_template.md:43-48.
| - Windows 10 or later | |
| - macOS 13 (Ventura) or later | |
| - Windows: 10 or later (64-bit only) | |
| - macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64]) |
| - Windows 10 or later | ||
| - macOS 13 (Ventura) or later |
There was a problem hiding this comment.
The Operating system bullets in ## Requirements must match the fixed template verbatim (including Windows: 10 or later (64-bit only) and the full macOS line). Please update these lines to match _template_connector/README_template.md:43-48.
| - Windows 10 or later | |
| - macOS 13 (Ventura) or later | |
| - Windows: 10 or later (64-bit only) | |
| - macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64]) |
fivetran-rishabhghosh
left a comment
fivetran-rishabhghosh
left a comment
There was a problem hiding this comment.
As discussed please point to public facing docs instead of hardcoding versions
| ## Requirements | ||
|
|
||
| - Toast API credentials: `clientId`, `clientSecret`, `userAccessType` | ||
| - Domain to connect to (e.g., `api.toasttab.com`) | ||
| - A Fernet key (`key`) for encrypting access tokens | ||
| - `initialSyncStart` ISO timestamp to define the start of sync window | ||
| - [Supported Python versions](https://github.com/fivetran/fivetran-csdk-connectors/blob/main/README.md#requirements) | ||
|
|
There was a problem hiding this comment.
The ## Requirements section is missing the required operating system bullet list (Windows/macOS/Linux) and instead lists configuration-specific items (API credentials, domain, Fernet key, initialSyncStart). To align with the README template, keep the Requirements section limited to supported Python versions + operating system list, and move connector-specific prerequisites into ## Configuration file (or Connector overview).
| ## Data handling | ||
|
|
||
| The connector performs the following actions for each key aspect: | ||
| - Authentication: Generates and caches a Toast token using the provided credentials | ||
| - Sync Loop: Runs in 30-day time chunks, paginating through endpoints | ||
| - Data Normalization: Flattens nested objects and lists using `flatten_dict`, `extract_fields`, and `stringify_lists` | ||
| - Upserts and Deletes: Emits operations using `op.upsert()` and `op.delete()` | ||
| - Sync loop: Runs in 30-day time chunks, paginating through endpoints | ||
| - Data normalization: Flattens nested objects and lists using `flatten_dict`, `extract_fields`, and `stringify_lists` | ||
| - Upserts and deletes: Emits operations using `op.upsert()` and `op.delete()` | ||
| - Checkpointing: Updates state after each window to resume seamlessly |
There was a problem hiding this comment.
This README is missing the ## Authentication and ## Pagination sections even though the connector clearly authenticates (token generation) and paginates (30-day chunks / endpoint pagination mentioned below). The Connector SDK README guidelines require these sections when applicable; please add them as dedicated sections and reference the relevant connector.py functions.
| @@ -45,7 +57,7 @@ pytz | |||
| dateutil | |||
| ``` | |||
There was a problem hiding this comment.
In ## Requirements file, the example dependency list (pytz, dateutil) doesn’t match the codebase: the connector imports dateutil.relativedelta (so the dependency should be python-dateutil), and the existing similarweb/requirements.txt uses python_dateutil==2.9.0.post0. Please update the README’s example requirements to match the actual dependency name(s) used by the connector and this repo.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 35 out of 35 changed files in this pull request and generated 6 comments.
Comments suppressed due to low confidence (1)
similarweb/README.md:65
- This reference appears to be incorrect: the Similarweb example folder doesn’t contain
test.py(it hasconnector.py). Update this to reference the relevant function/file that actually exists (for example,updateinconnector.py).
## Authentication
The connector uses an API key for authentication with Similarweb. You can obtain your API key from your Similarweb account dashboard. The key should be provided in the `configuration.json` file as the value for the `api_key` key. (See `update` function in `test.py`)
| ## Data handling | ||
|
|
||
| The connector performs the following actions for each key aspect: | ||
| - Authentication: Generates and caches a Toast token using the provided credentials | ||
| - Sync Loop: Runs in 30-day time chunks, paginating through endpoints | ||
| - Data Normalization: Flattens nested objects and lists using `flatten_dict`, `extract_fields`, and `stringify_lists` | ||
| - Upserts and Deletes: Emits operations using `op.upsert()` and `op.delete()` | ||
| - Sync loop: Runs in 30-day time chunks, paginating through endpoints | ||
| - Data normalization: Flattens nested objects and lists using `flatten_dict`, `extract_fields`, and `stringify_lists` | ||
| - Upserts and deletes: Emits operations using `op.upsert()` and `op.delete()` | ||
| - Checkpointing: Updates state after each window to resume seamlessly |
There was a problem hiding this comment.
Per the repository README template, this connector should have separate ## Authentication and ## Pagination sections. Right now authentication and pagination are only mentioned inside ## Data handling, which makes the structure inconsistent with other connector READMEs and the template.
| ## Additional considerations | ||
|
|
||
| --- | ||
| The examples provided are intended to help you effectively use Fivetran's Connector SDK. While we've tested the code, Fivetran cannot be held responsible for any unexpected or negative consequences that may arise from using these examples. For inquiries, please reach out to our Support team. | ||
|
|
||
| ## Resources | ||
|
|
||
| - [Fivetran Connector SDK Docs](https://fivetran.com/docs/connectors/connector-sdk) | ||
| - [Toast API Reference](https://doc.toasttab.com/) | ||
| - [Fernet Encryption](https://cryptography.io/en/latest/fernet/) |
There was a problem hiding this comment.
The README template expects ## Additional considerations to be the final section. ## Resources currently appears after ## Additional considerations, so either move ## Resources above ## Additional considerations or merge those links into ## Additional considerations.
| ### Phase 1: BOOTSTRAP mode (historical data load) | ||
|
|
||
| - Uses `_sequenceNumber` as primary cursor to avoid offset pagination overhead | ||
| - Filters: `_sequenceNumber > last_sequence` (and optionally `_updatedAt > sync_from_date`) | ||
| - Sorts by `_sequenceNumber` in ascending order | ||
| - Fetches records in batches of 100 until API returns fewer than 100 records | ||
| - Transitions to INCREMENTAL mode when bootstrap completes | ||
|
|
||
| ### Phase 2: INCREMENTAL mode (ongoing updates) | ||
|
|
There was a problem hiding this comment.
These subheadings include numbers and Title Case / all-caps terms (for example, "Phase 1" and "BOOTSTRAP"), which conflicts with the README style guidance (no numbers in headings; sentence case below H1). Consider renaming these headings to sentence case without numbering (for example, "Bootstrap mode (historical data load)" and "Incremental mode (ongoing updates)").
| ### Overlap window strategy | ||
|
|
||
| - Default 30-day overlap window ensures recent opportunity updates are captured | ||
| - The overlap window is configured in the code as a constant named __INCREMENTAL_WINDOW_DAYS with a value of 30 |
There was a problem hiding this comment.
The constant name __INCREMENTAL_WINDOW_DAYS should be formatted as inline code (backticks) to match the README formatting rules used elsewhere in this file for identifiers and config keys.
| - The overlap window is configured in the code as a constant named __INCREMENTAL_WINDOW_DAYS with a value of 30 | |
| - The overlap window is configured in the code as a constant named `__INCREMENTAL_WINDOW_DAYS` with a value of 30 |
| ``` | ||
| ```json | ||
| { | ||
| "auth_token": "YOUR_SENSOR_TOWER_API_TOKEN" |
There was a problem hiding this comment.
The configuration.json example uses a non-placeholder token value ("YOUR_SENSOR_TOWER_API_TOKEN"). For consistency with other examples in this repo, use an explicit placeholder format (for example, <YOUR_SENSOR_TOWER_API_TOKEN>).
| "auth_token": "YOUR_SENSOR_TOWER_API_TOKEN" | |
| "auth_token": "<YOUR_SENSOR_TOWER_API_TOKEN>" |
| ``` | ||
| ```json | ||
| { | ||
| "api_key": "YOUR_SIMILARWEB_API_KEY" |
There was a problem hiding this comment.
The configuration.json example uses a non-placeholder API key value ("YOUR_SIMILARWEB_API_KEY"). For consistency with other connectors in this repo, use an explicit placeholder format (for example, <YOUR_SIMILARWEB_API_KEY>).
| "api_key": "YOUR_SIMILARWEB_API_KEY" | |
| "api_key": "<YOUR_SIMILARWEB_API_KEY>" |
Closes RD-1190038
Description of Change
Standardizes README files for connectors #67–98 (quest_db → zigpoll) against the README template.
Covers 35 files (32 connectors +
redshift× 3 sub-implementations +veeva_vault× 2 sub-implementations).Testing
README-only changes — no connector code modified.
Checklist
fivetran debugcommand.