Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions google_trends/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,16 @@ The connector creates a single `google_trends` table with detailed interest scor

Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.

To initialize a new Connector SDK project using this connector as a starting point, run:

```
fivetran init --template google_trends
```

`fivetran init` initializes a new Connector SDK project by setting up the project structure, configuration files, and a connector you can run immediately with `fivetran debug`. For more information on `fivetran init`, refer to the [Connector SDK `init` documentation](https://fivetran.com/docs/connector-sdk/connector-development-and-configuration/connector-sdk-commands#fivetraninit).

> Note: Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.

## Features

- Full refresh sync strategy – Fetches complete timeframes on every sync to track historical changes
Expand Down Expand Up @@ -49,11 +59,11 @@ Configuration parameters:
- `timeframe` – Time period in Google Trends format (required)
- Relative format: "today 12-m" (last 12 months), "today 3-m" (last 3 months)
- Absolute format: "2024-01-01 2026-02-03" (specific date range)
Note: "today" keyword in absolute ranges is automatically converted to current date
> Note: "today" keyword in absolute ranges is automatically converted to current date
Comment thread
fivetran-JenasVimal marked this conversation as resolved.

You can also define the search array as a constant in `config.py`. The connector will use these defaults values if no `searches` key is provided in the `configuration,json`.
You can also define the search array as a constant in `config.py`. The connector will use these default values if no `searches` key is provided in the `configuration.json`.

Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
> Note: When submitting connector code as a [Community Connector](https://github.com/fivetran/fivetran-csdk-connectors/tree/main) in the open-source [Connector SDK repository](https://github.com/fivetran/fivetran-csdk-connectors/tree/main), ensure the `configuration.json` file has placeholder values. When adding the connector to your production repository, ensure that the `configuration.json` file is not checked into version control to protect sensitive information.

## Requirements file

Expand All @@ -67,7 +77,7 @@ pytrends==4.9.2
pandas==3.0.0
```

Note: The `fivetran_connector_sdk:latest` and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
> Note: The `fivetran_connector_sdk:latest`, `requests:2.33.0`, `grpcio:1.78.0`, and `grpcio-tools:1.78.0` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.

## Authentication

Expand Down
14 changes: 12 additions & 2 deletions goshippo/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Connector SDK Goshippo API Connector Example
# Goshippo Connector Example

## Connector overview

Expand All @@ -18,6 +18,16 @@ The connector fetches shipment records along with related data such as shipping

Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.

To initialize a new Connector SDK project using this connector as a starting point, run:

```
fivetran init --template goshippo
```

`fivetran init` initializes a new Connector SDK project by setting up the project structure, configuration files, and a connector you can run immediately with `fivetran debug`. For more information on `fivetran init`, refer to the [Connector SDK `init` documentation](https://fivetran.com/docs/connector-sdk/connector-development-and-configuration/connector-sdk-commands#fivetraninit).

> Note: Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.

## Features

- Incremental sync - Fetches only new or updated shipments since the last sync by tracking the `object_updated` timestamp
Expand All @@ -37,7 +47,7 @@ The connector requires the following configuration parameters:
}
```

Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
> Note: When submitting connector code as a [Community Connector](https://github.com/fivetran/fivetran-csdk-connectors/tree/main) in the open-source [Connector SDK repository](https://github.com/fivetran/fivetran-csdk-connectors/tree/main), ensure the `configuration.json` file has placeholder values. When adding the connector to your production repository, ensure that the `configuration.json` file is not checked into version control to protect sensitive information.

## Authentication

Expand Down
35 changes: 23 additions & 12 deletions greenplum_db/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,26 @@ This connector example demonstrates how to fetch data from a Greenplum database

## Requirements

* [Supported Python versions](https://github.com/fivetran/fivetran-csdk-connectors/blob/main/README.md#requirements)
* Operating system:
* Windows: 10 or later (64-bit only)
* macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64])
* Linux: Distributions such as Ubuntu 20.04 or later, Debian 10 or later, or Amazon Linux 2 or later (arm64 or x86_64)
- [Supported Python versions](https://github.com/fivetran/fivetran-csdk-connectors/blob/main/README.md#requirements)
- Operating system:
- Windows: 10 or later (64-bit only)
- macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64])
- Linux: Distributions such as Ubuntu 20.04 or later, Debian 10 or later, or Amazon Linux 2 or later (arm64 or x86_64)

## Getting started

Refer to the [Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.

To initialize a new Connector SDK project using this connector as a starting point, run:

```
fivetran init --template greenplum_db
```

`fivetran init` initializes a new Connector SDK project by setting up the project structure, configuration files, and a connector you can run immediately with `fivetran debug`. For more information on `fivetran init`, refer to the [Connector SDK `init` documentation](https://fivetran.com/docs/connector-sdk/connector-development-and-configuration/connector-sdk-commands#fivetraninit).

> Note: Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.

## Features

- Connects to Greenplum database using `psycopg2`
Expand All @@ -28,7 +38,7 @@ Refer to the [Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/se

The connector requires the following configuration parameters to connect to your Greenplum database:

```
```json
{
"HOST": "<YOUR_GREENPLUM_DATABASE_HOST>",
"PORT": "<YOUR_GREENPLUM_DATABASE_PORT>",
Expand All @@ -38,7 +48,7 @@ The connector requires the following configuration parameters to connect to your
}
```

Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
> Note: When submitting connector code as a [Community Connector](https://github.com/fivetran/fivetran-csdk-connectors/tree/main) in the open-source [Connector SDK repository](https://github.com/fivetran/fivetran-csdk-connectors/tree/main), ensure the `configuration.json` file has placeholder values. When adding the connector to your production repository, ensure that the `configuration.json` file is not checked into version control to protect sensitive information.

## Requirements file

Expand All @@ -48,28 +58,29 @@ The connector requires the following Python libraries:
psycopg2-binary
```

Note: The `fivetran_connector_sdk:latest` and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
> Note: The `fivetran_connector_sdk:latest`, `requests:2.33.0`, `grpcio:1.78.0`, and `grpcio-tools:1.78.0` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.

## Authentication

The connector authenticates with the Greenplum database using username and password credentials provided in the configuration file. Make sure the user has appropriate permissions to read from the tables or views you intend to query.

## Data handling

The connector performs the following data handling operations:
The connector performs the following data handling operations:
- Executes SQL queries to fetch data from Greenplum (refer to `GreenplumClient.upsert_data()`)
- Uses server-side cursors to stream large results without loading everything into memory
- Converts datetime objects to ISO format for proper serialization (refer to `GreenplumClient.convert_datetime_to_iso()`)
- Checkpoints progress based on the "`query_start`" field to support incremental syncing

## Error handling

The connector includes error handling for database connection issues. In the `GreenplumClient.connect()` method, connection errors are caught and raised.
The connector includes error handling for database connection issues. In the `GreenplumClient.connect()` method, connection errors are caught and raised.

The connector also performs validation of required configuration parameters in the `schema()` function to ensure all necessary credentials are provided before attempting to connect to the database.

## Tables created
The connector creates a single table, `SAMPLE_TABLE`, in the destination:

The connector creates a single table, `SAMPLE_TABLE`, in the destination:

```json
{
Expand All @@ -84,4 +95,4 @@ The connector creates a single table, `SAMPLE_TABLE`, in the destination:

## Additional considerations

The examples provided are intended to help you effectively use Fivetran's Connector SDK. While we've tested the code, Fivetran cannot be held responsible for any unexpected or negative consequences that may arise from using these examples. For inquiries, please reach out to our Support team.
The examples provided are intended to help you effectively use Fivetran's Connector SDK. While we've tested the code, Fivetran cannot be held responsible for any unexpected or negative consequences that may arise from using these examples. For inquiries, please reach out to our Support team.
34 changes: 27 additions & 7 deletions grey_hr/README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,33 @@
# Connector SDK GreytHR API Connector Example
# Grey HR Connector Example
Comment thread
fivetran-JenasVimal marked this conversation as resolved.

## Connector overview

This connector syncs Employee, Leave Transactions, and Attendance Insights data from the greytHR API to Fivetran destinations. greytHR is a cloud-based HR and payroll management platform that provides comprehensive employee data management capabilities. This connector implements incremental syncing with proper state management and handles the greytHR API's 31-day date range limitation for Leave and Attendance endpoints through intelligent date windowing.

## Requirements

- [Supported Python versions](https://github.com/fivetran/fivetran-csdk-connectors/blob/main/README.md#requirements)
- Operating system:
- Windows: 10 or later (64-bit only)
- macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64])
- Linux: Distributions such as Ubuntu 20.04 or later, Debian 10 or later, or Amazon Linux 2 or later (arm64 or x86_64)

## Getting started

Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.

To initialize a new Connector SDK project using this connector as a starting point, run:

```
fivetran init --template grey_hr
```

`fivetran init` initializes a new Connector SDK project by setting up the project structure, configuration files, and a connector you can run immediately with `fivetran debug`. For more information on `fivetran init`, refer to the [Connector SDK `init` documentation](https://fivetran.com/docs/connector-sdk/connector-development-and-configuration/connector-sdk-commands#fivetraninit).

> Note: Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.

## Features

- OAuth2 client credentials authentication for secure API access
- Automatic token refresh on expiration (401 errors) for uninterrupted syncing
- Incremental syncing for Employee data based on lastModified timestamp
Expand All @@ -37,19 +51,21 @@ Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/co
}
```

`api_username` - Your greytHR API username (client ID) obtained from greytHR API registration
`api_password` - Your greytHR API password (client secret) obtained from greytHR API registration
`greythr_domain` - Your organization's greytHR domain (e.g., moxemo6127dato.greythr.com)
`sync_start_date` - The starting date for syncing leave transactions and attendance data in YYYY-MM-DD format (e.g., 2020-01-01). If not provided, defaults to 1900-01-01
- `api_username` - Your greytHR API username (client ID) obtained from greytHR API registration
- `api_password` - Your greytHR API password (client secret) obtained from greytHR API registration
- `greythr_domain` - Your organization's greytHR domain (e.g., moxemo6127dato.greythr.com)
- `sync_start_date` - The starting date for syncing leave transactions and attendance data in YYYY-MM-DD format (e.g., 2020-01-01). If not provided, defaults to 1900-01-01

Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
> Note: When submitting connector code as a [Community Connector](https://github.com/fivetran/fivetran-csdk-connectors/tree/main) in the open-source [Connector SDK repository](https://github.com/fivetran/fivetran-csdk-connectors/tree/main), ensure the `configuration.json` file has placeholder values. When adding the connector to your production repository, ensure that the `configuration.json` file is not checked into version control to protect sensitive information.

## Requirements file

This connector does not require any additional Python packages beyond what is pre-installed in the Fivetran environment.

Note: The `fivetran_connector_sdk:latest` and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
> Note: The `fivetran_connector_sdk:latest`, `requests:2.33.0`, `grpcio:1.78.0`, and `grpcio-tools:1.78.0` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.

## Authentication

This connector uses OAuth2 client credentials flow for authentication. The authentication process is implemented in the `get_access_token()` function.

To obtain your API credentials:
Expand All @@ -61,9 +77,11 @@ To obtain your API credentials:
5. The connector automatically obtains and uses an access token for all API requests.

## Pagination

The Employee endpoint uses offset-based pagination implemented in the `sync_employees()` function. The connector fetches employees page by page with a configurable page size (default: 100 records per page). The pagination continues until the `hasNext` field in the API response is `false`, indicating all pages have been retrieved.

## Data handling

The connector processes data from three main greytHR API endpoints:

- Employee data is synced incrementally using the `modifiedSince` parameter based on the `lastModified` timestamp stored in state (refer to `sync_employees()` function).
Expand All @@ -73,6 +91,7 @@ The connector processes data from three main greytHR API endpoints:
Nested JSON objects are flattened using underscore notation (e.g., `employee.name` becomes `employee_name`). Arrays of objects are extracted into separate child tables with appropriate foreign key relationships to maintain data integrity.

## Error handling

The connector implements comprehensive error handling with exponential backoff retry logic and automatic token refresh (refer to `get_access_token()` and `make_api_request()` functions):

- HTTP 401 authentication errors automatically trigger token refresh and request retry
Expand All @@ -95,4 +114,5 @@ The connector implements comprehensive error handling with exponential backoff r
| `attendance_status` | `employee`, `sync_start_date`, `sync_end_date`, `type` | Contains attendance status counts (Present, Absent, etc.). Child table linked to `attendance_insight`. |

## Additional considerations

The examples provided are intended to help you effectively use Fivetran's Connector SDK. While we've tested the code, Fivetran cannot be held responsible for any unexpected or negative consequences that may arise from using these examples. For inquiries, please reach out to our Support team.
14 changes: 12 additions & 2 deletions gumroad/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,16 @@ This connector fetches sales, products, and subscriber data from the Gumroad API

Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.

To initialize a new Connector SDK project using this connector as a starting point, run:

```
fivetran init --template gumroad
```

`fivetran init` initializes a new Connector SDK project by setting up the project structure, configuration files, and a connector you can run immediately with `fivetran debug`. For more information on `fivetran init`, refer to the [Connector SDK `init` documentation](https://fivetran.com/docs/connector-sdk/connector-development-and-configuration/connector-sdk-commands#fivetraninit).

> Note: Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.

## Features

- Incremental syncing of sales data using date-based filtering
Expand All @@ -36,13 +46,13 @@ The connector requires the following configuration parameter:
}
```

Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
> Note: When submitting connector code as a [Community Connector](https://github.com/fivetran/fivetran-csdk-connectors/tree/main) in the open-source [Connector SDK repository](https://github.com/fivetran/fivetran-csdk-connectors/tree/main), ensure the `configuration.json` file has placeholder values. When adding the connector to your production repository, ensure that the `configuration.json` file is not checked into version control to protect sensitive information.

## Requirements file

The connector does not require any additional Python packages beyond those pre-installed in the Fivetran environment.

Note: The `fivetran_connector_sdk:latest` and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
> Note: The `fivetran_connector_sdk:latest`, `requests:2.33.0`, `grpcio:1.78.0`, and `grpcio-tools:1.78.0` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.

## Authentication

Expand Down
Loading
Loading