You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: google_trends/README.md
+14-4Lines changed: 14 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,6 +18,16 @@ The connector creates a single `google_trends` table with detailed interest scor
18
18
19
19
Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.
20
20
21
+
To initialize a new Connector SDK project using this connector as a starting point, run:
22
+
23
+
```
24
+
fivetran init --template google_trends
25
+
```
26
+
27
+
`fivetran init` initializes a new Connector SDK project by setting up the project structure, configuration files, and a connector you can run immediately with `fivetran debug`. For more information on `fivetran init`, refer to the [Connector SDK init documentation](https://fivetran.com/docs/connectors/connector-sdk/technical-reference/init).
28
+
29
+
> Note: Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.
30
+
21
31
## Features
22
32
23
33
- Full refresh sync strategy – Fetches complete timeframes on every sync to track historical changes
@@ -49,11 +59,11 @@ Configuration parameters:
49
59
-`timeframe` – Time period in Google Trends format (required)
- Absolute format: "2024-01-01 2026-02-03" (specific date range)
52
-
Note: "today" keyword in absolute ranges is automatically converted to current date
62
+
> Note: "today" keyword in absolute ranges is automatically converted to current date
53
63
54
-
You can also define the search array as a constant in `config.py`. The connector will use these defaults values if no `searches` key is provided in the `configuration,json`.
64
+
You can also define the search array as a constant in `config.py`. The connector will use these defaults values if no `searches` key is provided in the `configuration.json`.
55
65
56
-
Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
66
+
> Note: When submitting connector code as a [Community Connector](https://github.com/fivetran/fivetran-csdk-connectors/tree/main) in the open-source [Connector SDK repository](https://github.com/fivetran/fivetran-csdk-connectors/tree/main), ensure the `configuration.json` file has placeholder values. When adding the connector to your production repository, ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
57
67
58
68
## Requirements file
59
69
@@ -67,7 +77,7 @@ pytrends==4.9.2
67
77
pandas==3.0.0
68
78
```
69
79
70
-
Note: The `fivetran_connector_sdk:latest`and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
80
+
> Note: The `fivetran_connector_sdk:latest`, `requests:2.33.0`, `grpcio:1.78.0`, and `grpcio-tools:1.78.0` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
Copy file name to clipboardExpand all lines: goshippo/README.md
+12-2Lines changed: 12 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,4 @@
1
-
# Connector SDK Goshippo API Connector Example
1
+
# Goshippo Connector Example
2
2
3
3
## Connector overview
4
4
@@ -18,6 +18,16 @@ The connector fetches shipment records along with related data such as shipping
18
18
19
19
Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.
20
20
21
+
To initialize a new Connector SDK project using this connector as a starting point, run:
22
+
23
+
```
24
+
fivetran init --template goshippo
25
+
```
26
+
27
+
`fivetran init` initializes a new Connector SDK project by setting up the project structure, configuration files, and a connector you can run immediately with `fivetran debug`. For more information on `fivetran init`, refer to the [Connector SDK init documentation](https://fivetran.com/docs/connectors/connector-sdk/technical-reference/init).
28
+
29
+
> Note: Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.
30
+
21
31
## Features
22
32
23
33
- Incremental sync - Fetches only new or updated shipments since the last sync by tracking the `object_updated` timestamp
@@ -37,7 +47,7 @@ The connector requires the following configuration parameters:
37
47
}
38
48
```
39
49
40
-
Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
50
+
> Note: When submitting connector code as a [Community Connector](https://github.com/fivetran/fivetran-csdk-connectors/tree/main) in the open-source [Connector SDK repository](https://github.com/fivetran/fivetran-csdk-connectors/tree/main), ensure the `configuration.json` file has placeholder values. When adding the connector to your production repository, ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
- macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64])
13
+
- Linux: Distributions such as Ubuntu 20.04 or later, Debian 10 or later, or Amazon Linux 2 or later (arm64 or x86_64)
14
14
15
15
## Getting started
16
16
17
17
Refer to the [Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.
18
18
19
+
To initialize a new Connector SDK project using this connector as a starting point, run:
20
+
21
+
```
22
+
fivetran init --template greenplum_db
23
+
```
24
+
25
+
`fivetran init` initializes a new Connector SDK project by setting up the project structure, configuration files, and a connector you can run immediately with `fivetran debug`. For more information on `fivetran init`, refer to the [Connector SDK init documentation](https://fivetran.com/docs/connectors/connector-sdk/technical-reference/init).
26
+
27
+
> Note: Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.
28
+
19
29
## Features
20
30
21
31
- Connects to Greenplum database using `psycopg2`
@@ -28,7 +38,7 @@ Refer to the [Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/se
28
38
29
39
The connector requires the following configuration parameters to connect to your Greenplum database:
30
40
31
-
```
41
+
```json
32
42
{
33
43
"HOST": "<YOUR_GREENPLUM_DATABASE_HOST>",
34
44
"PORT": "<YOUR_GREENPLUM_DATABASE_PORT>",
@@ -38,7 +48,7 @@ The connector requires the following configuration parameters to connect to your
38
48
}
39
49
```
40
50
41
-
Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
51
+
> Note: When submitting connector code as a [Community Connector](https://github.com/fivetran/fivetran-csdk-connectors/tree/main) in the open-source [Connector SDK repository](https://github.com/fivetran/fivetran-csdk-connectors/tree/main), ensure the `configuration.json` file has placeholder values. When adding the connector to your production repository, ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
42
52
43
53
## Requirements file
44
54
@@ -48,28 +58,29 @@ The connector requires the following Python libraries:
48
58
psycopg2-binary
49
59
```
50
60
51
-
Note: The `fivetran_connector_sdk:latest`and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
61
+
> Note: The `fivetran_connector_sdk:latest`, `requests:2.33.0`, `grpcio:1.78.0`, and `grpcio-tools:1.78.0` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
52
62
53
63
## Authentication
54
64
55
65
The connector authenticates with the Greenplum database using username and password credentials provided in the configuration file. Make sure the user has appropriate permissions to read from the tables or views you intend to query.
56
66
57
67
## Data handling
58
68
59
-
The connector performs the following data handling operations:
69
+
The connector performs the following data handling operations:
60
70
- Executes SQL queries to fetch data from Greenplum (refer to `GreenplumClient.upsert_data()`)
61
71
- Uses server-side cursors to stream large results without loading everything into memory
62
72
- Converts datetime objects to ISO format for proper serialization (refer to `GreenplumClient.convert_datetime_to_iso()`)
63
73
- Checkpoints progress based on the "`query_start`" field to support incremental syncing
64
74
65
75
## Error handling
66
76
67
-
The connector includes error handling for database connection issues. In the `GreenplumClient.connect()` method, connection errors are caught and raised.
77
+
The connector includes error handling for database connection issues. In the `GreenplumClient.connect()` method, connection errors are caught and raised.
68
78
69
79
The connector also performs validation of required configuration parameters in the `schema()` function to ensure all necessary credentials are provided before attempting to connect to the database.
70
80
71
81
## Tables created
72
-
The connector creates a single table, `SAMPLE_TABLE`, in the destination:
82
+
83
+
The connector creates a single table, `SAMPLE_TABLE`, in the destination:
73
84
74
85
```json
75
86
{
@@ -84,4 +95,4 @@ The connector creates a single table, `SAMPLE_TABLE`, in the destination:
84
95
85
96
## Additional considerations
86
97
87
-
The examples provided are intended to help you effectively use Fivetran's Connector SDK. While we've tested the code, Fivetran cannot be held responsible for any unexpected or negative consequences that may arise from using these examples. For inquiries, please reach out to our Support team.
98
+
The examples provided are intended to help you effectively use Fivetran's Connector SDK. While we've tested the code, Fivetran cannot be held responsible for any unexpected or negative consequences that may arise from using these examples. For inquiries, please reach out to our Support team.
Copy file name to clipboardExpand all lines: grey_hr/README.md
+27-7Lines changed: 27 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,19 +1,33 @@
1
-
# Connector SDK GreytHR API Connector Example
1
+
# Grey HR Connector Example
2
2
3
3
## Connector overview
4
+
4
5
This connector syncs Employee, Leave Transactions, and Attendance Insights data from the greytHR API to Fivetran destinations. greytHR is a cloud-based HR and payroll management platform that provides comprehensive employee data management capabilities. This connector implements incremental syncing with proper state management and handles the greytHR API's 31-day date range limitation for Leave and Attendance endpoints through intelligent date windowing.
- macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64])
11
13
- Linux: Distributions such as Ubuntu 20.04 or later, Debian 10 or later, or Amazon Linux 2 or later (arm64 or x86_64)
12
14
13
15
## Getting started
16
+
14
17
Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.
15
18
19
+
To initialize a new Connector SDK project using this connector as a starting point, run:
20
+
21
+
```
22
+
fivetran init --template grey_hr
23
+
```
24
+
25
+
`fivetran init` initializes a new Connector SDK project by setting up the project structure, configuration files, and a connector you can run immediately with `fivetran debug`. For more information on `fivetran init`, refer to the [Connector SDK init documentation](https://fivetran.com/docs/connectors/connector-sdk/technical-reference/init).
26
+
27
+
> Note: Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.
28
+
16
29
## Features
30
+
17
31
- OAuth2 client credentials authentication for secure API access
18
32
- Automatic token refresh on expiration (401 errors) for uninterrupted syncing
19
33
- Incremental syncing for Employee data based on lastModified timestamp
@@ -37,19 +51,21 @@ Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/co
37
51
}
38
52
```
39
53
40
-
`api_username` - Your greytHR API username (client ID) obtained from greytHR API registration
41
-
`api_password` - Your greytHR API password (client secret) obtained from greytHR API registration
42
-
`greythr_domain` - Your organization's greytHR domain (e.g., moxemo6127dato.greythr.com)
43
-
`sync_start_date` - The starting date for syncing leave transactions and attendance data in YYYY-MM-DD format (e.g., 2020-01-01). If not provided, defaults to 1900-01-01
54
+
-`api_username` - Your greytHR API username (client ID) obtained from greytHR API registration
55
+
-`api_password` - Your greytHR API password (client secret) obtained from greytHR API registration
56
+
-`greythr_domain` - Your organization's greytHR domain (e.g., moxemo6127dato.greythr.com)
57
+
-`sync_start_date` - The starting date for syncing leave transactions and attendance data in YYYY-MM-DD format (e.g., 2020-01-01). If not provided, defaults to 1900-01-01
44
58
45
-
Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
59
+
> Note: When submitting connector code as a [Community Connector](https://github.com/fivetran/fivetran-csdk-connectors/tree/main) in the open-source [Connector SDK repository](https://github.com/fivetran/fivetran-csdk-connectors/tree/main), ensure the `configuration.json` file has placeholder values. When adding the connector to your production repository, ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
46
60
47
61
## Requirements file
62
+
48
63
This connector does not require any additional Python packages beyond what is pre-installed in the Fivetran environment.
49
64
50
-
Note: The `fivetran_connector_sdk:latest`and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
65
+
> Note: The `fivetran_connector_sdk:latest`, `requests:2.33.0`, `grpcio:1.78.0`, and `grpcio-tools:1.78.0` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
51
66
52
67
## Authentication
68
+
53
69
This connector uses OAuth2 client credentials flow for authentication. The authentication process is implemented in the `get_access_token()` function.
54
70
55
71
To obtain your API credentials:
@@ -61,9 +77,11 @@ To obtain your API credentials:
61
77
5. The connector automatically obtains and uses an access token for all API requests.
62
78
63
79
## Pagination
80
+
64
81
The Employee endpoint uses offset-based pagination implemented in the `sync_employees()` function. The connector fetches employees page by page with a configurable page size (default: 100 records per page). The pagination continues until the `hasNext` field in the API response is `false`, indicating all pages have been retrieved.
65
82
66
83
## Data handling
84
+
67
85
The connector processes data from three main greytHR API endpoints:
68
86
69
87
- Employee data is synced incrementally using the `modifiedSince` parameter based on the `lastModified` timestamp stored in state (refer to `sync_employees()` function).
@@ -73,6 +91,7 @@ The connector processes data from three main greytHR API endpoints:
73
91
Nested JSON objects are flattened using underscore notation (e.g., `employee.name` becomes `employee_name`). Arrays of objects are extracted into separate child tables with appropriate foreign key relationships to maintain data integrity.
74
92
75
93
## Error handling
94
+
76
95
The connector implements comprehensive error handling with exponential backoff retry logic and automatic token refresh (refer to `get_access_token()` and `make_api_request()` functions):
@@ -95,4 +114,5 @@ The connector implements comprehensive error handling with exponential backoff r
95
114
|`attendance_status`|`employee`, `sync_start_date`, `sync_end_date`, `type`| Contains attendance status counts (Present, Absent, etc.). Child table linked to `attendance_insight`. |
96
115
97
116
## Additional considerations
117
+
98
118
The examples provided are intended to help you effectively use Fivetran's Connector SDK. While we've tested the code, Fivetran cannot be held responsible for any unexpected or negative consequences that may arise from using these examples. For inquiries, please reach out to our Support team.
Copy file name to clipboardExpand all lines: gumroad/README.md
+12-2Lines changed: 12 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,6 +16,16 @@ This connector fetches sales, products, and subscriber data from the Gumroad API
16
16
17
17
Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.
18
18
19
+
To initialize a new Connector SDK project using this connector as a starting point, run:
20
+
21
+
```
22
+
fivetran init --template gumroad
23
+
```
24
+
25
+
`fivetran init` initializes a new Connector SDK project by setting up the project structure, configuration files, and a connector you can run immediately with `fivetran debug`. For more information on `fivetran init`, refer to the [Connector SDK init documentation](https://fivetran.com/docs/connectors/connector-sdk/technical-reference/init).
26
+
27
+
> Note: Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.
28
+
19
29
## Features
20
30
21
31
- Incremental syncing of sales data using date-based filtering
@@ -36,13 +46,13 @@ The connector requires the following configuration parameter:
36
46
}
37
47
```
38
48
39
-
Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
49
+
> Note: When submitting connector code as a [Community Connector](https://github.com/fivetran/fivetran-csdk-connectors/tree/main) in the open-source [Connector SDK repository](https://github.com/fivetran/fivetran-csdk-connectors/tree/main), ensure the `configuration.json` file has placeholder values. When adding the connector to your production repository, ensure that the `configuration.json` file is not checked into version control to protect sensitive information.
40
50
41
51
## Requirements file
42
52
43
53
The connector does not require any additional Python packages beyond those pre-installed in the Fivetran environment.
44
54
45
-
Note: The `fivetran_connector_sdk:latest`and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
55
+
> Note: The `fivetran_connector_sdk:latest`, `requests:2.33.0`, `grpcio:1.78.0`, and `grpcio-tools:1.78.0` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
0 commit comments