Skip to content

Commit 8c3fc1b

Browse files
re-raised new connector
1 parent cd313ea commit 8c3fc1b

4 files changed

Lines changed: 594 additions & 0 deletions

File tree

people_ai/README.md

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
# People.ai Connector Example
2+
3+
## Connector overview
4+
This example demonstrates how to extract activity and participant data from the People.ai API and load it into a destination using the Fivetran Connector SDK.
5+
The connector:
6+
- Authenticates with OAuth2 using the client credentials grant type.
7+
- Retrieves records from `/v0/public/activities` and `/v0/public/activities/{type}` endpoints.
8+
- Supports token refresh and exponential backoff on transient errors.
9+
- Upserts all records into their respective destination tables (`activity`, `participants`).
10+
11+
Related functions in `connector.py`:
12+
`schema`, `update`, `get_page`, `sync_base_activities`, `sync_activity_type`, `get_access_token`.
13+
14+
## Requirements
15+
- [Supported Python versions](https://github.com/fivetran/fivetran_connector_sdk/blob/main/README.md#requirements)
16+
- Operating system:
17+
- Windows: 10 or later (64-bit only)
18+
- macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64])
19+
- Linux: Ubuntu 20.04 or later, Debian 10 or later, or Amazon Linux 2 or later (arm64 or x86_64)
20+
21+
## Getting started
22+
Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) for setup instructions.
23+
24+
For local testing, this example includes a `__main__` block that reads `configuration.json` and runs `connector.debug(...)`.
25+
26+
## Features
27+
- Activity ingestion: Retrieves paginated activity data from `/v0/public/activities`.
28+
- Participants sync: Retrieves participant details from `/v0/public/activities/participants`.
29+
- Authentication: Automatically refreshes the access token when a 401 response is received.
30+
- Error handling: Retries failed requests with exponential backoff for transient 5xx and connection errors.
31+
- Schema: Defines two destination tables — `activity` and `participants`.
32+
33+
## Configuration file
34+
The `configuration.json` file provides API credentials required for authentication.
35+
36+
```json
37+
{
38+
"api_key": "<YOUR_PEOPLE_AI_API_KEY>",
39+
"api_secret": "<YOUR_PEOPLE_AI_API_SECRET>"
40+
}
41+
```
42+
- `api_key`: Your People.ai API key (client ID).
43+
- `api_secret`: Your People.ai API secret (client secret).
44+
45+
Note: Ensure that `configuration.json` is not committed to version control. Both configuration values are required; the connector will raise an error if either is missing.
46+
47+
## Requirements file
48+
This connector has no external dependencies and does not require a `requirements.txt` file.
49+
50+
Note: The `fivetran_connector_sdk:latest` and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.
51+
52+
## Authentication
53+
- Type: OAuth2 Client Credentials
54+
- Token URL: `https://api.people.ai/auth/v1/tokens`
55+
- Headers: `Content-Type: application/x-www-form-urlencoded`
56+
- Grant Type: `client_credentials`
57+
- Access Token Header: `Authorization: Bearer <access_token>`
58+
59+
Authentication is handled by `get_access_token`.
60+
The connector uses a reauthentication closure (`reauthenticate`) to refresh the token automatically when a `401` error occurs.
61+
62+
## Pagination
63+
Both `/activities` and `/activities/{type}` endpoints use offset-based pagination.
64+
65+
- The connector fetches data in pages using `limit` and `offset` query parameters.
66+
- Pagination continues until fewer than `limit` records are returned.
67+
- Each page is upserted into the destination table using `op.upsert(...)`.
68+
69+
Functions responsible for pagination:
70+
- `get_page`: Fetches a single page with retry and reauth logic.
71+
- `sync_base_activities`: Iterates through all pages for `/activities`.
72+
- `sync_activity_type`: Iterates through all pages for `/activities/{type}` (e.g., `participants`).
73+
74+
## Data handling
75+
- Schema definition: `schema(configuration)` defines two tables:
76+
- `activity` (primary key: `uid`)
77+
- `participants` (primary key: `uid`, `email`)
78+
- Renaming: The `subject` field (if present) is renamed to `api_subject` to avoid conflicts.
79+
- Upserts: All rows are written using `op.upsert(...)` to allow incremental updates.
80+
- Error resilience:
81+
- Retries up to five times for `5xx` and network errors with exponential backoff.
82+
- Refreshes the access token once upon a `401` error.
83+
84+
## Error handling
85+
- 401 Unauthorized: Triggers a single reauthentication attempt using `reauth_func`.
86+
- 502–599 Server Errors: Retries the request up to 5 times, with delays increasing exponentially (`2, 4, 8, 16, 32` seconds).
87+
- Connection errors: Retries similarly to `5xx` cases.
88+
- Configuration validation: Early failure if `api_key` or `api_secret` are missing.
89+
- Logging: Uses `fivetran_connector_sdk.Logging` for all status, error, and retry messages.
90+
91+
## Tables created
92+
This connector creates two tables, `ACTIVITY` and `PARTICIPANTS`.
93+
94+
### `ACTIVITY`
95+
- Primary key: `uid`
96+
- Selected columns (not exhaustive):
97+
`uid`, `sub_type`, `created_at`, `activity_type`, `updated_at`
98+
99+
### `PARTICIPANTS`
100+
- Primary key: `uid`, `email`
101+
- Selected columns (not exhaustive):
102+
`uid`, `email`, `status`, `name`, `ingested_at`, `phone_number`
103+
104+
## Additional files
105+
- `connector.py` – Contains all core logic: `schema`, `update`, `get_page`, `sync_base_activities`, `sync_activity_type`, `get_access_token`.
106+
- `configuration.json` – Contains API credentials (`api_key`, `api_secret`).
107+
- `requirements.txt` – Lists any third-party Python libraries required (e.g., `requests`).
108+
109+
## Additional considerations
110+
The examples provided are intended to help you effectively use Fivetran's Connector SDK.
111+
While we've tested the code, Fivetran cannot be held responsible for any unexpected or negative consequences that may arise from using these examples.
112+
For inquiries, please reach out to our Support team.

people_ai/configuration.json

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"api_key": "<YOUR_PEOPLE_AI_API_KEY>",
3+
"api_secret": "<YOUR_PEOPLE_AI_API_SECRET>"
4+
}

0 commit comments

Comments
 (0)