|
| 1 | +# People.ai Connector Example |
| 2 | + |
| 3 | +## Connector overview |
| 4 | +This example demonstrates how to extract activity and participant data from the People.ai API and load it into a destination using the Fivetran Connector SDK. |
| 5 | +The connector: |
| 6 | +- Authenticates with OAuth2 using the client credentials grant type. |
| 7 | +- Retrieves records from `/v0/public/activities` and `/v0/public/activities/{type}` endpoints. |
| 8 | +- Supports token refresh and exponential backoff on transient errors. |
| 9 | +- Upserts all records into their respective destination tables (`activity`, `participants`). |
| 10 | + |
| 11 | +Related functions in `connector.py`: |
| 12 | +`schema`, `update`, `get_page`, `sync_base_activities`, `sync_activity_type`, `get_access_token`. |
| 13 | + |
| 14 | +## Requirements |
| 15 | +- [Supported Python versions](https://github.com/fivetran/fivetran_connector_sdk/blob/main/README.md#requirements) |
| 16 | +- Operating system: |
| 17 | + - Windows: 10 or later (64-bit only) |
| 18 | + - macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64]) |
| 19 | + - Linux: Ubuntu 20.04 or later, Debian 10 or later, or Amazon Linux 2 or later (arm64 or x86_64) |
| 20 | + |
| 21 | +## Getting started |
| 22 | +Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) for setup instructions. |
| 23 | + |
| 24 | +For local testing, this example includes a `__main__` block that reads `configuration.json` and runs `connector.debug(...)`. |
| 25 | + |
| 26 | +## Features |
| 27 | +- Activity ingestion: Retrieves paginated activity data from `/v0/public/activities`. |
| 28 | +- Participants sync: Retrieves participant details from `/v0/public/activities/participants`. |
| 29 | +- Authentication: Automatically refreshes the access token when a 401 response is received. |
| 30 | +- Error handling: Retries failed requests with exponential backoff for transient 5xx and connection errors. |
| 31 | +- Schema: Defines two destination tables — `activity` and `participants`. |
| 32 | + |
| 33 | +## Configuration file |
| 34 | +The `configuration.json` file provides API credentials required for authentication. |
| 35 | + |
| 36 | +```json |
| 37 | +{ |
| 38 | + "api_key": "<YOUR_PEOPLE_AI_API_KEY>", |
| 39 | + "api_secret": "<YOUR_PEOPLE_AI_API_SECRET>" |
| 40 | +} |
| 41 | +``` |
| 42 | +- `api_key`: Your People.ai API key (client ID). |
| 43 | +- `api_secret`: Your People.ai API secret (client secret). |
| 44 | + |
| 45 | +Note: Ensure that `configuration.json` is not committed to version control. Both configuration values are required; the connector will raise an error if either is missing. |
| 46 | + |
| 47 | +## Requirements file |
| 48 | +This connector has no external dependencies and does not require a `requirements.txt` file. |
| 49 | + |
| 50 | +Note: The `fivetran_connector_sdk:latest` and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`. |
| 51 | + |
| 52 | +## Authentication |
| 53 | +- Type: OAuth2 Client Credentials |
| 54 | +- Token URL: `https://api.people.ai/auth/v1/tokens` |
| 55 | +- Headers: `Content-Type: application/x-www-form-urlencoded` |
| 56 | +- Grant Type: `client_credentials` |
| 57 | +- Access Token Header: `Authorization: Bearer <access_token>` |
| 58 | + |
| 59 | +Authentication is handled by `get_access_token`. |
| 60 | +The connector uses a reauthentication closure (`reauthenticate`) to refresh the token automatically when a `401` error occurs. |
| 61 | + |
| 62 | +## Pagination |
| 63 | +Both `/activities` and `/activities/{type}` endpoints use offset-based pagination. |
| 64 | + |
| 65 | +- The connector fetches data in pages using `limit` and `offset` query parameters. |
| 66 | +- Pagination continues until fewer than `limit` records are returned. |
| 67 | +- Each page is upserted into the destination table using `op.upsert(...)`. |
| 68 | + |
| 69 | +Functions responsible for pagination: |
| 70 | +- `get_page`: Fetches a single page with retry and reauth logic. |
| 71 | +- `sync_base_activities`: Iterates through all pages for `/activities`. |
| 72 | +- `sync_activity_type`: Iterates through all pages for `/activities/{type}` (e.g., `participants`). |
| 73 | + |
| 74 | +## Data handling |
| 75 | +- Schema definition: `schema(configuration)` defines two tables: |
| 76 | + - `activity` (primary key: `uid`) |
| 77 | + - `participants` (primary key: `uid`, `email`) |
| 78 | +- Renaming: The `subject` field (if present) is renamed to `api_subject` to avoid conflicts. |
| 79 | +- Upserts: All rows are written using `op.upsert(...)` to allow incremental updates. |
| 80 | +- Error resilience: |
| 81 | + - Retries up to five times for `5xx` and network errors with exponential backoff. |
| 82 | + - Refreshes the access token once upon a `401` error. |
| 83 | + |
| 84 | +## Error handling |
| 85 | +- 401 Unauthorized: Triggers a single reauthentication attempt using `reauth_func`. |
| 86 | +- 502–599 Server Errors: Retries the request up to 5 times, with delays increasing exponentially (`2, 4, 8, 16, 32` seconds). |
| 87 | +- Connection errors: Retries similarly to `5xx` cases. |
| 88 | +- Configuration validation: Early failure if `api_key` or `api_secret` are missing. |
| 89 | +- Logging: Uses `fivetran_connector_sdk.Logging` for all status, error, and retry messages. |
| 90 | + |
| 91 | +## Tables created |
| 92 | +This connector creates two tables, `ACTIVITY` and `PARTICIPANTS`. |
| 93 | + |
| 94 | +### `ACTIVITY` |
| 95 | +- Primary key: `uid` |
| 96 | +- Selected columns (not exhaustive): |
| 97 | + `uid`, `sub_type`, `created_at`, `activity_type`, `updated_at` |
| 98 | + |
| 99 | +### `PARTICIPANTS` |
| 100 | +- Primary key: `uid`, `email` |
| 101 | +- Selected columns (not exhaustive): |
| 102 | + `uid`, `email`, `status`, `name`, `ingested_at`, `phone_number` |
| 103 | + |
| 104 | +## Additional files |
| 105 | +- `connector.py` – Contains all core logic: `schema`, `update`, `get_page`, `sync_base_activities`, `sync_activity_type`, `get_access_token`. |
| 106 | +- `configuration.json` – Contains API credentials (`api_key`, `api_secret`). |
| 107 | +- `requirements.txt` – Lists any third-party Python libraries required (e.g., `requests`). |
| 108 | + |
| 109 | +## Additional considerations |
| 110 | +The examples provided are intended to help you effectively use Fivetran's Connector SDK. |
| 111 | +While we've tested the code, Fivetran cannot be held responsible for any unexpected or negative consequences that may arise from using these examples. |
| 112 | +For inquiries, please reach out to our Support team. |
0 commit comments