Skip to content

Commit 8e25fd7

Browse files
Alex-BairAlex-Bair
committed
docs: source-airtable-native
Documentation updates for estuary/connectors#3631. Marking `source-airtable` as deprecated since `source-airtable-native` is now the recommended way to capture data from Airtable.
1 parent 5849994 commit 8e25fd7

File tree

4 files changed

+113
-1
lines changed

4 files changed

+113
-1
lines changed

‎site/docs/reference/Connectors/capture-connectors/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,9 @@ These connectors are created by Estuary. We prioritize high-scale technology sys
1515
All Estuary connectors capture data in real time, as it appears in the source system.
1616

1717
- Airtable
18+
- [Configuration](./airtable-native.md)
19+
- Package - ghcr.io/estuary/source-airtable-native:dev
20+
- Airtable (deprecated)
1821
- [Configuration](./airtable.md)
1922
- Package - ghcr.io/estuary/source-airtable:dev
2023
- AlloyDB

‎site/docs/reference/Connectors/capture-connectors/airtable-native.md‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
# Airtable
2+
3+
This connector captures data from Airtable into Flow collections.
4+
5+
It is available for use in the Flow web application. For local development or open-source workflows, [`ghcr.io/estuary/source-airtable-native:dev`](https://ghcr.io/estuary/source-airtable-native:dev) provides the latest version of the connector as a Docker image. You can also follow the link in your browser to see past image versions.
6+
7+
## Supported data resources
8+
9+
This connector captures data from accessible bases and tables in an Airtable account. All available tables will appear after connecting to Airtable.
10+
11+
Resources are named using the pattern `{base_name}/{table_name}/{table_id}`.
12+
13+
:::info
14+
15+
If you rename a base or table in Airtable, the connector treats the renamed resource as a new resource. The connector stops populating collections under the old name and begins populating new collections with the updated name.
16+
17+
:::
18+
19+
## Prerequisites
20+
21+
### Authentication
22+
23+
There are two different ways to authenticate with Airtable when capturing data into Flow: using OAuth or an access token. Both require an active Airtable account.
24+
25+
### Setup
26+
27+
To create an access token, do the following:
28+
29+
1. Log into your Airtable account.
30+
2. In the navigation bar, click on "Builder Hub".
31+
3. In the "Developers" section, click on "Personal access tokens".
32+
4. Click on "Create token".
33+
5. Give your token a name and add the following scopes:
34+
- `data.records:read`
35+
- `data.recordComments:read`
36+
- `schema.bases:read`
37+
6. Under the "Access" section, choose the bases you want to capture data from.
38+
7. Click on "Create token".
39+
8. Copy the token for use in the connector configuration.
40+
41+
## Configuration
42+
43+
You configure connectors either in the Flow web app, or by directly editing the catalog specification file.
44+
See [connectors](../../../concepts/connectors.md#using-connectors) to learn more about using connectors. The values and specification sample below provide configuration details specific to the Airtable source connector.
45+
46+
### Properties
47+
48+
#### Endpoint
49+
50+
| Property | Title | Description | Type | Required/Default |
51+
|---|---|---|---|---|
52+
| **`/credentials/access_token`** | Access Token | The Airtable access token. | string | Required |
53+
| `/credentials/access_token_expires_at` | Access Token Expiration Datetime | The access token's expiration date and time in the format 2025-04-24T00:00:00Z. | string | Required if authenticating with OAuth |
54+
| `/credentials/client_id` | OAuth Client ID | The OAuth app's client ID. | string | Required if authenticating with OAuth |
55+
| `/credentials/client_secret` | OAuth Client Secret | The OAuth app's client secret. | string | Required if authenticating with OAuth |
56+
| **`/credentials/credentials_title`** | Authentication Method | Name of the credentials set. Set to `OAuth Credentials` if authenticating with OAuth or `Private App Credentials` if authenticating with an Airtable access token. | string | Required |
57+
| `/credentials/refresh_token` | Refresh Token | The refresh token received from the OAuth app. | string | Required if authenticating with OAuth |
58+
59+
#### Bindings
60+
61+
| Property | Title | Description | Type | Required/Default |
62+
|---|---|---|---|---|
63+
| **`/name`** | Data resource | Name of the data resource. | string | Required |
64+
| `/interval` | Interval | Interval between data syncs | string | |
65+
| `/schedule` | Formula Field Refresh Schedule | The schedule for refreshing this binding's [formula fields](#formula-fields). Accepts a cron expression. For example, a schedule of `55 23 * * *` means the binding will refresh formula fields at 23:55 UTC every day. If left empty, the binding will not refresh formula fields. | string | 55 23 * * * |
66+
67+
### Sample
68+
69+
```yaml
70+
captures:
71+
${PREFIX}/${CAPTURE_NAME}:
72+
endpoint:
73+
connector:
74+
image: ghcr.io/estuary/source-airtable-native:dev
75+
config:
76+
credentials:
77+
credentials_title: Private App Credentials
78+
access_token: <secret>
79+
bindings:
80+
- resource:
81+
name: base_name/table_name/table_id
82+
target: ${PREFIX}/base_name/table_name/table_id
83+
{...}
84+
```
85+
86+
## Incremental Replication
87+
88+
Tables that include a `lastModifiedTime` field are replicated incrementally, meaning only new and updated records are captured after the initial sync. Tables without this field are replicated using full refreshes, where all records are fetched on every sync.
89+
90+
To enable incremental replication for a table, add a `lastModifiedTime` field:
91+
92+
1. Click the **+** icon (Add field) in your table.
93+
2. Select **Last modified time** as the field type.
94+
3. Enter a name for the field.
95+
4. In the **Fields** section, select **All editable fields**.
96+
5. In the **Formatting** section, select **Include time**.
97+
6. Click **Create field**.
98+
99+
![](<../connector-images/airtable-add-last-modified-time-field.png>)
100+
101+
## Formula Fields
102+
103+
Airtable tables can contain [formula fields](https://support.airtable.com/docs/formula-field-overview), fields whose values are calculated at query time. Formula field updates do not cause `lastModifiedTime` fields to change. Since the connector uses `lastModifiedTime` fields to incrementally detect changes, formula field updates are not incrementally captured.
104+
105+
To address this challenge, the Airtable connector is able to refresh the values of formula fields on a schedule after the initial backfill completes. This is controlled at a binding level by the cron expression in the [`schedule` property](#bindings). When a scheduled formula field refresh occurs, the connector fetches every record's current formula field values and merges them into the associated collection using [`merge` reduction strategies](/reference/reduction-strategies/merge).
106+
107+
### Formula Field Errors
108+
109+
The connector skips formula fields containing [error codes](https://support.airtable.com/docs/common-formula-errors-and-how-to-fix-them) to avoid widening collections' inferred schemas and changing column data types in destinations.

‎site/docs/reference/Connectors/capture-connectors/airtable.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11

2-
# Airtable
2+
# Airtable (deprecated)
33

44
This connector captures data from Airtable into Flow collections.
55

‎site/docs/reference/Connectors/connector-images/airtable-add-last-modified-time-field.png‎

89.5 KB
Loading

0 commit comments

Comments
 (0)