Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions website/docs/components/data-connectors/abfs.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The Azure BlobFS (ABFS) Data Connector enables federated SQL queries on files st

When a folder path is provided, all the contained files will be loaded.

File formats are specified using the `file_format` parameter, as described in [Object Store File Formats](/docs/components/data-connectors/index.md#object-store-file-formats).
File formats are specified using the `file_format` parameter, as described in [File Formats](/docs/components/data-connectors#file-formats).

```yaml
datasets:
Expand Down Expand Up @@ -64,21 +64,21 @@ The dataset name cannot be a [reserved keyword](/docs/reference/spicepod/keyword

#### Basic parameters

| Parameter name | Description |
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `file_format` | Specifies the data format. Required if not inferrable from `from`. Options: `parquet`, `csv`. Refer to [Object Store File Formats](/docs/components/data-connectors/index.md#object-store-file-formats) for details. |
| `abfs_account` | Azure storage account name |
| `abfs_sas_string` | SAS (Shared Access Signature) Token to use for authorization |
| `abfs_endpoint` | Storage endpoint, default: `https://{account}.blob.core.windows.net` |
| `abfs_use_emulator` | Use `true` or `false` to connect to a local emulator |
| `abfs_authority_host` | Alternative authority host, default: `https://login.microsoftonline.com` |
| `abfs_proxy_url` | Proxy URL |
| `abfs_proxy_ca_certificate` | CA certificate for the proxy |
| `abfs_proxy_exludes` | A list of hosts to exclude from proxy connections |
| `abfs_disable_tagging` | Disable tagging objects. Use this if your backing store doesn't support tags |
| `allow_http` | Allow insecure HTTP connections |
| `hive_partitioning_enabled` | Enable partitioning using hive-style partitioning from the folder structure. Defaults to `false` |
| `schema_source_path` | Specifies the URL used to infer the dataset schema. Default to the most recently modified file |
| Parameter name | Description |
| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `file_format` | Specifies the data format. Required if not inferrable from `from`. Options: `parquet`, `csv`. Refer to [File Formats](/docs/components/data-connectors#file-formats) for details. |
| `abfs_account` | Azure storage account name |
| `abfs_sas_string` | SAS (Shared Access Signature) Token to use for authorization |
| `abfs_endpoint` | Storage endpoint, default: `https://{account}.blob.core.windows.net` |
| `abfs_use_emulator` | Use `true` or `false` to connect to a local emulator |
| `abfs_authority_host` | Alternative authority host, default: `https://login.microsoftonline.com` |
| `abfs_proxy_url` | Proxy URL |
| `abfs_proxy_ca_certificate` | CA certificate for the proxy |
| `abfs_proxy_exludes` | A list of hosts to exclude from proxy connections |
| `abfs_disable_tagging` | Disable tagging objects. Use this if your backing store doesn't support tags |
| `allow_http` | Allow insecure HTTP connections |
| `hive_partitioning_enabled` | Enable partitioning using hive-style partitioning from the folder structure. Defaults to `false` |
| `schema_source_path` | Specifies the URL used to infer the dataset schema. Default to the most recently modified file |

#### Authentication parameters

Expand Down Expand Up @@ -144,7 +144,7 @@ spice login abfs --tenant-id $TENANT_ID --client-id $CLIENT_ID

## Supported file formats

Specify the file format using `file_format` parameter. More details in [Object Store File Formats](/docs/components/data-connectors/index.md#object-store-file-formats).
Specify the file format using `file_format` parameter. More details in [File Formats](/docs/components/data-connectors#file-formats).

## Examples

Expand Down
12 changes: 6 additions & 6 deletions website/docs/components/data-connectors/file.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ description: 'File Data Connector Documentation'

The File Data Connector enables federated SQL queries on files stored by locally accessible filesystems. It supports querying individual files or entire directories, where all child files within the directory will be loaded and queried.

File formats are specified using the `file_format` parameter, as described in [Object Store File Formats](/docs/components/data-connectors/index.md#object-store-file-formats).
File formats are specified using the `file_format` parameter, as described in [File Formats](/docs/components/data-connectors#file-formats).

Example `spicepod.yml`

Expand Down Expand Up @@ -53,11 +53,11 @@ The dataset name cannot be a [reserved keyword](/docs/reference/spicepod/keyword

### `params`

| Parameter name | Description |
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `file_format` | Specifies the data file format. Required if the format cannot be inferred from the `from` path. Refer to [Object Store File Formats](/docs/components/data-connectors/index.md#object-store-file-formats) for details. |
| `hive_partitioning_enabled` | Enable partitioning using hive-style partitioning from the folder structure. Defaults to `false` |
| `schema_source_path` | Specifies the path used to infer the dataset schema. Default to the most recently modified file |
| Parameter name | Description |
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `file_format` | Specifies the data file format. Required if the format cannot be inferred from the `from` path. Refer to [File Formats](/docs/components/data-connectors#file-formats) for details. |
| `hive_partitioning_enabled` | Enable partitioning using hive-style partitioning from the folder structure. Defaults to `false` |
| `schema_source_path` | Specifies the path used to infer the dataset schema. Default to the most recently modified file |

For additional CSV, JSON, and Parquet specific parameters, see [File Formats](/docs/reference/file_format.md).

Expand Down
Loading