object-store-access-configuration.md

Object Store Access Configuration

This document provides a guide on how to configure access to object stores for the Iceberg Catalog Migrator.

Required Dependencies

The Iceberg Catalog Migrator CLI jar does not include cloud provider dependencies to keep the distribution size small. Users must supplement the appropriate Iceberg object store bundle jar based on the object store being used.

Download the required bundle jar from Maven Central

AWS S3

Required Dependencies

Users must include the Iceberg AWS bundle jar (can be downloaded from here) in the classpath:

java -cp iceberg-catalog-migrator-cli-0.1.0-SNAPSHOT.jar:iceberg-aws-bundle-x.x.x.jar \
  org.apache.polaris.iceberg.catalog.migrator.cli.CatalogMigrationCLI register \
  [your-options]

For more information on AWS integration, refer to the Iceberg AWS documentation.

Environment Variables

For AWS, use the following environment variables:

export AWS_ACCESS_KEY_ID=xxxxxxx
export AWS_SECRET_ACCESS_KEY=xxxxxxx
export AWS_S3_ENDPOINT=xxxxxxx

Azure Data Lake Storage (ADLS)

Required Dependencies

Users must include the Iceberg Azure bundle jar (can be downloaded from here) in the classpath:

java -cp iceberg-catalog-migrator-cli-0.1.0.jar:iceberg-azure-bundle-x.x.x.jar \
  org.apache.polaris.iceberg.catalog.migrator.cli.CatalogMigrationCLI register \
  [your-options]

Environment Variables

For ADLS, use the following environment variables:

export AZURE_SAS_TOKEN=xxxxxxx

Google Cloud Storage (GCS)

Required Dependencies

Users must include the Iceberg GCP bundle jar (can be downloaded from here) in the classpath:

java -cp iceberg-catalog-migrator-cli-0.1.0.jar:iceberg-gcp-bundle-x.x.x.jar \
  org.apache.polaris.iceberg.catalog.migrator.cli.CatalogMigrationCLI register \
  [your-options]

Notes

• Replace x.x.x with the Iceberg version matching the release version of the migrator tool.
• Multiple bundle jars can be included if users need to access multiple cloud providers.