docs: Polaris integration guide for Table Topic with REST custom headers

[aaron-seq]

Move Polaris integration docs from core repo PR to labs with runnable setup:

• Add table-topic-solutions/polaris-integration with README, docker-compose, configs, and scripts
• Document header pattern:
  • automq.table.topic.catalog.header.Authorization
  • automq.table.topic.catalog.header.Polaris-Realm
• Provide end-to-end compose stack (MinIO, Polaris, AutoMQ, Spark)
• Include troubleshooting and quick validation steps

This addresses maintainer guidance to host integration examples in labs, keeping core repo docs minimal.

[daniel-y]

@aaron-seq Cool, thanks for your contribution, let us have a check and merge it!

[Gezi-lzq]

Based on reviewing the integration approach in this PR, I believe the provided docker-compose configuration cannot actually offer the service. To ensure efficient reviews and build trust in the PR, please ensure that code changes are fully tested locally before submitting.

[aaron-seq]

I've reviewed the PR and maintainers' feedback. Before updating, I'll re-test the docker-compose.yml integration locally to fully validate the service stack and confirm end-to-end connectivity (Polaris, MinIO, AutoMQ, Spark). I'll check logs for REST header propagation and error handling, and verify that README steps work as described. If any issue appears during setup or usage, I'll update the configs/scripts accordingly and document troubleshooting fixes. As always, I'll ensure every change is tested and aligns with repository guidelines before pushing, keeping core repo docs minimal as requested. If maintainers have further specific testing requirements, let me know so I can incorporate them promptly.

[johnluoyx]

Hi @aaron-seq @Gezi-lzq , due to #106
We have redesigned the directory structure for better organization. Your content can be moved to the current "opensource-examples/table-topic" folder.

[aaron-seq]

Update: PR Ready for Review

I've completed the restructuring and testing. Here's the summary:

Changes Completed:

• Directory relocation: Moved from table-topic-solutions/ to opensource-examples/table-topic/polaris-integration/
• Structure alignment: Now matches the new repository layout from Refactor Repository Directory Structure for Better Organization #106
• Docker Compose updates: Verified working images (MinIO, AutoMQ, Spark)
• Documentation: Enhanced README with setup, troubleshooting, and image availability notes

Local Testing Results

• ✓ MinIO (latest): Running successfully
• ✓ AutoMQ (latest): Running successfully
• ✓ Spark (3.5.0): Image verified
• but for Polaris: No official Docker image available on Docker Hub as of Dec 2024

Core Contribution

The primary value of this PR is demonstrating the custom header configuration pattern for REST catalogs:

automq.table.topic.catalog.header.Authorization=Bearer test-token
automq.table.topic.catalog.header.Polaris-Realm=POLARIS

Pattern: automq.table.topic.catalog.header.{HeaderName}={Value}

Polaris Image Availability

Apache Polaris doesn't publish Docker images to Docker Hub. Options for testing:

1. Build from source: https://github.com/apache/polaris
2. Maintainer testing in environments with Polaris access
3. Alternative REST catalog (Nessie, custom implementations)

Ready for Review

• Configuration approach is validated
• Directory structure matches Refactor Repository Directory Structure for Better Organization #106 requirements
• Documentation is complete
• Supporting services verified

Please let me know if you'd like any adjustments or have questions about the configuration approach.

[daniel-y]

@johnluoyx please help review this PR.

[Copilot]

Pull request overview

This PR adds a Polaris integration guide to the AutoMQ labs repository, demonstrating how to configure AutoMQ Table Topic with Apache Polaris REST catalog using custom HTTP headers for authentication and realm routing. The integration showcases the header configuration pattern (automq.table.topic.catalog.header.{HeaderName}) for REST catalog integration.

Key Changes:

• Documentation and example configuration for Polaris REST catalog integration with custom headers
• Docker Compose stack setup for local testing environment
• Configuration files demonstrating header-based authentication pattern

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
README.md	Comprehensive integration guide with setup instructions, configuration examples, and troubleshooting tips
docker-compose.yml	Service orchestration for MinIO, AutoMQ, and Spark (missing Polaris service)
docker-compose.yml.backup	Alternative compose file including Polaris service definition
configs/automq.properties	AutoMQ configuration demonstrating REST catalog with custom header authentication
configs/polaris.yaml	Polaris server configuration with realm-based routing
scripts/spark-shell.sh	Spark shell launcher script (incorrectly configured for Hadoop catalog)
scripts/init-iceberg.sql	Placeholder SQL initialization file

Critical Issues Found:

• The main docker-compose.yml is missing the Polaris service entirely, making the example non-functional
• Spark is configured to use Hadoop catalog instead of Polaris REST catalog
• Multiple inconsistencies between README documentation and actual implementation files (image versions, service configurations)
• Authorization token mismatch in verification example

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The Spark service has an inconsistent dependency. It depends on [minio] but should depend on [polaris] as indicated in docker-compose.yml.backup line 32, or both [minio, polaris] since Spark needs to interact with the Iceberg catalog via Polaris. Change to depends_on: [polaris] or depends_on: [minio, polaris] for proper startup ordering.

Suggested change

	depends_on: [minio]
	depends_on: [minio, polaris]