Skip to content

Conversation

@MehulBatra
Copy link
Contributor

@MehulBatra MehulBatra commented Oct 11, 2025

Purpose

Linked issue: close #1727

Brief change log

  1. Added flink-iceberg.md for iceberg related fluss quickstart guide using hadoop catalog
  2. Modified flink.md to flink-paimon.md for seperation and changed header for the same
  3. Added a note explaining that the $lake suffix and metadata tables functionality (like
    $lake$snapshots) is currently in development and referenced issue Lake table decouple with Paimon #1559.
  4. Added both the files in quickstart section
  5. Added hadoop jar support in flink-dist plugins.xml for hadoop catalog

Tests

Verified the guide by running manually on my local

API and Format

Documentation

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a new Iceberg quickstart guide for Fluss, demonstrating real-time analytics with Flink and Apache Iceberg integration using Hadoop catalog. It separates the existing Flink guide to focus on Paimon, creating distinct documentation paths for different lake storage formats.

Key Changes:

  • Created new Flink-Iceberg quickstart guide with comprehensive tutorial including Docker setup, table creation, and lakehouse integration
  • Renamed existing Flink guide to specify Paimon integration and updated references
  • Added Hadoop JAR support in distribution plugins for Iceberg catalog functionality

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
website/docs/quickstart/flink-paimon.md Updated title to specify Paimon integration
website/docs/quickstart/flink-iceberg.md New comprehensive Iceberg quickstart guide with Docker setup and examples
website/docs/intro.md Updated quickstart link to point to Paimon-specific guide
fluss-dist/src/main/assemblies/plugins.xml Added Hadoop JAR dependency for Iceberg plugin support

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MehulBatra Thanks for the pr. Left some comments.

@MehulBatra MehulBatra changed the title Fluss iceberg quickstart guide [Docs] Fluss/flink - Iceberg quickstart guide Oct 13, 2025
@MehulBatra MehulBatra force-pushed the fluss-iceberg-quickstart-guide branch from 8526dc5 to 3542d23 Compare October 13, 2025 08:58
@MehulBatra
Copy link
Contributor Author

Thank you @luoyuxia for the review and feedback, addressed your comments, please have a look!

@MehulBatra MehulBatra requested a review from luoyuxia October 13, 2025 08:59
Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MehulBatra Thanks for update.
1: The ci fail
2: Also, I notice in the descrition #1727 (comment), it's suggested paimon to be a tab, iceberg to be another tab instead of pages. Is it possible to make them as tabs. I think tabs will be more unified to avoid introduce two many pages.

@luoyuxia
Copy link
Contributor

@MehulBatra Thanks for update. 1: The ci fail 2: Also, I notice in the descrition #1727 (comment), it's suggested paimon to be a tab, iceberg to be another tab instead of pages. Is it possible to make them as tabs. I think tabs will be more unified to avoid introduce two many pages.

If make them as tabs is not easy, I don't mind change them to tabs in another pr.

@MehulBatra
Copy link
Contributor Author

@MehulBatra Thanks for update. 1: The ci fail 2: Also, I notice in the descrition #1727 (comment), it's suggested paimon to be a tab, iceberg to be another tab instead of pages. Is it possible to make them as tabs. I think tabs will be more unified to avoid introduce two many pages.

If make them as tabs is not easy, I don't mind change them to tabs in another pr.

I think we can take that as a seperate PR as it will introduce more changes, I am thinking to get this merged before flink forward session

@MehulBatra
Copy link
Contributor Author

somehow it's passing in my local but failing here 😓

@luoyuxia luoyuxia force-pushed the fluss-iceberg-quickstart-guide branch 2 times, most recently from c5df208 to 77a696b Compare October 14, 2025 03:29
@luoyuxia luoyuxia force-pushed the fluss-iceberg-quickstart-guide branch from 77a696b to ecf0b27 Compare October 14, 2025 03:50
@luoyuxia luoyuxia requested a review from Copilot October 14, 2025 03:54
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@luoyuxia
Copy link
Contributor

@MehulBatra I revert to refer to flink.md, then every thing works well. I suggest to still use flink.md since in the following pr, we will just have one page flink.md, but two tabls for paimon and iceberg.

@MehulBatra MehulBatra requested a review from luoyuxia October 15, 2025 12:35
Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MehulBatra Left some comments again. PTAL

Copy link
Contributor

@luoyuxia luoyuxia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@luoyuxia luoyuxia merged commit 07847f5 into apache:main Oct 16, 2025
6 checks passed
polyzos pushed a commit to polyzos/fluss that referenced this pull request Oct 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create fluss/flink-Iceberg quickstart guide

2 participants