Skip to content

Conversation

@yzeng1618
Copy link
Contributor

Purpose of this pull request

  • Align HdfsFile sink with other file sinks (e.g. SftpFile/FtpFile/S3File/LocalFile) by exposing and documenting schema_save_mode and data_save_mode.
  • Add schema_save_mode / data_save_mode into HdfsFileSinkFactory OptionRule so configs using SaveMode are not rejected by option validation.

Does this PR introduce any user-facing change?

Yes.

  • Users can now configure schema_save_mode and data_save_mode for HdfsFile sink without failing option validation.
  • Documentation is updated accordingly.
  • Runtime semantics are unchanged (SaveMode behavior already exists); this PR mainly makes the options discoverable/usable and keeps docs consistent.

How was this patch tested?

No

Check list

davidzollo
davidzollo previously approved these changes Jan 6, 2026
Copy link
Contributor

@davidzollo davidzollo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1
Good job

Copy link
Contributor

@corgy-w corgy-w left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@davidzollo davidzollo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@davidzollo davidzollo merged commit 81834f2 into apache:dev Jan 7, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants