Skip to content

Conversation

@anurag2787
Copy link

Description

This PR implements a robust validation framework for Iceberg catalog configurations, addressing the lack of proper validation for different catalog types this enhancement prevents runtime failures by catching configuration errors early during initialization.

Fixes #478

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

AWS Glue Catalog

  • Valid Glue configuration with region
  • Missing region validation
  • AWS credentials validation (access key/secret key pairs)
  • Profile configuration validation

JDBC Catalog

  • Valid JDBC configuration with URL and credentials
  • JDBC URL format validation
  • Missing required credentials validation
  • Invalid URL format detection

REST Catalog

  • Token-based authentication validation
  • OAuth2 authentication validation
  • Missing authentication parameters
  • REST signing configuration validation
  • URL format validation

Hive Metastore

  • Valid Hive configuration with thrift URI
  • URI format validation
  • Client configuration limits
  • Missing required parameters

General Tests

  • Default value initialization
  • Required S3 path validation
  • Mixed credential configuration validation
  • Error message clarity and actionability

@CLAassistant
Copy link

CLAassistant commented Oct 14, 2025

CLA assistant check
All committers have signed the CLA.

@anurag2787
Copy link
Author

@vaibhav-datazip Could you please review this pr and let me know if any changes required

@vaibhav-datazip vaibhav-datazip added the hacktoberfest Issues open for Hacktoberfest contributors label Oct 14, 2025
@vaibhav-datazip
Copy link
Collaborator

vaibhav-datazip commented Oct 15, 2025

Hi @anurag2787 can you rebase this PR to staging branch, it's currently raised on master

@anurag2787
Copy link
Author

@vaibhav-datazip Could you please review now?

@vaibhav-datazip
Copy link
Collaborator

@anurag2787 , still you haven't rebased it to staging

@anurag2787
Copy link
Author

@vaibhav-datazip is now it is correct? Sorry for inconvenience as i am not familiar with staging

@vaibhav-datazip
Copy link
Collaborator

@vaibhav-datazip is now it is correct? Sorry for inconvenience as i am not familiar with staging

Screenshot 2025-10-30 at 12 08 19 PM

here you can see datazip:master, it should be datazip:staging

@Akshay-datazip
Copy link
Collaborator

hi @anurag2787 just a kind reminder to be on our slack if you are still facing issues related to pr would love to help there !
https://join.slack.com/t/getolake/shared_invite/zt-2uyphqf69-KQxih9Gwd4GCQRD_XFcuyw

@anurag2787 anurag2787 changed the base branch from master to staging October 30, 2025 17:50
@anurag2787
Copy link
Author

@vaibhav-datazip I have updated datazip:master to datazip:staging Thanks for the clarification!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hacktoberfest Issues open for Hacktoberfest contributors

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add validation for each catalog properly

4 participants