-
Notifications
You must be signed in to change notification settings - Fork 460
Numeric type widening converters #1877
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Numeric type widening converters #1877
Conversation
…orresponding unit tests in Fluss Client
…nit tests for various widening scenarios in Fluss Flink
…ening support in Fluss Java Client and DataStream API
|
@polyzos Can you please review the pull request. Thanks |
fluss-client/src/main/java/org/apache/fluss/client/converter/ConverterCommons.java
Outdated
Show resolved
Hide resolved
fluss-client/src/main/java/org/apache/fluss/client/converter/PojoToRowConverter.java
Outdated
Show resolved
Hide resolved
fluss-client/src/test/java/org/apache/fluss/client/converter/PojoToRowConverterTest.java
Outdated
Show resolved
Hide resolved
fluss-client/src/test/java/org/apache/fluss/client/converter/PojoToRowConverterTest.java
Show resolved
Hide resolved
|
@hemanthsavasere can you remove the documentation? This functionality is meant for internal use. |
|
@snuyanzin thank you for reviewing this and your thoughtful feedback |
Co-authored-by: Sergey Nuyanzin <[email protected]>
… (NaN, ±Infinity) throughout the Fluss data pipeline
|
@snuyanzin and @polyzos I have done the changes, can you please review PR. |
|
@snuyanzin @polyzos I have completed the requested changes. When you have the opportunity, could you please review the PR? |
Purpose
Linked issue: close #1844
Perform automatic type widening between numeric types when converting POJOs to internal rows. Change from strict and exact type validation to widening conversion from lower to higher data types in POJO field's Java type and the schema's Fluss type.
Implementation Sample Scenarios :
POJO with int field + Schema with BIGINT → SUCCEEDS ✅
POJO with short field + Schema with BIGINT → SUCCEEDS ✅
POJO with int field + Schema with FLOAT → SUCCEEDS ✅
POJO with Long field + Schema with SMALLINT → FAILS (narrowing not supported) ❌
Widening Primitive Conversions :
byte (TINYINT) → short, int, long, float, double
short (SMALLINT) → int, long, float, double
int (INTEGER) → long, float, double
long (BIGINT) → float, double
float (FLOAT) → double
fluss-client
fluss-flink-common
Tests
Added comprehensive tests in PojoToRowConverterTest.java for all widening combinations in fluss-client
Added comprehensive tests in PojoToRowConverterTest.java covering all widening scenarios in fluss-flink-common