Skip to content

fix!: DH-19276: Drop support for writing byte[] to Iceberg #6809

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
malhotrashivam wants to merge 3 commits into from
Closed

fix!: DH-19276: Drop support for writing byte[] to Iceberg #6809

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8d9ab79
Drop support for writing byte[] to Iceberg
malhotrashivam Apr 21, 2025
5a26b47
Merge branch 'main' into nightly/sm-DH-19276
malhotrashivam Apr 29, 2025
b485113
Added a test
malhotrashivam Apr 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions extensions/iceberg/src/main/java/io/deephaven/iceberg/base/IcebergUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,7 @@ public final class IcebergUtils {
DH_TO_ICEBERG_TYPE_MAP.put(LocalDateTime.class, Types.TimestampType.withoutZone());
DH_TO_ICEBERG_TYPE_MAP.put(LocalDate.class, Types.DateType.get());
DH_TO_ICEBERG_TYPE_MAP.put(LocalTime.class, Types.TimeType.get());
DH_TO_ICEBERG_TYPE_MAP.put(byte[].class, Types.BinaryType.get());
// TODO (deephaven-core#6327) Add support for more types like ZonedDateTime, Big Decimals, and Lists
// TODO (DH-18253): Add support for more types like ZonedDateTime, Big Decimals, and Lists
}

private static String path(@NotNull final String path, @NotNull final FileIO io) {
Expand Down
20 changes: 20 additions & 0 deletions extensions/iceberg/src/test/java/io/deephaven/iceberg/junit5/SqliteCatalogBase.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -454,6 +454,26 @@ void appendToCatalogTableWithAllDataTypesTest() {
assertTableEquals(source, tableAdapter.table());
}

@Test
void appendByteArrayToBinaryColTest() {
final Schema schema = new Schema(
Types.NestedField.required(1, "binaryCol", Types.BinaryType.get()));
final Namespace myNamespace = Namespace.of("MyNamespace");
final TableIdentifier myTableId = TableIdentifier.of(myNamespace, "appendToBinaryColTest");
catalogAdapter.catalog().createTable(myTableId, schema);
final Table source = TableTools.emptyTable(10)
.update("binaryCol = new byte[] {(byte) i}");
final IcebergTableAdapter tableAdapter = catalogAdapter.loadTable(myTableId);
try {
tableAdapter.tableWriter(writerOptionsBuilder()
.tableDefinition(source.getDefinition())
.build());
failBecauseExceptionWasNotThrown(TableDataException.class);
} catch (TableDataException e) {
assertThat(e).hasMessageContaining("Unsupported deephaven column type");
}
}

@Test
void testFailureInWrite() {
// Try creating a new iceberg table with bad data
Expand Down