Add TableSink operator with Java/Spark implementations

[harrygav]

Summary

This PR introduces a new TableSink operator for writing Record data into a database table via JDBC, with implementations for the Java and Spark platforms.

Opening as Draft to start discussion on the operator design and expected behavior.

Changes

• New operator: TableSink (in wayang-basic)

  • A UnarySink<Record> that targets a table name and accepts JDBC connection Properties
  • Supports a write mode (e.g. overwrite) and optional column names

• Java platform: JavaTableSink (in wayang-java)

  • JDBC-based implementation that can create the target table (if missing) and batch-insert records
  • Supports overwrite by dropping the target table first

• Spark platform: SparkTableSink (in wayang-spark)

  • Spark-side implementation of the same TableSink operator

Notes / open questions

• This started as a PostgreSQL sink, but the intention should likely be a generic JDBC sink that works across multiple databases.
• DDL generation is currently basic (e.g., columns are auto-created as VARCHARs)
• mode behavior (overwrite vs append, etc.) should be agreed on and formalized.

How to use / test

To run end-to-end locally, you currently need an external PostgreSQL instance available and provide JDBC connection details (driver/url/user/password) in the test setup/environment.

[juripetersen]

Thanks @harrygav, this is great!

Could we make TableSink generic over its input type and thus make DDL generation easier with reflections on the given type?

[novatechflow]

Thank you - just to make the tests running, how's about mocking the JDBC layer?

Wrap DriverManager.getConnection/Connection in a small interface (e.g., JdbcClient) and inject a fake in tests. Then assert SQL statements and batch parameters without a real DB.

[harrygav]

Thanks @harrygav, this is great!

Could we make TableSink generic over its input type and thus make DDL generation easier with reflections on the given type?

I will take a look and update the PR to continue the discussion!