Skip to content

[Fix][Connector-V2][OceanBase] oceanbase vector support simple vector index #9056

Closed
@xxsc0529

Description

@xxsc0529

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

ConstraintType class add VECTOR_INDEX_KEY enum,The error is reported in the buildConstrainKeySql method of OceanBaseMysqlCreateTableSqlBuilder. Currently, the vector index supported by oceanbase is very limited. It is not a good solution to not add an index under a large amount of data. Here is an appendix to a default index implementation.

SeaTunnel Version

all version

SeaTunnel Config

env {
  parallelism = 1
  job.mode = "BATCH"
  checkpoint.interval = 5000
}

source {
  Milvus {
    url = "http://ip:port"
    token = "root:Milvus"
    database = "default"
    collection="test"
  }
}

transform {
}
sink {
    jdbc {
        url = "jdbc:oceanbase://ip:port/db"
        driver = "com.oceanbase.jdbc.Driver"
        user = "user"
        password = "password"
        compatible_mode = "mysql"
        generate_sink_sql = true
        database = db
        table = test
    }
}

Running Command

bin/seatunnel.sh --config conf/jdbc_milvus_to_oceanbase.conf -e local

Error Exception

Caused by: java.lang.UnsupportedOperationException: Unsupported constraint type: VECTOR_INDEX_KEY
	at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.oceanbase.OceanBaseMysqlCreateTableSqlBuilder.buildConstraintKeySql(OceanBaseMysqlCreateTableSqlBuilder.java:269)
	at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.oceanbase.OceanBaseMysqlCreateTableSqlBuilder.buildColumnsIdentifySql(OceanBaseMysqlCreateTableSqlBuilder.java:173)
	at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.oceanbase.OceanBaseMysqlCreateTableSqlBuilder.build(OceanBaseMysqlCreateTableSqlBuilder.java:143)
	at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.oceanbase.OceanBaseMySqlCatalog.getCreateTableSql(OceanBaseMySqlCatalog.java:186)
	at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.AbstractJdbcCatalog.getCreateTableSqls(AbstractJdbcCatalog.java:442)
	at org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.AbstractJdbcCatalog.createTableInternal(AbstractJdbcCatalog.java:450)
	... 28 more

	at org.apache.seatunnel.engine.server.CoordinatorService.lambda$submitJob$6(CoordinatorService.java:656)
	at org.apache.seatunnel.api.tracing.MDCRunnable.run(MDCRunnable.java:43)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

Zeta or Flink or Spark Version

Java or Scala Version

No response

Screenshots

Image

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions