[refactor](fe) Merge fe-connector-api into fe-connector-spi (#66407)

### What problem does this PR solve?

Problem Summary:

The connector plugin contract was split across two modules whose names
were
inverted relative to common usage: `fe-connector-api` held the types a
plugin
IMPLEMENTS (`Connector`, `ConnectorMetadata` and its Ops sub-interfaces,
the
scan / write / procedure providers, handles, pushdown and ddl value
types) —
which is what "SPI" normally names — while `fe-connector-spi` held
mostly the
engine services a plugin CONSUMES (`ConnectorContext`,
`ConnectorStorageContext`,
`ConnectorConf`), which is what "API" normally names. Both modules'
package-info
already documented the inversion and asked readers to "read the content,
not the
name". The `fe-connector-metastore-api` / `-metastore-spi` pair right
next to
them follows the opposite (standard) convention, so one directory
carried two
contradictory naming rules.

Swapping the two names does not fix it. A strict split by "who
implements" is
circular: `ConnectorProvider.create` takes a `ConnectorContext` (spi ->
api),
while `ConnectorContext.createSiblingConnector` returns a `Connector`
(api -> spi). That cycle is exactly why Trino keeps its whole plugin
surface in
a single `trino-spi` module, and this framework is modelled on Trino.

This commit does the same: `fe-connector-api` is merged into
`fe-connector-spi`, and `org.apache.doris.connector.api` becomes
`org.apache.doris.connector.spi`. The merge is dependency-neutral —
every module
that depended on `fe-connector-api` (hive, hudi, maxcompute, paimon,
trino,
fe-core) already depended on `fe-connector-spi` as well.

Two consequences worth calling out:

- The connector plugin API version is bumped 2.0 -> 3.0. Every type on
the
contract changed its fully-qualified name, so a plugin built against 2.0
must
be refused by `ApiVersionGate` at load time rather than fail later with
  `NoClassDefFoundError`. Both recorded baselines
(`connector-metadata-methods.txt`, `connector-plugin-surface.txt`) and
the
version pinned in `ConnectorPluginSurfaceTest` are updated in this
commit.
`ConnectorPluginManagerTest` used to hardcode the compatible major as a
  literal; it now derives both the stale and the current major from the
  kernel's declared version, because what that test asserts is the gate
  (stale refused, current admitted), not which number is current.
  `ConnectorPluginSurfaceTest` remains the one deliberate speed bump.
- `ChildFirstClassLoader.DEFAULT_PARENT_FIRST_PACKAGES` listed only
  `org.apache.doris.connector.api.`, so `ConnectorProvider` and
  `ConnectorContext` were not parent-first. After the merge the single
  `org.apache.doris.connector.spi.` entry covers the whole contract.

The merged `package-info.java` keeps the existing Rule 1-7 design rules;
Rule 4
now explains why both directions live in one module instead of
disclaiming the
inverted names. README.md and AGENTS.md are updated accordingly.

Author: morningman

.licenserc.yaml

@@ -51,7 +51,7 @@ header:
     - "fe/fe-core/src/test/resources/stageUtilTest.txt"
     # Recorded method-surface baseline; ConnectorMetadataSurfaceTest reads every
     # non-blank line as a method signature, so a header would break the test.
-    - "fe/fe-connector/fe-connector-api/src/test/resources/connector-metadata-methods.txt"
+    - "fe/fe-connector/fe-connector-spi/src/test/resources/connector-metadata-methods.txt"
     # Recorded plugin API surface baselines, one per plugin family. Same reason as
     # above: each *PluginSurfaceTest.readBaseline() adds every non-blank line to the
     # expected signature set, so a header would be read back as phantom signatures.

build-support/check-fe-connector-imports.sh

@@ -25,7 +25,7 @@
 # datasource / qe / analysis / nereids / planner / persist / transaction / fs /
 # statistics / mysql / service). Anything they need from fe-core has to be
 # exposed through the SPI in
-#   org.apache.doris.connector.{api,spi,extension,...}
+#   org.apache.doris.connector.{spi,metastore,...}
 # or shared types in org.apache.doris.thrift / org.apache.doris.filesystem.
 #
 # The gate matches both plain and `import static` imports, scans src/main/java

build-support/tests/test-fe-connector-imports.sh

@@ -58,7 +58,7 @@ import org.apache.doris.mysql.privilege.Auth;
 import org.apache.doris.service.FrontendOptions;
 import org.apache.doris.thrift.TFoo;
 import org.apache.doris.filesystem.Bar;
-import org.apache.doris.connector.api.Baz;
+import org.apache.doris.connector.spi.Baz;
 import org.apache.doris.datasource.hive.HiveVersionUtil;
 import static org.apache.doris.datasource.hive.HiveVersionUtil.SOME_CONST;
 public class FakeConn {}
@@ -117,7 +117,7 @@ must_not_report() {
 }
 must_not_report 'org.apache.doris.thrift.TFoo'          # SPI-shared
 must_not_report 'org.apache.doris.filesystem.Bar'       # SPI-shared
-must_not_report 'org.apache.doris.connector.api.Baz'    # SPI
+must_not_report 'org.apache.doris.connector.spi.Baz'    # SPI
 must_not_report 'HiveVersionUtil;'                      # vendored (non-static)
 must_not_report 'HiveVersionUtil.SOME_CONST;'           # vendored (static) — E3
 

build.sh

@@ -723,8 +723,7 @@ if [[ "${BUILD_FE}" -eq 1 ]]; then
         fi
     done
     unset _fs_mod
-    # Connector API, SPI, and plugin modules (loaded at runtime as plugins)
-    modules+=("fe-connector/fe-connector-api")
+    # Connector SPI and plugin modules (loaded at runtime as plugins)
     modules+=("fe-connector/fe-connector-spi")
     for _conn_mod in es jdbc maxcompute trino hms hive paimon hudi iceberg; do
         if [[ -d "${DORIS_HOME}/fe/fe-connector/fe-connector-${_conn_mod}" ]]; then

fe/fe-connector/AGENTS.md

@@ -14,7 +14,7 @@ mvn -f fe/pom.xml -pl :fe-connector-<module> -am test
 
 # Run a single test class. Keep -DfailIfNoTests=false: with -am, upstream
 # modules have no matching tests and would fail the build otherwise.
-mvn -f fe/pom.xml -pl :fe-connector-api -am test \
+mvn -f fe/pom.xml -pl :fe-connector-spi -am test \
     -Dtest=ConnectorMetadataSurfaceTest -DfailIfNoTests=false
 ```
 
@@ -40,20 +40,25 @@ self-tests live in `build-support/` and `build-support/tests/`):
    wired into this directory's `pom.xml`. fe-connector modules must not
    import fe-core internals, in main OR test sources. When a connector needs
    something from the engine, the fix is to extend the SPI in
-   fe-connector-api / fe-connector-spi — never to import fe-core.
+   fe-connector-spi — never to import fe-core.
 2. **Metadata funnel** — `build-support/check-fe-core-metadata-funnel.sh`,
    wired into fe-core's `pom.xml`. Inside fe-core, only
    `PluginDrivenMetadata` may call `Connector#getMetadata`; exempt call
    sites carry a `getMetadata-funnel-exempt` marker, and deleting a marker
    auto-tightens the gate.
 
-**Changing the shared SPI surface (fe-connector-api):** regenerate
-`fe-connector-api/src/test/resources/connector-metadata-methods.txt` in the
-SAME commit — run `ConnectorMetadataSurfaceTest` and copy the "actual" set
-from the failure output. Always run fe-connector-api's OWN suite after an
-SPI change; running only consumer modules will not catch a stale baseline.
-The same suite pins the `@ConnectorMustImplement` set, so promoting a method
-into the minimum implementation set is a deliberate, reviewed change.
+**Changing the shared SPI surface (fe-connector-spi):** regenerate BOTH
+recorded baselines in the SAME commit — `connector-metadata-methods.txt`
+(`ConnectorMetadataSurfaceTest`) and `connector-plugin-surface.txt`
+(`ConnectorPluginSurfaceTest`), under
+`fe-connector-spi/src/test/resources/`; run the test and copy the "actual"
+set from the failure output. Any surface change is a MAJOR change: bump
+`connector.plugin.api.version` in `fe/fe-connector/pom.xml` and the version
+pinned in `ConnectorPluginSurfaceTest` in that same commit. Always run
+fe-connector-spi's OWN suite after an SPI change; running only consumer
+modules will not catch a stale baseline. The same suite pins the
+`@ConnectorMustImplement` set, so promoting a method into the minimum
+implementation set is a deliberate, reviewed change.
 
 ## Invariants Without a Gate
 
@@ -94,8 +99,9 @@ these has a concrete failure mode.
 
 ## Task Recipes
 
-- **Change the SPI surface**: edit fe-connector-api → regenerate the
-  baseline in the same commit → run fe-connector-api's suite → adjust
+- **Change the SPI surface**: edit fe-connector-spi → regenerate both
+  baselines and bump the plugin API version in the same commit → run
+  fe-connector-spi's suite → adjust
   affected connectors (grep for overrides) → run their module tests.
 - **Fix a connector bug**: module-scoped build + tests (recipes above;
   paimon via `install`). For user-visible behavior, extend the e2e suites

fe/fe-connector/README.md

@@ -40,8 +40,13 @@ Roles only — one line each. For anything deeper, read the module's javadoc
 
 | Module | Role |
 |---|---|
-| `fe-connector-api` | The engine <-> connector contract: `ConnectorMetadata` with its Ops sub-interfaces, plus handle / pushdown / mvcc / scan / write / ddl / procedure / event / rest types. The javadoc here **is** the API reference. |
-| `fe-connector-spi` | Bootstrap contract: `ConnectorProvider` (discovery identity + factory) and `ConnectorContext` (what the engine hands a connector, including sibling-connector creation). |
+| `fe-connector-spi` | The whole engine <-> connector contract, in both directions. What a connector implements: `ConnectorProvider` (discovery identity + factory), `Connector`, `ConnectorMetadata` with its Ops sub-interfaces, plus handle / pushdown / mvcc / scan / write / ddl / procedure / event / rest types. What the engine implements and hands down: `ConnectorContext` (including sibling-connector creation), `ConnectorStorageContext`, `ConnectorSession`, `ConnectorConf`. The javadoc here **is** the API reference. |
+
+The two directions are one module on purpose: the boundary is bidirectional
+(`ConnectorProvider.create` takes a `ConnectorContext`, and `ConnectorContext`
+hands back a `Connector`), so splitting it by "who implements" would be
+circular. Trino makes the same call with `trino-spi`. Contrast the metastore
+layer below, where the split is acyclic and the usual api/spi convention holds.
 
 **Metastore layer** (how connectors reach a metastore without hand-parsing
 endpoint properties)
@@ -99,8 +104,8 @@ exactly one `ConnectorMetadata` instance per catalog on the statement's
 `ConnectorStatementScope` and closes it deterministically at statement end.
 Scan planning follows the same shape: the generic `PluginDrivenScanNode`
 (fe-core) delegates all per-source planning to the connector's
-`ConnectorScanPlanProvider` — note the interface lives in fe-connector-api
-(`api.scan`), not in fe-core.
+`ConnectorScanPlanProvider` — note the interface lives in fe-connector-spi
+(`spi.scan`), not in fe-core.
 
 **Classloading.** Plugins load child-first, each carrying its own runtime
 closure. Wherever engine code crosses into a plugin — or a bundled library
@@ -119,17 +124,16 @@ This document never lists SPI methods. The truth lives in code, behind four
 mechanisms:
 
 1. **Javadoc is the API reference.** Start at `ConnectorMetadata` (and its
-   Ops sub-interfaces) in fe-connector-api, and `ConnectorProvider` in
-   fe-connector-spi. Every SPI method has a default body, so each
-   sub-interface's class javadoc states its minimum implementation set,
-   lifecycle, and threading rules.
+   Ops sub-interfaces) and `ConnectorProvider`, both in fe-connector-spi.
+   Every SPI method has a default body, so each sub-interface's class javadoc
+   states its minimum implementation set, lifecycle, and threading rules.
 2. **`@ConnectorMustImplement`** is the machine-readable half of the minimum
    implementation set: it marks the default methods a connector is
    nevertheless expected to override, with `when` naming the capability that
    triggers the obligation. A unit test pins the annotated set, so promoting
    a method is a deliberate, reviewed change.
 3. **The recorded surface.**
-   `fe-connector-api/src/test/resources/connector-metadata-methods.txt`
+   `fe-connector-spi/src/test/resources/connector-metadata-methods.txt`
    freezes the public method surface of `ConnectorMetadata`;
    `ConnectorMetadataSurfaceTest` fails on any drift. Adding, removing, or
    moving SPI methods must regenerate this baseline in the same commit (run
@@ -146,7 +150,7 @@ mechanisms:
    source; if a connector does not opt in, the feature stays off.
 
 Statement-scoped memoization has one framework-wide convention worth knowing
-before you read connector code: `ConnectorStatementScopes` (fe-connector-api)
+before you read connector code: `ConnectorStatementScopes` (fe-connector-spi)
 keys per-statement values by `(catalogId, db, table, queryId)` plus a
 connector-owned namespace constant prefixed with the connector's
 `getType()`. Each connector guards its own prefix with a unit test
@@ -239,10 +243,10 @@ metastore/shade/cache). For a write path, the richest example is
 ## Testing and Verification
 
 - **Unit tests** live in each module; build/test recipes are in `AGENTS.md`.
-- **The shared SPI surface** is guarded by fe-connector-api's own suite
-  (`ConnectorMetadataSurfaceTest`). Whenever you touch fe-connector-api, run
-  that module's tests — a consumer-only test run will not catch a stale
-  baseline.
+- **The shared SPI surface** is guarded by fe-connector-spi's own suite
+  (`ConnectorMetadataSurfaceTest` and `ConnectorPluginSurfaceTest`). Whenever
+  you touch fe-connector-spi, run that module's tests — a consumer-only test
+  run will not catch a stale baseline.
 - **Architecture gates** run in the `validate` phase of every FE build: the
   forbidden-import gate for this directory and the metadata-funnel gate for
   fe-core. Scripts and their self-tests live in `build-support/` and