[Refactor] Consolidate built-in connector registration

Signed-off-by: alvin-phoenix-ai <alvin.zhao@phoenixdata.ai>

Author: alvin-phoenix-ai

be/AGENTS.md

@@ -284,18 +284,11 @@ Reusable connector sink/file-writer helpers above connector contracts, including
 - Core tests: `connector_common_test`
 - Remediation: Keep ConnectorCommon limited to reusable connector sink/file-writer helpers expressed through connector contracts and compute/runtime lower layers; move concrete connectors, registry wiring, service, HTTP, storage-engine-specific behavior, and full Exec integration upward.
 
-### ConnectorBuiltinRegistry (`connectorbuiltinregistry`)
-Top-level built-in connector registration composition above connector contracts and concrete connector libraries.
-- Targets: `ConnectorBuiltinRegistry`
-- Allowed internal include prefixes: `connector/builtin_connector_registry.h`, `connector/`, `common/`, `base/`, `gutil/`, `gen_cpp/`
-- Allowed target deps: `Connector`, `ConnectorFile`, `ConnectorPrimitive`, `Common`, `Base`, `Gutil`, `StarRocksGen`
-- Remediation: Keep legacy built-in connector registration as top-level composition for unsplit connector libraries; split connector libraries register through ModuleBootstrap instead of depending back on this target.
-
 ### ModuleBootstrap (`modulebootstrap`)
-Default BE module bootstrap composition for built-in module registration, including filesystem provider registry installation and split connector registration.
+Default BE module bootstrap composition for built-in module registration, including filesystem provider registry installation and connector registration.
 - Targets: `ModuleBootstrap`
-- Allowed internal include prefixes: `module/`, `connector/benchmark/`, `connector/cache_stats/`, `connector/hive/`, `connector/iceberg/`, `connector/lake/`, `connector/jdbc/`, `connector/elasticsearch/`, `connector/mysql/`, `connector_primitive/`, `connector/connector_registry.h`, `fs/`, `fs_ext/hdfs/`, `compute_env/staros/starlet_filesystem.h`, `common/`, `base/`, `gutil/`, `gen_cpp/`
-- Allowed target deps: `FileSystem`, `HdfsFileSystem`, `ComputeEnv`, `Connector`, `ConnectorHive`, `ConnectorIceberg`, `ConnectorLake`, `ConnectorBenchmark`, `ConnectorCacheStats`, `ConnectorJDBC`, `ConnectorElasticsearch`, `ConnectorMySQL`, `ConnectorPrimitive`, `Common`, `Base`, `Gutil`, `StarRocksGen`
+- Allowed internal include prefixes: `module/`, `connector/benchmark/`, `connector/cache_stats/`, `connector/file/`, `connector/hive/`, `connector/iceberg/`, `connector/lake/`, `connector/jdbc/`, `connector/elasticsearch/`, `connector/mysql/`, `connector_primitive/`, `connector/connector_registry.h`, `fs/`, `fs_ext/hdfs/`, `compute_env/staros/starlet_filesystem.h`, `common/`, `base/`, `gutil/`, `gen_cpp/`
+- Allowed target deps: `FileSystem`, `HdfsFileSystem`, `ComputeEnv`, `Connector`, `ConnectorFile`, `ConnectorHive`, `ConnectorIceberg`, `ConnectorLake`, `ConnectorBenchmark`, `ConnectorCacheStats`, `ConnectorJDBC`, `ConnectorElasticsearch`, `ConnectorMySQL`, `ConnectorPrimitive`, `Common`, `Base`, `Gutil`, `StarRocksGen`
 - Core tests: `module_bootstrap_test`
 - Remediation: Keep ModuleBootstrap as top-level default module composition; module implementations should expose registration hooks here instead of depending on service startup directly.
 
@@ -335,15 +328,15 @@ Orchestration layer below Service for query, fragment, and ingestion lifecycle e
 Diagnostic script execution and command dispatch layer below Service, HttpService, Tools, and AgentServer, and above the remaining reusable BE modules.
 - Targets: `Script`
 - Allowed internal include prefixes: `script/`, `base/`, `cache/`, `column/`, `common/`, `connector/`, `compute_env/`, `data_workflows/`, `exec/`, `exec_primitive/`, `exprs/`, `formats/`, `fs/`, `gen_cpp/`, `geo/`, `gutil/`, `io/`, `orchestration/`, `platform/`, `runtime/`, `storage/`, `storage_primitive/`, `types/`
-- Allowed target deps: `Base`, `Gutil`, `Common`, `Cache`, `IO`, `FileSystem`, `Platform`, `Types`, `ColumnCore`, `ChunkCore`, `ColumnSortCore`, `Runtime`, `Runtime`, `Runtime`, `Formats`, `StoragePrimitive`, `StorageBase`, `Storage`, `ComputeEnv`, `DataWorkflows`, `Expr`, `ExprDict`, `ExprTableFunction`, `ExprUtility`, `ExecPrimitive`, `ExecRuntime`, `ExecSchemaScannerCore`, `ExecSchemaScanners`, `ExecJoinCore`, `Exec`, `ConnectorPrimitive`, `Connector`, `ConnectorBenchmark`, `ConnectorCacheStats`, `ConnectorElasticsearch`, `ConnectorMySQL`, `ConnectorBuiltinRegistry`, `Orchestration`, `Geo`, `StarRocksGen`
+- Allowed target deps: `Base`, `Gutil`, `Common`, `Cache`, `IO`, `FileSystem`, `Platform`, `Types`, `ColumnCore`, `ChunkCore`, `ColumnSortCore`, `Runtime`, `Runtime`, `Runtime`, `Formats`, `StoragePrimitive`, `StorageBase`, `Storage`, `ComputeEnv`, `DataWorkflows`, `Expr`, `ExprDict`, `ExprTableFunction`, `ExprUtility`, `ExecPrimitive`, `ExecRuntime`, `ExecSchemaScannerCore`, `ExecSchemaScanners`, `ExecJoinCore`, `Exec`, `ConnectorPrimitive`, `Connector`, `ConnectorBenchmark`, `ConnectorCacheStats`, `ConnectorElasticsearch`, `ConnectorMySQL`, `Orchestration`, `Geo`, `StarRocksGen`
 - Core tests: `script_test`
 - Remediation: Keep Script below Service, HttpService, Tools, and AgentServer; lower reusable behavior can live in lower BE modules that Script is allowed to depend on.
 
 ### Service (`service`)
 Shared service-layer target above Script, runtime, cache, compute, and AgentServer without owning ServiceBE bootstrap code.
 - Targets: `Service`
 - Allowed internal include prefixes: `service/`, `script/`, `agent/`, `base/`, `cache/`, `column/`, `common/`, `connector/`, `compute_env/`, `exec/`, `exec_primitive/`, `exprs/`, `formats/`, `fs/`, `gen_cpp/`, `gutil/`, `http/`, `io/`, `platform/`, `orchestration/`, `runtime/`, `storage/`, `storage_primitive/`, `types/`
-- Allowed target deps: `Script`, `Runtime`, `Orchestration`, `Runtime`, `Runtime`, `Cache`, `AgentServer`, `ComputeEnv`, `ExecRuntime`, `ExecPrimitive`, `Platform`, `Storage`, `StoragePrimitive`, `StorageBase`, `FileSystem`, `IO`, `HttpService`, `Common`, `Base`, `Gutil`, `StarRocksGen`, `Connector`, `ConnectorFile`, `ConnectorPrimitive`, `ConnectorBuiltinRegistry`, `Exec`, `Formats`, `ChunkCore`, `ColumnCore`, `Types`
+- Allowed target deps: `Script`, `Runtime`, `Orchestration`, `Runtime`, `Runtime`, `Cache`, `AgentServer`, `ComputeEnv`, `ExecRuntime`, `ExecPrimitive`, `Platform`, `Storage`, `StoragePrimitive`, `StorageBase`, `FileSystem`, `IO`, `HttpService`, `Common`, `Base`, `Gutil`, `StarRocksGen`, `Connector`, `ConnectorFile`, `ConnectorPrimitive`, `Exec`, `Formats`, `ChunkCore`, `ColumnCore`, `Types`
 - Remediation: Keep shared Service below ServiceBE and depend on checked module targets such as AgentServer instead of ad hoc lower-layer reach-through.
 <!-- END GENERATED: BE MODULE HARNESSES -->
 

be/CMakeLists.txt

@@ -550,14 +550,6 @@ set(STARROCKS_LINK_LIBS
     Orchestration
     DataWorkflows
     ${WL_START_GROUP}
-    Connector
-    ConnectorLake
-    ConnectorFile
-    ConnectorHive
-    ConnectorIceberg
-    # Keep legacy connector registration in the static library group so its
-    # concrete file connector registration is retained by static linking.
-    ConnectorBuiltinRegistry
     Exec
     ${WL_WHOLE_ARCHIVE_PREFIX} ${EXPR_EXTENSION_LIBS} ${WL_WHOLE_ARCHIVE_SUFFIX}
     ${WL_END_GROUP}
@@ -890,7 +882,6 @@ endif()
 set (TEST_LINK_LIBS
     Orchestration
     ${STARROCKS_LINK_LIBS}
-    ConnectorBuiltinRegistry
     TestUtil
     ${WL_START_GROUP}
     gmock

be/module_boundary_manifest.json

@@ -1119,34 +1119,17 @@
       "allowed_test_link_deps": ["ConnectorMySQL", "ConnectorPrimitive", "Expr", "Runtime", "ChunkCore", "ColumnCore", "Types", "Common", "Base", "Gutil", "StarRocksGen"],
       "remediation": "Keep ConnectorMySQL limited to MySQL read-side scan and scanner logic; move registration into ModuleBootstrap and avoid pulling Connector, storage, service, or full Exec code into the connector library."
     },
-    {
-      "id": "connectorbuiltinregistry",
-      "doc_label": "ConnectorBuiltinRegistry",
-      "summary": "Top-level built-in connector registration composition above connector contracts and concrete connector libraries.",
-      "owned_targets": ["ConnectorBuiltinRegistry"],
-      "additional_owned_globs": ["be/src/connector/builtin_connector_registry.h"],
-      "allowed_include_prefixes": [
-        "connector/builtin_connector_registry.h",
-        "connector/",
-        "common/",
-        "base/",
-        "gutil/",
-        "gen_cpp/"
-      ],
-      "forbidden_include_prefixes": ["exec/", "runtime/", "storage/", "service/", "http/", "cache/", "util/"],
-      "allowed_target_deps": ["Connector", "ConnectorFile", "ConnectorPrimitive", "Common", "Base", "Gutil", "StarRocksGen"],
-      "remediation": "Keep legacy built-in connector registration as top-level composition for unsplit connector libraries; split connector libraries register through ModuleBootstrap instead of depending back on this target."
-    },
     {
       "id": "modulebootstrap",
       "doc_label": "ModuleBootstrap",
-      "summary": "Default BE module bootstrap composition for built-in module registration, including filesystem provider registry installation and split connector registration.",
+      "summary": "Default BE module bootstrap composition for built-in module registration, including filesystem provider registry installation and connector registration.",
       "owned_targets": ["ModuleBootstrap"],
       "owned_roots": ["be/src/module"],
       "allowed_include_prefixes": [
         "module/",
         "connector/benchmark/",
         "connector/cache_stats/",
+        "connector/file/",
         "connector/hive/",
         "connector/iceberg/",
         "connector/lake/",
@@ -1164,7 +1147,7 @@
         "gen_cpp/"
       ],
       "forbidden_include_prefixes": ["agent/", "cache/", "connector/", "data_workflows/", "exec/", "http/", "runtime/", "script/", "service/", "storage/", "util/"],
-      "allowed_target_deps": ["FileSystem", "HdfsFileSystem", "ComputeEnv", "Connector", "ConnectorHive", "ConnectorIceberg", "ConnectorLake", "ConnectorBenchmark", "ConnectorCacheStats", "ConnectorJDBC", "ConnectorElasticsearch", "ConnectorMySQL", "ConnectorPrimitive", "Common", "Base", "Gutil", "StarRocksGen"],
+      "allowed_target_deps": ["FileSystem", "HdfsFileSystem", "ComputeEnv", "Connector", "ConnectorFile", "ConnectorHive", "ConnectorIceberg", "ConnectorLake", "ConnectorBenchmark", "ConnectorCacheStats", "ConnectorJDBC", "ConnectorElasticsearch", "ConnectorMySQL", "ConnectorPrimitive", "Common", "Base", "Gutil", "StarRocksGen"],
       "allowed_test_targets": ["module_bootstrap_test"],
       "allowed_test_link_deps": ["ModuleBootstrap"],
       "remediation": "Keep ModuleBootstrap as top-level default module composition; module implementations should expose registration hooks here instead of depending on service startup directly."
@@ -1331,7 +1314,6 @@
         "ConnectorCacheStats",
         "ConnectorElasticsearch",
         "ConnectorMySQL",
-        "ConnectorBuiltinRegistry",
         "Orchestration",
         "Geo",
         "StarRocksGen"
@@ -1397,7 +1379,6 @@
         "Connector",
         "ConnectorFile",
         "ConnectorPrimitive",
-        "ConnectorBuiltinRegistry",
         "Exec",
         "Formats",
         "ChunkCore",

be/src/connector/CMakeLists.txt

@@ -319,16 +319,3 @@ if (WITH_CONNECTOR_JDBC)
             libxml2
     )
 endif()
-
-ADD_BE_LIB(ConnectorBuiltinRegistry
-        connector_registry_init.cpp
-)
-
-target_link_libraries(ConnectorBuiltinRegistry PUBLIC
-        Connector
-        ConnectorFile
-        Common
-        Base
-        Gutil
-        StarRocksGen
-)

be/src/connector/builtin_connector_registry.h

@@ -489,4 +489,4 @@ ADD_BE_LIB(Exec
 )
 
 target_link_libraries(Exec PUBLIC ExecRuntime ExecPrimitive ExecSchemaScannerCore ExecSchemaScanners ExecJoinCore Runtime ComputeEnv Cache)
-target_link_libraries(Exec PRIVATE Formats Platform Connector ConnectorLake ConnectorFile ConnectorHive ConnectorIceberg ConnectorBuiltinRegistry)
+target_link_libraries(Exec PRIVATE Formats Platform Connector ConnectorLake ConnectorFile ConnectorHive ConnectorIceberg)

be/src/connector/connector_registry_init.cpp

@@ -49,9 +49,7 @@
 #include "compute_env/pipeline/driver_limiter.h"
 #include "compute_env/workgroup/scan_executor.h"
 #include "compute_env/workgroup/work_group_manager.h"
-#include "connector/builtin_connector_registry.h"
 #include "connector/common/connector_sink_executor.h"
-#include "connector/connector_registry.h"
 #include "exec/batch_write/batch_write_mgr.h"
 #include "exec/lookup_stream_mgr.h"
 #include "exec/pipeline/query_context.h"
@@ -193,7 +191,6 @@ Status ExecEnv::init(ProcessMetricsRegistry* process_metrics_registry, RuntimeEn
     if (_compute_env == nullptr) {
         return Status::InternalError("ComputeEnv is not attached");
     }
-    RETURN_IF_ERROR(connector::install_builtin_connectors(connector::ConnectorRegistry::default_instance()));
     _process_metrics_registry = process_metrics_registry;
     auto* process_metrics = process_metrics_registry->root_registry();
     _table_metrics_mgr = process_metrics_registry->table_metrics_mgr();

be/src/exec/CMakeLists.txt

@@ -28,6 +28,7 @@ target_link_libraries(ModuleBootstrap PRIVATE
         HdfsFileSystem
         ComputeEnv
         Connector
+        ConnectorFile
         ConnectorHive
         ConnectorIceberg
         ConnectorLake

be/src/exec/exec_env.cpp

@@ -19,6 +19,7 @@
 
 #include "connector/cache_stats/cache_stats_connector.h"
 #include "connector/connector_registry.h"
+#include "connector/file/file_connector.h"
 #include "connector/hive/hive_connector.h"
 #include "connector/lake/lake_connector.h"
 #include "connector_primitive/connector.h"
@@ -61,6 +62,7 @@ Status bootstrap_builtin_connectors() {
     auto* registry = ConnectorRegistry::default_instance();
     DCHECK(registry != nullptr);
     install_if_absent<HiveConnector>(registry, Connector::HIVE);
+    install_if_absent<FileConnector>(registry, Connector::FILE);
     install_if_absent<LakeConnector>(registry, Connector::LAKE);
     install_if_absent<CacheStatsConnector>(registry, Connector::CACHE_STATS);
 #ifndef __APPLE__