Skip to content

AGENTS.md

Latest commit

 

History

History
358 lines (298 loc) · 29.7 KB

AGENTS.md

File metadata and controls

358 lines (298 loc) · 29.7 KB
Raw

AGENTS.md - StarRocks Backend

Backend operating contract for agentic work in be/.

Read This First

Harness Commands

# Full BE architecture check
python3 build-support/check_be_module_boundaries.py --mode full

# PR-style changed-files check
python3 build-support/check_be_module_boundaries.py --mode changed --base origin/main

# PR-style check plus baseline shrink-only guard
python3 build-support/check_be_module_boundaries.py --mode changed --base origin/main --enforce-baseline-shrink

# Verify this file matches the manifest
python3 build-support/render_be_agents.py --check

# Rewrite the generated module section after manifest edits
python3 build-support/render_be_agents.py --write

Fast Loops

# Standard BE build
./build.sh --be

# Debug or ASAN build
BUILD_TYPE=Debug ./build.sh --be
BUILD_TYPE=ASAN ./build.sh --be

# Fast single-binary UT loop
./run-be-ut.sh --build-target <test_binary> --module <test_binary> --without-java-ext

Useful core binaries for fast iterations:

  • base_test
  • io_test
  • fs_test
  • types_test
  • column_test
  • runtime_test
  • expr_test

Core C++ Rules

  • Format with the repo .clang-format.
  • Prefer #pragma once.
  • Keep include order: corresponding header, C system, C++ stdlib, third-party, StarRocks.
  • Use Status and StatusOr for recoverable errors.
  • Treat hot-path allocations and row-by-row virtual dispatch as performance smells.

Architecture Workflow

  • If you change BE layering rules, edit be/module_boundary_manifest.json first.
  • After manifest changes, run python3 build-support/render_be_agents.py --write.
  • Before finishing BE work, run python3 build-support/check_be_module_boundaries.py --mode full.
  • If the check fails on a pre-existing edge, confirm it is already listed in build-support/be_module_boundary_baseline.json.
  • If the check fails on a new edge, fix the code or update the manifest. Do not add to the baseline unless the debt is deliberate and reviewed.
  • If you touch the baseline file, run the changed-files check with --enforce-baseline-shrink; fixed entries should be deleted, never rewritten into new debt.

Module Harness

This section is generated from be/module_boundary_manifest.json. Run python3 build-support/render_be_agents.py --write after changing the manifest. Run python3 build-support/check_be_module_boundaries.py --mode full to validate the same rules mechanically.

Base (base)

Lowest-level BE primitives. Keep it free of higher-level StarRocks module dependencies.

  • Targets: Base
  • Allowed internal include prefixes: base/, gutil/, gen_cpp/
  • Allowed target deps: Gutil
  • Core tests: base_test
  • Remediation: Move shared helpers into Base/Gutil or introduce a lower-level interface instead of pulling in higher-level BE modules.

Gutil (gutil)

Standalone utility substrate. Do not couple it to BE modules.

  • Targets: Gutil
  • Allowed internal include prefixes: gutil/
  • Allowed target deps: Common, Base, Gutil
  • Remediation: Keep Gutil independent; move BE-specific logic out instead of importing BE modules.

Common (common)

Core shared infrastructure above Base/Gutil and generated code only. Higher-level BE modules must not leak back into it.

  • Targets: Common
  • Allowed internal include prefixes: common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Base, Gutil, StarRocksGen
  • Core tests: common_test
  • Remediation: Move the dependency upward or add a lower-level abstraction; Common may only depend on Base, Gutil, and generated code.

Cache (cache)

Cache implementation module for DataCache facade, cache engines, scan read-buffer/cache stream wrappers, monitors, metrics, utilities, StarCache integration, and peer-cache RPC reads without service/bootstrap or ExecEnv singleton coupling.

  • Targets: Cache
  • Allowed internal include prefixes: cache/, common/brpc/, runtime/current_thread.h, runtime/mem_tracker.h, fs/, io/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Runtime, FileSystem, IO, Common, Base, Gutil, StarRocksGen
  • Core tests: cache_test
  • Remediation: Keep Cache self-contained within cache engines, scan read-buffer/cache stream wrappers, monitors, metrics, utilities, and injected peer-cache BRPC stubs; keep service startup, storage code, HTTP/admin code, and ExecEnv singleton access outside cache.

HttpService (httpservice)

BE HTTP pages, admin actions, load actions, download helpers, and web handlers.

  • Targets: HttpService
  • Allowed internal include prefixes: http/, cache/, compute_env/, data_workflows/, exec/, exec_primitive/, exprs/, fs/, io/, orchestration/, platform/, runtime/, storage/, storage_primitive/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Storage, DataWorkflows, Orchestration, Runtime, ComputeEnv, Exec, ExecRuntime, Expr, ExprDict, ExprTableFunction, ExprUtility, FileSystem, IO, Platform, StoragePrimitive, Cache, Common, Base, Gutil, StarRocksGen
  • Remediation: Keep BE HTTP pages, actions, download helpers, and web handlers in HttpService; keep reusable lower-level helpers in their owning modules such as Platform and Runtime.

IO (io)

Minimal IO foundation used by FS and upper layers.

  • Targets: IO
  • Allowed internal include prefixes: io/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Common, Base, Gutil
  • Core tests: io_test
  • Remediation: Keep IO free of higher FS/runtime/storage/exec/service code; move the dependency upward or add a lower-level interface.

FileSystem (filesystem)

Filesystem module on top of IO, including core abstractions and non-JNI filesystem providers.

  • Targets: FileSystem
  • Allowed internal include prefixes: fs/, io/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: IO, Common, Base, Gutil
  • Core tests: fs_test
  • Remediation: Keep FileSystem limited to IO plus filesystem code that does not require runtime/JNI integration; move runtime-backed filesystem providers to fs_ext.

HdfsFileSystem (hdfsfilesystem)

HDFS filesystem extension above FileSystem for libhdfs/JNI runtime integration.

  • Targets: HdfsFileSystem
  • Allowed internal include prefixes: fs_ext/hdfs/, fs/, io/, runtime/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: FileSystem, Runtime, IO, Common, Base, Gutil, StarRocksGen
  • Core tests: hdfs_filesystem_test
  • Remediation: Keep HDFS-specific libhdfs/JNI integration in fs_ext; shared filesystem contracts belong in FileSystem.

Platform (platform)

Shared BE platform utilities above IO/FS/Common and below Runtime/Exec/Storage/Service, including reusable HTTP client/server primitives.

  • Targets: Platform
  • Allowed internal include prefixes: platform/, fs/, io/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: FileSystem, IO, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: platform_test
  • Remediation: Keep Platform limited to reusable host, filesystem-adjacent, download, temp-file, environment, retry, HTTP primitive, and platform-level helpers; move runtime, exec, storage, and service integration upward.

Types (typecore)

Core type system without runtime/storage/exec coupling.

  • Targets: Types
  • Allowed internal include prefixes: types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Common, Base, Gutil
  • Core tests: types_test
  • Remediation: Keep Types independent of runtime/util/storage/exec layers; move integration code into higher layers.

ColumnCore (columncore)

Core column representations that must stay independent of ChunkCore and higher layers.

  • Targets: ColumnCore
  • Allowed internal include prefixes: column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Types, Common, Base, Gutil, StarRocksGen
  • Core tests: column_test
  • Remediation: Keep ColumnCore free of ChunkCore/Runtime/Exec/Storage coupling; move integration code upward or introduce an interface.

ChunkCore (chunkcore)

Core chunk, schema, field, and chunk-adjacent helpers on top of ColumnCore.

  • Targets: ChunkCore
  • Allowed internal include prefixes: column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: chunk_test
  • Remediation: Keep ChunkCore limited to chunk/schema helpers and free of Runtime/Exec/Storage-specific serialization coupling.

ColumnSortCore (columnsortcore)

Reusable column and chunk sorting primitives above ChunkCore without runtime, exec, storage, or expression coupling.

  • Targets: ColumnSortCore
  • Allowed internal include prefixes: column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: column_sort_core_test
  • Remediation: Keep ColumnSortCore limited to reusable sorting and permutation primitives; move runtime merge/cursor orchestration to ComputeEnv or higher layers.

Runtime (runtime)

Core runtime building blocks and process-scoped runtime environment resources without Exec/Storage/Service coupling.

  • Targets: Runtime
  • Allowed internal include prefixes: runtime/, platform/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Platform, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: runtime_test
  • Remediation: Keep Runtime restricted to core runtime infrastructure and process-scoped runtime environment resources; move execution, service, storage, stream-load, and connector integration into higher modules.

Format (format)

Format module for core format primitives and concrete CSV, JSON, Avro, ORC, and Parquet implementations.

  • Targets: Formats, FormatParquet, FormatOrc, FormatAvro, FormatJson, FormatCsv, FormatCore
  • Allowed internal include prefixes: formats/, cache/, compute_env/, exec_primitive/runtime_filter/, exprs/, runtime/, fs/, io/, storage_primitive/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: ComputeEnv, StoragePrimitive, ExecPrimitive, Expr, Runtime, FileSystem, IO, Cache, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen, orc
  • Core tests: format_test
  • Remediation: Keep format code inside the Format module; move connector orchestration and higher execution policy upward.

StoragePrimitive (storageprimitive)

Shared storage-access primitives, predicate contracts, predicate trees, iterator/merge/aggregation helpers, and value/schema types used by Storage, Exec, Connector, Format, and ComputeEnv without concrete Storage engine, tablet, rowset, lake, service, or full Exec coupling.

  • Targets: StoragePrimitive
  • Allowed internal include prefixes: storage_primitive/, exec_primitive/, exprs/, runtime/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: ExecPrimitive, Expr, Runtime, ColumnSortCore, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: storage_primitive_test
  • Remediation: Keep StoragePrimitive limited to reusable storage-access contracts, predicate contracts, predicate trees, iterator/merge/aggregation helpers, and value/schema types; move concrete storage engine, tablet, rowset, lake, service, concrete connector, and full Exec integration upward.

StorageBase (storagebase)

Base storage algorithms above StoragePrimitive and ComputeEnv without concrete Storage engine, tablet, rowset, lake, service, or full Exec coupling.

  • Targets: StorageBase
  • Allowed internal include prefixes: storage/base/, storage_primitive/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: ComputeEnv, StoragePrimitive, Runtime, ColumnSortCore, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: storage_base_test
  • Remediation: Keep StorageBase limited to reusable storage algorithms and helpers that may depend on ComputeEnv; move concrete storage engine, tablet, rowset, lake, service, connector, cache, and full Exec integration upward.

Storage (storage)

Concrete storage engine: tablet/rowset/segment management, compaction, persistent index, lake (cloud-native) tablets, binlog, inverted/vector indexes, and sstable, depending only on ComputeEnv's dependency closure plus StorageBase/StoragePrimitive and Platform HTTP helpers for replication transport, without Exec/Service/Connector/Agent reach-up or ExecEnv singleton coupling.

  • Targets: Storage
  • Allowed internal include prefixes: storage/, compute_env/, cache/, exec_primitive/, storage_primitive/, exprs/, runtime/, platform/, fs/, io/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: StorageBase, StoragePrimitive, ExecPrimitive, ComputeEnv, Cache, Expr, Runtime, Platform, FileSystem, IO, ColumnSortCore, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Remediation: Keep Storage limited to the concrete storage engine over ComputeEnv's dependency closure plus StorageBase/StoragePrimitive and Platform HTTP helpers; move Exec, Service, Connector, Agent, and DataWorkflows orchestration upward and route ExecEnv access through injected dependencies.

ComputeEnv (computeenv)

Shared compute-side BE environment boundary for process-scoped compute resources, StarOS/Starlet worker runtime, load-path management, query-scoped scan coordination helpers, query-cache primitives, WorkGroup scheduling/executor resources, StarCache coordination, and pipeline controls below full Exec/Storage and above Runtime.

  • Targets: ComputeEnv
  • Allowed internal include prefixes: compute_env/, cache/, exec_primitive/, storage_primitive/, exprs/, runtime/, platform/, fs/, io/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Cache, StoragePrimitive, ExecPrimitive, Expr, Runtime, Platform, FileSystem, IO, ColumnSortCore, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: compute_env_test, compute_env_sorting_test
  • Remediation: Keep ComputeEnv limited to process-scoped compute resources, StarOS/Starlet worker runtime, load-path management without ExecEnv or concrete storage engine coupling, query-scoped scan coordination helpers without concrete scan/storage policy, query-cache primitives without concrete storage policy, WorkGroup scheduling/executor resources, shared compute-side service contracts, stable execution primitives, reusable compute-side sorting algorithms, StarCache coordination, and spill infrastructure; move concrete Exec, Storage, Service, Connector, and Agent integration upward.

AgentServer (agentserver)

FE-agent task orchestration, heartbeat handling, agent metrics, and agent worker helpers below Service.

  • Targets: AgentServer
  • Allowed internal include prefixes: agent/, data_workflows/, cache/, compute_env/workgroup/, exec/pipeline/query_context.h, exec/runtime/query_context_manager.h, fs/, io/, runtime/, storage/, storage_primitive/, platform/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Runtime, DataWorkflows, Storage, Cache, ComputeEnv, Exec, ExecRuntime, StoragePrimitive, Platform, FileSystem, IO, Common, Base, Gutil, StarRocksGen
  • Remediation: Keep AgentServer as FE-agent task and heartbeat orchestration below Service; move service/bootstrap integration upward.

DataWorkflows (dataworkflows)

Data-plane workflow orchestration above concrete Storage and Exec for load, delete, schema-change, and tablet-write workflows without service/bootstrap ownership.

  • Targets: DataWorkflows
  • Allowed internal include prefixes: data_workflows/, connector/file/, storage/, storage_primitive/, exec/, exec_primitive/, runtime/, compute_env/, platform/, fs/, io/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Storage, Exec, ConnectorFile, Runtime, StorageBase, StoragePrimitive, ExecRuntime, ExecPrimitive, ComputeEnv, Runtime, Runtime, Platform, FileSystem, IO, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: data_workflows_test
  • Remediation: Keep DataWorkflows limited to data-plane orchestration over concrete Storage and Exec; move service/bootstrap, agent scheduling, connector registration, and lower reusable primitives to their owning layers.

Expr (expr)

Core expression infrastructure, UDF runtime helpers, builtin aggregate registry, and aggregate-state combinator helpers that depend only on Platform, Geo, Runtime, and lower layers.

  • Targets: Expr
  • Allowed internal include prefixes: exprs/, platform/, geo/, runtime/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Platform, Geo, Runtime, ColumnSortCore, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: expr_test, expr_java_test
  • Remediation: Keep Expr limited to core expression infrastructure, UDF runtime helpers, builtin scalar helpers, builtin aggregate registry and aggregate-state combinator helpers, and dependencies no higher than Platform, Geo, Runtime, and lower layers.

ExecPrimitive (execprimitive)

Primitive execution contracts, DataSink base contract, runtime-filter infrastructure, generic morsel queues, and stable pipeline operator/factory primitives without broader Exec runtime, scheduler, storage, service, or connector coupling.

  • Targets: ExecPrimitive
  • Allowed internal include prefixes: exec_primitive/, exprs/, runtime/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Expr, Runtime, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: exec_primitive_test
  • Remediation: Keep ExecPrimitive limited to execution contracts, the DataSink base contract, runtime filters, generic morsel queues, and stable operator/factory primitives; move runtime, scheduler, concrete operators, concrete sinks, storage, service, and connector integration upward.

ExecRuntime (execruntime)

Operator-tree execution framework for query and fragment contexts, driver lifecycle, execution groups, and scheduling-adjacent behavior above ComputeEnv and ExecPrimitive without concrete operators, storage, service, connector, cache, or broad Exec coupling.

  • Targets: ExecRuntime
  • Allowed internal include prefixes: compute_env/, exec/runtime/, exec_primitive/, exec/pipeline/operator_with_dependency.h, storage_primitive/, exprs/, runtime/, platform/, fs/, io/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: ComputeEnv, ExecPrimitive, StoragePrimitive, Expr, Runtime, Platform, FileSystem, IO, ColumnSortCore, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: exec_runtime_test
  • Remediation: Keep ExecRuntime limited to the operator-tree execution framework and runtime behavior that can be expressed through ComputeEnv and ExecPrimitive contracts; move concrete operators, storage, service, connector, cache, HTTP, and broad Exec integration upward.

ConnectorPrimitive (connectorprimitive)

Connector contracts, DataSource, DataSourceProvider default mechanics, and generic connector sink memory management interfaces without concrete connectors, sinks, registry composition, storage, service, or full Exec coupling.

  • Targets: ConnectorPrimitive
  • Allowed internal include prefixes: connector_primitive/, exec_primitive/, exprs/, runtime/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: ExecPrimitive, Expr, Runtime, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: connector_primitive_test
  • Remediation: Keep ConnectorPrimitive limited to connector contracts, default scan-range-to-morsel mechanics, and generic sink memory policies expressed through lower-level interfaces; move concrete connectors, reusable sink/file-writer helpers, registry wiring, storage, service, and full Exec integration upward.

ConnectorCommon (connectorcommon)

Reusable connector sink/file-writer helpers above connector contracts, including sink commit/profile types and shared writer-side utilities, without concrete connector registry composition, service, HTTP, or full Exec coupling.

  • Targets: ConnectorCommon
  • Allowed internal include prefixes: connector/common/, connector_primitive/, compute_env/, cache/, storage_primitive/, exec_primitive/, exprs/, runtime/, platform/, formats/, fs/, io/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: ComputeEnv, ConnectorPrimitive, Cache, StoragePrimitive, ExecPrimitive, Expr, Runtime, Platform, FormatCore, FileSystem, IO, ColumnSortCore, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: connector_common_test
  • Remediation: Keep ConnectorCommon limited to reusable connector sink/file-writer helpers expressed through connector contracts and compute/runtime lower layers; move concrete connectors, registry wiring, service, HTTP, storage-engine-specific behavior, and full Exec integration upward.

ModuleBootstrap (modulebootstrap)

Default BE module bootstrap composition for built-in module registration, including filesystem provider registry installation and connector registration.

  • Targets: ModuleBootstrap
  • Allowed internal include prefixes: module/, connector/benchmark/, connector/cache_stats/, connector/file/, connector/hive/, connector/iceberg/, connector/lake/, connector/jdbc/, connector/elasticsearch/, connector/mysql/, connector_primitive/, connector/connector_registry.h, fs/, fs_ext/hdfs/, compute_env/staros/starlet_filesystem.h, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: FileSystem, HdfsFileSystem, ComputeEnv, Connector, ConnectorFile, ConnectorHive, ConnectorIceberg, ConnectorLake, ConnectorBenchmark, ConnectorCacheStats, ConnectorJDBC, ConnectorElasticsearch, ConnectorMySQL, ConnectorPrimitive, Common, Base, Gutil, StarRocksGen
  • Core tests: module_bootstrap_test
  • Remediation: Keep ModuleBootstrap as top-level default module composition; module implementations should expose registration hooks here instead of depending on service startup directly.

ExecSchemaScannerCore (execschemascannercore)

Schema scanner base contract and shared mechanics without concrete scanner, pipeline, storage, service, or ExecEnv coupling.

  • Targets: ExecSchemaScannerCore
  • Allowed internal include prefixes: exec/schema_scanner.h, exprs/, runtime/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: Expr, Runtime, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: schema_scanner_core_test
  • Remediation: Keep SchemaScannerCore limited to the base SchemaScanner contract; move concrete scanner creation and service-specific logic into higher schema scanner modules.

ExecSchemaScanners (execschemascanners)

Clean concrete schema scanners that do not depend on SchemaHelper, FE RPC/client helpers, storage, service, cache, pipeline, or ExecEnv.

  • Targets: ExecSchemaScanners
  • Allowed internal include prefixes: exec/schema_scanner.h, exec/schema_scanner/schema_column_filler.h, runtime/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: ExecSchemaScannerCore, Runtime, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: exec_schema_scanners_test
  • Remediation: Keep this first schema scanner target limited to clean local/static scanners; leave SchemaHelper, storage, HTTP, cache, service, and ExecEnv users in higher compatibility modules until they get explicit boundaries.

ExecJoinCore (execjoincore)

Reusable exec join hash table algorithms without join nodes, pipeline, storage, service, or util coupling.

  • Targets: ExecJoinCore
  • Allowed internal include prefixes: exec/join/, exprs/, runtime/, column/, types/, common/, base/, gutil/, gen_cpp/
  • Allowed target deps: ComputeEnv, Expr, Runtime, ColumnSortCore, ChunkCore, ColumnCore, Types, Common, Base, Gutil, StarRocksGen
  • Core tests: exec_join_core_test
  • Remediation: Keep ExecJoinCore limited to reusable join hash table algorithms; leave join nodes, pipeline operators, storage/service integration, and util diagnostics in higher modules.

Orchestration (orchestration)

Orchestration layer below Service for query, fragment, and ingestion lifecycle entrypoints over concrete runtime and execution modules.

  • Targets: Orchestration
  • Allowed internal include prefixes: orchestration/, data_workflows/, connector_primitive/, column/, exec/, exec_primitive/, runtime/, compute_env/, platform/, common/, base/, gutil/, gen_cpp/, types/
  • Allowed target deps: DataWorkflows, Runtime, Exec, ExecRuntime, ExecPrimitive, ComputeEnv, ConnectorPrimitive, Platform, ColumnCore, Runtime, Common, Base, Gutil, StarRocksGen, Types
  • Core tests: orchestration_test
  • Remediation: Keep Orchestration below Service; move transport-specific RPC handling to Service and lower reusable execution/runtime or ingestion primitives to their owning modules.

Script (script)

Diagnostic script execution and command dispatch layer below Service, HttpService, Tools, and AgentServer, and above the remaining reusable BE modules.

  • Targets: Script
  • Allowed internal include prefixes: script/, base/, cache/, column/, common/, connector/, compute_env/, data_workflows/, exec/, exec_primitive/, exprs/, formats/, fs/, gen_cpp/, geo/, gutil/, io/, orchestration/, platform/, runtime/, storage/, storage_primitive/, types/
  • Allowed target deps: Base, Gutil, Common, Cache, IO, FileSystem, Platform, Types, ColumnCore, ChunkCore, ColumnSortCore, Runtime, Runtime, Runtime, Formats, StoragePrimitive, StorageBase, Storage, ComputeEnv, DataWorkflows, Expr, ExprDict, ExprTableFunction, ExprUtility, ExecPrimitive, ExecRuntime, ExecSchemaScannerCore, ExecSchemaScanners, ExecJoinCore, Exec, ConnectorPrimitive, Connector, ConnectorBenchmark, ConnectorCacheStats, ConnectorElasticsearch, ConnectorMySQL, Orchestration, Geo, StarRocksGen
  • Core tests: script_test
  • Remediation: Keep Script below Service, HttpService, Tools, and AgentServer; lower reusable behavior can live in lower BE modules that Script is allowed to depend on.

Service (service)

Shared service-layer target above Script, runtime, cache, compute, and AgentServer without owning ServiceBE bootstrap code.

  • Targets: Service
  • Allowed internal include prefixes: service/, script/, agent/, base/, cache/, column/, common/, connector/, compute_env/, exec/, exec_primitive/, exprs/, formats/, fs/, gen_cpp/, gutil/, http/, io/, platform/, orchestration/, runtime/, storage/, storage_primitive/, types/
  • Allowed target deps: Script, Runtime, Orchestration, Runtime, Runtime, Cache, AgentServer, ComputeEnv, ExecRuntime, ExecPrimitive, Platform, Storage, StoragePrimitive, StorageBase, FileSystem, IO, HttpService, Common, Base, Gutil, StarRocksGen, Connector, ConnectorFile, ConnectorPrimitive, Exec, Formats, ChunkCore, ColumnCore, Types
  • Remediation: Keep shared Service below ServiceBE and depend on checked module targets such as AgentServer instead of ad hoc lower-layer reach-through.

BE-Specific Sync Rules

  • BE configs are declared in be/src/common/config.h; update docs/en/administration/management/BE_configuration.md and docs/zh/ peer docs when config behavior changes.
  • BE metrics must update the matching monitoring docs when names, meanings, or labels change.
  • be/src/common has extra config-header rules in be/src/common/AGENTS.md.

Verification Before Handoff

Run the smallest relevant UT binary plus the architecture harness:

python3 build-support/check_be_module_boundaries.py --mode full
python3 build-support/render_be_agents.py --check

If you changed a core module, prefer its focused test binary before broader run-be-ut.sh.