Skip to content

[1928-a3] feat(config): remove input whitelist transitional constraints#2642

Draft
Takuka0311 wants to merge 6 commits into
alibaba:mainfrom
Takuka0311:feat/1928-a3-auto
Draft

[1928-a3] feat(config): remove input whitelist transitional constraints#2642
Takuka0311 wants to merge 6 commits into
alibaba:mainfrom
Takuka0311:feat/1928-a3-auto

Conversation

@Takuka0311

Copy link
Copy Markdown
Collaborator

背景

Epic #2595 · Discussion #1928 步骤 A3。依赖 A1 (#2596)、A2 (#2597) 均已合入。

A1 将 Input 类型白名单重构为「默认允许 + 黑名单(保持旧行为)」的过渡框架;A2 已按 Input 类型补齐 C++ 事件产出。本 PR 拆除该过渡闸门:任意注册在 PluginRegistry 的 Native Input,只要 Pipeline 含 Go Processor / Flusher,均可进入 Go——不再维护 Input 类型白名单

改动

  • core/config/CollectionConfig.cpp
    • 删除 GetNativeInputBlacklistWhenUsingGoPlugin / IsNativeInputBlockedWhenUsingGoPluginhasFileInput 分支(含 #ifdef APSARA_UNIT_TEST_MAIN 两支)
    • 删除 input_file 单 Input 过渡约束("more than 1 input_file …"),允许多个 Native 文件型 Input 共存
    • 移除处理器 / flusher 侧的黑名单共存闸门
    • 梳理并保留 Native + Go 多 Flusher 共存规则(有 Go flusher 时 native flusher 由 Go pipeline 供给,当前仅 flusher_sls 可承接)→ 至多 1 个 native flusher 且须为 flusher_sls;注释指向 B-line FlusherV2.Export([1928-b3] Extended Flusher 补齐 FlusherV2.Export #2623 / [1928-B4] Processor 迁移 + flusher_sls v2(按目录分批) #2603)解除
    • 移除随之不再使用的 #include <unordered_set>
  • core/collection_pipeline/CollectionPipeline.cpp
    • 文件型 Input 特例(DefaultLogQueueSize)指针改用 mInputs.back() 而非 mInputs[0],修正多 Input / 文件 Input 非首位时的错误 cast
  • core/unittest/config/CollectionConfigUnittest.cpp
    • NativeInputWithGoPlugin:由「黑名单拒绝」改为断言默认放行
    • 新增 MultiInputAndFlusherCoexistence:多 input_file + flusher_sls、Native+Go 多 Flusher 规则(成功与失败路径)
    • HandleInvalidInputs 移除已不再非法的「多个 input_file」用例

不改默认 StructureType。 变更仅放宽校验:原先合法的配置行为不变,原先被拒的多 Input / Native+Go 组合现予放行。

验收标准

  • CollectionConfig / CollectionPipeline 相关 UT 通过(本地实跑,见下)
  • 多 Input 与 flusher_sls 共存规则有测试或文档(新增 UT + 代码注释 + 本说明)
  • 不改默认 StructureType

Test plan(本地实跑,非「待 CI」)

Docker 镜像 loongcollector-build-linux:2.1.17-Werror 编译通过(warning-clean)。

collection_config_unittest : 12/12 PASSED
  ├ CollectionConfigUnittest.NativeInputWithGoPlugin            (default-allow)
  ├ CollectionConfigUnittest.MultiInputAndFlusherCoexistence    (新增)
  └ …(HandleValidConfig / HandleInvalid* / SelfMonitorInputWithGoFlusher / TestReplaceEnvVarRef)
pipeline_unittest          : 12/12 PASSED
  └ 含 OnInitVariousTopology / TestMultiFlusherAndRouter / OnInputFileWith*

Closes #2600

🤖 Generated with Claude Code

Remove the input-type whitelist/blacklist gate introduced as a transitional
bridge in A1 (alibaba#2596). Any registered native input may now coexist with Go
processors and/or Go flushers, realizing "no more Input-type whitelist"
(Discussion alibaba#1928, step A3). A2 (alibaba#2597) already ensures native inputs produce
proper C++ events, making this safe.

- drop GetNativeInputBlacklistWhenUsingGoPlugin / IsNativeInputBlockedWhenUsingGoPlugin
  and the hasFileInput branches; remove the "more than 1 input_file" single-input
  constraint so multiple native file inputs may coexist
- clarify the Native + Go multi-flusher coexistence rule (kept intentionally: only
  flusher_sls can currently be fed by the Go pipeline) with a forward reference to
  the B-line FlusherV2.Export work
- CollectionPipeline: capture the file-input pointer via mInputs.back() instead of
  mInputs[0] so detection is correct when a file input is not first
- update/add UTs: NativeInputWithGoPlugin now asserts default-allow;
  MultiInputAndFlusherCoexistence covers multi input_file + flusher_sls and the
  Native+Go multi-flusher rule (success and failure paths)

Does not change the default StructureType.

Closes alibaba#2600

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread core/collection_pipeline/CollectionPipeline.cpp Outdated
Comment thread core/config/CollectionConfig.cpp
Comment thread core/config/CollectionConfig.cpp Outdated
Comment thread core/config/CollectionConfig.cpp Outdated
Takuka0311 and others added 2 commits July 7, 2026 11:10
Replace mInputs.back() with a named raw pointer (currentInput) captured
before the move into mInputs, so the special-treatment binding for
input_file / input_container_stdio / input_static_file is self-explanatory.
Behavior is unchanged: both bind the input created in the current loop
iteration, which need not be mInputs[0] now that the input whitelist is
removed and multiple inputs may coexist.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review feedback: task-tracking labels like "A3" carry no meaning in
the source and confuse readers. Strip them from the input-coexistence
comments in CollectionConfig and CollectionConfigUnittest, keeping the
technical description of the removed input whitelist/blacklist gate.

Comment-only change; no behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Native file inputs (input_file / input_container_stdio /
input_static_file_onetime) register their per-input state in FileServer
keyed by config name only, so multiple such inputs in one pipeline would
overwrite each other's registration and silently lose data. Restore the
constraint that a native file input must be the sole input of its
pipeline until FileServer keys registrations by (config, index).

The A3 whitelist/blacklist removal is unaffected: a single native input
still coexists freely with Go processors/flushers.

Refs alibaba#2644
Fixes alibaba#2600
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread core/config/CollectionConfig.cpp
Takuka0311 and others added 2 commits July 7, 2026 12:04
A3 removes the input whitelist/blacklist gate but keeps two
transitional restrictions the reviewer flagged as still
unimplemented: the sole native file input rule (FileServer keys
by config name only, tracked in alibaba#2644) and the single flusher_sls
rule when Go flushers coexist. Restore their "remove this
restriction" TODO markers so the pending cleanup stays visible in
code, without epic/line labels.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round review: the two "remove these special restrictions" TODOs above the
hasFileInput detection blocks were dropped, but the file-input special
casing they mark is still in place (unimplemented). Restore them per
reviewer guidance to keep useful TODOs for restrictions not yet removed.

Fixes alibaba#2600

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Takuka0311 Takuka0311 changed the title feat(config): [1928-A3] remove input whitelist transitional constraints [1928-a3] feat(config): remove input whitelist transitional constraints Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[1928-A3] 删除 Input 白名单与过渡约束

1 participant