Skip to content

feat(inference): 引入串行/行并行/树并行推理后端并按后端拆分 CI#12

Open
huangbogeng wants to merge 4 commits into
mainfrom
dev-feature
Open

feat(inference): 引入串行/行并行/树并行推理后端并按后端拆分 CI#12
huangbogeng wants to merge 4 commits into
mainfrom
dev-feature

Conversation

@huangbogeng

Copy link
Copy Markdown
Owner

Summary

  • 增加编译期可选且互斥的推理后端特性:
    • infer-serial(默认)
    • infer-row-parallel
    • infer-tree-parallel
  • 将推理内核重构为后端分发模式,XgbModel::predict_dense 公共 API 保持不变。
  • 实现行级并行(微批优先,含小批量回退串行阈值)与树级并行(稳定优先:固定分块 + 确定性归并 + 补偿求和)。
  • 增加树并行重复运行确定性测试,确保稳定输出。
  • CI 从 --all-features 改为按后端矩阵分别执行 clippy/test,并同步更新 README、CONTRIBUTING、PR 模板和 CHANGELOG。

Motivation

面向低延迟/微批推理场景,需要在吞吐与稳定性之间提供可控选择。
通过编译期特性切换后端,可避免运行时策略分支开销,并确保部署形态可预测。

Validation

  • cargo fmt --all
  • cargo clippy --all-targets -- -D warnings
  • cargo test --all-targets
  • cargo clippy --all-targets --no-default-features --features infer-row-parallel -- -D warnings
  • cargo test --all-targets --no-default-features --features infer-row-parallel
  • cargo clippy --all-targets --no-default-features --features infer-tree-parallel -- -D warnings
  • cargo test --all-targets --no-default-features --features infer-tree-parallel

Notes

  • predict_dense 对外接口未变,更改集中在内部后端实现与构建特性。
  • 后端特性被显式约束为“仅可启用一个”,若配置冲突会在编译期报错。
  • 由于后端特性互斥,CI 不再使用 --all-features

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c3a6008de6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/inference.rs
Comment on lines +183 to +184
let chunk_count = usize::min(rayon::current_num_threads(), trees.len());
let chunk_ranges = tree_chunk_ranges(trees.len(), chunk_count);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Decouple tree chunking from Rayon thread count

predict_dense_scalar_tree_parallel derives chunk_count from rayon::current_num_threads(), so the chunk boundaries (and therefore floating-point summation order) change when the pool size changes, e.g. different RAYON_NUM_THREADS values or custom pool sizes across environments. Because IEEE-754 addition is non-associative, the same model/input can produce different predictions across deployments, which conflicts with the deterministic backend goal and can flip multi:softmax decisions near ties. Use a chunking strategy that is stable for a given model (or make the thread-count dependency explicit and tested) to keep outputs reproducible.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant