[TRTLLMINF-336][infra] BOLT profile merge: skip runtime tensorrt import gate (libs-only)

[mlefeb01]

Description

The cross-workload BOLT profile merge (scripts/bolt/internal/slurm_merge.sh) failed in setup_env.sh's runtime import tensorrt gate: the profiling base image (pytorch-…-skip-tritondevel) has no importable tensorrt before install. The merge doesn't need one — it only reconstructs the wheel's ELF .so files on disk to convert .fdata → .yaml (llvm-bolt <orig.so> -data -w <yaml>) and to re-BOLT via apply_bolt.py, and library discovery resolves the package by path (importlib.util.find_spec), not by importing it.

Changes

• Add a BOLT_SETUP_LIBS_ONLY=1 mode to setup_env.sh that skips the runtime import tensorrt / import tensorrt_llm checks while keeping the pip install --no-deps that lays the .sos on disk.
• Pass BOLT_SETUP_LIBS_ONLY=1 from slurm_merge.sh.

Test Coverage

Re-ran the merge end-to-end against real collected profiles: libs-only setup_env.sh → backup_libraries → merge-fdata → .fdata → .yaml → bundle packaging → apply_bolt.py re-BOLT all completed successfully.

Dev Engineer Review

• Added BOLT_SETUP_LIBS_ONLY=1 support to scripts/bolt/setup_env.sh.
• Libs-only mode skips TensorRT and TensorRT-LLM import checks.
• The mode still installs dependency-free wheels and preserves library files needed for BOLT processing.
• Updated scripts/bolt/internal/slurm_merge.sh to enable the mode.
• Default setup behavior remains unchanged.
• End-to-end profile merging and re-BOLT testing completed successfully.

QA Engineer Review

No test changes.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 33186b63-9430-477c-bcbe-cedec42ff848

📥 Commits

Reviewing files that changed from the base of the PR and between ef3124d and 061cab2.

📒 Files selected for processing (2)

• scripts/bolt/internal/slurm_merge.sh
• scripts/bolt/setup_env.sh

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

The setup script now supports libraries-only installation without TensorRT runtime checks. The Slurm merge job enables this mode before backing up libraries and converting profiling data. Normal setup retains its existing TensorRT validation.

Changes

Library-only environment setup

Layer / File(s)	Summary
Add libraries-only setup mode scripts/bolt/setup_env.sh	BOLT_SETUP_LIBS_ONLY=1 skips pre-install TensorRT validation and post-install TensorRT and TensorRT-LLM imports. Default mode retains runtime checks.
Enable mode for Slurm merge scripts/bolt/internal/slurm_merge.sh	The merge job sets BOLT_SETUP_LIBS_ONLY=1 before preparing libraries for profile conversion.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 061ca

The PR adds a libraries-only setup path for BOLT profile merging while preserving library installation and verified merge behavior; no actionable merge-blocking risk remains beyond normal checks and review.

Suggested reviewers: bowenfu

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly identifies the BOLT profile merge change and the new libs-only behavior. It uses a valid project-ticket and infra format.
Description check	✅ Passed	The description explains the failure, the implementation, the affected scripts, and the end-to-end test coverage. The required sections are present and the checklist is reviewed.

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2 files.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[mlefeb01]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #70094 [ run ] triggered by Bot. Commit: 061cab2 Link to invocation

[tensorrt-cicd]

PR_Github #70094 [ run ] completed with state SUCCESS. Commit: 061cab2
/LLM/main/L0_MergeRequest_PR pipeline #57361 completed with status: 'UNSTABLE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[mlefeb01]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #70113 [ run ] triggered by Bot. Commit: fee378c Link to invocation

[tensorrt-cicd]

PR_Github #70113 [ run ] completed with state SUCCESS. Commit: fee378c
/LLM/main/L0_MergeRequest_PR pipeline #57378 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[mlefeb01]

/bot skip --comment "Failure unrelated"

[tensorrt-cicd]

PR_Github #70164 [ skip ] triggered by Bot. Commit: a5f2049 Link to invocation

[tensorrt-cicd]

PR_Github #70164 [ skip ] completed with state SUCCESS. Commit: a5f2049
Skipping testing for commit a5f2049

Link to invocation